Capgemini-觀點：增強軟件團隊：如何從生成式人工智能中獲得最大幫助（英）

AI-AssistedCoding:

AugmentingSoftware

DevelopmentwithGenerativeAI

ExploringtheIntegrationofGenerativeAIinSoftwareEngineeringtoEnhanceCodingandTeamCollaboration

STNETNOC

04

05

07

07

08

08

09

09

11

12

13

14

GenerativeAIissoftwareengineering’slatestandgreatestevolution.

Bringingtomorrow’ssoftwareengineeringparadigmintoview:augmentedsoftwareteams

Howtomoveforward:Aprovenmethodforasoftwareengineeringtransformation

Assesstheorganization’smaturityanddeveloparoadmapwithclearobjectives.

Runreal-worldexperimentationsandmeasuretheimpactofGenerativeAI

DeployGenerativeAIforSoftwareEngineeringatscale

Capgemini’sexperienceinmeasuringGenerativeAIimpact

Measurementprotocolandreal-worldexperimentations:designedforactionableandreliableresults

Prerequisites&SuccessFactorsLessonslearned

Wecandrivesoftwareengineeringtransformationwithyou

Thedawningofaneweraforsoftwareengineering

2AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Disclaimer:Thispaperpresentsforward-lookingperspectivesgroundedinthecuΓΓe∩tla∩dscapeofGe∩eΓativeA/./tΓe卩ectstheΓapidadva∩ceme∩t—both

i∩teΓmsofouΓu∩deΓsta∩di∩ga∩dexpeΓie∩ceofGe∩eΓativeA/tech∩ologies./t

dΓawsi∩sightsfΓomexteΓ∩alΓeseaΓch,tech∩ologyleadeΓsaswellasouΓow∩

expeΓime∩tatio∩swithouΓteamsa∩dwithclie∩ts.Asalways,thefutuΓeΓemai∩su∩pΓedictable,butweca∩establisha∩dimagi∩elikelytΓajectoΓiesa∩doutlooks.

Softwareplaysacriticalroleinmodernbusiness,

whetherit’sintegratedintobusinessappsorproducts.Despiteitsubiquity,theultimatechallengeforsoftwareengineeringorganizationshasalwaysbeentorelease

qualitysoftwarefastenoughtokeepupwithconstantlyacceleratingdemand.

Overtheyears,productivityandqualitystandards

havebeenprogressivelyenhancedthroughthe

adoptionofnewmethodsandtechnologies.Agile

andcontinuousproduct-centricapproaches,software

lifecycleautomation,open-sourceecosystems,cloudnativeandcomposablesoftwarearchitectures,DevOpscontinuum,andlow-code/nocodedevelopmenthaveallbeenintroduced.Andeachhasbuiltontheadvancesofpastiterations.

Butdemandforquickertime-to-valueandbettercostefficiencyisstillaccelerating.Thisleavessoftware

organizationsstrugglingtodeliverattheexpectedpace,withtheexpectedquality,whilecontrollingtechnicaldebtandcosts.

3AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

GenerativeAIissoftwareengineering’slatestandgreatestevolution.

GenerativeAInowstirswonder

andexcitementacrossmany

applicationfields,notleast

softwareengineering.BillGatesiscallingitslatestadvancementthemostrevolutionarytechnologicalachievementinover40years,1

withthepotentialtodramaticallyimprovethewayorganizations

meetbusinessandITchallenges.

Theintersectionofbusiness

andtechnologywillbeattheheartofgenerativeAI’simpact.

Developmentsononeside

influencetheother.Forexample,

efficientsoftwareengineeringwithhighqualitycanreducetimeto

marketandthusprovidebusiness

valueearlier,givingleading

organizationsbothasoftwareandbusiness-orientededge.

OurCapgeminiResearch

Institutereportshowsthat61%

oforganizationsseeenabling

moreinnovativework,suchas

developingnewsoftwarefeaturesandservices,astheleadingbenefitgenerativeAI.Closebehindare

improvingsoftwarequality(49%)andincreasingproductivity(40%).Organizationsareutilizingtheseproductivitygainsoninnovativeworksuchasdevelopingnew

softwarefeatures(50%)and

upskilling(47%).Veryfewaimtoreduceheadcount(4%).2

AccordingtoanotherCapgeminiResearchInstitutesurveyacross

800organizations,67%of

executivesseethemostpotentialforgenerativeAIintheITfunctiontodriveinnovationandcreate

value3Moreover,accordingto

Forresterresearch,“Off-the-shelfandcustomAIsoftwarespend

willdoublefrom$33billionin

2021to$64billionin2025andwillgrow50%fasterthantheoverallsoftwaremarket,withanannualgrowthrateof18%.”4

Inbothlegacysoftware

modernizationandnewsoftwaredevelopmentcontexts,generativeAIgivesbacktimetosoftware

engineers.Theyarefreedup

tofocusonbusinessdemand,

softwarequality,security,andtheadvancedfeaturesrequiredby

newsoftware.

GenerativeAIalsohasapositive

impactonsoftwareprofessionals’jobsatisfaction.69%ofsenior

softwareprofessionalsand55%

ofjuniorsoftwareprofessionals

reporthighlevelsofsatisfaction

fromusinggenerativeAIfor

softwaredevelopment.78%

ofsoftwareprofessionalsare

optimisticaboutgenerativeAI’s

potentialtoenhancecollaborationbetweenbusinessandtechnologyteams5

GenerativeAIwillsupplement

bandwidth-usingthesame

capacity-forhigherproductivityandefficiency,alongwithfaster

timetomarket.Butonlyif

organizationsandtheiremployeescommittogettingonboard,

keepinginmindthatearlyadopterswilltakealeadingpositionin

thefield.

Inaddition,generativeAIpresentsanopportunitynotonlyto

transformbutalsotostandardizeandenhancethedeliveryof

software.Thebenefitsbrought

byusinggenerativeAIinsoftwareengineeringcanextendtoother

areasofthebusinesstocreateevenmorevalue,reduceITcostsand

minimizetechnicaldebt.

OrganizationsusinggenerativeAIhaveseen7%–18%productivity

improvement6inthesoftware

engineeringfunction,compared

totheirinitialestimates.Creatingliteratureanddocumentation,

andwritingcodeandscriptsshowthegreatesttimesavingwith35%maximumand10%average,and

34%maximumand9%average

respectively.7GenerativeAIalso

hasapositiveimpactonsoftwareprofessionals’jobsatisfaction.69%ofseniorsoftwareprofessionals

and55%ofjuniorsoftware

professionalsreporthighlevelsofsatisfactionfromusinggenerativeAIforsoftware.78%ofsoftwareprofessionalsareoptimistic

aboutgenerativeAI’spotentialtoenhancecollaborationbetweenbusinessandtechnologyteams.8

[1]

/The-Age-of-AI-Has-Begun

[2]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

[3]CapgeminiResearchInstitute“GenerativeAIinOrganizations”,July2023

[4]GlobalAISoftwareForecast,ForresterResearch,Inc.September29th,2022

[5]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

[6]Bytotalproductivityimprovementwemeanoverallimprovement

intheproductivityoftheindividualfromalltypesoftasksacceleratedbygenerativeAI.

[7]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

[8]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

4AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Bringingtomorrow’ssoftwareengineering

paradigmintoview:augmentedsoftwareteams

WebelievegenerativeAIwill

graduallytransformtheway

softwareisdeveloped.Theadventoflargelanguagemodels(LLMs)hasintroducedacompelling

rationaleforaparadigmshifttomoreAI-assisted(augmented)

softwareteams.TheintegrationofgenerativeAIintosoftware

engineeringoffersthepromiseofsignificantlyelevatedproductivityandenhancedquality.The

foundationalprinciplesand

methodsofAgileandDevOpsareretained,includingcollaboration,

adaptability,timetovalue,productcentricity,andcontinuousfeedbackloops.

ByworkingwithAIassistants

poweredbyLLMs,augmented

softwareteamscanautomate

mundanetasks,andpromote

morenuanced,datadriven

decisionmaking.Thisoptimizesthesoftwarelifecycleandhelpstoachievemoremilestonesalongtheway.

Wecallthisapproach

conversationalsoftware

engineering.Thisiswheresoftware

teamsprompttheAIassistant,

askingittogeneratesnippetsofcode,troubleshootissues,orevenhelpindesigningandarchitectingthesoftware.Theideaisto

makethesoftwaredevelopmentworkflowmoredynamicand

interactive,throughcontinuous

conversationswiththeAIassistantinthedevelopmentenvironment.It’sawaytostreamlinetasksandsolveproblemsmoreefficiently,

enablingsoftwareteamstofocusonmorecomplexandcreativeaspectsthatboostoverallproductivity.

Themodernevolutionofpair

programmingwithgenerative

AIAugmentedsoftwareteams

caneffectivelybenefitfromand

optimizeusageofgenerativeAI

througharevisedorganizationandnewwaysofworking,guidedby

thefollowingprinciples:

?Augmentedpairprogramming:

Thebasicunitofanaugmentedsoftwareteamisoneormultiplepairsofsoftwareengineers

workingintermittentlywitheachotherorindividuallywiththe

AIassistant.

?AI-humancollaboration:TeammembersworkindividuallywiththeAIassistanttoautomate

repetitivetasks,understandandsolveproblems,orbrainstorm

ideas,leveraginganLLM’sspeedandknowledge,whileensuringhumancreativity.

?Human-humancollaboration:

AfterconversationswithanAIassistant,humansreviewthegeneratedoutputandenhanceitsquality.

?Seniorcoordination:Aseniorleadoverseestheaugmentedteam,coordinatingefforts

andresolvingconflicts,while

ensuringcontrolandvalidationtopromoteasmoothworkflowandhealthyteamdynamic.

[9]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

AccordingtoCapgeminiResearchInstitute,thebiggestgapin

essentialprerequisitesisusually

accesstoplatformsandtools,

includingIDEs,automationand

testingtools,andcollaboration

tools.Only27%oforganizationsclaimtohaveaboveaverage

availabilityofthese.Oftesting

domainprofessionals,24%say

theyhaveaccesstothesetools,

comparedto19%ofproject

andprogrammanagement

professionals,furtherhighlightinggapswithinthesoftware

engineeringfunction.9

5AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

DeepdiveonCodingAssistants

Applyingthetechnologytosoftwareengineeringwillsignificantlyassistsoftwareteamsinthemultitudeoftaskstheyperformacrossthetraditionalsoftwaredevelopmentlifecycle(SDLC).

Forexample,softwareengineersusegenerativeAIwhentransformingbacklogstoriesintosoftware,throughdesignandcodingactivities.GenerativeAIcanbeharnessedtocreatedesignoutputssuchasuserinterfacemockups,

entitymodels,andapplicationprogramminginterfaces(APIs).Thisleadstoasignificantproductivityimprovementwithoutcompromisingquality,asdesignoutputsarealwaysreviewed,updated,andvalidatedbysoftwareengineersortechnologyleads.

AlthoughadoptionofgenerativeAIforsoftwareengineeringisstillinitsearlystages,functions.GenerativeAIisexpectedtoplayakeyroleinaugmentingsoftwareworkforcewithbetterexperience,toolsandplatforms,andgovernance,assistinginmorethan25%ofsoftwaredesign,development,andtestingworkby2026.10

GenerativeAIisalsopoisedtoredefineconventionalprogrammingpracticesbyshiftingthefocusfromcodingtopromptengineeringandcodeproofreading,asconfirmedbyAndrejKarpathy,anOpenAIcomputerscientist,whorecentlysaid:“thehottestnewprogramminglanguageisEnglish.”11Asanexample,usingplainlanguage,softwareengineerscandescribetheintendedfunctionalityofasoftwarefeature,thenreview,update,andvalidatethe

generatedoutput.Therearemanyotherexamples,suchasauto-completionofcode,generatingcodeforunittesting,(retro)documentation,andcodemigrationfromonelanguagetoanother.

AveryimportantvalueofgenerativeAIisthatitsupportsdevelopersalreadyduringcoding.Itcaneithersuggestcleancodedirectlyorevaluateexistingcodetoimprovesoftwarequalityifitfindsissues.

[10]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

[11]

/karpathy/status/1617979122625712128?lang=en-GB

6AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Howtomoveforward:

Aprovenmethodforasoftwareengineeringtransformation

Tostartthisjourneywith

generativeAI,organizations

willreaphugebenefitsfrom

partneringwithanexperiencedandtrustedadvisor.InadditiontohavingconsiderableAIexpertise,thispartnershouldbereadyto

overseeexperimentationsandformcollaborativeprojectteamsthateventuallyflourishintojointsoftwarehouses.

Avarietyoffactors,suchas

companysize,staffingpyramid,industry-specificgoals,meanthateachorganizationalunit

progressesatadifferentpaceandwithadifferentmodel.Therefore,eachwillhavedistinctneedsandchallengestoconsiderbefore

generativeAIcanbecomeapartofdailyoperations.

It’simportanttofocusonthreeareas:

?Assesstheorganization’smaturityanddeveloparoadmap.

?Runexperimentationsandmeasureimpact.

?Deploymentatscale.

Assesstheorganization’smaturityanddeveloparoadmapwithclearobjectives.

Withinacompany,thereare

usuallymanyvariationsofanSDLC.Typically,everyprogram/project/producthasitsownSDLCversion.Ameticulousassessmentand

deepunderstandingoftheSLDCofanITdomainwillrevealthe

currentmaturityofitssoftwareengineeringprocesses.And

whethertheyalignwithindustrybestpractices.Theevaluation

willalsoidentifyareasfor

improvement,particularlywherebottlenecksorinefficienciesoccur.

Basedonthisassessment,

objectivescanbedefinedby

selectingthemostpromising

transformationenablers,aswellasidentifyingtheirassociatedrisks

andchallenges.SomeenablerswillbepoweredbygenerativeAIwhile,forothers,itwillbemoreamatterofapplyingsoftware

engineeringbestpractices.It’simportanttodefinekeymetrics

tomeasuretheoutcomesofthefuturetransformationandinvolvestakeholdersearlytofostera

collaborativeenvironment.

Finally,acomprehensivevalue,

accessibility,andriskanalysiswillserveasabasisforestablishingaroadmapforchange.Alloptionswillbeweightedandprioritized

tomakeinformeddecisionsaboutresourceallocationasdifferent

domains,teams,androlesaredefinedforeachoption.

7AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Runreal-worldexperimentationsandmeasure

theimpactofGenerativeAI

Withtheprioritiesset,it’stime

toselectthebestgenerativeAI

toolstoachievethem.Andput-upguardrailstomanageanylegal

andcybersecurityrisks,while

controllingcosts.Measurementandcontinuousimprovementwillbepivotal.Asgenerative

AIisinsertedintomoreSDLC

processes,organizationsneed

togaugeitsimpact.Thismeansmeasuringcriticalaspectssuch

asproductivityenhancements,

softwarequality,time-to-market,anddeveloperexperience.A

feedbackloopshouldbeputinplacesothepaceandscopeofthedeploymentcanbeadaptedtoaccountforinefficiencies.Orrespondtoevolvingneedsandspecificobjectives.Critically,

theexperimentationsand

measurementsshouldberelevanttothebusinesscontextand

industryenvironmentoftheorganization.

Buttobetrulycompetitive,

organizationsneedmorethan

internalmeasurement.At

Capgemini,wehavedevelopedanindustrializedvaluemeasurementprotocolthatevaluatesthe

objectiveimpactofgenerative

AIacrossanorganization’smanySDLCs.Itisusedtomeasureandcompareanorganization’smetricsagainstourbenchmark,which

factorsinallourrelatedinternal

andexternalgenerativeAIprojects.Thisgivesorganizationsaclearviewofhowtheystackupagainsttheirpeers.

Toputthisintoperspective,

theadoptionofgenerativeAI

forsoftwareengineeringisstill

initsearlystageswith9in10

organizationsyettoscale.27%

oforganizationsarerunning

generativeAIpilots,and11%havestartedleveraginggenerativeAIintheirsoftwarefunctions.

Threeinfour(75%)large

organizations(annualrevenuegreaterthan$20billion)haveadopted(piloted/scaled)

generativeAIcomparedto23%oftheirsmallercounterparts(annualrevenuebetween$1–5billion).12

DeployGenerativeAIforSoftwareEngineering

atscale

Afterexperimentingwith

generativeAIthroughreal-world

pilotsandfullscaledelivery,

manyorganizationswillwantto

broadentheirapplicationsand

possiblyinvolvehundredsoreventhousandsofdevelopers.For

suchlargedeployments,careful

considerationmustbegiventotheorganizationalandHRimplications.

Sincevariousrolesmayneedto

change,it’simportanttointroduceagenerativeAIupskillingprogramthatwillhelpshapethenew

softwareengineeringpyramid,

andaddressthewaysofworkinginrespecttotheskillsrequired.Thiswilllaythefoundationfor

astrategicplanthatgradually

integratesgenerativeAIintothesoftwarelifecycle.Afurtherstep

maybetooffercoachingand

assistancetosteeremployeesastheypreparetoworkalongsideahostofnewgenerativeAItools.

Finally,adedicatedteamshould

besetuptodefinebusinesscases,measureprogress,andensuretheoutcomesmeetexpectations.Thisisessential,asthepaceandscopeofeachdeploymentwillinvariablyneedadjustment.

[12]CapgeminiResearchInstitute“Turbochargingsoftware”,June2024

8AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Capgemini’sexperienceinmeasuring

GenerativeAIimpact

Ourexperienceworkingwith

clientsacrossallsectorstells

usthatit’snotaquestionofif

generativeAI-poweredsoftwareengineeringwilldisruptand

reinventanorganization,but

ratherwhen,howquickly,andhowradically.Therearetwoquestionswehearthemostfromour

clients.First,howwillgenerative

AI-basedassistanceaffectsoftwareproductivityandquality.And

second,howwillsoftwareteams’organizationandwaysofworkingchangebecauseofit?

Atthebeginningof2023,

Capgeministartedalarge-scaleglobalprogramtoexperimentwithusecasestomeasure

generativeAI’simpactthrough

experimentations,bothinternallyandjointlywithclients.Theaimwastoseewhereandhowgenerative

AIcanaugmentthemanytasks

softwareteamstypicallyperform.Wedeployedareliableand

consistentmeasurementprotocolandnowweareprogressively

consolidatingourresultsintoa

repositoryforinternalandexternalusage/benchmarking.

Measurementprotocolandreal-worldexperimentations:designedforactionableand

reliableresults

Measuringproductivitywithin

softwaredevelopmentposes

inherentcomplexitiesdueto

themultifacetednatureoftheSDLC,thedynamicenvironmentitoperatesin,andthesubjective

andintangibleaspectsofmanyofitscomponents.Acomprehensivemeasurementapproachmust

encompassbothqualitativeand

quantitativefactors,tailoredtothespecificcontextoftheproject.

9AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Similarly,assessingsoftware

qualitypresentschallengesasit

involvesvariousdimensionssuchasfunctionality,performance,

reliability,usability,maintainability,security,andscalability,each

requiringitsownsetofmetricsandcriteria.

Moreover,solicitingfeedback

fromsoftwareengineersutilizingGenerativeAIonadailybasisis

crucial,consideringitsimpacton

thedevelopmentenvironmentandwoΓkHow.TheΓefoΓe,it’simpeΓativetodeviseandimplementapracticalmeasurementprotocol.Thisshouldbefocusedonunderstanding

Ge∩eΓativeAl’si∩Hue∩ceo∩codi∩9andunittestingwithinbespoke

applicationdevelopment,providingacleaΓi∩si9hti∩toitse阡ects.

Themeasurementprotocolisa

combi∩atio∩ofseveΓaldi阡eΓe∩ti∩9Γedie∩tsa∩dawell-de?∩ed

processtoproducetocreate

comprehensible,actionableandreliableresults.Thisincludesa

measurementapproach,metrics,team,a∩dawell-de?∩edpΓocess.

Measurementapproach:A

robustmeasurementapproach

includesstepsformeasuring

progress,includingplanning,

settingbaselines,andrunning

anexperimentation.Theright

measurementmetricsarekey,

andwellexploretheminthenextchapter.Tosupporttheapproach,metricstoolslikeSonarQube,

CAST,Jira,anddevelopersurveysareusedforcollectingand

analyzingdata.Prerequisites

andsuccessfactorsforaproperandconsistentmeasurement

areteamstability,duration,

technology,legalconsiderations,

andcybersecurity.Anormalization

processmusthandlevariability

duringexperimentationexecution,adjustingmetricstochangesin

teamsize,capacity,orcomplexity.

Metrics:Codingvelocityservesasapivotalmetrictomeasureteamproductivity,focusingoncoding

andunittestingactivities,typicallyqua∩ti?edbyimpleme∩tedstoΓypoints.Otherdimensionsmust

alsobeevaluated-likevelocityperdeveloperseniorityandvelocity

percomplexityofuserstories.ThiscompΓehe∩siveappΓoacho阡eΓs

insightsintohowGenerativeAIi∩Hue∩cespΓoductivityacΓoss

di阡eΓe∩tdevelopeΓskilllevelsandtheintricaciesofsoftwaredevelopmenttasks.

Besidesthat,whenitcomesto

testing,themetricforUnitTest

Coverageisessential.Ithelpstoassessthequalityandreliability

ofsoftware.Tokeepitsimplewefocusoninstructioncoverage(C0)asthisismeasuredbymostofthetools.

Ontopofthoseourprotocol

includesmanymoremetricslike

codee代cie∩cy,codesecuΓity,codesmells,codeduplication.

Team:Inthesingleteamapproach,oneteamsequentiallyexecutes

abacklogofuserstoriesof

consistentsizeorcomplexity,

comparingperformancewithandwithoutGenerativeAIassistance.

Ontheotherhand,themultiple

teamsapproachinvolvesparallel

executionofthesamebacklogbyatleasttwoteamswithdi阡eΓe∩t

toolsetups.Forexample,onewithGenerativeAItoolsandtheother

withoutGenerativeAItools.This

allowsforsimultaneousassessmentofGe∩eΓativeAl’se阡ectsacΓoss

di阡eΓe∩tteamdy∩amics.

Theseniorityorcapabilities

ofateamareimportantfor

normalization,andsoits

mandatorytoknowwhatkindofteammixisworkingonthede?∩edbacklo9.Wedisti∩9uishaseniorteamofhighlyskilledindividualsrepresentingthe

9oldsta∩daΓdofpΓo?cie∩cy.

Conversely,thewell-balanced

teamconsistsofagoodmixof

seniorsandjuniors,necessitatingsomelevelofcoachingtypically

facilitatedbyseniormembers

alongsidedailywork.Finally,the

juniorteamfeaturesfewseniors,withtheprimaryfocusoncoachingjuniormembersdevelopment

andproductivity.

Process:Oncealltheingredientshavebee∩de?∩ed,apΓocessisneededtoensurehighquality

Γesultsa∩dΓeducesidee阡ectsduetoestimationinaccuracyandthe

humanfactor.

1.De?∩etheteam(s)oΓ9a∩izatio∩andtheexperimentationscopeandtimeline.

2.De?∩ethemeasuΓeme∩tapproach.

3.Validatetheprerequisitesandthesuccessfactors.

4.Conductabaselineforthemetrics,without

GenerativeAIassistance.

5.Executetheexperimentationsprintswith

GenerativeAIassistance.

6.Collectandnormalizethemetricsandthefeedback.

7.Consolidateandreportthemeasuredresults.

10AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Prerequisitesandsuccessfactors

Successfulexperimentation

hingesonmanagingthefollowingprerequisiteseffectively:

?Teamstabilityisvitalfor

achievingcomparableresults.Thisincludesconsistentsize,seniority,andunwavering

processes.

?Baseliningnecessitatesa

minimumofthreesprints,whileGenerativeAIexperimentationsbenefitfrom6-9sprints.Awell-definedbacklogwithdiverse

userstoriesisessentialinputforthesprintsandshouldbeoperatedaccordingly.

?Measurementtoolsmustbe

readilyavailableforanaccurateassessment.Allteammembersneedaccess.Compliancewith

regulations,encompassinglegal,security,andprivacyaspects,

isimperativethroughouttheprocess.

Baseliningiscrucialfor

understandingthecurrentvelocityandqualityofthetraditionalwaysofworkingwithoutGenerative

AItools.It’srecommendedthat

insightsaregatheredfromthe

lastthreesprints,excluding

GenerativeAIassistance.Coding

velocitydatashouldbegatheredfromteambacklogmanagementandcollaborationtools,aswell

ascodequalityreportsfrom

staticcodeanalysistools.Ifdata

fromthelastthreesprintsisn’t

available,threesprintswithout

GenerativeAIassistancemustbeconductedbeforeproceeding

totheGenerativeAIphase.The

baselinemustbecalculatedbasedonthis.Additionally,thedeveloperexperiencesurvey,basedonthe

experiencewithoutGenerativeAI,isconductedduringthe

baselinephase.

Forasuccessfulexperimentation,it’simportanttomaintaina

constantteamsizeandcarefully

curatethebacklogtoincludeusecasesofvaryingcomplexities.Theteammustbeequippedwiththenecessarysetuptoeffectively

utilizetheGenerativeAItool.

Additionally,assigningadedicatedtoolexperttoeachteamcan

significantlyenhancesuccessandefficiency.Monitoringtheteam

involvesensuringtimelytask

updatesonplatformslikeJIRAorAzure,withteamleadsresponsiblefortrackingandensuring

compliance.Inanaugmented

teamapproach,peerreviewsareessential,andtheGenerative

AItoolshouldbeutilizedasan

assistantthroughouttheprocess.

11AugmentedSoftwareTeams:HowtodrivemaximumassistancefromGenerativeAI

Lessonslearned

Likeallorganizations,weare

learningfromGenerativeAIinrealworldandrealsizeconditions.

Belowareourhigh-levellearnings:

ExperimentationFramework:OurexperimentationswithgenerativeAIincustomsoftwareengineeringtypicallyinvolveonetomultiple

teams,spanningaminimum

durationofsixsprints(preferablynine),withadiversebacklogof

userstoriesandcomplexitymix.

Differentteamconfigurations

offervaluableinsights.Typical

configurationsmightinclude,

existingteamstransitioningto

usinggenerativeAI.Newteams

integratinggenerativeAIlater.Andshadowteamsworkingalongsideexistingones.

BaselineEstablishment:Solid

baseliningiscrucialtoensure

robustandrepresentative

comparisonsacrossvarious

metrics.Theseincludevelocity,quality,security,time-to-market,anddeveloperexperience.

Quantifiableresultsfrom

experimentationshowcaseproductivityimprovements.

ToolingPerspective:

Experimentationsinthecodingandunittestingstagesofthesoftwarelifecyclecovermultipletechnologycombinations.Thisincludesvendor-packagedsolutionswithgenerativeAIextensionsandfoundational

LLMsenhancedwithcontextualizedpromptengineeringtechniques.

Pro