test approach faqs v2測試方法常見問題解答

TestApproachFAQs6/14/2011MajorTestingPhasesStringCycle2Cycle1Cycle3TestPhase~600FTScripts~50~75E2EScripts~3000FTScriptsAugSeptOctNovDecJanFebMarApr~1700FTScripts~2400FTScriptsCycle1Prep~0E2EScriptsQAEnvt(410)QAEnvt(410)~20~30E2EScriptsTrainingEnvt(510)QAEnvt(410)UVT~100FTScripts~50~75E2EScriptsDREnvt(TBD)Throughoutthetestingtimeframe:TestswillshiftfocusfromFunctionTestswithinaBPDtoEndtoEndtestsacrossBPDsandsystemsasthephaseprogressesTestExecutionInventoryforeachcyclewillbeselectedbasedon“Risk”ofagivenscriptandavailabilityofdevelopmentobjectsDataSetupwillshiftfrom“Creation”to“Selection”asconverteddataesavailableEntranceandExitCriteriawillbeestablishedforbothFunctionalandE2ETestsforeachcyclee.g.IngeneralE2EtestsshouldnotstartuntiltheassociatedcomponentlevelfunctiontestshavepassedExitcriteriaforSolutionPassrateswillbeestablishedExitcriteriaforIssueBacklog(durationbypriority)willbesetandmeasuredbyfixteamMock2Mock1Mock4Mock5FTScripts=FunctionalTestScripts(i.e.,whatiswrittenattheBPDLevel)E2EScripts=CompleteEndtoEndScripts(maybeequivalentto~20-40FunctionalScripts)SecurityRoleandRegressiontestingwillbescheduledduringtheendofCycle2andthroughoutCycle3.ThetablebelowdescribestheassumptionsabouttestcoverageandscriptcountsacrosseachoftheprogramphasesItisexpectedthattherewillbe~16coreE2Escenarioseachwithafewvariationsleadingto~50-75totalEndtoEndScriptsFunctionTestscriptswillbeexecutedinparallelwithE2EstoverifyspecificfunctionalityIngeneral,DataseedingwillbelimitedtoacasewheredatadoesnotalreadyexisttoselectfromScriptswillbeorderedsuchthattheygeneratedatathatcanbeusedbyfuturescriptsduringdatathescriptdataselection/setupTestPhaseExecutionAssumptionsDatesPhase%ofSolutionTestedFunctionScriptsE2EScriptsDataSeedingBatch8/1-9/2StringTest~20~30%600~10000AllManual9/5-10/28Cycle1~50%17000TransactionalManual11/1-1/27Cycle2100%3000~20~30If

neededManual+Automated2/1-3/30Cycle3100%~2400~50~75IfneededManual+Automated3/19-4/27UVT~30%100~50~75IfneededAutomatedTestApproachFAQsQuestionApproachWhatisthedifferencebetweenST,Cycle1,Cycle2,Cycle3?StringTestingwillfocusonvalidationofconfigurationobjects,transactioncodes,masterdatasetup,toprepareforCycleTesting.Validationmayoccurwithinindividualsystemse.g.,itwouldbeacceptableinstringtesttomanuallyseedaninterfacingapplicationtotestit’sownfunctionalityCycle1willfocusonfunctionaltestingroughly50%ofthetotalscriptinventorythathasbeendeveloped.TheexactinventorywillbeselectedbasedonScriptRiskTierandDevelopmentAvailability.Cycle2willincorporatebothfunctionaltestingscriptsandbegintoencompassEndToEndTesting.Cycle2willbetheprogramsfirstattempttotest100%oftotalsolutionscope.Cycle3willfocuson100%EndtoEndtestscriptfocused,andsecondpassatselectedfunctiontestingscriptstoensurea2ndpassofcriticalfunctionalityandoverallregressiontestingofmajorchanges.UVTwilllargelybe100%EndtoEndtestscriptfocusedwithasmallnumberofkeyfunctiontestingscriptslargelyexecutedbybusinessSMEs,toensureanyfinalregressiontestinghasbeenvalidated.TestApproachFAQsQuestionApproachHowdoyoudeterminethescopeofeachtestpass?ScopewillbedrivenbyselectingtestscriptsbasedonRiskTierandavailabledevelopmentobjects.RisktierwillbebasedontheProcessPriority,anyassociatedRICEFWComplexity,andInterfacingApplicationSupportTier.TheearlytestcyclesshouldfocusontestingmaximumavailablecustomdevelopmentobjectswithrestofthepoolcomingfromnonRICEFbasedscriptsTestpassesstartingwithcycle2willalsoincludeEndtoEndtestingWillyoualwaysdependoninterfaceobjecttobereadybeforetestingalltheassociatedfunctionality?Withinthetestenvironment,Testingforsystemconnectivity/datatransferforinterfaceswillnothappenunlessallthecomponents(SAP,Legacy,Middleware)arereadyfortestingScriptstargetingfollowonsystemfunctionalityshouldbetestedbymanualdatacreationinearlytestpasseseveniftheinterfaceisnotyetavailabletotestDoweexecuteeverytestscriptineachtestpass?Needanideaonhowmetricswillbemeasured.WedonotneedtoexecuteeveryscriptineachtestpassOnescriptcouldbeexecutedmultipletimestotestcriticaldatavariations–multipleexecutionrecordswillbecreatedforascriptinthatcaseDuringlatertestpasses,somescriptswillformEndtoEndscenarios.MetricswillbemeasuredforoverallExecutionRateandPassRate;groupedbythe“HealthChart”categoryaswellasEndtoEndscenarios.AdditionallymetricsforIssuebacklogbyteamandprioritywillbemanaged.Whatisapproachonuseraccessandsecuritytesting?AlltestersininitialtestCycleswillhaveallinclusivefunctionalaccessRolebasedtestingwillstartatunitlevelduringCycle2andprogressto100%rolebasedtestingduringUVT.(Detailedapproachisbeingworkedon).TestApproachFAQsQuestionApproachHowtohandletestscenariosinwhichweneedstepstooccurfromdifferentteams?Ensuretestchampionsareworkingacrossteams,anddrivingtheinclusionofstepswithintherelevantscriptsIfthescriptbelongingtootherteamisalreadycompletedandneedstobeupdated,wecancreatea“scriptupdate”IssuewithinQCtorequestanupdatetoacompletedscript.Mapcrossteamrequirementsdirectlytotestscriptswhereitisapplicable.Datastewards(extensivelyworkingcrossteam)willberesponsibleforupdatingtheactualvalidationvaluesaspartofexecutionpreparationIfateamcreatesitsownvalidationscript,itshouldmentionthepre-requisitescriptownedbytheotherteamsothatthisdependencyiswellunderstoodduringexecutionsequenceplanningHowwillwestringscriptsacrossSAPandLegacysystems?TestteamtoorganizeaweeklymeetingwithalltheLegacyandSAPTestchampionstoprovidecrossteamcollaborationforumFDandinterfaceApp.mappingagainstexistingSAPandLegacyscriptsexpectedtocompletebyJuly8TestChampionsshouldgothroughalltheinterfacerelatedscriptstoensuretherearecorrespondingscriptsonbothSAPandLegacysideandcaptureexecution/datacreationdependenciesTestChampionstocollaboratewithcrossteamcounterpartstoensurecoherenttestingTestApproachFAQsQuestionApproachWhatisthedatapreparationstrategyforeachtestpass?ThetestscriptexecutionwillbesequencedperthenormalbusinessprocessflowtoensurethatsubsequentscriptsusethedatacreatedbypreviousscriptswhereverpossibleDatastewards(2-3resourcesperteam)willbeidentifiedtocaptureanddocumentdatarelatedtoexecuteandvalidatethescriptsandwillneedextensivecrossteamcollaborationDatastewardswillberesponsibleforattachingdatasheetstothescriptspriortoexecutionStringTesting–AlldataexpectedtobecreatedmanuallypriortoscriptexecutionCycle1–Mock1conversionwillfeedtheCycle1environmenthoweverscopewillbelimitedtosmallnumberofmasterdataobjects.MostofthetransactionaldataaswellasmasterdatanotconsideredbyMock1conversionswillhavetobecreat