視頻分析中的人工智能 基于機(jī)器學(xué)習(xí)的分析注意事項(xiàng) 和深度學(xué)習(xí)

WHITEPAPER

AIinvideoanalytics

Considerationsforanalyticsbasedonmachinelearninganddeeplearning

March2021

PAGE

10

PAGE

13

TableofContents

Summary

3

Introduction

4

AI,machinelearning,anddeeplearning

4

Machinelearning

4

Deeplearning

5

Classicalmachinelearningvs.deeplearning

6

Thestagesofmachinelearning

6

Datacollectionanddataannotation

7

Training

7

Testing

8

Deployment

9

Edge-basedanalytics

9

Hardwareacceleration

9

AIisstillinitsearlydevelopment

9

Considerationsforoptimalanalyticsperformance

10

Imageusability

10

Detectiondistance

11

Alarmsandrecordingsetup

11

Maintenance

12

Privacyandpersonalintegrity

13

Appendix

14

Neuralnetworks

14

Convolutionalneuralnetworks(CNN)

15

Summary

AI-basedvideoanalyticsisoneofthemostdiscussedtopicsinthevideosurveillanceindustry.Someoftheapplicationscansubstantiallyspeedupdataanalysisandautomaterepetitivetasks.ButAIsolutionstodaycannotreplacethehumanoperator’sexperienceanddecision-makingskills.Thestrengthliesinsteadinacombination:takingadvantageofAIsolutionstoimproveandincreasehumanefficiency.

TheAIconceptincorporatesmachinelearningalgorithmsanddeeplearningalgorithms.Bothtypesautomaticallybuildamathematicalmodel,usingsubstantialamountsofsampledata(trainingdata),togaintheabilitytocalculateresultswithoutbeingspecificallyprogrammedforit.AnAIalgorithmisdevelopedthroughaniterativeprocess,inwhichacycleofcollectingtrainingdata,labelingtrainingdata,usingthelabeleddatatotrainthealgorithm,andtestingthetrainedalgorithm,isrepeateduntilthedesiredqualitylevelisreached.Afterthis,thealgorithmisreadytouseinananalyticsapplicationwhichcanbepurchasedanddeployedonasurveillancesite.Atthispoint,allthetrainingisdoneandtheapplicationwillnotlearnanythingnew.

AtypicaltaskforAI-basedvideoanalyticsistovisuallydetecthumansandvehiclesinavideostreamanddistinguishwhichiswhich.Amachinelearningalgorithmhaslearnedthecombinationofvisualfeaturesthatdefinestheseobjects.Adeeplearningalgorithmismorerefinedandcan-iftrainedforit-detectmuchmorecomplexobjects.Butitalsorequiressubstantiallylargereffortsfordevelopmentandtrainingandmuchmorecomputationresourceswhenthefinalizedapplicationisused.Forwell-specifiedsurveillanceneeds,itshouldthereforebeconsideredwhetheradedicated,optimizedmachinelearningapplicationcanbesufficient.

AlgorithmdevelopmentandincreasingprocessingpowerofcamerashavemadeitpossibletorunadvancedAI-basedvideoanalyticsdirectlyonthecamera(edgebased)insteadofhavingtoperformthecomputationsonaserver(serverbased).Thisenablesbetterrealtimefunctionalitybecausetheapplicationshaveimmediateaccesstouncompressedvideomaterial.Withdedicatedhardwareaccelerators,suchasMLPU(machinelearningprocessingunit)andDLPU(deeplearningprocessingunit),inthecameras,edge-basedanalyticscanbemorepower-efficientlyimplementedthanwithaCPUorGPU(graphicsprocessingunit).

BeforeanAI-basedvideoanalyticsapplicationisinstalled,themanufacturer’srecommendationsbasedonknownpreconditionsandlimitationsmustbecarefullystudiedandfollowed.Everysurveillanceinstallationisunique,andtheapplication’sperformanceshouldbeevaluatedateachsite.Ifthequalityisfoundto

belowerthanexpected,investigationsshouldbemadeonaholisticlevel,andnotfocusonlyontheanalyticsapplicationitself.Theperformanceofvideoanalyticsisdependentonmanyfactorsrelatedtocamerahardware,cameraconfiguration,videoquality,scenedynamics,andillumination.Inmanycases,knowingtheimpactofthesefactorsandoptimizingthemaccordinglymakesitpossibletoincreasevideoanalyticsperformanceintheinstallation.

AsAIisincreasinglyappliedinsurveillance,theadvantagesofoperationalefficiencyandnewusecasesmustbebalancedwithamindfuldiscussionaboutwhenandwheretoapplythetechnology.

Introduction

AI,artificialintelligence,hasbeendevelopedanddebatedeversincethefirstcomputerswereinvented.Whilethemostrevolutionaryincarnationsarenotyethere,AI-basedtechnologiesarewidelyusedtodayforcarryingoutclearlydefinedtasksinapplicationssuchasvoicerecognition,searchengines,andvirtualassistants.AIisalsoincreasinglyemployedinhealthcarewhereitprovidesvaluableresourcesin,forexample,x-raydiagnosticsandretinascananalysis.

AI-basedvideoanalyticsisoneofthemostdiscussedtopicsinthevideosurveillanceindustryandexpectationsarehigh.ThereareapplicationsonthemarketthatuseAIalgorithmstosuccessfullyspeedupdataanalysisandautomaterepetitivetasks.Butinawidersurveillancecontext,AItodayandinthenearfutureshouldbeviewedasjustoneelement,amongseveralothers,intheprocessofbuildingaccuratesolutions.

Thiswhitepaperprovidesatechnologicalbackgroundonmachinelearninganddeeplearningalgorithmsandhowtheycanbedevelopedandappliedforvideoanalytics.ThisincludesabriefaccountofAIaccelerationhardwareandtheprosandconsofrunningAI-basedanalyticsontheedgecomparedtoonaserver.ThepaperalsotakesalookathowthepreconditionsforAI-basedvideoanalyticsperformancecanbeoptimized,takingawidescopeoffactorsintoaccount.

AI,machinelearning,anddeeplearning

Artificialintelligence(AI)isawideconceptassociatedwithmachinesthatcansolvecomplextaskswhiledemonstratingseeminglyintelligenttraits.DeeplearningandmachinelearningaresubsetsofAI.

Artificialintelligence

Machinelearning

Deeplearning

Machinelearning

MachinelearningisasubsetwithinAIthatusesstatisticallearningalgorithmstobuildsystemsthathavetheabilitytoautomaticallylearnandimproveduringtrainingwithoutbeingexplicitlyprogrammed.

Inthissection,wedistinguishbetweentraditionalprogrammingandmachinelearninginthecontextofcomputervision—thedisciplineofmakingcomputersunderstandwhatishappeninginascenebyanalyzingimagesorvideos.

Traditionallyprogrammedcomputervisionisbasedonmethodsthatcalculateanimage’sfeatures,forexample,computerprogramslookingforpronouncededgesandcornerpoints.Thesefeaturesneedtobemanuallydefinedbyanalgorithmdeveloperwhoknowswhatisimportantintheimagedata.Thedeveloperthencombinesthesefeaturesforthealgorithmtoconcludewhatisfoundinthescene.

Machinelearningalgorithmsautomaticallybuildamathematicalmodelusingsubstantialamountsofsampledata–trainingdata–togaintheabilitytomakedecisionsbycalculatingresultswithout

specificallybeingprogrammedtodoso.Thefeaturesarestillhand-craftedbuthowtocombinethesefeaturesislearnedbythealgorithmitselfthroughexposuretolargeamountsoflabeled,orannotated,trainingdata.Inthispaper,werefertothistechniqueofusinghand-craftedfeaturesinlearnedcombinations,asclassicalmachinelearning.

Inotherwords,foramachinelearningapplicationweneedtotrainthecomputertogettheprogramwewant.Dataiscollectedandthenannotatedbyhumans,sometimesassistedwithpre-annotationbyservercomputers.Theresultisfedintothesystemandthisprocessgoesonuntiltheapplicationhaslearnedenoughtodetectwhatwewanted,forexample,aspecifictypeofvehicle.Thetrainedmodelbecomestheprogram.Notethatwhentheprogramisfinishedthesystemdoesnotlearnanythingnew.

Traditionalprogramming:

Dataiscollected.Programcriteriaaredefined.Theprogramiscoded(byahuman).Done.

Machinelearning:

Dataiscollected.Dataislabeled.Themodelundergoesaniterativetrainingprocess.Thefinalizedtrainedmodelbecomestheprogram.Done.

TheadvantageofAIovertraditionalprogramming,whenbuildingacomputervisionprogram,istheabilitytoprocessextensivedata.Acomputercangothroughthousandsofimageswithoutlosingfocus,whereasahumanprogrammerwillgettiredandunfocusedafterawhile.Thatway,theAIcanmaketheapplicationsubstantiallymoreaccurate.However,themorecomplicatedtheapplication,theharderitisforthemachinetoproducethewantedresult.

Deeplearning

Deeplearningisarefinedversionofmachinelearninginwhichboththefeatureextractionandhowtocombinethesefeatures,indeepstructuresofrulestoproduceanoutput,arelearnedinadata-drivenmanner.Thealgorithmcanautomaticallydefinewhatfeaturestolookforinthetrainingdata.Itcanalsolearnverydeepstructuresofchainedcombinationsoffeatures.

Thecoreofthealgorithmsusedindeeplearningisinspiredbyhowneuronsworkandhowthebrainusesthesetoformhigher-levelknowledgebycombiningtheneuronoutputsinadeephierarchy,oranetwork,

ofchainedrules.Thebrainisasysteminwhichthecombinationsthemselvesarealsoformedbyneurons,erasingthedistinctionbetweenfeatureextractionandthecombinationoffeatures,makingthemthesameinsomesense.Thesestructuresweresimulatedbyresearchersintosomethingcalledartificialneuralnetworks,whichisthemostwidelyusedtypeofalgorithmindeeplearning.Seetheappendixofthisdocumentforabriefoverviewofneuralnetworks.

Usingdeeplearningalgorithms,itispossibletobuildintricatevisualdetectorsandautomaticallytrainthemtodetectverycomplexobjects,resilienttoscale,rotation,andothervariations.

Thereasonbehindthisflexibilityisthatdeeplearningsystemscanlearnfromamuchlargeramountofdata,andmuchmorevarieddata,thanclassicalmachinelearningsystems.Inmostcases,theywillsignificantlyoutperformhand-craftedcomputervisionalgorithms.Thismakesdeeplearningespecially

suitedforcomplexproblemswherethecombinationoffeaturescannoteasilybeformedbyhumanexperts,suchasimageclassification,languageprocessing,andobjectdetection.

Objectdetectionbasedondeeplearningcanclassifycomplexobjects.Inthisexample,theanalyticsapplicationcannotonlydetectvehicles,butalsoclassifythetypeofvehicle.

Classicalmachinelearningvs.deeplearning

Whiletheyaresimilartypesofalgorithms,adeeplearningalgorithmtypicallyusesamuchlargersetoflearnedfeaturecombinationsthanaclassicalmachinelearningalgorithmdoes.Thismeansthatdeeplearning-basedanalyticscanbemoreflexibleandcan-iftrainedto-learntoperformmuchmorecomplextasks.

Forspecificsurveillanceanalytics,however,adedicated,optimizedclassicalmachinelearningalgorithmcanbesufficient.Inawellspecifiedscope,itcanprovidesimilarresultsasadeeplearningalgorithmwhilerequiringlessmathematicaloperationsandcanthereforebemorecost-efficientandlesspowerconsumingtouse.Itfurthermorerequiresmuchlesstrainingdataandthisgreatlyreducesthedevelopmenteffort.

Thestagesofmachinelearning

Thedevelopmentofamachinelearningalgorithmfollowsaseriesofstepsanditerations,roughlyvisualizedbelow,beforeafinalizedanalyticsapplicationcanbedeployed.Attheheartofananalyticsapplicationis

oneormorealgorithms,forexampleanobjectdetector.Inthecaseofdeeplearningbasedapplicationsthecoreofthealgorithmisthedeeplearningmodel.

Preparation:Definingthepurposeoftheapplication.

Training:Collectingtrainingdata.Annotatingthedata.Trainingthemodel.Testingthemodel.Ifqualityisnotasexpected,thepreviousstepsaredoneagaininaniterativeimprovementcycle.

Deployment:Installingandusingthefinishedapplication.

Datacollectionanddataannotation

TodevelopanAI-basedanalyticsapplicationyouneedtocollectlargeamountsofdata.Invideosurveillance,thistypicallyconsistsofimagesandvideoclipsofhumansandvehiclesorotherobjectsofinterest.Inordertomakethedatarecognizableforamachineorcomputeradataannotationprocessisnecessary,wheretherelevantobjectsarecategorizedandlabeled.Dataannotationismainlyamanualandlabor-intensetask.Theprepareddataneedstocoveralarge-enoughvarietyofsamplesthatarerelevantforthecontextwheretheanalyticsapplicationwillbeused.

Training

Training,orlearning,iswhenthemodelisfedannotateddataandatrainingframeworkisusedtoiterativelymodifyandimprovethemodeluntilthedesiredqualityisreached.Inotherwords,themodelisoptimizedtosolvethedefinedtask.Trainingcanbedoneaccordingtooneofthreemainmethods.

Supervisedlearning:themodellearnstomakeaccuratepredictions

Unsupervisedlearning:Themodellearnstoidentifyclusters

Reinforcementlearning:Themodellearnsfrommistakes

Supervisedlearning

Supervisedlearningisthemostusedmethodinmachinelearningtoday.Itcanbedescribedaslearningbyexamples.Thetrainingdataisclearlyannotated,meaningthattheinputdataisalreadypairedwiththedesiredoutputresult.

Supervisedlearninggenerallyrequiresaverylargeamountofannotateddataandtheperformanceofthetrainedalgorithmisdirectlydependentonthequalityofthetrainingdata.Themostimportantqualityaspectistouseadatasetthatrepresentsallpotentialinputdatafromarealdeploymentsituation.Forobjectdetectors,thedevelopermustmakesuretotrainthealgorithmwithawidevarietyofimages,withdifferentobjectsinstances,orientations,scales,lightsituations,backgrounds,anddistractions.Onlyifthetrainingdataisrepresentativefortheplannedusecase,thefinalanalyticsapplicationwillbeabletomakeaccuratepredictionsalsowhenprocessingnewdata,unseenduringthetrainingphase.

Unsupervisedlearning

Unsupervisedlearningusesalgorithmstoanalyzeandgroupunlabeleddatasets.Thisisnotacommontrainingmethodinthesurveillanceindustry,becausethemodelrequiresalotofcalibrationandtestingwhilethequalitycanstillbeunpredictable.

Thedatasetsmustberelevantfortheanalyticsapplicationbutdonothavetobeclearlylabeledormarked.Themanualannotationworkiseliminated,butthenumberofimagesorvideosneededforthetrainingmustbegreatlyincreased,byseveralordersofmagnitude.Duringthetrainingphase,theto-be-trainedmodelisidentifying,supportedbythetrainingframework,commonfeaturesinthedatasets.Thisenablesitto,duringthedeploymentphase,groupdataaccordingtopatternswhilealsoallowingittodetectanomalieswhichdonotfitintoanyofthelearnedgroups.

Reinforcementlearning

Reinforcementlearningisusedin,forexample,robotics,industrialautomation,andbusinessstrategyplanning,butduetotheneedforlargeamountsoffeedback,themethodhaslimiteduseinsurveillancetoday.Reinforcementlearningisabouttakingsuitableactiontomaximizethepotentialrewardinaspecificsituation,arewardthatgetslargerwhenthemodelmakestherightchoices.Thealgorithmdoesnotusedata/labelpairsfortraining,butisinsteadoptimizedbytestingitsdecisionsthroughinteractionwiththeenvironmentwhilemeasuringthereward.Thegoalofthealgorithmistolearnapolicyforactionsthatwillhelpmaximizethereward.

Testing

Oncethemodelistrained,itneedstobethoroughlytested.Thissteptypicallycontainsanautomatedpartcomplementedwithextensivetestinginreal-lifedeploymentsituations.

Intheautomatedpart,theapplicationisbenchmarkedwithnewdatasets,unseenbythemodelduringitstraining.Ifthesebenchmarksarenotwheretheyareexpectedtobe,theprocessstartsoveragain:newtrainingdataiscollected,annotationsaremadeorrefinedandthemodelisretrained.

Afterreachingthewantedqualitylevel,afieldteststarts.Inthistest,theapplicationisexposedtorealworldscenarios.Theamountandvariationdependonthescopeoftheapplication.Thenarrowerthescope,thelessvariationsneedtobetested.Thebroaderthescope,themoretestsareneeded.

Resultsareagaincomparedandevaluated.Thisstepcanthenagaincausetheprocesstostartover.Anotherpotentialoutcomecouldbetodefinepreconditions,explainingaknownscenarioinwhichtheapplicationisnotoronlypartlyrecommendedtobeused.

Deployment

Thedeploymentphaseisalsocalledinferenceorpredictionphase.Inferenceorpredictionistheprocessofexecutingatrainedmachinelearningmodel.Thealgorithmuseswhatitlearnedduringthetrainingphasetoproduceitsdesiredoutput.Inthesurveillanceanalyticscontext,theinferencephaseistheapplicationrunningonasurveillancesystemmonitoringreallifescenes.

Toachievereal-timeperformancewhenexecutingamachinelearningbasedalgorithmonaudioorvideoinputdata,specifichardwareaccelerationisgenerallyrequired.

Edge-basedanalytics

High-performancevideoanalyticsusedtobeserverbasedbecausetheyrequiredmorepower,andcooling,thanacameracouldoffer.ButalgorithmdevelopmentandincreasingprocessingpowerofedgedevicesinrecentyearshavemadeitpossibletorunadvancedAI-basedvideoanalyticsontheedge.

Thereareobviousadvantagesofedgebasedanalyticsapplications:theyhaveaccesstouncompressedvideomaterialwithverylowlatency,enablingrealtimeapplicationswhileavoidingtheadditionalcostandcomplexityofmovingdataintothecloudforcomputations.Edgebasedanalyticsalsocomewithlowerhardwareanddeploymentcostssincelessserverresourcesareneededinthesurveillancesystem.

Someapplicationsmaybenefitfromusingacombinationofedgebasedandserverbasedprocessing,withpreprocessingonthecameraandfurtherprocessingontheserver.Suchahybridsystemcanfacilitatecost-efficientscalingofanalyticsapplicationsbyworkingonseveralcamerastreams.

Hardwareacceleration

Whileyoucanoftenrunaspecificanalyticsapplicationonseveraltypesofplatforms,usingdedicatedhardwareaccelerationachievesamuchhigherperformancewhenpowerislimited.Hardwareacceleratorsenablepower-efficientimplementationofanalyticsapplications.Theycanbecomplementedbyserverandcloudcomputeresourceswhensuitable.

GPU(graphicsprocessingunit).GPUsweremainlydevelopedforgraphicsprocessingapplicationsbutarealsousedforacceleratingAIonserverandcloudplatforms.Whilesometimesalsousedinembeddedsystems(edge),GPUsarenotoptimal,fromapowerefficiencystandpoint,formachinelearninginferencetasks.

MLPU(machinelearningprocessingunit).AnMLPUcanaccelerateinferenceofspecificclassicalmachinelearningalgorithmsforsolvingcomputervisiontaskswithveryhighpowerefficiency.Itisdesignedforreal-timeobjectdetectionofalimitednumberofsimultaneousobjecttypes,forexample,humansandvehicles.

DLPU(deeplearningprocessingunit).Cameraswithabuilt-inDLPUcanaccelerategeneraldeeplearningalgorithminferencewithhighpowerefficiency,allowingforamoregranularobjectclassification.

AIisstillinitsearlydevelopment

ItistemptingtomakeacomparisonbetweenthepotentialofanAIsolutionandwhatahumancanachieve.Whilehumanvideosurveillanceoperatorscanonlybefullyalertforashortperiodoftime,acomputercankeepprocessinglargeamountsofdataextremelyquicklywithoutevergettingtired.

ButitwouldbeafundamentalmisunderstandingtoassumethatAIsolutionswouldreplacethehuman

operator.Therealstrengthliesinarealisticcombination:takingadvantageofAIsolutionstoimproveandincreasetheefficiencyofahumanoperator.

Machinelearningordeeplearningsolutionsareoftendescribedashavingthecapabilitytoautomaticallylearnorimprovethroughexperience.ButAIsystemsavailabletodaydonotautomaticallylearnnewskillsafterdeploymentandwillnotrememberspecificeventsthathaveoccurred.Toimprovethesystem’sperformance,itneedstoberetrainedwithbetterandmoreaccuratedataduringsupervisedlearningsessions.Unsupervisedlearningtypicallyrequiresalotofdatatogenerateclustersandisthereforenotusedinvideosurveillanceapplications.Itisinsteadusedtodaymainlyforanalyzinglargedatasetstofindanomalies,forexampleinfinancialtransactions.Mostapproachesthatarepromotedas“self-learning”withinvideosurveillancearebasedonastatisticaldataanalysisandnotonactuallyretrainingthedeeplearningmodels.

HumanexperiencestillbeatsmanyAI-basedanalyticsapplicationsforsurveillancepurposes.Especiallythosewhicharesupposedtoperformverygeneraltasksandwherecontextualunderstandingiscritical.Amachinelearningbasedapplicationmightsuccessfullydetecta“runningperson”ifspecificallytrainedforitbutunlikeahumanwhocanputthedataintocontext,theapplicationhasnounderstandingofwhythepersonisrunning–tocatchthebusorfleefromthenearbyrunningpoliceofficer?DespitepromisesfromcompaniesapplyingAIintheiranalyticsapplicationsforsurveillance,theapplicationcannotyetunderstandwhatitseesonvideowithremotelythesameinsightasahumancan.

Forthesamereason,AI-basedanalyticsapplicationscanalsotriggerfalsealarmsormissalarms.Thiscouldtypicallyhappeninacomplexenvironmentwithalotofmovement.Butitcouldalsobeabout,forexample,apersoncarryingalargeobject—effectivelyobstructingthehumancharacteristicstotheapplication,makingacorrectclassificationlesslikely.

AI-basedanalyticstodayshouldbeusedinanassistingway,forexample,toroughlydeterminehowrelevantanincidentisbeforealertingahumanoperatortodecideabouttheresponse.Thisway,AIisusedtoreachscalabilityandthehumanoperatoristheretoassesspotentialincidents.

Considerationsforoptimalanalyticsperformance

TonavigatethequalityexpectationsofanAI-basedanalyticsapplication,itisrecommendedtocarefullystudyandunderstandtheknownpreconditionsandlimitations,typicallylistedintheapplication’sdocumentation.

Everysurveillanceinstallationisuniqueandtheapplication’sperformanceshouldbeevaluatedateachsite.Ifthequalityisnotattheexpectedoranticipatedlevel,itisstronglyrecommendedtonotonlyfocustheinvestigationontheapplicationitself.Allinvestigationsshouldbemadeonaholisticlevelbecausetheperformanceofananalyticsapplicationdependsonsomanyfactors,mostofwhichcanbeoptimizedifweareawareoftheirimpact.Thesefactorsinclude,forexample,camerahardware,videoquality,scenedynamics,illuminationlevel,aswellascameraconfiguration,position,anddirection.

Imageusability

Imagequalityisoftensaidtodependonhighresolutionandhighlightsensitivityofthecamera.Whiletheimportanceofthesefactorscannotbequestioned,therearecertainlyothersthatarejustasinfluentialfortheactualusabilityofanimageoravideo.Forexample,thebestqualityvideostreamfromthemostexpensivesurveillancecameracanbeuselessifthesceneisnotsufficientlylitatnight,ifthecamerahasbeenredirected,orifthesystemconnectionisbroken.

Theplacementofthecamerashouldbecarefullyconsideredbeforedeployment.Forvideoanalyticstoperformasexpected,thecameraneedstobepositionedtoenableaclearview,withoutobstacles,oftheintendedscene.

Imageusabilitymayalsodependontheusecase.Videothatlooksgoodtoahumaneyemaynothavetheoptimalqualityfortheperformanceofavideoanalyticsapplication.Infact,manyimageprocessingmethodsthatarecommonlyusedtoenhancevideoappearanceforhumanviewingarenotrecommendedwhenusingvideoanalytics.Thismayinclude,forexample,appliednoisereductionmethods,widedynamicrangemethods,orautoexposurealgorithms.

VideocamerastodayoftencomewithintegratedIRilluminationwhichenablesthemtoworkincompletedarkness.Thisispositiveasitmayenablecamerastobeplacedondifficult-lightsitesandreducetheneedforinstallingadditionalillumination.However,ifheavyrainorsnowfallareexpectedonasite,itishighlyrecommendednottorelyonlightcomingfromthecameraorfromalocationveryclosetothecamera.

Toomuchlightmaybedirectlyreflectedbacktothecamera,againstraindropsandsnowflakes,makingtheanalyticsunabletoperform.Withambientlight,ontheotherhand,thereisabetterchancethattheanalyticswilldeliversomeresultsevenindifficultweather.

Detectiondistance

ItisdifficulttodetermineamaximumdetectiondistanceofanAI-basedanalyticsapplication—anexactdatasheetvalueinmetersorfeetcanneverbethewholetruth.Imagequality,scenecharacteristics,weatherconditions,andobjectpropertiessuchascolorandbrightnesshaveasignificantimpactonthedetectiondistance.Itisevident,forexample,thatabrightobjectagainstadarkbackgroundduringasunnydaycanbevisuallydetectedatmuchlongerdistancesthanadarkobjectonarainyday.

Thedetectiondistancealsodependsonthespeedoftheobjectstobedetected.Toachieveaccurateresults,avideoanalyticsapplicationneedsto“see”theobjectduringasufficientlylongperiodoftime.Howlongthatperiodneedstobedependsontheprocessingperformance(framerate)oftheplatform:thelowertheprocessingperformance,thelongertheobjectneedstobevisibleinordertobedetected.Ifthecamera’sshuttertimeisnotwellmatchedwiththeobjectspeed,motionblurappearingintheimagemayalsolowerthedetectionaccuracy.

Fastobjectsmaybemoreeasilymissediftheyarepassingbyclosertothecamera.Arunningpersonlocatedfarfromthecamera,forexample,mightbewelldetected,whileapersonrunningveryclosetothecameraatthesamespeedmaybeinandoutofthefieldofviewsoquicklythatnoalarmistriggered.

Inanalyticsbasedonmovementdetection,objectsmovingdirectlytowardsthecamera,orawayfromit,presentanotherchallenge.Detectionwillbeespeciallydifficultforslow-movingobjects,whichwillonlycauseverysmallchangesintheimagecomparedtomovementacrossthescene.

Ahigherresolutioncameratypicallydoesnotprovidealongerdetectiondistance.Theprocessingcapabilitiesneededforexecutingamachinelearningalgorithmareproportionaltothesizeoftheinputdata.Thismeansthatth