車端AI實(shí)時(shí)分析系統(tǒng)-2024-06-智能網(wǎng)聯(lián)

REAL-TIMEANALYTICSFORENDPOINTAIIN

AUTOMOTIVESYSTEMS

JUNE2024

KURTDEKOSKI

BUSINESSDEVELOPMENTANDPRODUCTMANAGEMENTFORAUTOMOTIVESWSEPBD

RENESASELECTRONICSCORPORATION

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

from

AplatformpoweringEdgeAI

onRenesasprocessors

MLModel

Development

Explainability+

HardwareAnalytics

Solutions+

ReferenceDesigns

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page2

REALITYAIPLATFORMCOMBINESADVANCEDSIGNALPROCESSINGANDMACHINELEARNINGONMCU/MPUEDGENODES

AdvancedSignalProcessing

Automaticallysearchesawiderangeofsignal-processingtransformstocreateacustom,optimizedfeaturetransform

ArtificialIntelligenceandAnomalyDetection

Automaticallygeneratesmachinelearningmodels,explanatoryvisualizationsand

hardwaredesignanalytics

MCU/MPUEdgeNodes

RunsonalmosteveryMCUandMPUcoreavailablefromRenesas,withnewones

addedconstantly.RealityAIalsosupportsRenesasmotorcontrolboards.

SCALABLEFROM16-BITto64-BITCORES

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page3

AI/MLTECHNOLOGY

ChangingIndustryNeedsDrivingAutomotive

ElectrificationAutonomous

Driving

Increaseddrivingsafety

Decreasedrelianceondriver

SoftwareDefinedVehicleDigitizationandAI

Software-firstdevelopmentstructure

Sensorfusion

BenefitsofAIandML

Reducedhumanerror,especiallyinprocessesthatinvolveanalyzingnumericalorsignaldata.AIwon’tmakeoversightsduetofatigueorerrorasahumanmight.

Increaseoperationalefficiencybyautomatingmorefunctionsandidentifyingandconsolidatingredundantprocesses.

RealityAIprovidesthesmallestmodelsinthemarketintermsofRAM,Flashandprocessingcycles

RealityAImodelsaresupportedforRenesasMCUs

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page4

AUTONOMOUSMEGATREND

INCREASEDLEVELOFAUTONOMY,REDUCEDLEVELOFDRIVERENGAGEMENT

INCREASEDLEVELOFFEATURESTOMIMICDRIVERPERCEPTION

AIsoftware:AIforself-drivingcarsusingsensor-fedmachinelearningalgorithms

AVhardware:Sensorsprovidethekeydatainputstothecar’sAIsystems

Level5Autonomy

FullAutomation

Thevehicleiscapableof

performingalldrivingfunctionsunderallconditions.Thedrivermayhavetheoptiontocontrolthevehicle.

Level4Autonomy

HighAutomation

Thevehicleiscapableof

performingalldrivingfunctionsundercertainconditions.Thedrivermayhavetheoptiontocontrolthevehicle.

Level2Autonomy

PartialAutomation

Vehiclehascombined

automatedfunctionslike

accelerationandsteering,butthedrivermustremain

engagedwiththedrivingtaskandmonitortheenvironmentatalltimes.

Level3Autonomy

ConditionalAutomation

Driverisanecessitybutisnot

requiredtomonitorthe

environment.Thedrivermustbereadytotakecontrolofthevehicleatalltimeswithnotice.

LevelsofvehicularautonomydefinedbytheSocietyofAutomotiveEngineers(SAE)

Page5

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

MostlyuseAIfordifficultproblemswheretraditionalmethodsaren’tworkingandtherearenoobvioussolutions

Instrumentationandhardwareissuesaddcomplexity,mostoptimizeforcost

Cannotdeploy“blackbox”solutions

REALITYAISOFTWAREANDSOLUTIONSADDRESSCOMMONENGINEERINGFRUSTRATIONSWITHAI/ML

Engineers’frustrationswithAI/ML

HowRealityAImakesitbetter

Builtspecificallyfornon-visualsensingbasedonadvancedsignalprocessingmathandedge

deploymentonRenesasMCUs

Analyticstosupporthardwaredesign(notjustalgorithmsandmodelbuilding)

Explainabilitybasedontime-frequencycharacteristicswithfulltransparency

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page6

REALITYAIPRODUCTPORTFOLIOOFFERSBOTHSOFTWAREANDSOLUTIONS

Software

RealityAI

Softwareformodelcreationandhardwareoptimizationon

RenesasMCUs,MPUsand

motorcontrolkits

RealityAITools?

Buildmodels,visualizeexplanations,optimizehardwarebuild

RealityCheckTMMotor

Forembeddedpredictivemaintenanceandcontrolfeedback

RealityCheckTMAD

Forcloud-basedanomalydetectioninfactoryandon-line

Completeframeworkforspecificusecases,includinghardware,firmware,softwareandML

referencedesigns

RealityAISolutions

RealityCheckTMHVAC

Completeframeworkforsmart,self-diagnosingHVAC

AutomotiveSWS

Completeframeworkforaudio-basedADASsensingforvehicles

RenesasCore

ArmCore

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page7

REALITYAIPRODUCTPORTFOLIOOVERVIEW

RealityCheckMotor

AnomalyDetection

SWS-EVD

GettingStartedPackage

墓RealtyAIToolse

RealityCheckAD

DemoVideos

SoftwareManual

ApplicationNotes

SignalClassification

RealityCheckHVAC

Regression

(continuousvalueprediction)

NativedeploymentavailablenowforallRenesasRA,RX,RZ,RL78,RH850,andR-Carfamiliesofdevices

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page8

REALITYAIADDRESSESTHEFULLAIOTDESIGNLIFECYCLE

Only5%oftypicalprojectcostsarespentonbuildingmodels

?RealityAlToolse

Softwarehelpswiththeother95%

SensorSelection

andBOM

Optimization

Algorithmically-DrivenFeatureDiscovery

墓RealityAI'Tools?software

DataReadiness

AIExploreTM(AutoML)

EdgeAI/TinyMLCodeOptimization

Instrumentation&hardware

development,datacollection,

validationandinternal

confidencebuilding

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page9

RENESAS/REALITYAIFAMILY

InputSignal

Application

?AutoML

(nocoding)

?Explainability

AIExploreTM(AutoML)

Application

Software

Stack

SensorSelection

andBOM

Optimization

Customer's

?Cost-optimizedspecifications

?Minimumsensorset

ML/AI

Generated

Code

DataReadiness

?Automated

?Consistency

?Quality

?Coverage

EdgeAI/TinyMLCodeOptimization

OutputApplicationTriggers

?Embeddedcode

generation–C,C++

?Easeofdeployment

?MATLABcompatibility

NativedeploymentavailablenowforallRenesasRA,RX,RZ,RL78,RH850,andR-Carfamiliesofdevices

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page10

REALITYAITOOLS?–APPLICATIONINPUTSIGNAL

Fullyexplainable.“Whentheresonancepeakat146hz

disappears,weknowthefanisblocked”

30+dBshiftin

100dBSiren@15m

100dBSiren@0m

overallnoiselevel

Image

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page11

“GETSTARTED”WITHREALITYAIPRODUCTPORTFOLIO

Mostcustomersstartwithoneofour“GetStarted”packagesthatcombinesoftwarewithenhancedonboardingandservices,beforeswitchingtoanannualsoftwaresubscription

StarterPoCPrototype

Idon’thavedataIhavedata

4-monthPrototypePoC

?8-weekTrialRunwithaccessextendedtofourmonths

?EnhancedsupportfromRealityAI,includingpre-processing,model

constructionandpost-processing

?Constructionoflaptop-basedorothernon-embeddedprototype/demo

6-monthEdgeAIPrototype

?4-monthPrototypePoC,withaccessextendedtosixmonths

?DeliveryofEdgeAImodelcompiledforCortexM-classarchitectureorLinux

?Assistancewithhardwareand

applicationintegration/debugging

8-weekTrialRun

?Requirementsanddataassessment

?Onboardingandsetup

?TwoiterationsrunbyRealityAIteam

?Usertrainingonowndataandmodels

?Uptofourweeksofaccessfor

evaluationandcontinueddevelopment

5-monthPoC

?8-week“Phase0”

?Monitoringofclientdatacollection

?FouriterationsrunbyRealityAIteam

?Usertrainingonowndataandmodels

?ValidationstatisticsforevaluationofPoCresults

7-monthWorkingPrototype

?8-week“Phase0”

?Monitoringofclientdatacollection

?5-monthPOC

?Constructionoflaptop-basedoranothernon-embeddedprototype/demo

8-week“Phase0”

?Requirementsanddataassessment

?Datacoverageandcollectionplan

?Instrumentation/hardwareplan

?FeasibilityPoCplan

?R&Droadmapwithmachinelearning

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page12

REALITYAIDEVELOPMENT

WITHRENESASAUTOMOTIVEMCUsandSoCs

RenesasCore

16-bitMCU32-bitMCU

POWER

SUPERLOWPOWER

EFFICIENCY

RenesasmeetsthevariedrequirementsofourcustomerswitharichlineupofautomotiveMCUsandSoCs

Arm?Core

32-bitMCU

SCALABLEWITHSoC

64-bitSoC

HIGHPERFORMANCE

Embedded

AI-enabled

Solution

RealityCheck

Motor

CBM

EVDriveMotor

SWS-EVD

RealityAITools?

RealityCheckAD

OD-GR

BMS

RealityCheck

HVAC

MassProduction

Sampling

Development

Concept

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page13

AUTOMOTIVE,CONSUMERANDINDUSTRIALIOTAPPLICATIONS

RealityAITinyMLmodelsrunoneveryRenesasMCU

AutomotiveSWS

Completeframeworkforaudio-basedADASsensingforvehicles

RealityCheckHVAC

Completeframeworkforsmart,self-diagnosingHVAC(nowengagingwithearlyadoptercustomers)

Diagnosethesourceof

Remotelivestockbehaviormonitoring

Airconditionersthatpredicttheirownmaintenanceneeds

Hearapproachingemergencyvehicles,carsandbicycles

powerqualityissuesbasedon3-phaseAC

Detectweather,terrain,roadconditionsand

micro-collisions

Maketiresthatreporttheirstateofwear

HomeApplianceelectric

motorsthatmonitorthehealthofthesystemstheydrive

Predicte-bikechaindeteriorationPumpsthatknowtheyaredry

orcavitating

RealityCheckMotor

Forembeddedpredictivemaintenanceandcontrolfeedback

Filtersthatknowhowlonguntiltheywillclog

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page14

TARGETUSECASE

SEEINGWITHSOUND

Environmentparticipantsemitsoundsthatcanoftenbeheardbeforetheyareseen

?Aroundcorners

?Obstructedviews

?Atadistance

?Vehicleblindspots

?Whereothersensorscannotsee

?Wheredriversarenotfocused

Usingsounds,theAItechnologyforSWS

providesearlywarningofemergencysirens,on-

comingvehiclesandothersoundsourcestoaugmentexistingautonamousdrivingandADASsystems

SWS-EVD

WhySWS–Contributingfactorsandcauses:

DistractedDriving

AutomotiveApplicationFeatures

EmergencyVehicleDetection+BlindSpotDetection

+OtherVehicles+HornHonks

ActiveandPassiveNoiseCancelation

+Pedestrians+Cyclists

AutonomousEmergencyBraking–Pedestrian/Cyclist

CyclistDooringPrevention

+PavementConditions/Terrain

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page15

TARGETUSECASE

SEEINGWITHSOUND

Vehicles30m–obstructedview/intersectionwithmoderatenoise

DemonstrationVideo

Blindspots

Sirens1km–normaloperatingconditionsandspeeds

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page16

TARGETUSECASE:CONDITIONMONITORING

ROADCONDITIONSANDTERRAIN

Targetingvariousnoisepatternsthatindicatedifferingroadsurfacesthataffectdrivinghabits

DryRoadSurface

WetRoadSurface(howmuch)

Snowy/IcyRoadSurface

GravelorSandRoadSurfaces

Cobblestonesurfaces

DevelopmentLevel=PoC

墓RealityAIToolse

Arm?Core

RenesasCore

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page17

TARGETUSECASE

MICRO-COLLISION

Detectionofsmallimpactsagainstthevehicle

GlassBreakage

Scrapes/Scratches

SmallDings/Dents

DevelopmentLevel=PoC

墓RealityAIToolse

Arm?Core

RenesasCore

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page18

TARGETUSECASE

AUDIOANDVOICECOMMANDS

Identifyoccupantsforindividualizedsoundzone

Clearcommunicationfrom

fronttorearoccupants

PresenceDetectionofeachoccupantandseatinglocation.

AllowsfortargetedSoundZoneforaudioperformance

Allowsforcontrolledin-cabincommunication

Anti-spoofing

Recordedverseactual

AI/MLtrainedVoicebasedUserAccessControl

Voice

AntiSpoofing

Anti-SpoofingDemo

Arm?Core

墓RealtyAIToolse

RenesasCore

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page19

TARGETUSECASES

PRESENCEDETECTIONANDDRIVER/OCCUPANTBEHAVIORANDPROFILING

GestureRecognition

OccupantMoving

OccupantBreathing

UseAI/MLtolearnGestures

OccupantDetection

AI/MLtrainedtorecognizehumansounds

PresenceDetection

Method1:Sound

Video

RenesasCore

PresenceDetectionandOccupantBehaviorandProfilingMethod2:DopplerRadar

Video

Method3:UWB

UsesdopplerradarorUWBtodistinguishbetweenhumans,petsandinanimateobjects(shiftinginseat,jostling,breathing,gestures,etc.)

?2024RenesasElectronicsCorporation.Allrightsreserved.

RENESASCONFIDENTIAL

Page20

RealityCheckMotor

AI/MLUnbalancedLoadDetection+Sensor-lessFOC

UnbalancedDetection

NOADDITIONALSENSORS!

LeveragesAvailableElectrical&MotorSignals

BalancedUnbalanced

12mmM4screw,11mmoffcentertocreateunbalancedcondition

Motor

Speed/Torque

AI/MLMotor

Speed/Torque+Sensor-less

FOC

LowSpeedHighToque

HighSpeedLowTorque

TARGETUSECASES:CONDITIONMONITORINGELECTRICMOTOR–(RealityCheckMotor)

AI/MLMotor

Mis-alignment+Sensor-less

FOC

Aligned

Mis-Aligned

Motor

Mis-alignment

DevelopmentLevelforSmallmotors=Pre-Production

DemoVideo

Enablessensor-lessMLmodelstobedeployeddirectlytothemotorcontrolboard:

?Useselectricalinformationalreadyavailableontheboard

?Takeshighsample-ratecurrentandvoltageasaproxyextracomponents(like