2024歐盟人工智能法案對歐洲中東和非洲地區(qū)醫(yī)療科技領域的影響分析報告 How the EU AI Act is Reshaping Medtech in EMEA

HowtheEUAIActisReshapingMedtechinEMEA

Alphasense

What’sInside

1Introduction

3CHAPTER1

ANeedforNewRules

5CHAPTER2

UnderstandingtheAIAct

7CHAPTER3

RiskClassi?cationUndertheAIAct

11CHAPTER4

ExistingRegulationOverlapswiththeAct

15CHAPTER5

StayonTopoftheEvolvingAILandscapewithAlphaSense

17CHAPTER6

LookingForward

Alphasense

Introduction

SincetheCOVID-19pandemicgrippedtheworld,globalmedical

marketshavehadtodramaticallyreinventthemselvestowithstandanumberofunpredictableandvolatileevents.In2020,once

unshakeablesupplychain-ef?ciencystoodatastandstillinthewake

ofworldwidelockdowns.Newgeopoliticalconlictsthataroseafterthepandemic’sinception(e.g.,Russia-Ukrainewar,Israel-

Palestineconlict,etc.)posedthechallengeofprovidinglife-saving

medicalcareinthemidstofwarfare.Likewise,demandforchronic

conditionstreatmentsskyrocketed.Thebottomline:innovationwithin

healthcarewasdesperatelyneeded,astraditionalorexistingmedicalinterventionwasfallingshort.

Cuearti?cialintelligence(AI)andthequick-pacerolloutofgenerative

AI(genAI).Inamatterofmonths,thistechnologyreimaginedhowdrugdiscovery,R&D,massivedataanalysis,vaccinedevelopment,

andmuchmorecouldbeconductedmoreef?cientlyandeffectively.ExcitementremainspalpableforgenAIasresearchersandmedtech

developerswieldittoful?lloncetedious,time-consumingprocesses.

However,thisexcitementhasledtosomespeculationaboutwherethistechcouldbetroublesomefortheentireindustry.

ToputparametersaroundtheuseofAIinhealthcareandquash

qualmsofpotentialprejudicialbiasorprivacyrisksassociatedwithit,theEUrecentlyintroducedtheArti?cialIntelligenceAct(AIAct),“theworld’s?rstcomprehensiveAIlaw.”HavingbeenpassedthisMaybytheCounciloftheEuropeanUnion,theActisleadingmedtechleaderstonotonlyquestionhowthesestipulationsaffecttoolsalreadyin

1

circulationwithinthehealthcaresystems,butwhatitmeansforafuturewhereAIisattherootoftheindustry.

UsingtheAlphaSenseplatform,weunveilhowtheAIActwillreshape

medtechdevelopmentinthecomingmonthsandyears.Discoverinsightsthatwillequipyouwiththeintelligenceneededtostay

attheforefrontofthemedtechindustryandremainagiletorisingtechnologyandnewlegislation.

2

CHAPTER1

ANeedforNewRules

Simplyput,theAIActaimstobuildtrustamongEuropeans

regardingthebene?tsofAI.AlthoughmanyAIsystemspresent

littletonoriskandhavethepotentialtoaddressvarious

societalchallenges,someAItechnologiesintroducerisksthatneedtobemanagedtopreventnegativeconsequences.

AstheEUCommissionstates,“itisoftennotpossibleto

?ndoutwhyanAIsystemhasmadeadecisionorpredictionandtakenaparticularaction.So,itmaybecomedif?culttoassesswhethersomeonehasbeenunfairlydisadvantaged,suchasinahiringdecisionorinanapplicationforapublicbene?tscheme.Althoughexistinglegislationprovides

someprotection,itisinsuf?cienttoaddressthespeci?cchallengesAIsystemsmaybring.”

TheActaimstoaccomplishthefollowing:

RecognizeanddefuserisksgeneratedbyAIapplicationsDisarmAIpracticesthatposeunacceptablerisks

Identifyalistofhigh-riskAIapplications

EstablishrequirementsforAIsystemsthatentailhigh-riskapplications

Setspeci?cobligationsfordeployersandprovidersofhigh-riskAIapplications

EnactaconformityassessmentbeforeagivenAIsystemisputintoserviceorplacedonthemarket

Alphasense

3

EnforcelegislationforAIsystemsoncethey’veenteredthemarket

BuildagovernancestructureattheEuropeanandnationallevel

Buthowdoesthisimpactmedtechcompanies?

Essentially,itinluencestheircomplianceresponsibilitiesandsafeguardstheir?nancialinterests.AnybusinesswhoseAI

system’soutputisusedintheEUmustadheretothislaw,regardlessofthecompany’sorsystem’slocation.

“Inthatsense,thereisreallynowayarounditasa

developer,[they]willhavetocomplywiththisact.It

reallywillbeanewlyregulatedspaceforthesespeci?c

AIproducts,withtheonlyexceptionthatthisonlyappliestothehigh-riskAIproducts,whichisthatmostofthe

productsthatareusedinhealthcarearehighrisk.

Productscanalsobelowrisk,forexample,ifit’sonly

usedinthehospitaladministrationprocessanddoesn’tengagedirectlywithpatients,andthenyoudon’thavetocomplywitheverypartoftheAIAct,youjusthavetoshowthatit’sasafeproductlikeanyotherproductsyouwouldputonamarket.”

—IndustryExpert|ExpertTranscript

Andformedtechcompanies,thetimeisnowtofamiliarizethemselveswiththeActandadoptresponsibleAIpractices.AccordingtotheWorldEconomicForum,implementing

responsibleAIprocessestypicallytakesaroundthreeyears.Companiesthatstartearlywillbeinastrongerposition

forcompliance,gainingstakeholdertrust,optimizingAI’sbusinessadvantages,andmore.

Alphasense

4

Act

CHAPTER2

UnderstandingtheAI

Thewide-spreaddistributionofgenAIanditsapplication

totoolswieldedbynearlyeveryprofessionaltoday(i.e.

lawyersusingChatGPTtocitecasesforcourt)hasonly

raisedconcernsaboutthelimitationofthistech—orlack

thereof.Withinthehealthcareindustry,manyskepticshaveraisedprivacyconcernsregardingpatientinformation,

biasedorprejudicialalgorithms,andunethicalusageof

genAI,leadingtheEUtocraftalegislativeframeworkthatsafeguardsbothpatientsandproviders.

Atthemostfundamentallevel,theAIActsetsparameters

forhowarti?cialintelligenceandgenAIisleveragedand

distributedwithinthehealthcaresectors.TheNational

LibraryofMedicinestatesthattheAct“aimstosetout(a)

uniforminternalmarketrules‘forthedevelopment,placingonthemarket,puttingintoserviceandtheuseofarti?cial

intelligence’inamannerthat(b)promotes‘humancentric

andtrustworthyarti?cialintelligence’and(c)‘ensuresahighlevelofprotectionofhealth,safety,fundamentalrights.’”

Inits?rstyearofenactment,theActwillinitiallyreform

general-purposeAIsystems,andintwoyears’time,extendtothosethatincludeAI-enableddigitalhealthtools(DHTs),andeventuallycoverhigh-riskAIsystems(i.e.DHTs).

Itsimultaneouslyintroducesarangeofnewobligations

fordevelopers,deployers,noti?edbodies,andregulators,particularlythoseworkingwiththenewlycreatedAIOf?ceandtheEuropeanCommission.LikeotherEUregulations,itpertainstoDHTsdevelopedthataremarketedandused

5

Alphasense

outsidetheEU,ormorespeci?cally,“AIsystemsandmodelsspeci?callydevelopedandputintoserviceforthesole

purposeofscienti?cresearchanddevelopment.”

Regulatorsandmedtechtitansalikeremainconfusedabouttheexactandde?niteparametersoftheAct.According

toNatureMedicine:“Duetothehorizontalnature,widescope,andrapidlychangingnatureofhealthcareAI,

therewillbemanyproblemscreatedbyambiguitiesand

uncertainintersectionswithexistinglaws.Here,lexibility

andintelligenceoftheresponsiblebodies,includingthe

Commission,theEUAIOf?ce,thecommitteesresponsible

forcreatingguidelinesformedicaldevicesoftwareunder

theMDR,andevenofthecourtswillbeessentialtomeetingthestatedaimoftheAIAct,topromoteratherthanto

decimateAIsectorentrepreneurialactivityandinnovation.”

6

Al

phasen

se

CHAPTER3

RisksClassi?cations

UndertheAct

TheregulatoryframeworkoftheAIActde?nesfourlevelsofriskforAIsystems:

UnacceptableRisk

Socialscoring,facialrecognition,darkpatternAI,manipulation

HighRisk

Safetycomponentsincriticalinfrastructure,

employmentandperformanceinwork,accessto

education,accesstopublicservices,useininsurance,creditscoring,bordercontrol,justicesystems

LimitedRisk

GeneralpurposeAIandAIsystemswith

speci?ctransparencyrequirementssuchaschatbots,emotionrecognitionsystems

MinimalRisk

AI-enabledvideogames,spam?lters

GraphicsourcedfromForvisMazars

UnacceptableRisk

AnunacceptableriskisattributedtoAIsystemsthatviolatecoreEuropeanUnionprinciples,includingrespectforhumandignity,freedom,equality,democracy,andtheruleoflaw,aswellas

fundamentalrightsrecognizedbytheUnion.Consequently,AIsystemscategorizedunderthisriskarebanned.

Alphasense

7

Inotherwords,thesesystemsconductprocesses,leverageorproduceinformationdeemedunlawfulbytheCommissioninthefollowingways:

AIsystemsthatmanipulateindividualsintoperformingdangerousorunintendedactions,ormakingdecisionstheyotherwisewouldnot,areproblematic.

Additionally,systemsthattakeadvantageofthe

vulnerabilitiesofcertaingroups,suchasthosebasedonage,disabilities,orspeci?csocialoreconomic

conditions,poseethicalconcerns.

Biometriccategorizationsystems,whichanalyzedatalikefacialfeaturesor?ngerprintstoinfercharacteristicssuchaspoliticalviews,religiousbeliefs,orsexualorientation,alsoraiseissues.

Similarly,theuseofreal-timebiometricidenti?cation

inpublicspacesforlawenforcementpurposes,thoughtherearesomeexceptions,iscontroversial.

Systemsthatclassifypeopleovertimebasedontheirsocialbehavior,knownassocialscoring,areanother

concern.Moreover,AItechnologiesdesignedtoassessemotionalstatesinworkplaceoreducationalcontextsshouldbebanned.

HighRisk

TheseAIsystemstypicallypossesshighimpactcapabilities

and,therefore,carrysystemicrisksthatcanaffectthehealth,safety,andfundamentalrightsofpersonsintheUnion.

Highrisksystemsareusedinavarietyoftoolsrelevant

tothemedtechandhealthcareindustry,andinclude

medicaldevices(i.e.anyAIsystemthatisaClassIIaor

highermedicaldeviceorusesAIasasafetycomponent),emergencytriageAIsystems(usedforemergencypatienttriagetreatmentorsurgery),andAIsystemsmonitoringorassessingemotionrecognition.

Alphasense

8

Morespeci?cexamplesofthesesystemsinclude:

AI-assistedX-RayDiagnosis

AsylumHealthRiskScreening

MedicalTrainingAssessmentAISoftware

HealthcareWorkforceManagementAISystems

High-riskAIsystemsmustadheretotheobligationssetforthbytheCommissionbeforeenteringthemarket.Soastoensuretheef?cacyandreliabilityofthesesystems,developersmustincorporateadequateriskassessmentandmitigationsystems,aswellashighqualityreferencedatasetstominimizerisksanddiscriminatoryoutcomes.

Foraccountabilitypurposes,asystemmustalsologallactivitytoensuretraceabilityofitsresults,whiledevelopersmust

providedetaileddocumentationonthesystemanditspurposeforauthoritiestoassesscompliance.Whenitcomesto

consumerorend-userprotection,appropriatehumanoversightmeasuresoverasystemarerequiredtominimizerisk,in

tandemwithahighlevelofrobustness,securityandaccuracy.

LimitedRisk

LimitedriskreferstosystemsthatlacktransparencyintheirAIusage.TheActinstillstransparencyobligationstoensurethatconsumersareinformedwhennecessaryinorderto

fostertrustwithanAIsystem,likeachatbot.Forinstance,

anend-usershouldbemadeawarethattheyareinteractingwithamachinesotheycanmakeaninformeddecisionon

howtoassesstheinformationbeingpresentedtothem.

ProvidersalsohavetoensurethatAI-generatedcontentisidenti?able.AnyAI-generatedtextpublishedintendedtoinformonmattersofpublicinterestmustbelabeledasarti?ciallygenerated,includingaudioandvideocontentconstitutingdeepfakes.

Alphasense

9

MinimalorNoRisk

AIsystemsunderminimalriskdonothaveanyrestrictions

ormandatoryobligations.ExamplesofminimalornoriskAIsystemsincludespam?ltersandAI-enabledvideogames.

AccordingtotheEU,“avastmajorityofAIsystemscurrentlyusedintheEUfallintothiscategory.”TheCommissionandBoard,however,encouragedrawingupcodesofconduct

intendedtofosterthevoluntaryapplicationofrequirementstolow-riskAIsystems.

Ultimately,theCommission’srisk-basedstrategyrequires

organizationstoassesstheAIsystemstheycurrentlyuseorplantoimplementanddeterminetheriskleveleachsystempresents.CreatinganinventoryofAIsystemscanserveasaninitialstepinthisprocess.Oncetherisklevelsareidenti?ed,conductingagapassessmentinrelationtotheAIActhelpsorganizationsevaluatehowtheircurrentpracticescanbe

adaptedtomeetcompliancerequirements,allowingforthedevelopmentofastructuredplan.Additionally,publicsectorentitiesmustconsidergovernment-issuedguidelinesontheethicalandresponsibleapplicationofAItechnologies.

10

Alphasense

CHAPTER4

ExistingRegulation

OverlapswiththeAct

WhiletheAIActsetsoutnobleintentionstosafeguardsocietyfrompotentialAIblunders,confusionamongst

medtechprofessionalsregardingtheproposedbroad-

basedAIActandhowitwillcoexistorwork-in-tandem

withexistinglegislation—andnothinderproductionandinnovation—isn’tnew.

TheAIActbroadlyappliestocorporationswholeverage

AIandgenAIforconsumeruse,itsetsparametersin

accordancewithEUpolicyto“protectandimprovehealth,givingequalaccesstomodernandef?cienthealthcareforallEuropeans,andcoordinatinganyserioushealththreatsinvolvingmorethanoneEUcountry.”

However,manyexpertsandindustryparticipantsbelievethatinsteadofaone-size-?ts-allapproach,theexistingsector-

speci?cregulations—namely,theMedicalDeviceRegulation(MDR)andtheInVitroDiagnosticRegulation(IVDR)—couldbealteredtoaddressthenuancesofAItechnologies.

Currently,thesesector-speci?clawsclassifyAI-powered

softwareasatypeofmedicaldeviceifitisdesignedto

assistinmakingmedicaldecisions.Thisincludesfunctions

relatedtodiagnosing,preventing,monitoring,predicting,

prognosing,treating,oralleviatinghealthconditions,aswellassupportinglifestylechangesrelatedtohealth.

11

Alphasense

TheActincorporatesmanyprovisionsthatechothosefoundintheMDRandEUIVDR,suchasthemandatesforaqualitymanagementsystem,technicaldocumentation,andusage

instructions.Medtechmanufacturerswhohavealready

achievedcerti?cationundertheEUMDRandIVDRwilllikelyneedtoupdatetheirtechnicaldocumentationtoalignwiththenewstipulationsoftheEUAIAct.

InadditiontoexistingEUMDRandIVDRrequirements,theActintroducesseveralnewstipulationsforAIsystems,including:

Governanceanddatamanagementprotocolsfortrainingandtestingdatasets

Enhancedrecord-keepingpractices,suchasautomatic

eventloggingthroughoutthesystem’slifecycle

Designtransparencytoensurethatuserscanunderstand

andproperlyutilizethesystem’soutputs

RequirementsforhumanoversightindesignStandardsforaccuracyandcybersecurity

Butwhatmedtecharegripingaboutisthetime-consuming,costlyburdensofMDRandIVDRcerti?cation,andhowtheAIActcouldcomplicatethisprocess.Accordingtoglobal

law?rmLoganHovell:

“Theconformityassessmentofmedicaldevicesunderthe

MDRorIVDRcanbealengthy(andcostly)process(18

monthsormoreonaverage).Whiletheindustryislooking

attheEUregulatorsto?ndsolutionstomakethecurrentCEmarkingprocessmoreef?cient,therearesomefearsthattheapplicationoftheAIActmayaddanadditionalburdentothereviewbyNoti?edBodiesandnegativelyaffectthetimelinenecessarytoaf?xtheCEmarktomedicaldevicesinthe

EU.ConsideringthecurrentworkloadofcertainMDR/IVDRNoti?edBodies,suchpotentialriskcannotbedismissed.”

WhilethegoaloftheAIActistobalancethepromotion

ofinnovationandthenecessityofsafeguardingusersfrom

potentialrisksposedbyAI,italsoleavesagrayareainitsriskclassi?cationfordevicesbetweentheAIAct,MDR,andIVDR

Alphasense

12

SubjecttothrdpartyassesmentunderUnion

Harmonisation

law(i.e.anoti?edbody)

ManufacturersofhighriskAI

systemswillbe

requiredto

complywiththeMDR/IVDR+AIA

AIusedasasafetycomponentortheAIsystemisitselfaproduct

CoveredunderUnion

Harmonisationlaw(i.e.theMDR)

HighriskAI

medicaldevicesystem

!!HighriskundertheAIAdoesnotmeanriskundertheMDR!!

AI-poweredmedicaldevicesmaybeclassi?edashigh-riskAI

systemsiftheyfunctionasasafetycomponentofaproductoriftheyareconsideredproductsthemselvesunderUnionHarmonisationLaw,asoutlinedinAnnexIoftheAIAct.This

includesregulationssuchastheMDRandIVDR,especiallywhenthird-partyassessmentsarerequired.Insuchcases,theAI

medicaldevicemustthencomplywithboththeAIActandtheMDRorIVDR.

GraphicsourcedfromLoganHowell

Despitelawmakers’effortstostreamlineoverlapping

regulations,manyquestionsremainunresolved.For

example,therelationshipbetweentheAIAct’sruleson

substantialmodi?cationsandthemodi?cationrequirements

intheMDRandIVDRisstillunclear.It’salsouncertain

whetherdevicesundergoingtrials,suchasperformance

evaluationsorclinicalinvestigations,willneedcerti?cationundertheAIActbeforeuseinthesetrials.

Alphasense

13

“Oneofthechallengesformedtechmanufacturers

isthatsomuchoftheAIActdoesnotdirectlyapplytomedicaldevices.OnenoteworthyprovisioninthedrafttextisfoundinRecital30,whichstipulatesthatanymedicaldeviceincorporatingAIcomponentsandrequiringtheinvolvementofanoti?edbodyduring

aconformityassessmentundertheMDRorInVitro

DiagnosticsRegulation(IVDR)wouldautomaticallybeclassi?edasahigh-riskdeviceundertheAIAct.”

–ClarivatePlc|‘RegulatoryLasagne’andtheImpactoftheEuropeanAIActonMedtech

Inthecomingmonthsandyears,itisexpectedthatthe

differencesbetweentheEUAIAct,MDR,andIVDRwill

becomeclearer.HowcanmedtechcompaniessuccessfullylaunchanAIsystemthatcomplieswithallthreeregulatoryframeworks?Thesolutionliesinregulatorysandboxes.

TheEUAIActproposescreatingcoordinatedAI“regulatorysandboxes”toencourageinnovationacrossEurope.

Thesesandboxesallowbusinessestotestandre?netheirnewproducts,services,orbusinessmodelsunderthe

supervisionofregulators.Thiscontrolledenvironment

givesinnovatorsasafespacetoexperimentwhilehelpingregulatorsunderstandnewtechnologiesandultimately

expandconsumerchoices.However,topreventmisuseorexploitation,regulatorysandboxesneedasolidlegalframeworktoensuretheirproperfunctioning.

“RegulatoryoverlapwiththeMDRandIVDRislikelytopersistinthe?nalizedAIActtext.Howeverthoroughpreparationbymedtechcompaniesandtheuseof

regulatorysandboxescouldhelpaddresstheneedfordualconformity.”

–MedtechInsight|EUAIActRegulatoryOverlap“LikelytoPersist”:ExpertPresentsSolutionsforDualConformity

Alphasense

14

CHAPTER5

StayonTopoftheEvolvingAI

LandscapewithAlphaSense

It’struethatintoday’smedtechmarket,ifyou’renotabreastofeverynewdevelopment,legislation,andcompetitor,

you’llnotonlystruggletotakealeadinyourrespectiveindustry,butalsomatchthecaliberofcompetition.

InthecaseoftheAIAct,wherelawmakersanddevelopers

arenavigatingthelegislation’scomplexitiesonadaily

basis,traditionalresearchmethodswillleaveyououtofthisevolvingindustryconversation.That’swhymoreandmore

professionalsareembracingtheAIerabyinvestinginmarketintelligenceplatformslikeAlphaSense.

AlphaSensesurfacesalltheindustryinsightsyouneedsothat

youcanbenchmarkagainstpeers,?ndoutaboutanyemergingcompetitors,legislation,ortechnologies,generateideasfor

newproductdevelopment,andidentifyrisks—allinoneplace.

AlphaSensehousesavastuniverseofover10,000trustedcontentsources,includingbutnotlimitedto:

Companydocumentssuchaseventtranscripts,global?lings,pressreleases,companypresentations,productbrochures,andESGreports,inadditiontoprivate

companydocuments

Industryresearchfromtheworld’stop?nancialresearch?rms,includingCowen,MorganStanley,BankofAmerica,Jefferies,andJPMorgan,aswellasourproprietary

15

equityresearchcontentsetWallStreetInsights?thatisspeci?callytailoredforcorporateclients

Alphasense

Uniqueindustryexpertperspectivesfromcustomers,competitors,medicalexperts,doctors,andpartners

Thousandsofcuratednewssources,tradepublications,andregulatoryinformationsuchas510(k)s

TheAlphaSenseplatformalsodeliversunmatchedAI-search

capabilitiesandfeaturesforanalyzingqualitativeand

quantitativeresearch,andcanmineunstructureddataforthemostcriticalinsights,including:

Automatedandcustomizablealertsfortrackingregulatory?lings,companies,industries,andpotentialinvestments

TableexporttoolsthatsupportM&Aworklowsliketargetlistsandduediligence

SmartSynonyms?technologythatensuresyounevermissasourceimportanttoyourresearch

SmartSummaries,our?rstgenerativeAIfeature,summarizeskeyinsightsfromearningscallsforfasteranalysis

e

Alphasens

16

CHAPTER6