人工智能對網絡威脅的近期影響（英文版）

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

Thenear-termimpactofAIonthecyberthreat

AnNCSCassessmentfocusingonhowAIwillimpacttheefficacyofcyber

operationsandtheimplicationsforthecyberthreatoverthenexttwoyears.

UKCyberPolicycomment

DuringtheBletchleyAISafetySummitinNovember2023,internationalleaderscame

togethertodiscussthevastpotentialofAImodelsinpromotingeconomicgrowth,

propellingscientificadvances,andprovidingawiderangeofpublicbenefits.TheyalsounderscoredthesecurityrisksthatcouldarisefromtheirresponsibledevelopmentanduseofAItechnologies.TheUKgovernmentisevaluatingandaddressingthepotentialthreatsandrisksassociatedwithAI.

WhileitisessentialtofocusontherisksposedbyAI,wemustalsoseizethesubstantialopportunitiesitpresentstocyberdefenders.Forexample,AIcanimprovethedetectionandtriageofcyberattacksandidentifymaliciousemailsandphishingcampaigns,

ultimatelymakingthemeasiertocounteract.

TheSummitDeclarationhighlightedtheimportanceofensuringthatAIisdesigned,

developed,deployed,andusedinamannerthatissafe,human-centric,trustworthy,andresponsibleforthebenefitofall.TheNCSCcontinuestoworkwithinternationalpartnersandindustrytoprovideguidanceonthesecuredevelopmentanduseofAI,sothatwe

canrealisethebenefitsthatAIofferstosociety,publishing

GuidelinesforSecureAI

SystemDevelopment

inNovember2023.

NCSCAssessment

NCSCAssessment(NCSC-A)istheauthoritativevoiceonthecyberthreattotheUK.Wefuseall-sourceinformation–classifiedintelligence,industryknowledge,academicmaterialandopensource–toprovideindependentkeyjudgementsthatinformpolicydecisionmakingandimproveUKcybersecurity.Wework

closelywithgovernment,industryandinternationalpartnersforexpertinputintoourassessments.

.uk/report/impact-of-ai-on-cyber-threat

1/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

NCSC-AispartoftheProfessionalHeadsofIntelligenceAssessment(PHIA).PHIA

leadsthedevelopmentoftheprofessionthroughanalyticaltradecraft,professionalstandards,andbuildingandsustainingacross-governmentcommunity.

Thisreportusesformalprobabilisticlanguage(seeyardstick)fromNCSC-A

producttoinformreadersaboutthenear-termimpactonthecyberthreatfromAI.TolearnmoreaboutNCSC-A,pleasecontact

enquiries@.uk

.

Howlikelyisa'realisticpossibility'?

ProfessionalHeadofIntelligenceAssessment(PHIA)probabilityyardstick

NCSCAssessmentusesthePHIAprobabilityyardstickeverytimewemakeanassessment,judgement,orprediction.Thetermsusedcorrespondtothe

likelihoodrangesbelow:

Keyjudgements

·Artificialintelligence(AI)willalmostcertainlyincreasethevolumeandheightentheimpactofcyberattacksoverthenexttwoyears.However,theimpactonthecyberthreatwillbeuneven(

seetable1

).

.uk/report/impact-of-ai-on-cyber-threat

2/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

·Thethreatto2025comesfromevolutionandenhancementofexistingtactics,techniquesandprocedures(TTPs).

·Alltypesofcyberthreatactor–stateandnon-state,skilledandlessskilled

–arealreadyusingAI,tovaryingdegrees.

·AIprovidescapabilityupliftinreconnaissanceandsocialengineering,

almostcertainlymakingbothmoreeffective,efficient,andhardertodetect.

·MoresophisticatedusesofAIincyberoperationsarehighlylikelytobe

restrictedtothreatactorswithaccesstoqualitytrainingdata,significantexpertise(inbothAIandcyber),andresources.Moreadvancedusesareunlikelytoberealisedbefore2025.

·AIwillalmostcertainlymakecyberattacksagainsttheUKmoreimpactful

becausethreatactorswillbeabletoanalyseexfiltrateddatafasterandmoreeffectively,anduseittotrainAImodels.

·AIlowersthebarrierfornovicecybercriminals,hackers-for-hireand

hacktiviststocarryouteffectiveaccessandinformationgathering

operations.Thisenhancedaccesswilllikelycontributetotheglobalransomwarethreatoverthenexttwoyears.

·Movingtowards2025andbeyond,commoditisationofAI-enabledcapabilityincriminalandcommercialmarketswillalmostcertainlymakeimproved

capabilityavailabletocybercrimeandstateactors.

Context

ThisassessmentfocusesonhowAIwillimpacttheeffectivenessofcyber

operationsandtheimplicationsforthecyberthreatoverthenexttwoyears.It

doesnotaddressthecybersecuritythreattoAItools,northecybersecurityrisksofincorporatingthemintosystemarchitecture.

TheassessmentassumesnosignificantbreakthroughintransformativeAIinthistimeperiod.Thisassumptionshouldbekeptunderreview,asanybreakthrough

couldhavesignificantimplicationsformalwareandzero-dayexploitdevelopmentandthereforethecyberthreat.

.uk/report/impact-of-ai-on-cyber-threat

3/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

TheimpactofAIonthecyberthreatwillbeoffsetbytheuseofAItoenhance

cybersecurityresiliencethroughdetectionandimprovedsecuritybydesign.

MoreworkisrequiredtounderstandtheextenttowhichAIdevelopmentsincybersecuritywilllimitthethreatimpact.

Assessment

1.TheimpactofAIonthecyberthreatisuneven;bothintermsofitsusebycyberthreatactorsandintermsofupliftincapability.

2.Table1:ExtentofcapabilityupliftcausedbyAIovernexttwoyears.

.uk/report/impact-of-ai-on-cyber-threat

4/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

Highlycapable

statethreatactors

Capablestateactors,commercialcompaniessellingtostates,

organisedcybercrime

groups

Less-skilledhackers-for-hire,opportunisticcybercriminals,

hacktivists

Intent

High

High

Opportunistic

Capability

HighlyskilledinAIandcyber,wellresourced

Skilledincyber,someresourceconstraints

Novicecyberskills,limitedresource

Reconnaissance

Moderateuplift

Moderateuplift

Uplift

Social

engineering,phishing,

passwords

Uplift

Uplift

Significantuplift(fromlowbase)

Tools(malware,exploits)

Realisticpossibilityofuplift

Minimaluplift

Moderateuplift(fromlowbase)

Lateralmovement

Minimaluplift

Minimaluplift

Nouplift

Exfiltration

Uplift

Uplift

Uplift

Implications

Bestplacedto

harnessAI'spotentialinadvancedcyber

operationsagainst

networks,forexampleuseinadvanced

malwaregeneration.

Mostcapabilityupliftinreconnaissance,socialengineeringand

exfiltration.WillproliferateAI-enabledtoolstonovicecyberactors.

Lowerbarriertoentrytoeffectiveandscalableaccessoperations-

increasingvolumeof

successfulcompromise

ofdevicesandaccounts.

KEY:MINIMALUPLIFT囚MODERATEUPLIFT囚UPLIFT囚SIGNIFICANTUPLIFT

3.AIwillprimarilyofferthreatactorscapabilityupliftinsocialengineering.

GenerativeAI(GenAI)canalreadybeusedtoenableconvincinginteractionwithvictims,includingthecreationofluredocuments,withoutthe

translation,spellingandgrammaticalmistakesthatoftenrevealphishing.Thiswillhighlylikelyincreaseoverthenexttwoyearsasmodelsevolveanduptakeincreases.

.uk/report/impact-of-ai-on-cyber-threat

5/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

4.AI’sabilitytosummarisedataatpacewillalsohighlylikelyenablethreatactorstoidentifyhigh-valueassetsforexaminationandexfiltration,

enhancingthevalueandimpactofcyberattacksoverthenexttwoyears.

5.Threatactors,includingransomwareactors,arealreadyusingAItoincreasetheefficiencyandeffectivenessofaspectsofcyberoperations,suchas

reconnaissance,phishingandcoding.Thistrendwillalmostcertainly

continueto2025andbeyond.Phishing,typicallyaimedeitheratdelivering

malwareorstealingpasswordinformation,playsanimportantrolein

providingtheinitialnetworkaccessesthatcybercriminalsneedtocarryoutransomwareattacksorothercybercrime.Itisthereforelikelythatcyber

criminaluseofavailableAImodelstoimproveaccesswillcontributetotheglobalransomwarethreatinthenearterm.

6.AIislikelytoassistwithmalwareandexploitdevelopment,vulnerability

researchandlateralmovementbymakingexistingtechniquesmore

efficient.However,inthenearterm,theseareaswillcontinuetorelyon

humanexpertise,meaningthatanylimitedupliftwillhighlylikelyberestrictedtoexistingthreatactorsthatarealreadycapable.AIhasthepotentialto

generatemalwarethatcouldevadedetectionbycurrentsecurityfilters,butonlyifitistrainedonqualityexploitdata.Thereisarealisticpossibilitythat

highlycapablestateshaverepositoriesofmalwarethatarelargeenoughtoeffectivelytrainanAImodelforthispurpose.

7.Cyberresiliencechallengeswillbecomemoreacuteasthetechnology

develops.To2025,GenAIandlargelanguagemodels(LLMs)willmakeit

difficultforeveryone,regardlessoftheirlevelofcybersecurity

understanding,toassesswhetheranemailorpasswordresetrequestis

genuine,ortoidentifyphishing,spoofingorsocialengineeringattempts.Thetimebetweenreleaseofsecurityupdatestofixnewlyidentified

vulnerabilitiesandthreatactorsexploitingunpatchedsoftwareisalreadyreducing.Thishasexacerbatedthechallengefornetworkmanagersto

patchknownvulnerabilitiesbeforetheycanbeexploited.AIishighlylikelytoacceleratethischallengeasreconnaissancetoidentifyvulnerabledevicesbecomesquickerandmoreprecise.

8.Expertise,equipment,timeandfinancialresourcingarecurrentlycrucialtoharnessmoreadvancedusesofAIincyberoperations.Onlythosewho

.uk/report/impact-of-ai-on-cyber-threat

6/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

investinAI,havetheresourcesandexpertise,andhaveaccesstoqualitydatawillbenefitfromitsuseinsophisticatedcyberattacksto2025.Highlycapablestateactorsarealmostcertainlybestplacedamongstcyber

threatactorstoharnessthepotentialofAIinadvancedcyberoperations.Otherstateactorsandmostcommercialcompaniesthatoffercapabilitytostatesworldwidewillgainmoderatecapabilityupliftoverthenext

eighteenmonthsinsocialengineering,reconnaissanceandexfiltration.Capableandestablishedcriminalgroupsarealsolikelytohaveenoughtrainingdataandresourcetogainsomeuplift.

9.However,itisarealisticpossibilitythatthesefactorsmaybecomeless

importantovertime,asmoresophisticatedAImodelsproliferateand

uptakeincreases.PubliclyavailableAImodelsalreadylargelyremovethe

needforactorstocreatetheirownreplicatechnologies,especiallyinlow-sophisticationoperationssuchasspear-phishing.Less-skilledcyberactorswillalmostcertainlybenefitfromsignificantcapabilityupliftsinthistypeofoperationto2025.Commoditisationofcybercrimecapability,forexample‘as-a-service’businessmodels,makesitalmostcertainthatcapable

groupswillmonetiseAI-enabledcybertools,makingimprovedcapabilityavailabletoanyonewillingtopay.

10.To2025,trainingAIonqualitydatawillremaincrucialforitseffectiveuseincyberoperations.Thescalingbarriersforautomatedreconnaissanceof

targets,socialengineeringandmalwareareallprimarilyrelatedtodata.Butto2025andbeyond,assuccessfulexfiltrationsoccur,thedatafeedingAIwillalmostcertainlyimprove,enablingfaster,moreprecisecyberoperations.

11.Increasesinthevolumeandheightenedcomplexityandimpactofcyber

operationswillindicatethatthreatactorshavebeenabletoeffectively

harnessAI.ThiswillhighlylikelyintensifyUKcyberresiliencechallengesintheneartermforUKgovernmentandtheprivatesector.

Glossary

Artificialintelligence

.uk/report/impact-of-ai-on-cyber-threat

7/8

3/4/24,11:35AMThenear-termimpactofAIonthecyberthreat-NCSC.GOV.UK

Computersystemswhichcanperformtasksusuallyrequiringhumanintelligence.Thiscouldin