OWASP LLM人工智能網(wǎng)絡安全與治理清單（英文版）

LLMAICybersecurity&GovernanceChecklist

FromtheOWASPTop10forLLMApplicationsTeam

Version:1.0

Published:February19,2024

RevisionHistory

Revision

Date

Author(s)

Description

0.1

2023-11-01

SandyDunn

initialdraft

0.5

2023-12-06

SD,Team

publicdraft

0.9

2023-02-15

SD,Team

pre-releasedraft

1.0

2024-02-19

SD,Team

publicreleasev1.0

Theinformationprovidedinthisdocumentdoesnot,andisnotintendedto,constitutelegaladvice.Allinformationisforgeneralinformationalpurposesonly.

Thisdocumentcontainslinkstootherthird-partywebsites.SuchlinksareonlyforconvenienceandOWASPdoesnotrecommendorendorsethecontentsofthethird-partysites.

1

Overview

5

1.1

ResponsibleandTrustworthyArti?cialIntelligence

6

1.2

WhoisThisFor?

7

1.3

WhyaChecklist?

7

1.4

NotComprehensive

7

1.5

LargeLanguageModelChallenges

7

1.6

LLMThreatCategories

8

1.7

Arti?cialIntelligenceSecurityandPrivacyTraining

9

1.8

IncorporateLLMSecurityandgovernancewithExisting,EstablishedPracticesandControls9

1.9

FundamentalSecurityPrinciples

9

1.10

Risk

10

1.11

VulnerabilityandMitigationTaxonomy

10

2

DeterminingLLMStrategy

11

2.1

DeploymentStrategy

13

3

Checklist

14

3.1

AdversarialRisk

14

3.2

ThreatModeling

14

3.3

AIAssetInventory

14

3.4

AISecurityandPrivacyTraining

15

3.5

EstablishBusinessCases

15

3.6

Governance

16

3.7

Legal

17

3.8

Regulatory

18

3.9

UsingorImplementingLargeLanguageModelSolutions

19

3.10

Testing,Evaluation,Veri?cation,andValidation(TEVV)

19

3.11

ModelCardsandRiskCards

20

3.12

RAG:LargeLanguageModelOptimization

21

3.13

AIRedTeaming

21

4

Resources

22

A

Team

32

Overview

Everyinternetuserandcompanyshouldpreparefortheupcomingwaveofpowerfulgenerativearti?cialintelligence(GenAI)applications.GenAIhasenormouspromiseforinnovation,ef?ciency,andcommercialsuccessacrossavarietyofindustries.Still,likeanypowerfulearlystagetechnology,itbringsitsownsetofobviousandunexpectedchallenges.

Arti?cialintelligencehasadvancedgreatlyoverthelast50years,inconspicuouslysupportingavarietyofcorporateprocessesuntilChatGPT’spublicappearancedrovethedevelopmentanduseofLargeLanguageModels(LLMs)amongbothindividualsandenterprises.Initially,thesetechnologieswerelimitedtoacademicstudyortheexecutionofcertain,butvital,activitieswithincorporations,visibleonlytoaselectfew.However,recentadvancesindataavailability,computerpower,GenAIcapabilities,andthereleaseoftoolssuchasLlama2,ElevenLabs,andMidjourneyhaveraisedAIfromanichetogeneralwidespreadacceptance.TheseimprovementshavenotonlymadeGenAItechnologiesmoreaccessible,buttheyhavealsohighlightedthecriticalneedforenterprisestodevelopsolidstrategiesforintegratingandexploitingAIintheiroperations,representingahugestepforwardinhowweusetechnology.

?Arti?cialintelligence(AI)isabroadtermthatencompassesall?eldsofcomputersciencethatenablemachinestoaccomplishtasksthatwouldnormallyrequirehumanintelligence.MachinelearningandgenerativeAIaretwosubcategoriesofAI.

?MachinelearningisasubsetofAIthatfocusesoncreatingalgorithmsthatcanlearnfromdata.Machinelearningalgorithmsaretrainedonasetofdata,andthentheycanusethatdatatomakepredictionsordecisionsaboutnewdata.

?GenerativeAIisatypeofmachinelearningthatfocusesoncreatingnewdata.

?Alargelanguagemodel(LLM)isatypeofAImodelthatprocessesandgenerateshuman-liketext.Inthecontextofarti?cialintelligencea"model"referstoasystemthatistrainedtomakepredictionsbasedoninputdata.LLMsarespeci?callytrainedonlargedatasetsofnaturallanguageandthenamelargelanguagemodels.

OrganizationsareenteringunchartedterritoryinsecuringandoverseeingGenAIsolutions.TherapidadvancementofGenAIalsoopensdoorsforadversariestoenhancetheirattackstrategies,introducingadualchallengeofdefenseandthreatescalation.

Businessesusearti?cialintelligenceinmanyareas,includingHRforrecruiting,emailspamscreening,SIEMforbehavioralanalytics,andmanageddetectionandresponseapplications.However,thisdocument’sprimaryfocusisonLargeLanguageModelapplicationsandtheirfunctionincreatinggeneratedcontent.

ResponsibleandTrustworthyArti?cialIntelligence

Aschallengesandbene?tsofArti?cialIntelligenceemerge-andregulationsandlawsarepassed-theprinciplesandpillarsofresponsibleandtrustworthyAIusageareevolvingfromidealisticobjectsandconcernstoestablishedstandards.The

OWASPAIExchangeWorkingGroup

ismonitoringthesechangesandaddressingthebroaderandmorechallengingconsiderationsforallaspectsofarti?cialintelligence.

Figure1.1:Imagedepictingthepillarsoftrustworthyarti?cialintelligence

WhoisThisFor?

TheOWASPTop10forLLMApplicationsCybersecurityandGovernanceChecklistisforleadersacrossexecutive,tech,cybersecurity,privacy,compliance,andlegalareas,DevSecOps,MLSecOps,

andCybersecurityteamsanddefenders.Itisintendedforpeoplewhoarestrivingtostayaheadin

thefast-movingAIworld,aimingnotjusttoleverageAIforcorporatesuccessbutalsotoprotectagainsttherisksofhastyorinsecureAIimplementations.Theseleadersandteamsmustcreatetacticstograbopportunities,combatchallenges,andmitigaterisks.

Thischecklistisintendedtohelpthesetechnologyandbusinessleadersquicklyunderstandtherisksandbene?tsofusingLLM,allowingthemtofocusondevelopingacomprehensivelistofcriticalareasandtasksneededtodefendandprotecttheorganizationastheydevelopaLargeLanguageModelstrategy.

ItisthehopeoftheOWASPTop10fortheLLMApplicationsteamthatthislistwillhelporganizationsimprovetheirexistingdefensivetechniquesanddeveloptechniquestoaddressthenewthreatsthatcomefromusingthisexcitingtechnology.

WhyaChecklist?

Checklistsusedtoformulatestrategiesimproveaccuracy,de?neobjectives,preserveuniformity,andpromotefocuseddeliberatework,reducingoversightsandmisseddetails.Followingachecklistnotonlyincreasestrustinasafeadoptionjourney,butalsoencouragesfutureorganizationsinnovationsbyprovidingasimpleandeffectivestrategyforcontinuousimprovement.

NotComprehensive

AlthoughthisdocumentintendstosupportorganizationsindevelopinganinitialLLMstrategyinarapidlychangingtechnical,legal,andregulatoryenvironment,itisnotexhaustiveanddoesnotcovereveryusecaseorobligation.WhileusingthisdocumentisOrganizationsshouldextendassessmentsandpracticesbeyondthescopeoftheprovidedchecklistasrequiredfortheirusecaseorjurisdiction.

LargeLanguageModelChallenges

LargeLanguagemodelsfaceseveralseriousanduniqueissues.OneofthemostimportantisthatwhileworkingwithLLMs,thecontrolanddataplanescannotbestrictlyisolatedorseparable.Anothersigni?cantchallengeisthatLLMsarenondeterministicbydesign,yieldingadifferentoutcomewhenpromptedorrequested.LLMsemploysemanticsearchratherthankeywordsearch.Thekeydistinctionbetweenthetwoisthatthemodel’salgorithmprioritizesthetermsinitsresponse.Thisisasigni?cantdeparturefromhowconsumershavepreviouslyusedtechnology,andithasanimpactontheconsistencyandreliabilityofthe?ndings.Hallucinations,emergingfromthegapsandtraining?awsinthedatathemodelistrainedon,aretheresultofthismethod.

Therearemethodstoimprovereliabilityandreducetheattacksurfaceforjailbreaking,modeltricking,andhallucinations,butthereisatrade-offbetweenrestrictionsandutilityinbothcostandfunctionality.

LLMuseandLLMapplicationsincreaseanorganization’sattacksurface.Somerisksassociated

withLLMsareunique,butmanyarefamiliarissues,suchastheknownsoftwarebillofmaterials(SBoM),supplychain,datalossprotection(DLP),andauthorizedaccess.TherearealsoincreasedrisksnotdirectlyrelatedtoGenAI,butGenAIincreasestheef?ciency,capability,andeffectivenessofattackerswhoattackandthreatenorganizations.

AdversariesareincreasinglyharnessingLLMandGenerativeAItoolstore?neandexpeditetraditional

methodsofattackingorganizations,individuals,andgovernmentsystems.LLMfacilitatestheirabilitytoenhancetechniquesallowingthemtoeffortlesslycraftnewmalware,potentiallyembeddedwithnovelzero-dayvulnerabilitiesordesignedtoevadedetection.Theycanalsogeneratesophisticated,unique,ortailoredphishingschemes.Thecreationofconvincingdeepfakes,whethervideooraudio,furtherpromotestheirsocialengineeringploys.Additionally,thesetoolsenablethemtoexecuteintrusionsanddevelopinnovativehackingcapabilities.Inthefuture,more“tailored”andcompounduseofAItechnologybycriminalactorswilldemandspeci?cresponsesanddedicatedsolutionsfor

anorganization’sappropriatedefenseandresiliencecapabilities.

OrganizationsalsofacethethreatofNOTutilizingthecapabilitiesofLLMssuchasacompetitivedisadvantage,marketperceptionbycustomersandpartnersofbeingoutdated,inabilitytoscalepersonalizedcommunications,innovationstagnation,operationalinef?ciencies,thehigherriskofhumanerrorinprocesses,andinef?cientallocationofhumanresources.

UnderstandingthedifferentkindsofthreatsandintegratingthemwiththebusinessstrategywillhelpweighboththeprosandconsofusingLargeLanguageModels(LLMs)againstnotusingthem,makingsuretheyaccelerateratherthanhinderthebusiness’smeetingbusinessobjectives.

LLMThreatCategories

Figure1.2:ImagedepictingthetypesofAIthreats

Arti?cialIntelligenceSecurityandPrivacyTraining

Employeesthroughoutorganizationsbene?tfromtrainingtounderstandarti?cialintelligence,generativearti?cialintelligence,andthefuturepotentialconsequencesofbuilding,buying,orutilizingLLMs.Trainingforpermissibleuseandsecurityawarenessshouldtargetallemployeesaswellasbemorespecializedforcertainpositionssuchashumanresources,legal,developers,datateams,andsecurityteams.

Fairusepoliciesandhealthyinteractionarekeyaspectsthat,ifincorporatedfromtheverystart,willbeacornerstonetothesuccessoffutureAIcybersecurityawarenesscampaigns.Thiswillnecessarilyprovideuserswithknowledgeofthebasicrulesforinteractionaswellastheabilitytoseparategoodbehaviorfrombadorunethicalbehavior.

IncorporateLLMSecurityandgovernancewithExisting,EstablishedPracticesandControls

WhileAIandgeneratedAIaddanewdimensiontocybersecurity,resilience,privacy,andmeetinglegalandregulatoryrequirements,thebestpracticesthathavebeenaroundforalongtimearestillthebestwaytoidentifyissues,?ndvulnerabilities,?xthem,andmitigatepotentialsecurityissues.

?Con?rmthemanagementofarti?cialintelligencesystemsisintegratedwithexistingorganizationalpractices.

?Con?rmAIMLsystemsfollowexistingprivacy,governance,andsecuritypractices,withAIspeci?cprivacy,governance,andsecuritypracticesimplementedwhenrequired.

FundamentalSecurityPrinciples

LLMcapabilitiesintroduceadifferenttypeofattackandattacksurface.LLMsarevulnerabletocomplexbusinesslogicbugs,suchaspromptinjection,insecureplugindesign,andremotecodeexecution.Existingbestpracticesarethebestwaytosolvetheseissues.Aninternalproductsecurityteamthatunderstandssecuresoftwarereview,architecture,datagovernance,andthird-partyassessmentsThecybersecurityteamshouldalsocheckhowstrongthecurrentcontrolsareto?ndproblemsthatcouldbemadeworsebyLLM,suchasvoicecloning,impersonation,orbypassingcaptchas.Givenrecentadvancementsinmachinelearning,NLP(NaturalLanguageProcessing),NLU(NaturalLanguageUnderstanding),DeepLearning,andmorerecently,LLMs(LargeLanguageModels)andGenerativeAI,itisrecommendedtoincludeprofessionalspro?cientintheseareasalongsidecybersecurityanddevopsteams.Theirexpertisewillnotonlyaidinadoptingthesetechnologiesbutalsoindevelopinginnovativeanalysesandresponsestoemergingchallenges.

Risk

ReferencetoriskusestheISO31000de?nition:Risk="effectofuncertaintyonobjectives."LLMrisksincludedinthechecklistincludesatargetedlistofLLMrisksthataddressadversarial,safety,legal,regulatory,reputation,?nancial,andcompetitiverisks.

VulnerabilityandMitigationTaxonomy

Currentsystemsforclassifyingvulnerabilitiesandsharingthreatinformation,likeOVAL,STIX,CVE,andCWE,arestilldevelopingtheabilitytomonitorandalertdefendersaboutvulnerabilitiesandthreatsspeci?ctoLargeLanguageModels(LLMs)andPredictiveModels.Itisexpectedthatorganizationswillleanontheseestablishedandrecognizedstandards,suchasCVEforvulnerabilityclassi?cationandSTIXfortheexchangeofcyberthreatintelligence(CTI),whenvulnerabilitiesorthreatstoAI/MLsystemsandtheirsupplychainsareidenti?ed.

DeterminingLLMStrategy

TherapidexpansionofLargeLanguageModel(LLM)applicationshasheightenedtheattentionandexaminationofallAI/MLsystemsusedinbusinessoperations,encompassingbothGenerativeAIandlong-establishedPredictiveAI/MLsystems.Thisincreasedfocusexposespotentialrisks,suchasattackerstargetingsystemsthatwerepreviouslyoverlookedandgovernanceorlegalchallengesthatmayhavebeendisregardedintermsoflegal,privacy,liability,orwarrantyissues.ForanyorganizationleveragingAI/MLsystemsinitsoperations,it’scriticaltoassessandestablishcomprehensivepolicies,governance,securityprotocols,privacymeasures,andaccountabilitystandardstoensurethesetechnologiesalignwithbusinessprocessessecurelyandethically.

Attackers,oradversaries,providethemostimmediateandharmfulthreattoenterprises,people,andgovernmentagencies.Theirgoals,whichrangefrom?nancialgaintoespionage,pushthemtostealcriticalinformation,disruptoperations,anddamagecon?dence.Furthermore,theirabilitytoharnessnewtechnologiessuchasAIandmachinelearningincreasesthespeedandsophisticationofattacks,makingitdif?cultfordefensestostayaheadofattacks.

Themostpressingnon-adversaryLLMthreatformanyorganizationsstemfrom"ShadowAI":

employeesusingunapprovedonlineAItools,unsafebrowserplugins,andthird-partyapplicationsthatintroduceLLMfeaturesviaupdatesorupgrades,circumventingstandardsoftwareapprovalprocesses.

Figure2.1:Imageofoptionsfordeploymentstrategy

DeploymentStrategy

Thescopesrangefromleveragingpublicconsumerapplicationstotrainingproprietarymodelsonprivatedata.Factorslikeusecasesensitivity,capabilitiesneeded,andresourcesavailablehelpdeterminetherightbalanceofconveniencevs.control.However,understandingthese?vemodeltypesprovidesaframeworkforevaluatingoptions.

Figure2.2:Imageofoptionsfordeploymenttypes

Checklist

AdversarialRisk

AdversarialRiskincludescompetitorsandattackers.

□Scrutinizehowcompetitorsareinvestinginarti?cialintelligence.AlthoughtherearerisksinAIadoption,therearealsobusinessbene?tsthatmayimpactfuturemarketpositions.

□Investigatetheimpactofcurrentcontrols,suchaspasswordresets,whichusevoicerecognitionwhichmaynolongerprovidetheappropriatedefensivesecurityfromnewGenAIenhancedattacks.

□UpdatetheIncidentResponsePlanandplaybooksforGenAIenhancedattacksandAIMLspeci?cincidents.

ThreatModeling

Threatmodelingishighlyrecommendedtoidentifythreatsandexamineprocessesandsecuritydefenses.Threatmodelingisasetofsystematic,repeatableprocessesthatenablemakingreasonablesecuritydecisionsforapplications,software,andsystems.ThreatmodelingforGenAIacceleratedattacksandbeforedeployingLLMsisthemostcosteffectivewaytoIdentifyandmitigaterisks,protectdata,protectprivacy,andensureasecure,compliantintegrationwithinthebusiness.

□Howwillattackersaccelerateexploitattacksagainsttheorganization,employees,executives,orusers?Organizationsshouldanticipate"hyper-personalized"attacksatscaleusingGenerativeAI.LLM-assistedSpearPhishingattacksarenowexponentiallymoreeffective,targeted,andweaponizedforanattack.

□HowcouldGenAIbeusedforattacksonthebusiness’scustomersorclientsthroughspoo?ngorGenAIgeneratedcontent?

□CanthebusinessdetectandneutralizeharmfulormaliciousinputsorqueriestoLLMsolutions?

□CanthebusinesssafeguardconnectionswithexistingsystemsanddatabaseswithsecureintegrationsatallLLMtrustboundaries?

□Doesthebusinesshaveinsiderthreatmitigationtopreventmisusebyauthorizedusers?

□CanthebusinesspreventunauthorizedaccesstoproprietarymodelsordatatoprotectIntellectualProperty?

□Canthebusinesspreventthegenerationofharmfulorinappropriatecontentwithautomatedcontent?ltering?

AIAssetInventory

AnAIassetinventoryshouldapplytobothinternallydevelopedandexternalorthird-partysolutions.

□CatalogexistingAIservices,tools,andowners.Designateataginassetmanagementforspeci?cinventory.

□IncludeAIcomponentsintheSoftwareBillofMaterial(SBOM),acomprehensivelistofallthesoftwarecomponents,dependencies,andmetadataassociatedwithapplications.

□CatalogAIdatasourcesandthesensitivityofthedata(protected,con?dential,public)

□EstablishifpentestingorredteamingofdeployedAIsolutionsisrequiredtodeterminethecurrentattacksurfacerisk.

□CreateanAIsolutiononboardingprocess.

□EnsureskilledITadminstaffisavailableeitherinternallyorexternally,followingSBoMrequirements.

AISecurityandPrivacyTraining

□ActivelyengagewithemployeestounderstandandaddressconcernswithplannedLLMinitiatives.

□Establishacultureofopen,andtransparentcommunicationontheorganization’suseofpredictiveorgenerativeAIwithintheorganizationprocess,systems,employeemanagementandsupport,andcustomerengagementsandhowitsuseisgoverned,managed,andrisksaddressed.

□Trainallusersonethics,responsibility,andlegalissuessuchaswarranty,license,andcopyright.

□UpdatesecurityawarenesstrainingtoincludeGenAIrelatedthreats.Voicecloningandimage

cloning,aswellasinanticipationofincreasedspearphishingattacks

□AnyadoptedGenAIsolutionsshouldincludetrainingforbothDevOpsandcybersecurityforthedeploymentpipelinetoensureAIsafetyandsecurityassurances.

EstablishBusinessCases

SolidbusinesscasesareessentialtodeterminingthebusinessvalueofanyproposedAIsolution,balancingriskandbene?ts,andevaluatingandtestingreturnoninvestment.Thereareanenormousnumberofpotentialusecases;afewexamplesareprovided.

□Enhancecustomerexperience

□Betteroperationalef?ciency

□Betterknowledgemanagement

□Enhancedinnovation

□MarketResearchandCompetitorAnalysis

□Documentcreation,translation,summarization,andanalysis

Governance

CorporategovernanceinLLMisneededtoprovideorganizationswithtransparencyandaccountability.IdentifyingAIplatformorprocessownerswhoarepotentiallyfamiliarwiththetechnologyorthe

selectedusecasesforthebusinessisnotonlyadvisedbutalsonecessarytoensureadequate

reactionspeedthatpreventscollateraldamagestowellestablishedenterprisedigitalprocesses.

□Establishtheorganization’sAIRACIchart(whoisresponsible,whoisaccountable,whoshouldbeconsulted,andwhoshouldbeinformed)

□DocumentandassignAIrisk,riskassessments,andgovernanceresponsibilitywithintheorganization.

□Establishdatamanagementpolicies,includingtechnicalenforcement,regardingdataclassi?cationandusagelimitations.Modelsshouldonlyleveragedataclassi?edfortheminimumaccesslevelofanyuserofthesystem.Forexample,updatethedataprotectionpolicytoemphasizenottoinputprotectedorcon?dentialdataintononbusiness-managedtools.

□CreateanAIPolicysupportedbyestablishedpolicy(e.g.,standardofgoodconduct,dataprotection,softwareuse)

□PublishanacceptableusematrixforvariousgenerativeAItoolsforemployeestouse.

□DocumentthesourcesandmanagementofanydatathattheorganizationusesfromthegenerativeLLMmodels.

Legal

ManyofthelegalimplicationsofAIareunde?nedandpotentiallyverycostly.AnIT,security,andlegalpartnershipiscriticaltoidentifyinggapsandaddressingobscuredecisions.

□Con?rmproductwarrantiesareclearintheproductdevelopmentstreamtoassignwhoisresponsibleforproductwarrantieswithAI.

□ReviewandupdateexistingtermsandconditionsforanyGenAIconsiderations.

□ReviewAIEULAagreements.End-userlicenseagreementsforGenAIplatformsareverydifferentinhowtheyhandleuserprompts,outputrightsandownership,dataprivacy,compliance,liability,privacy,andlimitsonhowoutputcanbeused.

□OrganizationsEULAforcustomers,Modifyend-useragreementstopreventtheorganizationfromincurringliabilitiesrelatedtoplagiarism,biaspropagation,orintellectualpropertyinfringementthroughAI-generatedcontent.

□ReviewexistingAI-assistedtoolsusedforcodedevelopment.Achatbot’sabilitytowritecodecanthreatenacompany’sownershiprightstoitsproductifachatbotisusedtogeneratecodefortheproduct.Forexample,itcouldcallintoquestionthestatusandprotectionofthegeneratedcontentandwhoholdstherighttousethegeneratedcontent.

□Reviewanyriskstointellectualproperty.Intellectualpropertygeneratedbyachatbotcouldbeinjeopardyifimproperlyobtaineddatawasusedduringthegenerativeprocess,whichissubjecttocopyright,trademark,orpatentprotection.IfAIproductsuseinfringingmaterial,itcreatesariskfortheoutputsoftheAI,whichmayresultinintellectualpropertyinfringement.

□Reviewanycontractswithindemni?cationprovisions.Indemni?cationclausestrytoputtheresponsibilityforaneventthatleadstoliabilityonthepersonwhowasmoreatfaultforitorwhohadthebestchanceofstoppingit.EstablishguardrailstodeterminewhethertheprovideroftheAIoritsusercausedtheevent,givingrisetoliability.

□ReviewliabilityforpotentialinjuryandpropertydamagecausedbyAIsystems.

□Reviewinsurancecoverage.Traditional(D&O)liabilityandcommercialgeneralliabilityinsurancepoliciesarelikelyinsuf?cienttofullyprotectAIuse.

□Identifyanycopyrightissues.Humanauthorshipisrequiredforcopyright.Anorganizationmayalsobeliableforplagiarism,propagationofbias,orintellectualpropertyinfringementifLLMtoolsaremisused.

□EnsureagreementsareinplaceforcontractorsandappropriateuseofAIforanydevelopmentorprovidedservices.

□RestrictorprohibittheuseofgenerativeAItoolsforemployeesorcontractorswhereenforceablerightsmaybeanissueorwherethereareIPinfringementconcerns.

□AssessandAIsolutionsusedforemployeemanagementorhiringcouldresultindisparatetreatmentclaimsordisparateimpactclaims.

□MakesuretheAIsolutionsdonotcollectorsharesensitiveinformationwithoutproperconsentorauthorization.

Regulatory

TheEUAIActisanticipatedtobethe?rstcomprehensiveAIlawbutwillapplyin2025attheearliest.TheEU?GeneralDataProtectionRegulation(GDPR)doesnotspeci?callyaddressAIbutincludesrulesfordatacollection,datasecurity,fairnessandtransparency,accuracyandreliability,andaccountability,whichcanimpactGenAIuse.IntheUnitedStates,AIregulationisincludedwithinbroaderconsumerprivacylaws.TenUSstateshavepassedlawsorhavelawsthatwillgointoeffectbytheendof2023.

FederalorganizationssuchastheUSEqualEmploymentOpportunityCommission(EEOC),theConsumerFinancialProtectionBureau(CFPB),theFederalTradeCommission(FTC),andtheUSDepartmentofJustice?CivilRightsDivision(DOJ)arecloselymonitoringhiringfairness.

□DetermineCountry,State,orotherGovernmentspeci?cAIcompliancerequirements.

□Determinecompliancerequirementsforrestrictingelectronicmonitoringofemployeesandemployment-relatedautomateddecisionsystems(Vermont,California,Maryland,NewYork,NewJersey)

□DeterminecompliancerequirementsforconsentforfacialrecognitionandtheAIvideoanalysisrequired(Illinois,Maryland,Washington,Vermont)

□ReviewanyAItoolsinuseorbeingconsideredforemployeehiringormanagement.

□Con?rmthevendor?compliancewithapplicableAIlawsandbestpractices.

□AskanddocumentanyproductsusingAIduringthehiringprocess.Askhowthemodelwastrained,andhowitismonitored,andtrackanycorrectionsmadetoavoiddiscriminationandbias.

□Askanddocumentwhataccommodationoptionsareincluded.

□Askanddocumentwhetherthevendorcollectscon?dentialdata.

□Askhowthevendorortoolstoresanddeletesdataandregulatestheuseoffacialrecognitionandvideoanalysistoolsduringpre-employment.

□Reviewotherorganization-speci?cregulatoryrequirementswithAIthatmayraisecomplianceissues.TheEmployeeRetirementIncomeSecurityActof1974,forinstance,has?duciarydutyrequirementsforretirementplansthatachatbotmightnotbeabletomeet.

UsingorImplementingLargeLanguageModelSolutions

□ThreatModelLLMcomponentsandarchitecturetrustboundaries.

□DataSecurity,verifyhowdataisclassi?edandprotectedbasedonsensitivity,includingpersonalandproprietarybusinessdata.(Howareuserpermissionsmanaged,andwhatsafeguardsareinplace?)

□AccessControl,implementleastprivilegeaccesscontrolsandimplementdefense-in-depthmeasures

□TrainingPi