Europa-AI and digital tools in workplace management and evaluation-An assessment of the EUs legal framework-report (2022-07-26)（英文）行業(yè)資料

PanelfortheFutureofScienceandTechnologyEPRS|EuropeanParliamentaryResearchServiceScientificForesightUnit(STOASTUDYAIanddigitaltoolsinworkplacemanagementandevaluationThisstudyfocusesonoptionsforregulatingtheuseofAIenabledandalgorithmicmanagementsystemsintheworldofworkunderEUlaw.Thefirstpartdescribeshowthesetechnologiesarealreadybeingdeployed,particularlyinrecruitment,staffappraisal,taskdistributionanddisciplinaryprocedures.Itdiscussessomenear-termpotentialdevelopmentprospectsandpresentsanimpactassessment,highlightingsomeofthesetechnologies'mostsignificantimplications.Thesecondpartaddressestheregulatoryfield.ItexaminesthedifferentEUregulationsanddirectivesthatarealreadyrelevanttoregulatingtheuseofAIinemployment.Subsequently,itanalysesthepotentiallabourandemploymentimplicationsoftheEuropeanCommission'sproposalforaregulationlayingdownharmonisedrulesonartificialintelligence(AIact).Finally,itsummarisestheotherongoingEUpolicydebatesrelevanttotheregulationofAIatwork.ThethirdandfinalpartofthisstudyreflectsindetailupontheAIactanditspotentialimpactontheexistingEUsocialacquis.Onthisbasis,itadvancespotentialpolicyoptionsacrossdifferentEUlegislativefiles,includingbutnotlimitedtotheAIact,toensurethatregulationkeepspacewithtechnologicaldevelopment.ItalsoarguesthattheAIactshould'serve'andcomplement–ratherthanover-ride–otherregulatorystandardsthatcanalreadygoverntheintroductionanduseofAI-enabledandalgorithmic-managementsystemsatwork.STOA|PanelfortheFutureofScienceandTechnologyAUTHORSThisstudyhasbeenwrittenbyProfessorValerioDeStefanoofOsgoodeHallLawSchool,YorkUniversity,Toronto,OntarioandtheKULeuven(UniversityofLeuven),andDrMathiasWoutersofKULeuven(UniversityofLeuven)attherequestofthePanelfortheFutureofScienceandTechnology(STOA)andmanagedbytheScientificForesightUnit,withintheDirectorate-GeneralforParliamentaryResearchServices(EPRS)oftheSecretariatoftheEuropeanParliament.Theresearchleadingtothisstudywasalsocarriedoutintheframeworkofthe'Employmentrightsandlabourprotectionintheon-demandeconomy'grantawardedbytheFWOResearchFoundation–FlanderstoProfessorDeStefano.TheauthorswouldalsoliketoexpresstheirgratitudetoProfessorFrankHendrickxandSimonTaesfortheircommentsandsuggestions.ADMINISTRATORRESPONSIBLEPhilipBoucher,ScientificForesightUnit(STOA)Tocontactthepublisher,pleasee-mailstoa@ep.europa.euLINGUISTICVERSIONOriginal:ENManuscriptcompletedinMarch2022.DISCLAIMERANDCOPYRIGHTThisdocumentispreparedfor,andaddressedto,theMembersandstaffoftheEuropeanParliamentasbackgroundmaterialtoassistthemintheirparliamentarywork.Thecontentofthedocumentisthesoleresponsibilityofitsauthor(s)andanyopinionsexpressedhereinshouldnotbetakentorepresentanofficialpositionoftheParliament.Reproductionandtranslationfornon-commercialpurposesareauthorised,providedthesourceisacknowledgedandtheEuropeanParliamentisgivenpriornoticeandsentacopy.usselsEuropeanUnionISBN:978-92-846-9458-7doi:10.2861/305539QA-07-22-330-EN-Nhttp://www.europarl.europa.eu/stoa(STOAwebsite)http://www.eprs.ep.parl.union.eu(intranet)http://www.europarl.europa.eu/thinktank(internet)http://epthinktank.eu(blog)AIanddigitaltoolsinworkplacemanagementandevaluationExecutivesummaryArtificialintelligence(AI)isspreadingintoworkplacesinmanyways.WorkersandemployerswillincreasinglybeconfrontedwithAIapplicationsandAI-inducedsoftwarethatencroachonlabourrelations.TheseAIsystemscouldbeintroducedwithanexplicitaimtoimproveworkingconditions;however,thiscannotbeexpectedingeneral,andevidencesuggeststhatthiswillinsteadbetheexception.Rather,forpolicy-makersandlegislators,itwillbeaquestionofensuringthattheintroductionandoperationofAIintheworkplacearecarriedoutinamannerthatismindfulofAI'simpactonworkingconditions.FurthermoreregulationwillhavetoforcebusinessestominimiseAI'snegativeimpactonlabourprotections.Tothisend,thisstudypresentspolicyoptionsintentonpreservinganadequateleveloflabourandsocialprotectionsinlightoftheintroductionofAIsystemsatwork.ThesepolicyoptionsarelimitedtoreformsthattheEuropeanUnion'sinstitutionscanintroduce.Theyaredevelopedthroughoutthisstudyby,first,exploringandassessingtheimpactsofhownewrobots,'smart'workequipment,software,wearablesandothertechnologicaltoolswillincreasinglycollectworkers'datausedbyvariousformsofAItopursuetheirhuman-definedorganisationalgoals.ThestudythenexaminestheextenttowhichcurrentEuropeanlawsgoverntheuseofAIatworkandanalysestheinteractionbetweenexistinglawsandtheEuropeanCommission'sproposallayingdownharmonisedrulesonAI(AIact)andthelatter'srelevanceconcerningworkingconditions,providingthebuildingblocksforthepolicyoptions.Thekeyfindingsofthisstudyincludethefollowing.AIwillbecomeprominentintheworkplace–Overtime,manyformsofsoftware,bothindustry-specificandgeneralist,willacquireAIfunctionalitiesthatwillimpactworkingconditions.AI,asageneraltechnology,islikelytobecomeaprominentfeatureinmanypeople'sjobsinthenextfewyears.Atthesametime,amorelimitedgroupofAItechnologieswithparticularlyfar-reachingeffectsonworkerswillinitiallybecomeprominentinsomeindustriesmorethanothers.Atleastshortly,notallindustrieswillbecharacterisedbyequallyintrusiveAIsystems.Certainbusinesses,forinstance,engagedinlogisticsandstockmanagement,seemtoalreadyextensivelyrelyonAI-enabledtoolsandapplicationstodrivetheireconomicactivities.However,inothersectors,developmentsaresofarmorelimited,evenifthiscouldchangeintime.Thiswillbemadepossible,amongotherthings,bytheelaborationofgenericAIproducts,whichwillbemarketedasbeingadaptableacrossindustries.Likewise,consultancycompanieswillbecomemoreexperiencedinproposingAIsolutionstobusinesschallenges.Moreover,genericsoftwarepackages,suchasthoseforhumanresourcesmanagement(HRM),willalsoincreasinglyincludesomestandardAIfeatures.Hence,AIwillbecomemorecommonplaceinmanagingpeopleatworkthroughouttheeconomiesofindustrialisedcountries.IntroducingandoperatingAIatworkposeschallengesforbusinesses–AlthoughitmightseemloorbystormsomeofourfindingssuggesttheoppositeFirstlyemployerswillhavetocarefullyreflectonwhetherandhowAIcouldbestservetheirneedsbyconsideringexistingemploymentandlabourprotectionsandrights,includingfundamentalones.LeavingasideAIapplicationsofminorimportance,companieswillhavetothinklongandhardabouthowtheycanincorporateAIintotheirbusinessstructuretoobtainthebestresults.Manyofthebesttoolswilllikelycomewithaheftypricetag,requireadditionalinvestmentinmaterialandpersonnel,ordemandadjustmentstocompanies'workorganisation.Therefore,beforecommittingtofar-reachingAI-basedsystems,employerswillhavetoexaminethebusinesscaseforit.Thisalsopresentsanopportunityforlawandpolicytointervene.AsalreadyhappensinmanyEUcountries,thelawcandemandcompaniestofullyconsiderworkers'concerns,interestsandrightsbeforeimplementingnewtechnologies,somethingwhichwouldbeentirelyjustifiedbyseveralreasonsdiscussedinthestudy.STOA|PanelfortheFutureofScienceandTechnologyThefunctionalityandutilityofmanyformsofAIatworkremainindoubt–Forexample,manyoftheAItoolscurrentlydeployedinrecruitmentremainuntested.Astheliteraturepointsout,therearemanyvalidreasonstoadoptacautiousandcriticalattitudetowardsAI-poweredrecruitmentapplications.Similarly,inthefuture,AIwilllikelyleadtomorecontinuousstaffappraisal.Insteadofevaluatingworkers'performanceonlyacoupleoftimesayear,AIsystemsareprojectedtoprovidemorecontinuousfeedback.Yet,despitethetheoreticalpromiseofAIinthisarea,itseemsrathercomplicatedtoeffectivelyincorporateAIinstaffappraisalinpractice.Forinstance,howcouldtherebeanycertaintythatAIisobtainingandusingallnecessarydatatoreachacertainconclusion?Currentsystemseffectivelycollectandcompiledata,butinterpretingsuchdataismuchtrickier,especiallyifhumansarenotallowedtocorrectasystem'sinferences.FurthermoresomecompaniesalreadyrelyonalgorithmsandAItodistributetasks,monitorworkersandevaluatetheirperformanceandattitudes.Relianceonsuchsweeping'surveillanceloops'isstillsomehowlimited,but,intime,thesemayspread.ManymorebusinesseswillrelyonAItoevaluateworkers,afterwhichtheAI'sfindingswilldirectlyinfluencethedecisionsofmanagers.AcriticalissueinthisrespectisthatbusinessesrisktrustingAIdisproportionatelyandunwarrantedly.Onecrucialwaytobetterassesswhetherthesesystemsfunctionadequatelywillbebyinforming,consultingandnegotiatingwithworkersandtheirrepresentatives.AIispromisingfortheworldofwork,butitisnotasilverbullet.Itrequiresalotofefforttobeimplementedsuccessfullywithoutproducingexcessiveadverseeffectsthat,amongotherthings,arealsohighlydetrimentaltoworkers.BusinessesthatleadindevelopingAItechnologieswillsetthelinesthatotheremployersfollow;therefore,actionisurgentlyneeded–Manyemployersstilllackthehardware,data,staff,etc.,requiredtorelyonAIpurposefully.However,somebusinessesattheforefrontofthistechnologyarealreadypacingahead,settingoutthelinesthatmanyotheremployerswillsoonfollow.Therefore,wemusturgentlypayattentiontothebusinessesandthesectorsmostaffectedbythesedevelopmentsandhavebroaderdiscussionsaboutwhatgoalsAIshouldserve.Thetimetosetandenforceclearboundariestodissuadeinvestmentinillegal,unethicalandundesirableAIapplicationsisnow.ThisisachallengethatEuropeaninstitutionscouldfaceimmediately.AIhasevidentdrawbacksintermsofitsimpactonworkingconditions–AIwill,amongotherthings,bolsterthedataficationofwork,makingdataprotectionandprivacyrightsallthemoreimportantasacounterweight.AIislikewisepoisedtomakeworkmoreprecarious,boostbusinesses'surveillancecapabilities,and,insomeinstances,evenfrustrateunionactivities.Furthermore,despitesomeproclaimedpromisesofAIinoccupationalsafetyandhealth(OSH)andnon-discrimination,thesetechnologiesalsopresentclearthreatsintheseareas.Overall,therearegenuineriskstotheincorporationofAIatwork.Drawbacksthatcanbecounteractedthroughregulation–Despitetheseverepotentialerosionofworkingconditions,ourreviewoftheEU'sregulatorycontextindicatesthat,withthenecessaryadjustments,muchoftheexistingregulationcancontinuetofunction.Indeed,itcansteertheimplementationoffutureAIatworkinadesirabledirection.Forthistooccur,weoutlineheresomeofthemostrelevantoptions–amuchmoreextensivelistcanbefoundinChapter5.BasedontheEuropeanUnion'sprimarylaw,theEuropeanresponsetoAIatworkneedsto,amongotherthings,primarilyfocusonfundamentalandhumanrights.'AIethics'isimportant;however,theprotectionofestablishedrightsshouldnotbewatereddownbygenericdiscoursesaboutethics.TheEuropeanUnion'sCharterofFundamentalRightsenshrinesacrucialrangeofpersonal,civil,political,economicandsocialrights.AnyrethinkingoftheEU'ssecondarylawinresponsetoAIshouldalsopaydueregardtothetensionbetweencertainAIapplicationsandthoserights.ThepotentialbenefitsofAIdonotjustifyinfringingtheserights.ItisuptotheEuropeaninstitutionsandtheMemberStatestokeepAIfromendangeringdecentandjustworkingconditions.AIanddigitaltoolsinworkplacemanagementandevaluationVForexample,theprinciplesenshrinedintheGeneralDataProtectionRegulation(GDPR)provideanessentialframeworktomitigatethenegativeconsequencesofAIatwork,especiallybutnotsolelyfromaprivacyperspective.AnyAI'sgoalsandfunctioninginevitablydependonthedataavailable.Dataprotectionlawscanthuseffectivelyestablishboundaries.Assumingtheserulesareadequatelyenforced,whichistherealchallenge,theGDPR'sprinciplesoflawfulness,fairness,transparency,purposelimitation,dataminimisationandaccuracycansignificantlymitigatetheriskthatharmfulAIsystemsareelaboratedandimplementedintheEU.Atthesametime,nothingintheGDPRpreventsAIdevelopersandemployersfromcarefullyintroducingAIatworkaslongasthefundamentalrightsandinterestsofallthedatasubjects,particularlytheworkers,areconsidered.BesidestheGDPR,whichisfundamentallyimportantfortheprotectionofworkersagainstundesirableAI,EUlawalsoprovidesworkers'representativesandsocialpartnerswithamuch-neededlegalbasistoclaimconsultationrightswhenAIisintroducedtotheworkplace.Mostnotably,EUDirective2002/14/ECenshrinesarighttoinformationandconsultationthatcanoperate,forinstance,whenanAIoranalgorithmictoolcanleadtosubstantialchangesinworkorganisation.Inthisrespect,however,MemberStates'nationallawsneedtobereviewedtoensurethedirectiveisadequatelyimplemented.EUlawprovidessomebasistosubjectAIto'systematic'socialdialogue.Yet,atpresent,MemberStatesoftenprotectandpromoteinformationandconsultationrightsstemmingfromEUlawsomewhatunsatisfactorily.ConcerningAI,thiscouldbeaddressedbybetterclarifyingtheextentandscopeoftheserights.Similarly,undercurrentrules,employersshouldalreadyperformriskassessmentstoidentifyandaddressOSHriskswhenintroducingAIintotheworkplace.Someanalyses,however,havewarnedthattotheextentitmerelyconcernssoftware,employersmayunderestimatethepotentialOSHrisks,includingpsychosocialrisks,broughtforthbyAI.Forthisreason,futureguidanceonhowtoassessAIfromanOSHperspective,beitintheformofasoftlaworahardlawinstrument,couldbehighlyvaluable.ThedutyofemployerstoassessandreacttotheOSHrisksassociatedwithcertainformsofAImust,indeed,bestressed.Itisaveryreasonableobjectiveforpolicy-makerstopursuebybuildingontheexistingOSHframework.EUnon-discriminationlawsarelikewisealreadysomehowbeingusedtovetAIsystemsintheworkplace.However,theselaws'effectivenesslargelydependsontheeasewithwhichtheburdenofproofincourtisplacedontheuseroftheAIsysteminsteadofontheworkers.Inprinciple,claimantsmustbringprimafacieevidenceofdiscriminationbeforetheburdenofproofshiftstothedefendant.LegislaturesandcourtswillplayavitalpartindeterminingatwhichpointtheentityinchargeoftheAIisobligedtodisproveallegationsofdiscrimination.AlternativeproceduresavailableoutsidecourttoscrutiniseAIsystems,suchascertificationorspecialisedauditing,willalsobecrucialincombatingdiscriminatoryapplicationsofAIandalgorithmic-managementsystems.Themosteffectiveofsuchprocedurescouldeventuallyfindtheirplaceintheregulations.Theburdenofproofforclaimantsincourtshould,forexample,beloweredincaseanAIsystemisnotcertified.Varioussuchinteractionsareconceivable.Regulatorychangeswillberequiredinvariousfieldsoflaw–Byandlarge,acrossmultipleareasoflabourandemploymentlegislation,onecouldconcludethatgoverningAIatworkisnotunfeasible,providedthattherulesareadequatelyadaptedtothechallengesAIposes.However,thissomewhatpositiveoverallassessmentofcurrentEUlawdoesnotmeanwecanbeunconcernedaboutanyofthesematters.Infact,withoutthenecessaryregulatorychangesandinvestmentsintoeffectiveenforcementmechanismsinvariousareas,AImaywellspiraloutofcontrolatworkandelsewhere.Moreover,evenwithsolidgovernancestructures,thingsmaystillbecomeunpredictable.Forinstance,asnotedinthestudy,inthelongrun,howAIsystemswillimpactworkingtimeandworkers'contractualrelationsisstilllargelyunknown.Inthissense,eventhoughsomeactions,suchaslimitingtheuseofsuccessiveshort-termemploymentcontracts,canalsoalreadybetaken,policy-makersshouldarguablyadoptarealisticapproachtotheregulationofSTOA|PanelfortheFutureofScienceandTechnologyAI.Concretemeasuresshouldarguablybetakensoon.Thisstudyarguesthatamendmentsarepossibleanddesirabletovariouslabourlaws;addedtogether,theywouldresultinalegalenvironmentthattakenasawholebecomesmuchmorecapableofgoverningtheuseofAIatworkTheAIactshouldbolstertheeffectivenessofotherlawsingoverningAI–Withthisinmind,lookingatthecurrentsituationfromanemploymentperspective,webelievethattheEuropeanCommission'sproposedAIactcouldstrengthenexistingregulatorymechanismstoprotectworkingconditions.However,quitetothecontrary,thecurrentdraftAIactarguablyimposesaregulatory'ceiling',implicitlyweakeningthecapacityofvariousexistingregulatorymechanisms.Indeed,thecurrentdraftwouldissueanall-encompassingframeworkmeanttodeterminewhatAIistrustworthy,afterwhichsuch'trustworthy'AIreceivesalmostfreereigntoentertheworkplace.Manyexistingworkprotectionsystemswillbe'gutted'ratherthanenhancedbythisapproach.ForallthereasonsdiscussedinChapter4,thisstudyarguesthatitisvitaltoputtheAIactattheserviceratherthanabovetheotherlawsthatcouldgoverntheintroductionanduseofAIandalgorithmicmanagementsystemsintheworkenvironment.Thiswillrequirematerialamendmentstothecurrentdraft,somethingwhichwarrantsseriousscrutinyofthisproposedinstrument.Additionally,theAIactintersectswithmanyotherEUpolicyinitiativesthatwillframethefutureofAIsystemsinEuropeanworkplaces.Therefore,itisimportanttocoordinatethesevariousinitiatives.Inthisregard,forexample,therecentdraftdirectiveonplatformworkcontainsachapteronalgorithmicmanagementthatoffersvaluableandyetimprovableprotectionsforplatformworkers.However,unlesstheAIactincludessimilarmeasures,allworkersbeyondplatformworkerswillremainmuchmorevulnerabletotherisksposedbyAIandalgorithmicmanagementatwork.OnecanalsoquestionwhetherthedraftregulationisentirelyinlinewiththeGDPR.TheAIactshouldbeaparticipantinsteadofanarbiterinthefieldoflabourprotection–TheAIactmightsucceedinregulatingAIassuch.Stilliftheaimistoregulate'AIatwork'adequately,wearguethatEUpolicy-makersshouldkeepthemanyexistinglawsinmindandoptforacoherentsetofmeasuresacrossdifferentfieldsoflaw.SalvationwillnotcomefromtheAIactalone.ThegovernanceofAIatworkcoulddrawonmanymoreareasoflaw.Therefore,Section5.2concludesthisstudybydrawingamateriallistofpolicyoptionsinthecontextofdifferentlegislativefiles,includingbutnotlimitedtotheAIact,that–itistheopinionoftheauthors–mightallowthebenefitsofAIatworktobereapedbyadequatelycounteringitsrelevantrisks.AIanddigitaltoolsinworkplacemanagementandevaluationTableofcontents1.Introduction:Settingthescene 12.Methodology 43.ReviewofAIforHRmanagement 53.1.Thetechnologiesandtools 53.2.TheapplicationofAIinvariouscontexts 103.2.1.Recruitment 103.2.2.Performancemanagement:Staffappraisalandprofessionaldevelopment 133.2.3.Taskdistribution,managementandevaluation 163.2.4.Retention,rewardsandpromotion 193.2.5.Disciplinaryprocedures 203.3.Near-termpossibledevelopmentprospects(nextfiveyears) 233.4.Assessmentofimpacts 264.Reviewoftheregulatorycontext 334.1.ResolutionsoftheEuropeanParliament 334.2.CurrentEUlegislation 354.2.1.EUPrimarylaw 354.2.2.SecondaryLaw.TheGeneralDataProtectionRegulation 354.2.3.Transparencyandworkerinvolvement 424.2.4.Occupationalsafetyandhealth 434.2.5.Workingconditions 444.2.6.Anti-discrimination 474.3.OngoingnegotiationsrelatedtotheAIAct 494.4.Policydebates 565.Policyoptions 605.1.Criticalpointsinthecontextofdifferentlegislativefiles 60STOA|PanelfortheFutureofScienceandTechnology5.2.Anoverviewofpolicyoptions__________________________________________________636.Conclusions__________________________________________________________________70AIanddigitaltoolsinworkplacemanagementandevaluationListofabbreviationsAGIArtificialgeneralintelligenceIArtificialintelligenceAIaaSAIasaserviceAIactRegulationoftheEuropeanParliamentandoftheCouncillayingdownharmonisedrulesonartificialintelligence(ArtificialIntelligenceact)AndamendingcertainunionlegislativeactsAIDASpecialCommitteeonArtificialIntelligenceinaDigitalAgeBrain-computerinterfaceCDEICentreforDataEthicsandInnovationCharterEUCharterofFundamentalRightsCIPDCharteredInstituteofPersonnelandDevelopmentCJEUCourtofJusticeoftheEuropeanUnionCNILCommissionNationaledel'InformatiqueetdesLibertésDPADataprotectionauthorityDPIADataProtectionImpactAssessmentDPODataprotectionofficerECHREuropeanConventiononHumanRightsECtHREuropeanCourtofHumanRightsEDPBEuropeanDataProtectionBodyEDPSEuropeanDataProtectionSupervisorEESCEuropeanEconomicandSocialCommitteeTUCEuropeanTradeUnionConfederationEU-OSHAEuropeanOccupationalSafetyandHealthatWorkAuthorityFLIFutureofLifeInstituteGDPRGeneralDataProtectionRegulationHBSHarvardBusinessSchoolHumanresourcemanagementOInformationCommissioner'sOfficeInternationalLabourOrganizationInternetprotocolSTOA|PanelfortheFutureofScienceandTechnologyXMachinelearningNFCNear-fieldcommunicationOECDOrganisationforEconomicCo-operationandDevelopmentOSHOccupationalsafetyandhealthPwCPricewaterhouseCoopersMSSocialmediascreeningTUCTradesUnionCongressFWorldEconomicForumAIanddigitaltoolsinworkplacemanagementandevaluation11.Introduction:SettingthesceneIn2016DanielKahnemanandcolleaguespublishedanarticleintheHarvardBusinessReviewdetailinghowunreliablehumansareasdecision-makers.Astheauthorscontended,'[r]esearchhasconfirmedthatinmanytasks,experts'decisionsarehighlyvariable:valuingstocks,appraisingrealestate,sentencingcriminals,evaluatingjobperformance,auditingfinancialstatements,andmore'(Kahnemanetal.,2016,40).Theseinherenterrorsinhumanjudgements,leadingtovariabilityequentbooktheseauthorsarguedthatalgorithms1andartificialintelligence(AI)mayconstituteavaluableimprovementinthisrespectbecausethesetoolshardlysufferfrom'noise'(Kahneman,SibonyandSunstein,2021).Inprinciple,anAIwillreachthesameoutcomesbasedonthesameinputs.PreciselythisisoneofthereasonswhyAImaybecomeofcrucialimportanceinbusinessesaroundtheworld.Itpromisesconsistency,objectivityand,insomeinstances,explicability.Inthisregard,onecouldthinkofAIasabrilliantcalculator:'theyarebothmachinesdesignedtoconvertinputintooutputinwaysthathumans–whohaveminds–choosetointerpretasmeaningful'(Heaven,2021,70).However,thisfactalsocomeswithitsownsetofrisks.Awell-knownexampleisCOMPAS,whichstandsfor'CorrectionalOffenderManagementProfilingforAlternativeSanctions'.Lawenforcementusedthiscomputerprogramtopredictdefendants'likelihoodofcommittingafuturecrime.ThisAItoolwasusedeventhough'[t]healgorithmunderlyingCOMPASisheldasatradesecretbyitsmanufacturer,Northpointe(nowEquivant),whichmeansthatwedon'tknowhowCOMPASgeneratesitspredictions,nordowehaveaccesstothedatathealgorithmistrainedon—sowecannoteveninquireintoitsrationale'(Babicetal.,2020,65).Asitturnsout,aProPublicainvestigationdiscoveredthealgorithmswerebiasedagainstAfricanAmericans(Angwinetal.,2016);inotherwords,this'intelligent'calculatorwasbroken.Inasimilarvein,astheupcomingchapterswillillustrate,awiderangeofAIapplicationsarebeingusedintheworldofworkthatcould–intheory–improvehumanjudgementandarepurportedtodoso.Ifthetoolsworked,theycouldsignificantlyimproveworkinglife.Alongthoselines,personscouldalsobemanagedthroughAI-basedsystems.Thiswouldallegedlyimplyfewerbiasesandcronyism;staffappraisalswouldbeentirelytransparentandobjective;managementcouldalsobewarnedbyanAIwhensomeoneshowsearlysignsofburnout;staffschedulingcouldeventakeanindividual'swork-lifebalanceintoaccount.However,oneofthebiggestproblemsisthat,inreality,mostofthesetoolsdonotwork.Sometoolsmightshowtoomanyfalsepositivesornegatives.Forselyidentifysomeonewhoisatriskofburnoutwhileatthesametimeneglectingapersonwhotrulyneedshelpbutdoesnotreceiveitsinc