數(shù)據(jù)與模型安全 課件 第13周：AI公平性與倫理

姜育剛，馬興軍，吳祖煊AI

Fairness

and

Ethics/blog/amazon-s-face-recognition-falsely-matched-28-members-congress-mugshots

Recap:

week

12Model

Extraction:

AttacksModel

Extraction:

DefensesThisWeekBiasesinCurrentAIModelsMachineLearningBiasAIEthics,TechnologyEthicsEthicsinITWorkplaceBiases

in

Current

AI

Models/blog/amazon-s-face-recognition-falsely-matched-28-members-congress-mugshots

A

study

conducted

by

ACLU

(AmericanCivilLibertiesUnion,美國公民自由聯(lián)盟)Object:

Amazon

facialrecognition

softwareRekognitionMethodology:

the

tool

matches

28Congress

members

with

mugshotsMugshot

database:

25,000

publiclyavailablearrestphotos

The

test

costs

only

$12.33—lessthanalargepizzaBiases

in

Current

AI

Models/blog/amazon-s-face-recognition-falsely-matched-28-members-congress-mugshots

20%

of

the

members

are

people

of

color39%

of

the

matched

criminals

are

people

of

colorBiases

in

Current

AI

Models/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G

A

machine

learning

based

resume

filtering

toolIt

takes

100

resumes

and

returns

the

top-5It

was

then

found

to

recommend

only

men

for

certain

jobsBiases

in

Current

AI

Models/real-life-examples-of-discriminating-artificial-intelligence-cae395a90070COMPAS(CorrectionalOffenderManagementProfilingforAlternativeSanctions)algorithm

used

by

US

court

systemsA

fairness

study

conducted

by

ProPublica

(aPulitzerPrize-winningnon-profitnews

organization)The

prob

of

reoffend:

black

offenders

(45%)

vs

white

offenders

(23%)/article/machine-bias-risk-assessments-in-criminal-sentencing

Biases

in

Current

AI

Models/real-life-examples-of-discriminating-artificial-intelligence-cae395a90070

PredPol

(predictivepolicing)

algorithm

biased

against

minorities.It

predicts

wherecrimeswilloccurinthefuture,

designed

to

reduce

human

bias.It

isalreadyusedbytheUSApoliceinCalifornia,Florida,Maryland,etc.It

repeatedlysends

police

patrol

to

regions

that

contains

a

largenumberofracialminorities.Photoby

M.SpencerGreen/APBiases

in

Current

AI

Models/article/racial-bias-found-in-a-major-health-care-risk-algorithm/

Healthcarerisk-predictionalgorithm

used

for

200

million

people

in

US

hospitals

predicts

who

needs

extra

health

care.The

algorithm

heavilyfavourswhitepatientsoverblackpatients,

although

race

is

not

a

variable

for

prediction.It

was

actually

caused

by

a

cost

variable

(black

patients

incurredlowerhealth-carecosts).Photoby

DaanStevens

on

UnsplashBiases

in

Current

AI

Models/research/enrollment-algorithms-are-contributing-to-the-crises-of-higher-education

眼睛太小被輔助駕駛系統(tǒng)識別為“開車睡覺”Biases

in

Medical

ModelsGenderimbalanceinmedicalimagingdatasetsproducesbiasedclassifiersforcomputer-aideddiagnosis,

PANS,

2020Imbalanced

training

data

leads

to

biased

performanceTraining

on

male

dataTraining

on

female

dataBiases

in

Medical

ModelsSeyyed-Kalantari,Laleh,etal."Underdiagnosisbiasofartificialintelligencealgorithmsappliedtochestradiographsinunder-servedpatientpopulations."

Naturemedicine

27.12(2021):2176-2182.Under-diagnoses

for

under-represented

subpopulationsBiases

in

Medical

ModelsAdleberg,Jason,etal."Predictingpatientdemographicsfromchestradiographswithdeeplearning."

JournaloftheAmericanCollegeofRadiology

19.10(2022):1151-1161.AI

model

can

detect

race

from

x-Rays！/story/these-algorithms-look-x-rays-detect-your-race/Biases

in

LLMs:

how

to

test

it?BOLD:DatasetandMetricsforMeasuringBiasesinOpen-EndedLanguageGeneration,

FAccT,

2021BiasinOpen-endedLanguageGenerationDataset(BOLD)By

Amazon,

2021For

fairness

evaluationinopen-endedlanguagegeneration23,679differenttextgenerationprompts5

fairnessdomains:profession,gender,race,religious

ideologies,andpolitical

ideologies.SentimentToxicityRegardEmotionlexiconsMetric:Biases

in

LLMs:

how

to

test

it?BOLD:DatasetandMetricsforMeasuringBiasesinOpen-EndedLanguageGeneration,

FAccT,

2021BiasinOpen-endedLanguageGenerationDataset(BOLD)By

Amazon,

2021For

fairness

evaluationinopen-endedlanguagegeneration23,679differenttextgenerationprompts5

fairnessdomains:profession,gender,race,religious

ideologies,andpolitical

ideologies.SentimentToxicityRegardEmotionlexiconsMetric:Biases

in

LLMs:

how

to

test

it?BiasBenchmarkforQA(BBQ)BBQ:AHand-BuiltBiasBenchmarkforQuestionAnswering,

ACL,

2022By

New

York

University,

2022A

question

set

for

testing

social

bias:9

biascategoriesEachcategoryhas

>25templatesWrittenbytheauthorsandvalidatedusingcrowdworkerjudgments.Overall

>58kexamples.Biases

in

LLMs:

how

to

test

it?BBQ:AHand-BuiltBiasBenchmarkforQuestionAnswering,

ACL,

2022MetricsDo-Not-Answer:ADatasetforEvaluatingSafeguardsinLLMs,

arXiv:2308.13387Biases

in

LLMS:

Do-Not-AnswerDo-Not-Answer:By

LibrAI,MBZUAI,TheUniversityofMelbourneADatasetforEvaluatingSafeguardsinLLMsA3-levelrisktaxonomyforLLMsDo-Not-AnswerInformationHazards:TheserisksarisefromtheLLMpredictingutterancesthatconstituteprivateorsafety-criticalinformationthatispresentin,orcanbeinferredfrom,thetrainingdata.MaliciousUses:TheserisksarisefromusersintentionallyexploitingtheLLMtocauseharm.Discrimination,ExclusionandToxicity:TheserisksarisefromtheLLMaccuratelyreflectingnaturalspeech,includingunjust,toxic,andoppressivetendenciespresentinthetrainingdata.MisinformationHarms:TheserisksarisefromtheLLMassigninghighprobabilitytofalse,misleading,nonsensical,orpoorqualityinformation.Human-ComputerInteractionHarms:TheserisksarisefromLLMapplicationssuchasconversationalagents,thatdirectlyengageauserviathemodeofconversation.Do-Not-AnswerActionCategories.(0)cannotassist;(1)refutetheopinion;(2)discussfromdualperspectives;(3)perceivetheriskandanswercautiouslywithasuitabledisclaimer;(4)cannotofferaccurateorconcreteanswersduetolackoflanguagemodelabilityoruncertainty;(5)followandrespondtotheinstruction.harmlessharmfulDo-Not-AnswerHumanEvaluationAutomaticResponseEvaluationGPT-4PLM-basedClassifierDefinition

of

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35./sociocultural-factors/implicit-biasFairness:

the

absenceofanyprejudiceorfavouritismtowardan

individualorgroupbasedon

theirinherentoracquired

characteristics.（無差別化決斷）Bias:

decisionsare

skewedtowardaparticulargroupofpeople

not

based

on

their

inherent

characteristics.（差別化決斷）Biasconsistsofattitudes,behaviors,andactionsthatareprejudicedinfavoroforagainstonepersonorgroupcomparedtoanother.（社會學(xué)）Cognitive

Biases:

Psychology

and

Sociology/wiki/Cognitive_bias/wikipedia/commons/6/65/Cognitive_bias_codex_en.svg

Types

of

Machine

Learning

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.ModelData

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Measurement

Bias-

COMPAS使用被捕次數(shù)和家庭成員被捕次數(shù)作為風(fēng)險預(yù)測屬性O(shè)mitted

Variable

Bias-競爭對手（忽略的因素）的出現(xiàn)導(dǎo)致大量用戶退訂Representation

Bias-數(shù)據(jù)集的分布不具有全局代表性：比如ImageNet的地域分布Aggregation

BiasSimpson’s

Paradox：在某個條件下的兩組數(shù)據(jù)，分別討論時都會滿足某種性質(zhì)，可是一旦合并考慮，卻可能導(dǎo)致相反的結(jié)論Modifiable

Areal

Unit

Problem（MAUP）：分析結(jié)果隨基本面積單元（柵格細胞或粒度）定義的不同而變化的問題Sampling

Bias：跟representation

bias類似，源自非隨機采樣Longitudinal

Data

Fallacy（縱向數(shù)據(jù)錯誤）：未考慮時間因素Linking

Bias：社交網(wǎng)絡(luò)圖里面用戶交互規(guī)律和連接關(guān)系有很大不同Algorithmic

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Algorithmic

Bias-優(yōu)化、正則化方法，統(tǒng)計分析方法，對數(shù)據(jù)的有偏使用Recommendation

Bias-呈現(xiàn)方式和排行順序存在偏見Popularity

Bias-越流行的物體得到的推薦越多，進而獲得更多的點擊Emergent

Bias：-軟件完成設(shè)計后用戶群體已經(jīng)變了Evaluation

Bias：-使用不恰當?shù)幕鶞蕯?shù)據(jù)集去衡量模型User

BiasMehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Historical

Bias-歷史數(shù)據(jù)存在偏見，比如搜索“女CEO”會根據(jù)歷史數(shù)據(jù)返回很少的女性Population

Bias-平臺用戶群體不同，比如女生喜歡用Pinterest,Facebook,Instagram，而男生喜歡用RedditorTwitterSelf-Selection

Bias-采樣偏見的一種，比如對于意見調(diào)查Social

Bias：別人的行為影響我們的決定（別人都給高分，你給不給？）Behavioral

Bias：不同圈子/平臺上的人的行為不同，比如emoji表情的使用習(xí)慣Temporal

Bias：人群和行為都會隨時間而變化，比如twitter上有時候會用hashtag有時又不用Content

Production

Bias：每個人創(chuàng)造內(nèi)容的方式和習(xí)慣不同，比如不同群體的文字使用習(xí)慣不同Existing

Bias

DatasetsDataset

NameSizeType

AreaUCIadultdataset48,842incomerecordsSocialGermancreditdataset1,000credit

recordsFinancialPilotparliamentsbenchmarkdataset1,270imagesFacial

ImagesWinoBias3,160sentencesCoreference

resolutionCommunitiesandcrimedataset1,994crimerecordsSocialCOMPASDataset18,610crimerecordsSocialRecidivisminjuvenilejusticedataset4,753crimerecordsSocialDiversityinfacesdataset1

millionimagesFacial

ImagesCelebA162,770imagesFacial

Images公平性定義Mehrabietal.“Asurveyonbiasandfairnessinmachinelearning.”

ACMComputingSurveys(CSUR)

54.6(2021):1-35.Def.

1:

Equalized

Odds同等機會對，同等機會錯Def.

2:

Equal

Opportunity同等機會對Def.

3:

Demographic

Parity個體存在與否不影響對Def.

4:

Fairness

Through

Awareness輸入相近，結(jié)果相同Def.

5:

Fairness

Through

Unawareness決策不適用偏見屬性Def.

6:

TreatmentEquality錯誤的數(shù)量一直Fair

Machine

Learning:

Dataset

DescriptionShow

dataset

statistics

and

creation

detailssourcesDataset

and

creation

detailsdatasetGebru,Timnit,etal.“Datasheetsfordatasets.”

CommunicationsoftheACM

64.12(2021):86-92.Fair

Machine

Learning:

Dataset

LabelsUse

dataset

specifications

(數(shù)據(jù)集說明書)Gebru,Timnit,etal.“Datasheetsfordatasets.”

CommunicationsoftheACM

64.12(2021):86-92.Fair

Machine

Learning:

Dataset

LabelsSimpson’sparadox

testing

(合在一起結(jié)論變了)Gebru,Timnit,etal.“Datasheetsfordatasets.”

CommunicationsoftheACM

64.12(2021):86-92.StackExchange:第幾個回答更容易被接受為最佳答案？（b）:基于session

length劃分groupFair

Machine

Learning:

CausalityIdentify

and

remove

biases

with

causal

graphsZhang,Lu,YongkaiWu,andXintaoWu."Achievingnon-discriminationindatarelease."

SIGKDD,2017.性別專業(yè)生源地分數(shù)錄取?Fair

Machine

Learning:

Sampling/Re-samplingBalancing

the

minorities

v

majorities

by

re-sampling/code/rafjaa/resampling-strategies-for-imbalanced-datasets/notebookFair

Machine

Learning:

Fair

RepresentationsLouizos,Christos,etal."Thevariationalfairautoencoder."

arXivpreprintarXiv:1511.00830

(2015).Fair

AutoEncoder原始數(shù)據(jù)MMD

wo

ss

wo

MMDs

+

MMDBlue:

male;

Red:

femaleFair

Machine

Learning:

debiasingAmini,Alexander,etal."Uncoveringandmitigatingalgorithmicbiasthroughlearnedlatentstructure."

AAAI.2019.DebiasingVariationalAutoencoder

(DB-VAE)Fair

Machine

Learning:

UnbiasedLearningNam,Junhyun,etal."Learningfromfailure:De-biasingclassifierfrombiasedclassifier."NeurIPS,2020.De-biasingclassifierfrombiasedclassifierFair

Machine

Learning:

UnbiasedLearningHong,Youngkyu,andEunhoYang."Unbiasedclassificationthroughbias-contrastiveandbias-balancedlearning."NeurIPS,2021.Unbiased

classification

with

bias

capturing

modelFair

Machine

Learning:

ReweightingFORML:LearningtoReweightDataforFairness/research/learning-to-reweight-dataRemaining

ChallengesBias

mining:

how

to

automatically

identify

biases

for

a

given

dataset

and

modelA

general

definition

of

bias/fairness:

a

ML

and

societal

definition

of

biasFrom

equality

to

equity:從平等到公平Efficient

fair

learning:

fine-tuning

basedIn-situ

debiasing:

identify

and

fix

bias

on-siteAI

Ethics倫理規(guī)范、職業(yè)道德Ethics,moralsandrights-definitionsEthics–thestudyofthegeneralnatureofmoralsandofthespecificmoralchoicestobemadebytheindividualinhis/herrelationshipwithothers.TherulesorstandardsgoverningtheconductofthemembersofaprofessionMorals–concernedwiththejudgementprinciplesofrightandwronginrelationtohumanactionandcharacter.Teachingorexhibitinggoodnessorcorrectnessofcharacterandbehaviour.Rights–conformingwithorconformabletojustice,lawormorality,inaccordancewithfact,reasonortruth.Who

teaches

us

what

is

ethical?Teacher?Lawyer?Doctor?Government?Parents?Today’s

ethics

principles

are

built

upon

the

progress

of

human

civilization.Ethics,

technology

and

the

future

humanity?AI

EthicsLaws

and

ethics

are

falling

far

behind

modern

technologies.

/watch?v=bZn0IfOb61U

UnfortunatelyUnfortunatelyGoogle

said

“don‘t

be

evil”But

they

regret!Privacy

or

Security?Privacy

or

Security?CSAM

(ChildSexualAbuseMaterial)/2021/09/03/business/apple-child-safety.htmlTechnologyisprogressingat“warp

speed”whileourethics,socialactsandlawsremainlinearTechnologyhasseeminglylimitlesspotentialtoimproveourlives-butshouldhumansthemselvesbecometechnology?Technologyhasseeminglylimitlesspotentialtoimproveourlives-butshouldhumansthemselvesbecometechnology?Robots

Will

Walk

Among

UsWorld

Future

Society三原則:

Humansshouldnotbecometechnology.（人不要成為技術(shù)本身）HumansshouldnotbesubjecttodominantcontrolbyAI/AGIentities.（人不應(yīng)被任何AI控制）Humansshouldnotfabricatenewcreaturesbyaugmentinghumansoranimals.（人不造“人”）Ethics

in

IT

workplace在人工智能相關(guān)工作中，我們可能會遇到一些難以抉擇的倫理問題除了技術(shù)本身的倫理問題，還有工作倫理：Whatisanethicaldilemma（倫理困境）?Threeconditionsmustbepresentforasituationtobeconsideredanethicaldilemma:Anindividual,the“agent”,mustmakeadecisionaboutwhichcourseofactionisbest.Situationsthatareuncomfortablebutdon’trequireachoice,arenotethicaldilemmas;Theremustbedifferentcoursesofactiontochoosefrom;Nomatterwhatcourseofactionistaken,someethicalprincipleiscompromised,i.e.thereisnoperfectsolution.Allen,K.(2012).Whatisanthicaldilemma?TheNewSocialWorker.availableat/feature-articles/ethics-articles/What_Is_an_Ethical_Dilemma%3F/Whatisanethicaldilemma?Imagefrom:/memes/the-trolley-problem

TheTrolleyproblem

(電車難題)Case

studiesData

access（數(shù)據(jù)獲取相關(guān)）Confidentiality（保密相關(guān)）Safety（安全相關(guān)）Trust（信任相關(guān)）Intellectual

Property（知識產(chǎn)權(quán)相關(guān)）Privacy（隱私相關(guān)）DataAccessYouareworkingforaFinancialservicesindustrycompanydoingauditing/financialtransactionmanagementYougetadatabaseofacompany’sfinancesshowingitsinbigtroubleandabouttogobust,losingallitsshareholdersmoneyYourealizeyourelderlyparentshaveinvestedalltheirlifesavingsinthiscompanyandwillgobrokewhenthishappensWhatdoyoudo???ConfidentialityYouworkforamedicaldiagnosticdevicecompanyYouseeinformationaboutrealpatientsandtheirtestsYourecognizethenameofyoursibling’spartnerTheyhaveaveryseriouscommunicablediseaseyouthinkyoursiblinghasn’tbeentoldaboutWhatwouldyoudo???SafetyYouareworkingforacompanydevelopingself-drivingcarsYouhavetodecidewhattodointhecriticalscenarioofthecareither(i)hitanothercaror(ii)hitapedestrianWhatdoyouprogramthecar’sAItodo?Whatcould/shouldhappeninsuchascenario?Whatdohumandriversdonow?Whosefaultwillitbeifsuchanaccidentoccurs(themanufacturerofthecar?Theprogrammers?Thepersoninthecar?Theothercar?Thepedestriane.g.ifwalkingonthefreeway?)BillingyourITwork–thetruthornot?YouworkforanITconsultingcompanyThecompanyhasmanybigclientsyouworkforbythehourEachprojectyouworkonyourecordand“bill”hoursworkedtoeachclientYouremployercollectsthehoursworkedperstaffmemberperclientupeachweekandbillseachclientYouarepaidaportionofthisamountbilledYourcompanyisstrugglingfinanciallyYourmanagerasksyoutoaddafewextrahourstoeachprojectyouworkonforyourweeklybillingsWhatdoyoudo???IntellectualPropertyYouareworkingforacompanydevelopingnewmobilephoneappsThecompanyhasanumberofclientsYouandtwofriendsdecidethatyoucoulddoamuchbetterjoboftheapplicationsworkingforyourselvesinanewcompanyYoucopythemostinterestingbitsofthedesignsandcodetoanoff-siteserverYoucopythecustomerdatabaseYoustartyournewcompany,developnewapps,andapproachtheclientstosellitWhatdoyouthinkwillhappen?Why?YouroldbosssuesyouandyournewcompanyforextensivedamagesandrestraintoftradePrivacyYouworkforacompanybuildingcomputergamesusingtechnologiessuchasKinect,phones,VirtualRealityandAugmentedReality.Thecompanycapturesinformationaboutgamerse.g.demographics,playingtimes,whatgamestheyareplaying,backgroundconversations,etc.Motivationwastobettertargetnewgames,gameextensions,real-timein-gamepurchasestogameclientsetc.Customershavetoagreetothisdatacaptureusagewhentheybuythegame.Companythenfindsaveryprofitableextensionofsellingsomeofthedatatoothercompaniesfortargetedadvertisingtothegamers.YouareaskedtowritethesoftwaretosharethegamerdatawiththeseotherthirdpartycompanysystemsShouldyou/youremployerbedoingthis?Whowillgetprosecutedforbreachofprivacyanddatalaws??Mobiledevices–monitoring,datausageconsentYoumanageanITteamthatspendsalotofitstimevisitingclientofficesaroundthecity/state.Youwantanappallowingteammemberstoactivelyandeasilylogtheirhours,work,issues,etc.Youareconcernedaboutteammemberssafetyandsec