人工智能不確定性

手寫數(shù)字識別文本分類圖像分割第八章Uncertainty

不確定性對應(yīng)教材第13章本章大綱Uncertainty不確定性Probability概率SyntaxandSemantics語法與語義Inference推理IndependenceandBayes‘Rule

—獨立性及貝葉斯法則

不確定性智能體幾乎從來無法了解關(guān)于其環(huán)境的全部事實。因此其必須在不確定的環(huán)境下行動。概率推理

得到了某一證據(jù)，那么有多大的幾率結(jié)論為真？

例如：我頸部痛;我得腦膜炎的可能有多大?不確定性假如有如下規(guī)則:

iftoothache（牙疼）then原因是cavity（牙齒有洞）但并不是所有牙疼的病人都是因為牙齒有洞，所以我們可以建立如下規(guī)則:

iftoothacheand?gum-disease（牙齦疾?。゛nd

?filling（補牙）and...thenproblem=cavity以上規(guī)則是復(fù)雜的;更好的方法:

iftoothachethenproblemiscavitywith0.8probability

orP(cavity|toothache)=0.8

theprobabilityofcavityis0.8giventoothacheisobserved不確定性 LetactionAt=離起飛時間提前t分鐘動身去機場

At會使我準(zhǔn)時到達機場嗎?

Problems:

1.partialobservability/部分可觀察性(roadstate,otherdrivers‘plans)

2.noisysensors(trafficreports)

3.行動結(jié)果的不確定性(flattire,etc.)

4.immensecomplexityofmodelingandpredictingtraffic

因此一個純粹的邏輯描述方法：

1.risksfalsehood（錯誤風(fēng)險）:“A25

willgetmethereontime”,or

2.leadstoconclusionsthataretooweakfordecisionmaking:

“A25

willgetmethereontimeifthere’snoaccidentonthebridgeanditdoesn‘trainandmytiresremainintactetcetc.”

(A1440

mightreasonablybesaidtogetmethereontimebutI’dhavetostayovernightintheairport…)世界與模型中的不確定性Trueuncertainty:rulesareprobabilisticinnature

擲骰子，拋硬幣惰性:把所有意外的規(guī)則都列舉出來是很困難的

花費太多時間來確定所有的相關(guān)因素

這些規(guī)則過于繁雜而難以使用理論的無知:某些領(lǐng)域中還沒有完整的理論

(e.g.,medicaldiagnosis)實踐的無知:掌握了所有規(guī)則但是

并不是所有的相關(guān)信息都能被收集到處理不確定性的方法概率理論作為一種正式的方法for：

不確定知識的表示和推理

命題中的模型信度(event,conclusion,diagnosis,etc.)

給定可獲得的證據(jù),

A25

willgetmethereontimewithprobability0.04概率是不確定性的語言

現(xiàn)代AI的中心支柱Probability概率概率理論提提供了一種種方法以概概括來自我我們的惰性性和無知的的不確定性性。ProbabilisticassertionssummarizeeffectsofLaziness（惰性）:failuretoenumerateexceptions（例外）,qualifications（條件）,etc.Ignorance（理論的無無知）:lackofrelevantfacts,initialconditions,etc.Subjectiveprobability（主觀概率）:

Probabilitiesrelatepropositions（命題）toagent'sownstateofknowledge

e.g.,P(A25|noreportedaccidents)=0.06Thesearenotassertions（斷言）abouttheworld命題的概率率隨著新證證據(jù)的發(fā)現(xiàn)現(xiàn)而改變:

e.g.,P(A25|noreportedaccidents,5a.m.)=0.15不確定條件件下的決策策假設(shè)下述概概率是真的的:P(A25getsmethereontime|……)=0.04

P(A90getsmethereontime|……)=0.70

P(A120getsmethereontime|……)=0.95

P(A1440getsmethereontime|……)=0.9999Whichactiontochoose?Dependsonmypreferences（偏好）formissingflightvs.timespentwaiting,etc.Utilitytheory（效用理論論）用來對偏好好進行表示示和推理Decisiontheory=probabilitytheory+utilitytheory決策理論=概率理論+效用理論Syntax語法基本元素:randomvariable（隨機變量量）Arandomvariableissomeaspectoftheworldaboutwhichwe(may)haveuncertainty通常大寫e.g.,Cavity,Weather,Temperature類似于命題題邏輯:未知世界被被隨機變量量的賦值所所定義Booleanrandomvariables（布爾隨機機變量）e.g.,Cavity（牙洞）(doIhaveacavity?)Discreterandomvariables（離散隨機機變量）e.g.,Weatherisoneof<sunny,rainy,cloudy,snow>定義域mustbeexhaustive（窮盡的））andmutuallyexclusive（互斥的））Continuousrandomvariables（連續(xù)隨機機變量）e.g.,Temp=21.6;alsoallow,e.g.,Temp<22.0SyntaxElementaryproposition（命題）constructedbyassignmentofavaluetoarandomvariable:e.g.,Weather=sunny,Cavity=false(簡寫為?cavity)Complexpropositionsformedfromelementarypropositionsandstandardlogicalconnectivese.g.,Weather=sunny∨Cavity=falseSyntaxAtomicevent:Acompletespecificationofthestateof

theworldaboutwhichtheagentisuncertain原子事件：對智能體體無法確定定的世界狀狀態(tài)的一個個完

整的詳細描述述。E.g.,iftheworldconsistsofonlytwoBooleanvariablesCavityandToothache,thenthereare4distinctatomicevents:Cavity=false∧Toothache=falseCavity=false∧Toothache=trueCavity=true∧Toothache=falseCavity=true∧Toothache=trueAtomiceventsaremutuallyexclusiveandexhaustive窮盡和互斥斥概率公理對任意命題題A,B0≤P(A)≤1

P(true)=1andP(false)=0

P(A∨B)=P(A)+P(B)-P(A∧B)Priorprobability（先驗概率率）Priororunconditionalprobabilities（無條件概概率）ofpropositions在沒有任何何其它信息息存在的情情況下關(guān)于于命題的信信度e.g.,P(Cavity=true)=0.1andP(Weather=sunny)=0.72correspondtobeliefpriortoarrivalofany(new)evidenceProbabilitydistributiongivesvaluesforallpossibleassignments:概率分布給出一個隨隨機變量所所有可能取取值的概率率P(Weather)=<0.72,0.1,0.08,0.1>(normalized（歸一化的的）,i.e.,sumsto1)Jointprobabilitydistributionforasetofrandomvariablesgivestheprobabilityofeveryatomiceventonthoserandomvariables(i.e.,everysamplepoint)聯(lián)合概率分分布給出一一個隨機變變量集的值值的全部組組合的概率率P(Weather,Cavity)=a4××2matrixofvalues:Everyquestionaboutadomaincanbeansweredbythejointdistributionbecauseeveryeventisasumofsamplepoints連續(xù)變量的的概率Expressdistributionasaparameterized（參數(shù)化的的）functionofvalue:P(X=x)=U[18,26](x)=uniform（均勻分布布）densitybetween18and26連續(xù)變量的的概率MarginalDistributions（邊緣概率率分布）Marginaldistributionsaresub-tableswhicheliminatevariablesMarginalization(summingout):CombinecollapsedrowsbyaddingConditionalprobability（條件概率率）Conditionalorposteriorprobabilities（后驗概率率）P(a|b)證據(jù)累積過過程的形式式化和發(fā)現(xiàn)現(xiàn)新證據(jù)后后的概率更更新當(dāng)一個命題題為真的條條件下，指指定命題的的概率e.g.,P(cavity|toothache)=0.8i.e.,鑒于牙疼是是已知證據(jù)據(jù)(Notationforconditionaldistributions（條件概率率分布）:P(cavity|toothache)=asinglenumberP(Cavity,Toothache)=2x2tablesummingto1P(Cavity|Toothache)=2-elementvectorof2-elementvectorsIfweknowmore,e.g.,cavityisalsogiven,thenwehaveP(cavity|toothache,cavity)=1新證據(jù)可能能是不相關(guān)關(guān)的，可以以簡化,e.g.,P(cavity|toothache,sunny)=P(cavity|toothache)=0.8條件概率定義條件概率為為:

P(a|b)=P(a∧∧b)/P(b)ifP(b)>0Productrule（乘法規(guī)則則）givesanalternativeformulation:

P(a∧b)=P(a|b)P(b)=P(b|a)P(a)Ageneralversionholdsforwholedistributions,e.g.,P(Weather,Cavity)=P(Weather|Cavity)P(Cavity)(Viewasasetof4××2equations,notmatrixmultiplication)Chainrule（鏈?zhǔn)椒▌t則）isderivedbysuccessiveapplicationofproductrule:條件概率條件概率跟跟標(biāo)準(zhǔn)概率率一樣,forexample:0<=P(a|e)<=1conditionalprobabilitiesarebetween0and1inclusiveP(a1|e)+P(a2|e)+...+P(ak|e)=1conditionalprobabilitiessumto1wherea1,…,akareallvaluesinthedomainofrandomvariableAP(?a|e)=1-P(a|e)negationforconditionalprobabilities通過枚舉的的推理Startwiththejointprobabilitydistribution（全聯(lián)合概概率分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一個命題的的概率等于于所有當(dāng)它它為真時的的原子事件件的概率和和通過枚舉的的推理Startwiththejointprobabilitydistribution（全聯(lián)合概概率分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一個命題的的概率等于于所有當(dāng)它它為真時的的原子事件件的概率和和通過枚舉的的推理Startwiththejointprobabilitydistribution（全聯(lián)合概概率分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一個命題的的概率等于于所有當(dāng)它它為真時的的原子事件件的概率和和通過枚舉的的推理Startwiththejointprobabilitydistribution（全聯(lián)合概概率分布））:Normalization（歸一化））Denominator（分母）canbeviewedasanormalizationconstantαP(Cavity|toothache)=αP(Cavity,toothache)

=α[P(Cavity,toothache,catch)+P(Cavity,toothache,?catch)]

=α[<0.108,0.016>+<0.012,0.064>]

=αα<0.12,0.08>=<0.6,0.4>Generalidea:computedistributiononqueryvariablebyfixingevidencevariables（證據(jù)變量量）andsummingoverhidden

variables（未觀測變變量）通過枚舉的的推理Typically,weareinterestedintheposteriorjointdistributionofthequeryvariables（查詢變量量）Ygivenspecificvaluesefortheevidencevariables（證據(jù)變量量）ELetthehiddenvariables（未觀測變變量）beH=X-Y–EThentherequiredsummationofjointentriesisdonebysummingoutthehiddenvariables:P(Y|E=e)=αP(Y,E=e)=αΣhP(Y,E=e,H=h)ThetermsinthesummationarejointentriesbecauseY,EandHtogetherexhaustthesetofrandomvariables（Y,E,H構(gòu)成了域中中所有變量量的完整集集合）Obviousproblems:

1.Worst-casetimecomplexityO(dn)wheredisthelargestarity2.SpacecomplexityO(dn)tostorethejointdistribution3.HowtofindthenumbersforO(dn)entries?Independence（獨立性））AandBareindependentiffP(A|B)=P(A)orP(B|A)=P(B)orP(A,B)=P(A)P(B)E.g:rollof2die:P({1},{3})=1/6*1/6=1/36P(Toothache,Catch,Cavity,Weather)=P(Toothache,Catch,Cavity)P(Weather)32entriesreducedto12;fornindependentbiasedcoins,O(2n)→O(n)Absoluteindependencepowerfulbutrare絕對獨立強強大但罕見見Dentistry（牙科領(lǐng)域域）isalargefieldwithhundredsofvariables,noneofwhichareindependent.Whattodo?獨立的濫用用天真的數(shù)學(xué)學(xué)笑話：一個著名統(tǒng)統(tǒng)計學(xué)家永永遠不會坐坐飛機旅行行,因為他研究究了航空旅旅行和估計計,任何給定的的航班上有有炸彈的可可能性是一一百萬分之之一,他不準(zhǔn)備接接受這些可可能性。有一天，一一位同時在在遠離家鄉(xiāng)鄉(xiāng)的會議上上遇到他。?！澳阍趺疵吹竭@里的的？坐火車車嗎？”“不，我飛過過來的”“Whataboutthepossibilityofabomb?”“Well,Ibeganthinkingthatiftheoddsofonebombare1:million,thentheoddsoftwobombsare(1/1,000,000)x(1/1,000,000).Thisisavery,verysmallprobability,whichIcanaccept.SonowIbringmyownbombalong!”Conditionalindependence條件獨立性性Randomvariablescanbedependent,butconditionallyindependentExample:YourhousehasanalarmNeighborJohnwillcallwhenhehearsthealarmNeighborMarywillcallwhenshehearsthealarmAssumeJohnandMarydon’’ttalktoeachotherIsJohnCallindependentofMaryCall?No–IfJohncalled,itislikelythealarmwentoff,whichincreasestheprobabilityofMarycallingP(MaryCall|JohnCall)≠≠P(MaryCall)條件件獨獨立立性性But,ifweknowthestatusofthealarm,JohnCallwillnotaffectwhetherornotMarycallsP(MaryCall|Alarm,JohnCall)=P(MaryCall|Alarm)WesayJohnCallandMaryCallareconditionallyindependentgivenAlarmIngeneral,““AandBareconditionallyindependentgivenC””means:P(A|B,C)=P(A|C)P(B|A,C)=P(B|C)P(A,B|C)=P(A|C)P(B|C)條件件獨獨立立性性P(Toothache,Cavity,Catch)has23-1=7independententries專業(yè)業(yè)領(lǐng)領(lǐng)域域知知識識:Cavitydirectlycausestoothacheandprobe-catches.IfIhaveacavity,theprobabilitythattheprobecatchesinitdoesn‘‘tdependonwhetherIhaveatoothache:(1)P(catch|toothache,cavity)=P(catch|cavity)ThesameindependenceholdsifIhaven’’tgotacavity:(2)P(catch|toothache,?cavity)=P(catch|??cavity)CatchisconditionallyindependentofToothachegivenCavity:P(Catch|Toothache,Cavity)=P(Catch|Cavity)Equivalentstatements:P(Toothache|Catch,Cavity)=P(Toothache|Cavity)P(Toothache,Catch|Cavity)=P(Toothache|Cavity)P(Catch|Cavity)條件件獨獨立立性性Writeoutfulljointdistributionusingchainrule:P(Toothache,Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch|Cavity)P(Cavity)=P(Toothache|Cavity)P(Catch|Cavity)P(Cavity)I.e.,2+2+1=5independentnumbersInmostcases,theuseofconditionalindependencereducesthesizeoftherepresentationofthejointdistributionfromexponentialinntolinearinn.在大多多數(shù)情情況下下，使使用條條件獨獨立性性能將將全聯(lián)聯(lián)合概概率的的表示示由n的指數(shù)數(shù)關(guān)系系減為為n的線性性關(guān)系系。Conditionalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainenvironments.Bayes’Rule（貝葉葉斯法法則））Bayes’Rule（貝葉葉斯法法則））乘法原原則?Bayes‘rule:orindistributionform為什么么該法法則非非常有有用?將條件件倒轉(zhuǎn)轉(zhuǎn)通常一一個條條件是是復(fù)雜雜的，，一個個是簡簡單的的許多系系統(tǒng)的的基礎(chǔ)礎(chǔ)(e.g.語音識識別)現(xiàn)代AI基礎(chǔ)??！Bayes’Rule（貝葉葉斯法法則））Usefulforassessingdiagnosticprobability（診斷概率））fromcausalprobability（因果概率））:E.g.,letMbemeningitis（腦膜膜炎））,Sbestiffneck（脖子子僵硬硬）:Note:腦膜炎炎的后后驗概概率依依然非非常小小!Note:依然要要先檢檢測脖脖子僵僵硬!Why?Bayes’RuleinPractice使用貝貝葉斯斯法則則:IH=““havingaheadache““頭痛F=““comingdownwithFlu”流感P(H)=1/10P(F)=1/40P(H|F)=1/2有一天天你早早上醒醒來發(fā)發(fā)現(xiàn)頭頭很痛痛，于于是得得到以以下結(jié)結(jié)論：：“因因為得得了流流感以以后50%的幾率率會引引起頭頭痛，，所以以我有有50%的幾率率得了了流感感”Isthisreasoningcorrect?使用貝貝葉斯斯法則則:IH="havingaheadache““F="comingdownwithFlu"P(H)=1/10P(F)=1/40P(H|F)=1/2TheProblem:P(F|H)=?使用貝貝葉斯斯法則則:IH="havingaheadache““F="comingdownwithFlu"P(H)=1/10P(F)=1/40P(H|F)=1/2TheProblem:P(F|H)=P(H|F)P(F)/P(H)=1/8≠≠P(H|F)使用貝貝葉斯斯法則則:II在一個個包裹裹里有有2個信封封一個信信封里里有一一個紅紅球(worth$100)和一個個黑球球另一個個信封封里有有2個黑球球.黑球一一文不不值然后你你隨機機拿出出一個個信封封，并并隨機機拿出出一個個球–it’’sblack此時此此刻給給你個個機會會換一一個信信封.是換呢呢還是是換呢呢還是是換呢呢?使用貝貝葉斯斯法則則:IIE:envelope,1=(R,B),2=(B,B)B:theeventofdrawingablackballP(E|B)=P(B|E)*P(E)/P(B)WewanttocompareP(E=1|B)vs.P(E=2|B)P(B|E=1)=0.5,P(B|E=2)=1P(E=1)=P(E=2)=0.5P(B)=P(B|E=1)P(E=1)+P(B|E=2)P(E=2)=(.5)(.5)+(1)(.5)=.75P(E=1|B)=P(B|E=1)P(E=1)/P(B)=(.5)(.5)/(.75)=1/3P(E=2|B)=P(B|E=2)P(E=2)/P(B)=(1)(.5)/(.75)=2/3因此在在已發(fā)發(fā)現(xiàn)一一個黑黑球后后,該信封封是1的后驗驗概率率(thusworth$100)比信封封是2的后驗驗概率率低所以還還是換換吧課堂測測驗一名醫(yī)醫(yī)生做做了一一個具具有99%可靠性性的測測試：：也就就是說說,99%的病人人其檢檢測呈呈陽性性,99%的健康康人士士檢測測呈陰陰性.該醫(yī)生生估計計1%的人類類病了了。。。。Question:一個患患者檢檢測呈呈陽性性.該患者者得病病的幾幾率是是多少少?0-25%,25-75%,75-95%,or95-100%?課堂測測驗Adoctorperformsatestthathas99%reliability,i.e.,99%ofpeoplewhoaresicktestpositive,and99%ofpeoplewhoarehealthytestnegative.Thedoctorestimatesthat1%ofthepopulationissick.Question:Apatienttestspositive.Whatisthechancethatthepatientissick?0-25%,25-75%,75-95%,or95-100%?Intuitiveanswer:99%;Correctanswer:50%Bayes’rulewith多重證證據(jù)和和條件件獨立立性P(Cavity|toothache∧catch)=αP(toothache∧catch|Cavity)P(Cavity)=αP(toothache|Cavity)P(catch|Cavity)P(Cavity)Thisisanexampleofana??veBayesmodel（樸素素貝葉葉斯模模型））:Totalnumberofparameters（參數(shù)數(shù)）islinearinn鏈?zhǔn)椒ǚ▌t全聯(lián)合合分布布using鏈?zhǔn)椒ǚ▌t:P(Toothache,Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch|Cavity)P(Cavity)=P(Toothache|Cavity)P(