問(wèn)題為什么現(xiàn)在的計(jì)算機(jī)處理智能信息效率很低

Prof.LiqingZhangDept.ComputerScience&Engineering,ShanghaiJiaotongUniversityStatisticalLearning

&Inference第一頁(yè)，共二十五頁(yè)。BooksandReferencesTrevorHastie

RobertTibshirani

JeromeFriedman,TheElementsofstatisticalLearning:DataMining,Inference,andPrediction,2001,Springer-VerlagVladimirN.Vapnik,TheNatureofStatisticalLearningTheory,2nded.,Springer,2000S.Mendelson,AfewnotesonStatisticalLearningTheoryinAdvancedLecturesinMachineLearning:MachineLearningSummerSchool2002,S.MendelsonandA.J.Smola(eds),LectureNotesinComputerScience,2600,Springer,2003M.Vidyasagar,Learningandgeneralization:withapplicationstoneuralnetworks,2nded.,Springer,2003第二頁(yè)，共二十五頁(yè)。2023/5/62StatisticalLearningandInferenceOverviewoftheCourseIntroductionOverviewofSupervisedLearningLinearMethodforRegressionandClassificationBasisExpansionsandRegularizationKernelMethodsModelSelectionsandInferenceSupportVectorMachineBayesianInferenceUnsupervisedLearning第三頁(yè)，共二十五頁(yè)。2023/5/63StatisticalLearningandInferenceWhyStatisticalLearning?我門(mén)被信息淹沒(méi)，但卻缺乏知識(shí)。----R.Roger恬靜的統(tǒng)計(jì)學(xué)家改變了我們的世界；不是通過(guò)發(fā)現(xiàn)新的事實(shí)或者開(kāi)發(fā)新技術(shù)，而是通過(guò)改變我們的推理、實(shí)驗(yàn)和觀點(diǎn)的形成方式。----I.Hacking問(wèn)題：為什么現(xiàn)在的計(jì)算機(jī)處理智能信息效率很低？圖像、視頻、音頻認(rèn)知語(yǔ)言第四頁(yè)，共二十五頁(yè)。2023/5/64StatisticalLearningandInferenceML:SARS

RiskPredictionSARSRiskAgeGenderBloodPressureChestX-RayPre-HospitalAttributesAlbuminBloodpO2WhiteCountRBCCountIn-HospitalAttributes第五頁(yè)，共二十五頁(yè)。2023/5/65StatisticalLearningandInferenceML:AutoVehicleNavigationSteeringDirection第六頁(yè)，共二十五頁(yè)。2023/5/66StatisticalLearningandInferenceProteinFolding第七頁(yè)，共二十五頁(yè)。2023/5/67StatisticalLearningandInferenceTheScaleofBiomedicalData第八頁(yè)，共二十五頁(yè)。2023/5/68StatisticalLearningandInference計(jì)算科學(xué)與腦科學(xué)計(jì)算機(jī)信息處理基于邏輯的計(jì)算ＣＰＵ和數(shù)據(jù)分離數(shù)據(jù)處理與存儲(chǔ)簡(jiǎn)單智能信息處理復(fù)雜、慢認(rèn)知能力弱信息處理模式：邏輯－概念－統(tǒng)計(jì)信息大腦信息處理基于統(tǒng)計(jì)信息的計(jì)算計(jì)算和數(shù)據(jù)集成一體數(shù)據(jù)處理與存儲(chǔ)未知智能信息處理簡(jiǎn)單、快速認(rèn)知能力強(qiáng)信息處理模式：統(tǒng)計(jì)信息－概念－邏輯第九頁(yè)，共二十五頁(yè)。2023/5/69StatisticalLearningandInferenceFunctionEstimationModelTheFunctionEstimationModeloflearningexamples:Generator(G)generatesobservationsx(typicallyinRn),independentlydrawnfromsomefixeddistributionF(x)Supervisor(S)labelseachinputxwithanoutputvalueyaccordingtosomefixeddistributionF(y|x)LearningMachine(LM)“l(fā)earns”fromani.i.d.l-sampleof(x,y)-pairsoutputfromGandS,bychoosingafunctionthatbestapproximatesSfromaparameterisedfunctionclassf(x,),whereisintheparameterset第十頁(yè)，共二十五頁(yè)。2023/5/610StatisticalLearningandInferenceFunctionEstimationModelKeyconcepts:

F(x,y),ani.i.d.k-sampleonF,functionsf(x,)andtheequivalentrepresentationofeachfusingitsindexxGSLMyy^第十一頁(yè)，共二十五頁(yè)。2023/5/611StatisticalLearningandInferenceThelossfunctional(L,Q)theerrorofagivenfunctiononagivenexampleThe

riskfunctional(R)theexpectedlossofagivenfunctiononanexampledrawnfromF(x,y)the(usualconceptof)generalisationerrorofagivenfunction

TheProblemofRiskMinimization第十二頁(yè)，共二十五頁(yè)。2023/5/612StatisticalLearningandInferenceTheProblemofRiskMinimizationThreeMainLearningProblemsPatternRecognition:RegressionEstimation:DensityEstimation:第十三頁(yè)，共二十五頁(yè)。2023/5/613StatisticalLearningandInferenceGeneralFormulationTheGoalofLearningGivenani.i.d.k-samplez1,…,zkdrawnfromafixeddistributionF(z)Forafunctionclass’lossfunctionalsQ(z,),within

Wewishtominimisetherisk,findingafunction*第十四頁(yè)，共二十五頁(yè)。2023/5/614StatisticalLearningandInferenceGeneralFormulationTheEmpiricalRiskMinimization(ERM)InductivePrincipleDefinetheempiricalrisk(sample/trainingerror):Definetheempiricalriskminimiser:ERMapproximatesQ(z,*)withQ(z,k)theRempminimiser…thatisERMapproximates*withkLeast-squaresandMaximum-likelihoodarerealisationsofERM第十五頁(yè)，共二十五頁(yè)。2023/5/615StatisticalLearningandInference4IssuesofLearningTheoryTheoryofconsistencyoflearningprocessesWhatare(necessaryandsufficient)conditionsforconsistency(convergenceofRemptoR)ofalearningprocessbasedontheERMPrinciple?Non-asymptotictheoryoftherateofconvergenceoflearningprocessesHowfastistherateofconvergenceofalearningprocess?GeneralizationabilityoflearningprocessesHowcanonecontroltherateofconvergence(thegeneralizationability)ofalearningprocess?Constructinglearningalgorithms(i.e.theSVM)Howcanoneconstructalgorithmsthatcancontrolthegeneralizationability?第十六頁(yè)，共二十五頁(yè)。2023/5/616StatisticalLearningandInferenceChangeinScientificMethodologyTRADITIONALFormulatehypothesisDesignexperimentCollectdataAnalyzeresultsReviewhypothesisRepeat/PublishNEWDesignlargeexperimentsCollectlargedataPutdatainlargedatabaseFormulatehypothesisEvaluatehypothesisondatabaseRunlimitedexperimentsReviewhypothesisRepeat/Publish第十七頁(yè)，共二十五頁(yè)。2023/5/617StatisticalLearningandInferenceLearning&AdaptationInthebroadestsense,anymethodthatincorporatesinformationfromtrainingsamplesinthedesignofaclassifieremployslearning.Duetocomplexityofclassificationproblems,wecannotguessthebestclassificationdecisionaheadoftime,weneedtolearnit.Creatingclassifierstheninvolvespositingsomegeneralformofmodel,orformoftheclassifier,andusingexamplestolearnthecompleteclassifier.第十八頁(yè)，共二十五頁(yè)。2023/5/618StatisticalLearningandInferenceSupervisedlearningInsupervisedlearning,ateacherprovidesacategorylabelforeachpatterninatrainingset.Thesearethenusedtotrainaclassifierwhichcanthereaftersolvesimilarclassificationproblemsbyitself.第十九頁(yè)，共二十五頁(yè)。2023/5/619StatisticalLearningandInferenceUnsupervisedlearningInunsupervisedlearning,orclustering,thereisnoexplicitteacherortrainingdata.Thesystemformsnaturalclustersofinputpatternsandclassifiersthembasedonclusterstheybelongto.第二十頁(yè)，共二十五頁(yè)。2023/5/620StatisticalLearningandInferenceReinforcementlearningInreinforcementlearning,ateacheronlysaystoclassifierwhetheritisrightwhensuggestingacategoryforapattern.Theteacherdoesnottellwhatthecorrectcategoryis.第二十一頁(yè)，共二十五頁(yè)。2023/5/621StatisticalLearningandInferenceClassificationThetaskoftheclassifiercomponentistousethefeaturevectorprovidedbythefeatureextractortoassigntheobjecttoacategory.Classificationisthemaintopicofthiscourse.Theabstractionprovidedbythefeaturevectorrepresentationoftheinputdataenablesthedevelopmentofalargelydomain-independenttheoryofclassification.Essentiallytheclassifierdividesthefeaturespaceintoregionscorrespondingtodifferentcategories.第二十二頁(yè)，共二十五頁(yè)。2023/5/622StatisticalLearningandInferenceClassificationThedegreeofdifficultyoftheclassificationproblemdependsonthevariabilityinthefeaturevaluesforobjectsinthesamecategoryrelativetothefeaturevaluevariationbetweenthecategories.Variabilityisnaturalorisduetonoise.Variabilitycan