圖像檢索中相關(guān)反饋的半監(jiān)督主動(dòng)學(xué)習(xí)研究的中期報(bào)告

圖像檢索中相關(guān)反饋的半監(jiān)督主動(dòng)學(xué)習(xí)研究的中期報(bào)告AbstractThisreportpresentstheprogressofourresearchonsemi-supervisedactivelearningwithrelevantfeedbackforimageretrieval.Imageretrievalhasbecomeoneofthemostimportantapplicationsinthefieldofcomputervisionduetotheincreasingnumberofimagesavailableinvariousdomains.However,theperformanceofimageretrievalalgorithmsheavilyreliesonthequalityoftheannotatedtrainingdatasincemanualannotationisatime-consumingandlabor-intensivetask.Toaddressthisissue,activelearninghasbeenproposedtoautomaticallyselectthemostinformativesamplesforannotation.Meanwhile,relevantfeedbackfromuserscanbeincorporatedintoactivelearningtofurtherenhancetheperformanceofimageretrievalsystems.Inthisreport,wereviewrelatedworkonsemi-supervisedactivelearningandrelevantfeedback,anddiscussourproposedmethodsandpreliminaryexperimentalresults.IntroductionImageretrievalhasbeenwidelyusedinmanyfields,suchase-commerce,socialmedia,andvideosurveillance.However,thequalityofimageretrievalalgorithmsreliesheavilyontheavailabilityandqualityofannotatedtrainingdata.Manualannotationisatime-consumingandlabor-intensivetask,whichmakesitdifficulttoscaleuptolargedatasets.Inaddition,theannotationqualitymaybeinconsistentduetodifferentannotatorsorsubjectivecriteria.Toaddresstheseissues,activelearninghasbeenproposedtoautomaticallyselectthemostinformativesamplesforannotation.Activelearningisaniterativeprocessthatstartswithasmalllabeleddatasetandgraduallyexpandsittoreducetheannotationcostwhilemaintainingorevenimprovingtheperformanceofthemodel.Thekeyideaistoselectthesamplesthataremostinformativeforthemodel'straining,i.e.,thesamplesthatthemodelismostuncertainorconfidentbutwrongabout.Therearevariousselectioncriteriaforactivelearning,suchasuncertaintysampling,diversity,density,andrepresentativeness.However,activelearningonlyconsiderstheintrinsicpropertiesofthesamplesandignorestheadditionalinformationthatexternalsourcescouldprovide.Relevantfeedbackfromusers,suchasrelevancefeedback,preferencefeedback,orsimilarityfeedback,canbeveryvaluableforimageretrievalsinceitcapturestheusers'subjectivepreferencesandinterests.Incorporatingrelevantfeedbackintoactivelearningcouldfurtherimprovetheimageretrievalperformanceandreducetheannotationcost.Therefore,thecombinationofactivelearningandrelevantfeedbackhasattractedconsiderableattentioninrecentyears.Inthisreport,wereviewrelatedworkonsemi-supervisedactivelearningandrelevantfeedbackforimageretrievalanddiscussourproposedmethodsandpreliminaryexperimentalresults.RelatedWorkActivelearninghasbeenwidelystudiedinthefieldofmachinelearningandhasbeenappliedtovarioustasks,suchasclassification,regression,andclustering.Inactivelearning,themodelistrainedonasmalllabeleddatasetandthenselectsthemostinformativesamplesforannotation.Theselectioncriteriaaimtomaximizethegaininmodelperformancewithalimitedannotationbudget.Themostcommonselectioncriteriaareuncertaintysampling,diversitysampling,densitysampling,andrepresentativenesssampling.Uncertaintysamplingselectsthesamplesforwhichthemodel'spredictionismostuncertain,i.e.,thesampleswiththehighestentropyormargin.Diversitysamplingselectsthesamplesthataremostdissimilarfromtheexistinglabeledsamples,i.e.,thesampleswiththelowestdistanceormaximumdiversity.Densitysamplingselectsthesamplesinthesparseregionsofthefeaturespace,i.e.,thesampleswiththelowestdensityormaximumdensity.Representativenesssamplingselectsthesamplesthataremostrepresentativeoftheunderlyingdistributionofthedata,i.e.,thesampleswiththemaximumdisagreementorconsensus.Activelearninghasbeenappliedtoimageretrievalaswell.Themostcommonapproachistousethequery-by-committee(QBC)method,whichtrainsmultipleclassifiersontheavailablelabeleddataandselectsthesamplesforwhichtheclassifiersdisagreethemost.However,QBCmaysufferfromthediversityproblem,wheretheclassifiersarebiasedtowardsacertainsubsetofthedataandcannotcapturetheentireunderlyingdistribution.Toaddressthisissue,somemethodshavebeenproposedtoincorporatediversityorrepresentativenessintoactivelearningforimageretrieval.Relevantfeedbackhasalsobeenwidelystudiedinthefieldofinformationretrievalandhasbeenappliedtovarioustasks,suchaswebsearch,recommendation,andimageretrieval.Relevantfeedbackcapturestheusers'subjectivepreferencesandinterestsandcanbeusedtorefinethesearchresultsorrecommendersystems.Themostcommontypesofrelevantfeedbackarerelevancefeedback,preferencefeedback,andsimilarityfeedback.Relevancefeedbackaskstheuserstoprovidefeedbackontherelevanceoftheretrievedresults,e.g.,bylabelingthemasrelevantorirrelevant.Preferencefeedbackaskstheuserstoprovidefeedbackonthepreferencesbetweenpairsofitems,e.g.,byindicatingwhichitemismorepreferred.Similarityfeedbackaskstheuserstoprovidefeedbackonthesimilaritybetweenpairsofitems,e.g.,byindicatingwhichpairismoresimilar.Relevantfeedbackhasbeenappliedtoimageretrievalaswell.ThemostcommonapproachistousetheRocchioalgorithm,whichupdatesthequerybasedontherelevantandirrelevantfeedbackandre-rankstheretrievedresults.However,Rocchiomaysufferfromthecold-startproblem,wherethereisnorelevantfeedbackavailableforanewqueryorauser.Toaddressthisissue,somemethodshavebeenproposedtoincorporateactivelearningintorelevantfeedbackforimageretrieval.Semi-supervisedactivelearningwithrelevantfeedbackcombinesactivelearningandrelevantfeedbacktoenhancetheperformanceofimageretrieval.Thekeyideaistousethelabeleddataandtherelevantfeedbacktotrainthemodelandthenselectthemostinformativesamplesforannotation.Theselectioncriteriaaimtomaximizethegaininmodelperformancewithalimitedannotationbudget,giventheavailablerelevantfeedback.Themostcommonapproachesareco-training,multi-viewlearning,andquery-by-committeewithfeedback.Co-trainingtrainsmultipleclassifiersondifferentfeaturesorviewsandthenselectsthesamplesforwhichtheclassifiersdisagreethemost,giventheavailablerelevantfeedback.Multi-viewlearningcombinesmultiplefeaturesorviewsintoaunifiedrepresentationandthenselectsthesamplesthataremostinformativeforeachview,giventheavailablerelevantfeedback.Query-by-committeewithfeedbacktrainsmultipleclassifiersonthelabeleddataandtherelevantfeedbackandthenselectsthesamplesforwhichtheclassifiersdisagreethemost,giventheavailablerelevantfeedback.ProposedMethodsInourresearch,weproposeseveralmethodstoenhancetheperformanceofimageretrievalwithsemi-supervisedactivelearningandrelevantfeedback.Thefirstmethodistoincorporaterepresentationlearningintosemi-supervisedactivelearningandrelevantfeedback.Representationlearningaimstolearnalow-dimensionalrepresentationofthedatathatcapturestheunderlyingstructureandsimilarity.Weusedeepneuralnetworkstolearnsuchrepresentationsandthencombinethemwithactivelearningandrelevantfeedbacktoselectthemostinformativesamplesforannotation.Thesecondmethodistoincorporateclusteringintosemi-supervisedactivelearningandrelevantfeedback.Clusteringaimstogroupsimilarsamplesintoclustersandthenselectthemostrepresentativesamplesforeachcluster.Weusek-meansclusteringtogrouptheunlabeleddataintoclustersandselectthemostinformativesamplesfromeachcluster,giventheavailablerelevantfeedback.Thethirdmethodistoincorporatediversityintosemi-supervisedactivelearningandrelevantfeedback.Diversityaimstoselectthemostrepresentativesamplesthatcovertheentireunderlyingdistribution,ratherthanabiasedsubset.Weusethemaximummarginaldiversitycriteriontoselectthemostdiversesamples,giventheavailablerelevantfeedback.PreliminaryExperimentalResultsWeconductedpreliminaryexperimentsontheCIFAR-10datasettoevaluatetheproposedmethods.TheCIFAR-10datasetconsistsof60,00032x32colorimagesin10classes,with6,000imagesperclass.Werandomlyselected100imagesperclassastheinitiallabeleddataandusedtheremainingimagesastheunlabeleddata.Wecomparedtheproposedmethodswiththerandomsamplingbaseline,wherethesamplesforannotationarerandomlyselected.Theresultsshowthattheproposedmethodsoutperformedtheran