cs209搜索廣告平臺第一周lesson

A B C杜聿明 D陳數(shù)據(jù)結數(shù)據(jù)結SearchAds(搜索NativeAds(原生DisyAds(顯示SearchAds(搜索logic:match touser’sadsformat:text,adsposition:mainline,sidebar,top,bottomofsearch NativeAds(原生logic:match towebpageorAPP’sadsformat:text,image,styleshouldalsomatchcontextofwebpageorAPPadsposition:embeddedinoriginalcontentofwebpageorDisyAds(顯示 logic:matchuser’sdemographic,intereststoad’scategoryinterestscollectedfromuserbehavior:pagedwelltime,click,engagementadsformat:image,animation(gif)，,adsposition:sidebar,top,bottomofpageoryou’re475timesmorelikelytosurviveanecrashthanclickabanner互互聯(lián)What’sinCommonlogic:matchAdstosomething:user’squery,interests,contextofwebpageor互互聯(lián)Quiz:Whichonehashigherclickratenativeads>searchads>y數(shù)據(jù)結數(shù)據(jù)數(shù)據(jù)結數(shù)據(jù)數(shù)據(jù)結n●●●●Anfocusesonathemeoragroupofsetachooseyouraudiencewriteyourad,ad數(shù)據(jù)結數(shù)據(jù)結數(shù)據(jù)結given603click,16Kimpression,what’stheclick$300budgetdaily,$1.4perclickwhydailypotentialclicksis

數(shù)據(jù)結

數(shù)據(jù)結Queryremovestopwords:a,an,theremoveendings,queryintentpredictuser’sbuyharrypotterdvdonline->harrypotterneedqueryhistoryquerynikerunningshoe->nikerunningsoftwaredeveloper->softwareengineerresult:alistofquerySelectsendqueryunderstandresulttoindexandselectasmuchcandidatesasapplyinformationretrievalalgorithmonindexhowtomakeitscalablewhatifQPS>10K Rankgivenadcandidates,howtocalculaterankagoodrankscoreshould relevancebetweenqueryand click bid howtochargeisitfairtochargethembybidcostperclickincreaseisgoodnot 數(shù)據(jù)結Webcrawlingisaboutmakingrequestsandextractingdatafromtheresponsestep1httpstep2step3extractdatafromHTMLParseHTML<style><script><div><div>搜索平臺實attributename:attributevalue 搜索平臺實搜索平臺實戰(zhàn)搜索平臺實WheretostartNetworkI/OistheAvoidBotCrawlerneededtobeWheretostartNetworkI/OistheAvoidBotCrawlerneededtobeWheretostartcrawlingstartwithfeedsfilealistofwebsiteurlrequesturlwithdifferentQueamplencornerleatherwhiteleather web

crawledNetworkI/Oistheifasitetakes2-3secondstorespond,thenyou’relookingatmaking20-30requestsaminutewasteCPUwhenwaitingforresponsefrommakeyourcrawlermulti-AvoidBotWhatifwebsite blockyourIPQuiz:Whatstatus5035xxServer503ServiceSpoofHttpHeaders:UserAgent,Accept,Accept-Service:RotateIPsusingalistofover100+CrawlerneededtobeWhatifcrawlercrashduetouncaughtHowtohandleurlfailedtocrawlHowtoavoidcrawlsameurllogcrawledlogfailedlog數(shù)據(jù)結DeviceDeviceSessionAdQuery配置開發(fā)環(huán)Java，Python,JavaIDE(InlijorEclipse),MemCached(optional)，MySQL(optional),Spark(optional)參見預習資準備feeds文件，發(fā)sofa,12.4,deadline:12/31/20164PMPacific課程項目郵：

，獲得太閣信 搜 ●●●配置開發(fā)環(huán)Java，Python,JavaIDE(InlijorJavalib:Jsoup1.10.1，jacksoncrawldedupegenerateclicklogforqueryintentpredictionandclickreverseDeviceDeviceSessionAdQueryHowtodevelopfacerecognitionalgorithmwithfacewhatifwedon’thavefaceimageatWealsoneednegativePositiveQueryCategry_AdsCategoryQuery_nIdQuery_AdIdNegativemismatchedquery_categorymismatched mismatched nIdweight,lowestadIdweightperqueryStep1:Segmentusersto410000IP,5deviceidperIP,user=level0:5%ofuserwhoclickforeachlevel1:5%ofIPwhose1st2deviceidclickforeachquery,70%ofquery,restof30%querynoclick

rest3device_idclicklevel2:40%ofIPfromwherewerandomselect1device_idclickfor50%ofquery,restof50%querynoclick,restof4device_idnoclicklevel3:50%neverStep2:assign nID,AdId,Click(0/1),Adcategory_Querycategory(0/1)touserassignweight nID,AdgroupaddatabyQuerygroupngroup1corner1leather1whiteleather1officecomputeradjustableofficehomeofficemetalofficenAdHowtocalculateweightforeach Foreachad n,calculaterelevancebetweenadandquery,=relevance_i/sum(relevance_i),iisindexofadfor Foreach nperquerygroup, n_weight_j=sumofadrelevancein n/total_relevance_cross_ ngroupId1corner1leather1whiteleather1officecomputeradjustableofficehomeofficemetalofficenAdHowtoassign nID,AdIdtoWeightedRandomGivennitem,eachonehasweightw[i],theseweightsformtheunnormalizedprobabilitydistributionwewanttosamplefrom,each