18python協(xié)程編程18 4使用Gevent完成多任務(wù)圖片

第十八章Python本節(jié)所講內(nèi)容Greenlet使用Gevent協(xié)程的概defyield簡單理解：協(xié)程是也是實(shí)現(xiàn)多任務(wù)的式必須自己承擔(dān)調(diào)度的責(zé)任，同時(shí)，協(xié)程也失去了標(biāo)準(zhǔn)線程使用多CPU的能力）；234CPU1、無法利用多核資源：協(xié)程的本質(zhì)是個(gè)單線程,它不能同時(shí)將單個(gè)CPU的多個(gè)核用上,協(xié)程需要和進(jìn)程配合才能運(yùn)行在多CPU上.cpu密集型應(yīng)用；yielddefwhiledefwork2():whiledefw1=work1()w2=work2()whileTrue:ifname=="main":Greenlet模使用如下命令安裝greenlet:pip3installimporttimeimportgreenletdefforiinrange(5):#切換到協(xié)程2里面執(zhí)行對應(yīng)的任#任務(wù)defforiinrange(5):#切換到第一個(gè)協(xié)程執(zhí)行對應(yīng)的任ifname=='main#創(chuàng)建協(xié)程指定的對應(yīng)任g1=greenlet.greenlet(work1)g2=greenlet.greenlet(work2)switchsw?t?]:轉(zhuǎn)換Gevent的使GreenletGreenlet切換任務(wù)的第庫，那就是Gevent。輸入輸出，由于IO操作非常耗時(shí)，經(jīng)常使程序處于等待狀態(tài)，有了GeventGreenletIO。安裝：pip3installgevent1.geventimportgeventdefwork(n):foriin#獲取當(dāng)前協(xié)print(gevent.getcurrent(),i)g1=gevent.spawn(work,5)g2=gevent.spawn(work,5)g3=gevent.spawn(work,5)<Greenletat0x10e49f550:f(5)><Greenletat0x10e49f550:f(5)><Greenletat0x10e49f550:f(5)><Greenletat0x10e49f550:f(5)><Greenletat0x10e49f550:f(5)><Greenletat0x10e49f910:f(5)>可以看到，3個(gè)greenletspawn[sp??n]：產(chǎn)卵;;引起;導(dǎo)致;造geventimportgeventdefwork(n):foriin#獲取當(dāng)前協(xié)程print(gevent.getcurrent().namei)g1=gevent.spawn(work,5)g2=gevent.spawn(work,5)g3=gevent.spawn(work,5)Greenlet-0Greenlet-1Greenlet-2Greenlet-0Greenlet-1Greenlet-2Greenlet-0Greenlet-1Greenlet-2Greenlet-0Greenlet-1Greenlet-2Greenlet-0Greenlet-1Greenlet-2spawn[sp??n]：產(chǎn)卵;;引起;導(dǎo)致;造18.3.3給程序打補(bǔ)丁importgeventimporttimefromgeventimport#打補(bǔ)丁，讓gevent框架識別耗時(shí)操作，比如：time.sleep，網(wǎng)絡(luò)請求延#任務(wù)defforiinrange(num): gevent.sleep(0.2)#任務(wù)1defforiinrange(num): #ifname=='main#創(chuàng)建協(xié)程指定對應(yīng)的任g1=gevent.spawn(work1,3)g2=gevent.spawn(work2,#主線程等待協(xié)程執(zhí)行完成以后程序再退18.3.4注意當(dāng)前程序是一個(gè)死循環(huán)并且還能有耗時(shí)操作，就不需要加上join方法了,因?yàn)槌绦蛐枰恢边\(yùn)行不importgeventimporttimefromgeventimport#打補(bǔ)丁，讓gevent框架識別耗時(shí)操作，比如：time.sleep，網(wǎng)絡(luò)請求延#任務(wù)defforiinrange(num): gevent.sleep(0.2)#任務(wù)1defforiinrange(num): #ifname=='main#創(chuàng)建協(xié)程指定對應(yīng)的任g1=gevent.spawn(work1,3)g2=gevent.spawn(work2,3)whileTrue:主線執(zhí)主線執(zhí)主線執(zhí)主線執(zhí) 使用Gevent完成多任多任務(wù)fromgeventimport#打補(bǔ)?。鹤実event使用網(wǎng)絡(luò)請求的耗時(shí)操作，讓協(xié)程自動切換執(zhí)行對應(yīng)的任importgeventimporturllib.request#網(wǎng)絡(luò)請求模#根據(jù)地址對應(yīng)defdownload_img(img_url,img_name):#根據(jù)地址打開網(wǎng)絡(luò)資源數(shù)response=#創(chuàng)建文件把數(shù)據(jù)寫入到指定文件里withopen(img_name,"wb")asimg_file:whileTrue:#網(wǎng)絡(luò)數(shù)img_data=response.read(1024)ifimg_data:#把數(shù)據(jù)寫入到指定文件里exceptExceptionasprint("異常:",e)print("成功:%s"%img_name)ifname=='main':#準(zhǔn)備地img_url1= /bdr/576img_url2= /bdr/594 img_url3= /bdr/864#創(chuàng)建協(xié)程指派對應(yīng)的任g1=gevent.spawn(download_img,img_url1,g2gevent.spawn(download_imgimg_url2,2.jpg")g3gevent.spawn(download_imgimg_url3,3.jpg")線程等待所有的協(xié)程執(zhí)行完成以后程序再退出gevent.joinall([g1,g2,g3]) /bdr/576 /bdr/594 /bdr/5942.jpg1.jpg從上面的結(jié)果可以看到依次根據(jù)地址去，但是收到數(shù)據(jù)的先后順序不一定與發(fā)送順序相同，fromgeventimportmonkeyimportgevent,requests#pipinstallfromurllibimport#def#開print('start#的url地url= #模擬瀏覽'Referer': 'User-Agent':'Mozilla/5.0(WindowsNT10.0;Win64;x64)AppleWebKit/537.36(KHTML,likeGecko)Chrome/71.0.3578.80Safari/537.36'}#模擬瀏覽器服務(wù)器發(fā)送內(nèi)str_data='''h:beautysn:30listtype:newtemp:1'''send_datafordatainline_data=data.split(':')#返回一個(gè)列iflen(line_data)==2:#如果這個(gè)列表當(dāng)中有兩個(gè)數(shù)據(jù)key,value=line_data#a,b=[1,2]#進(jìn)行序列解包賦值！ifkeyandvalue:#如果兩者keyvalue都有值我就進(jìn)行搭建send_data[key]=#end_data=send_data['sn']=eval(str(num)+'*'+'30')#eval（'3*4'）#z字典的修改#requests這個(gè)方法進(jìn)行網(wǎng)求，模擬劉燃?xì)夥?wù)器返回結(jié)果response=requests.get(url,headers=headers,params=send_data)#json()方法轉(zhuǎn)換為python操作對象{‘a(chǎn)";1}json_data=#利用么酷的方法，序列解包賦forindex,srcin#獲取url地image_url=src['qhimg_url']#給定本地地image_name='./360_image/'+image_url[-#吧網(wǎng)的到本exceptExceptionase:#format格式print('{}isdo