虛擬化與云計算課程實驗報告——Hadoop平臺搭建

1、虛擬化與云計算實驗報告目錄一、實驗?zāi)繕?二、實驗內(nèi)容1三、實驗步驟1四、實驗遇到的問題及其解決方法24五、實驗結(jié)論25一、實驗?zāi)康? .實驗題目：配置和使用SAN存儲掌握在Linux上配置iSCSItarget服務(wù)的方法。2 .實驗題目：Hadoop&MapReduce安裝、部署、使用Hadoop-HDFS配置運行MapReduce程序，使用MapReduce編程二、實驗內(nèi)容1 .實驗題目：配置和使用SAN存儲配置在Linux上iSCSI實現(xiàn)兩臺機器間的共享存儲。2 .實驗題目：Hadoop&MapReduce1 .掌握在集群上（使用虛擬機模擬）安裝部署Hadoop-HDFS的

2、方法。2 .掌握在HDFS運行MapReduce任務(wù)的方法。3 .理解MapReduce編程模型的原理，初步使用MapReduce模型編程。三、實驗步驟及實驗結(jié)果1 .實驗題目：配置和使用SAN存儲在實驗1中我作為主機提供共享存儲空間，實驗地點是在機房，但是由于我當(dāng)時沒有截圖所以回寢室在自己的電腦上重做，以下為主機步驟：1.1 確定以root身份執(zhí)行以下步驟sudosu一1.2 安裝iSCSITarget軟件款件中心-IB主文片為圖片Z.DGBI一R口工依4計埠機壬箕dj事過i?S«iiTK、晉而©eALibs加戒悍甲心常有乾樣IfifiltHB3cRas»文忤K

3、Sttl席培出海苴J%.武$11磯1林新科eR由就1酬叩11寸QlOptrrH£9-Utt5gd£5£lEntarprluTdrgtkATMlfnodultuturtdGQjisa*ptt-HQ*xffaiS£EIEntdrprlbtTirgtkKtrUfKltook>53t*petHghpcrronnMKK,tramporl:dependiSCElInplcmentatlaraOpen-f«3.i&CElEnterprlukcrrulfnoddesaur-cedhmc乜出31的.bfUMSCSItJretuu-f>ipjc

4、etiXklsG%tBt.moduUr,tvEnidnwnInltrimr&InrrAstructure»networkfu叩口rtBdrjruil-nftwvfrSbor-tgeNamingSeivlce皿A，ArittfnetS£»r-kgeNjmangEctyIcc-dt-cllentHBEKtClustefL.VMDiernonifcirLvTnZ1.3 修改/etc/default/iscsitargetISCSITARGET_ENABLE=true03F*1則14&4-魚4KeiKirgvi:(/tit/dtfAvlE.-4tdlc克忤

5、IH璃即口&索國I工Jt文港|時福助MFFT開HJM慢件3*HM.知racsitaretttISCSITARGETEMABLE«trut*K+-t-RirW24»A 創(chuàng)建共享存儲共享存儲可以是logicalvolumes,imagefiles,harddrives,harddrivepartitionsorRAIDdevices例如使用imagefile的方法，創(chuàng)建一個10G大小的LUN：ddif=/dev/zeroof=/storage/lun1.imgbs=1024kcount=10240 修改/etc/iet/ietd.conf添加：Targetiqn.200

6、1-.example:storage.lun1IncomingUserusernamepasswordOutgoingUserLun0Path=/storage/lun1.img,Type=fileioAliasLUN1文件M殖量TACn克科忸*助4打開Ml保存fl*ma，*看0-£Md.confMftMdxOutstandtfigRliriflUjtaP1Dl>nOrder7es題己心Mqu電網(wǎng)葡nOr*?fres喜ErurIRiK口vcftLeNa-#He-Bderl>iLgeitSDdtaDtgestM盯$-31峭&#i4C4PIntfrvalfflCrTl

7、nwuEItone.CRCazCrtone.CRCJZC£tillR，hujpuiL年MaxoutstandingRZTspercmdDat«iniPDUsIsD«l&rEdHeH如齡的rt6*dercd*pflutIn調(diào)*x-dered餐Ha.Mycupprt3d百年Heonlysupportleve-l0PDUheaderchecksw4I90listMomorCHC3ZC»ifonlyonet&setthen學(xué)電¥nttt4trnmtag曹.taItortheconnectionMtItfAtlPDUd-atftcheck

8、sunnig口list4Sanea，Wv。fnaxiRsj*flufiber53»thtt-5號SendaInptngc-jch*rt»rthatnarvysecondsIftheconn<sotberwiiEIdle-0-off“Hdteth4cnd自*lec4ndsftrt-0»EM軸”0*1«hop-InpigIfaar>MCWTn.-tifVALprHWInttrv4lIsused!.鼻V-BrlctFitargetper-vwtersikMt解”也H#Nurtbar單r10th，$AueuedCawiands32掌Numberorq

9、ueiiedccmandxTBrgctiqn.2001-04-hxi（上七展此stor-age.Lun3irwwtuserhzbl兄m如*索黑身窗受證將用戶名配碼加在此行WT21時“4TLun4Patfiv/stcr-age/'Lun3,tny,Tyw）e-"filetnAHaeLUN1liHaM.CDHne'ctkuns0#MaxConnections61.6修改/etc/iet/initiators.allow如果只允許特定IP的initiator訪問LUN,則如下設(shè)置iqn.2001-.example:storage.lun100如果任意i

10、nitiator均可以訪問，則：ALLALL力wMfrwgtqii坤-g+dli!文性in植2置M也素聞TJtCD宜朗時*助<打開/保帝凸*./m電e.R組Ifr*1ii)140*A"ImpQ.inrtrjtors.allowttflTheSiEHanticsarei*Withoutanyentriesn。tnltlatorc-ansee。connectt。日mytarget1.Ft*ey&+r劑it聘制18alltnttteterstecenecttoalltargets“thdth»"LLALL*ntT,.aQClwrrd*vhichIndtla

11、tcirscan-connecttowhichtargets-byputtinganefttfyTbra1*41>T-tori>eTby由iirnft*iep4fdt-Adlist0rtIntttatorSjeitherlipaddressmIQHm昭卓(UasU電裝ferTQKnantsir#Acc*pt«dl*av«aut*ar>d$f,pKlAlti-ttha-yarttnpllctt)flseparatethetwowithHhltesp4ce.fl11Eh'fcrypmceai-lngitoipaatthe"ALL"t

12、argetwitfyLfprearnt>,Lqni124S1-IH.conexanplr；3tD-rag«dta-kl$第sJ耨te192.145,-tl.0/lOB.*ihsciI-1,-.e-Bflple-V.c.on11MM2ifll-H.C4R.eaFVl*；stfrf白市?MIMISy&2,。七STfeiJoa;ll;li：2l;4)rr；r«l”白打dSTFef5H；2!；1：；FXr1K.10%22,w歲fUp,2M1-M,eflft-vxapvl*:tWagpi已屈i-s”3alldlqn.2BB384.con.icxapleisto-ragi

13、c.dlfkt,*>±4-Nyz(lqn.'西1、上.ve»u二.*."Q片flALLALLALL魏文才*!特出段：方frrir?J1»A1.6啟動/重啟動iSCSItarget/etc/init.d/iscsitargetstart/etc/init.d/iscsitargetrestart燧*ffilcs>th則lL0Jgp0尺件由配HBM商人址格式會格®ZAffl忸？電也也3,1dI/1里*7*%匕*aam匚=，、電iGinn2/3|nu|su>：hb：>|ha|wi|ddd33&a

14、mp;«|im|¥|HU，p|ttawawawftanaft丁|A且IWllISfil孱1匿J叵叵*,B3a,|，1.2.,如果看到sshdKia：hk|ma|附Iiddooj«|im那ssh-server還沒有啟動，需要/etc/init.d/sshstart那說明ssh-server已經(jīng)啟動了。2.實驗題目：Hadoop&MapReduce安裝JDK在實驗中安裝為OpenJDK6Runtime安裝openssh-server,命令為：sudoapt-getinstallopenssh-server并檢查sshserver是否已經(jīng)啟動：ps-e|grep

15、ssh,如果只有ssh-agent000000000000 .免密碼ssh設(shè)置，確認能否不輸入口令就用ssh登錄localhost:sshlocalhost如果不輸入口令就無法用ssh登陸localhost,執(zhí)行下面的命令：$ssh-keygen-trsa-f/.ssh/id_rsacat/.ssh/id_rsa.pub>>/.ssh/authorized_keys再次執(zhí)行sshlocalhost完成后請登入確認不用輸入密碼111111111111 .安裝Hadoop解壓Hadoop安裝包到安裝目錄，在本實驗中我新建了一個用戶hadoop并把hadoop解壓到home/hadoop

16、文件夾下，在偽分布模式下使用的版本為0.20.203,在分布模式下因為要與Eclipse兼容所以最終使用版本為0.20.2,下面部分截圖為版本為0.20.203。并編輯conf/hadoop-env.sh將JAVA_HOME設(shè)置為Java安裝根路徑運行運行bin目錄下的hadoop,如顯示hadoop腳本的使用文檔，說明安裝已經(jīng)完成。如下圖所示：AM占diz.心gkd&ap/h&dg尸1mL之時.好抽出口-VkM£9E，Ijj.而22:22£L收£>0*工HE密法I升修辭區(qū)4改8CICW)HHpKIHIT三口人/t通等，*；>喜Biia

17、dw-sclTlcmirwhT用獨I電e卜電第ThennlyreqpLredErru,r«iHntvanabLeisJAVAHOd.Jilletheriareclonal.*enrunnii中/dlsirlbuctdiconTlfluriticmIt1&tesrtofsetJAVAHOHEinthis-file-,sqthatitiscnrrectlvdefinedonmatenodesThejavainpleRntBlionguse.He?fired.eijMrtJAVAHOK-/usr/tfi!/jmb/java1.6.9-apenjdk以fExtraJby-bCLASS

18、PATHcleniits,mHo411aL了exportHMXMP=CLASSPA1H=-TheMximnwiunluf巾卬touminra.Def-BulEla1BN.it-exportHMXMPHEAPEIZE-2WAEilrnJB¥-arunE1monptimi.E«pW防deFault.XexportHMXMP_DPT5=-wrrt-rCowandspecificUpliBi%appendedgNADQQP的丹wticnipetifiedei網(wǎng)rtHADOOPIFEMDDE-Dea.sw.tsnaqtKnt.rente5HIE40PHWENOC-EOPTS-tapd

19、ftHA跳”一轉(zhuǎn)亡口的陋岫MEWUF0打谷曰電口r皿rw叫“nF.WWW飛E。口曲院MHE肥口FWiBrpfl.tIWPWFOAfAMNtDtM.iw*.Rinflaewttjre1W5tWW!PDATiWfOPTS*apartHADOOPBALAN£EE-Den.sm.HnaqEBent.kkrEHte5HIEODPBAUAZEF.OPTS-M|MftHADWJOflTIUCiCifti»T5-x-D£OB.ium.tnagnHr.t.jixrwteWDCiOP»IRA£KERDPT1BT祐JtIliefolltNinq'iippli

20、es-toiultiplecoiwandsdtfs,fsckPdiistcpetc).HpHtHAD«PELIEKTOPTSE»tras>hoptions.Enp»bydcf-ault.exportHAD*5£HOPTS*'a.CannectTaKiut-l2itfidEnv-HAfCOPCOHFDTft'ffMierelogfil«are.£HWOOFKWE/logifcrdefault.exportHWOCPLOGUg*HUMPHWEl/LagsFilenmngrewiteil-mre-hosts.5HA&a

21、mp;MPHDME/'csnf/slaves-hydefault.exportHWOCP5UiVE$HUH0PHfflEi/conf/slmshost:pathwhs-rehadtKrpcodeshouldbe-rsync'dfrm.UnMtbydefault.a.iW博*文件陰搜玳履葡n冊mr(»tubfvntu：/lud&Dp/hAdMp-B.Abchnod777btA/hadaafigt電ubuntu二Jhzdaip/hradgpnB-Z&.203.4*0himJh逐必叩ipul-T-.“UBaiftJs4ge:hjdoop-conf-Lgcon

22、fdlrcamAJWireCQIWUMOIsunemmatnani疝1描工KOTMM.Imaftedf-r&rsiisg皿3¥2i必ndFiervodc-datarwdedfs-adhinfifidnlftFfiCkTornatthe好$fllefiyitcRhflrtnohart'arril二&：IWrt«U.Bifimin布網(wǎng)EHMMiqbdlarKe-rfE-tcMtjobcr-BCkerta-ktricktrihiitoryterMie-r，由queueversteftjar<)a.r>dlstu中X5ircurl>fwru

23、riirunirunfiihrilfiEriruntMDF5MAtnX.theDFSn-aw-node13i3fetchrunrun-EFSdat-finMle-DFS-adnlincllentMdg?-ftedMjeeadnliiclLifttOFSflUsystesMe*工yumttygimiflcuse-rcltentclusterbdlanclngiutilityadelcq-Btiontok«ifrontheHaRecedeHdpReduCc。曲TrackernodeaPipetJobEH*aiMjpn«du<a1卡箏kTr4cMrnaflt-runJobh

24、lst&rserversastanddlorHdaiNinanipulateMapRedirce1由sgmtInjRoTzHQne日din口JQueuesprtncthe腿rMonn>aHm*4de-5.turl>copyTileordirectoriesr-ecurslwclya匯htyR5icidvcHaMNftftE-p*pjieffit口。七11*匯號Kduist*ufEituatiadoap4clhi-jE白國推廠的產(chǎn)品4.UAJ.CMYI侵用VM*型安置HadE醛逆轉(zhuǎn)-haiW亞12面口旭冏予欄-押音籟從七月中工月捌;*:模幅，而E«*聿苜-XX：于1

25、5自整文"花照內(nèi)同,解開苗用通戶一.SJ直岳北也不sudopaaswtiE里國中HS田，NqqtE<*i犯UnihiingiEwlE取出割口】計1口-百里第E助西山1CU口上harigu的能立（定展廊母云壯毒rfeve粒奉冏皚LftAh.BEi&aun4SMiIH中.ll"Fg旗祖3餐.聚下./胸thn.VHl#I注或SEF可“我示M不皿忸瓶31皿帽中叼跖括2511fcD-U-目.11無1fffilB；nEiVbaSPeirhsai皿融川El：株至不曜：）苜或空間比SiI憫Mbi：gvndXEOEHMi亞密不副J：M*bull：呼所1&5防4«

26、;1值慎口停率1nfl的華0I|梗廈華。|里冏*墟I喉is癌的M»£l.5.安裝部署單機偽分布式模式Hadoop-HDFSHadoop可以在單節(jié)點上以偽分布式模式運行.編輯/conf/core-s讓e.xml<configuration><property><name></name><value>hdfs:/localhost:9000</value></property><property><name>hadoop.tmp.dir<

27、;/name><value>/tmp/hadoop/hadoop-$</value></property></configuration>&/ME*1.前25:201L*4又的弓3偏IBCTI唱曲外狀海區(qū)佃BQCW3Help(fl就此.T3口'一七小屯/口*，劭<prnperly>*naK«tfii-dle-fault.nwonurci叫玩廿八*仃:"localUt:加加”"Lu。c/pTDirertip-"Tiraperry><ridrF4i

28、itap-1iBp,<iir<.'i«VBlue>/tR»/hadDDp/>uftM)p-StuMir.riieJ/vHVue*</piTDpert)f><,':ii|-|TlijiirHFji.編輯/opt/hadoop/conf/hdfs-site.xml<configuration><property><name>dfs.replication</name><value>1</value></property>i-II-iSJ:

29、5-J-5t-?：I-id-Il-U-IB-14it1<,出白”?£VlmItatEdftu武忤(D/MBMItiAU柜式砂衰fl!®工反>宜口皿用B3回油陰*2編輯/opt/hadoop/conf/mapred-site.xml<configuration><property><name>mapred.job.tracker</name><value>localhost:9001</value></property></configuration>格式化HDFSbi

30、n/hadoopnamenodeformat啟動Hadoop,啟動后可使用命令jps查看已啟動進程，觀察是否namenodedatanode已啟動，關(guān)閉所有服務(wù)，使用命令stop-all.sh,bin/start-all.shR主豆片央即片T。度第一31叱日五件至3m.p嗎S<3|.png5HL期.即g式力,王文件患3|9hpnc|心fU聯(lián)二豆件雁螳事口小rootfpubvntu:/hadop/hadoap-G.ZD.ZOXQ白口itaq-apache,MH,serwt*r.n.fsfsj七網(wǎng)事：i之刈atorg.Apache.Ivado-op.hdfs.server_n.anmiode

31、.FSInage-tfornat(FSIugie-.3fl)at。口量p4clitKfld?ciphdfa-.senrcr*力jheqb.NdmNdv.Ferna1(HaiHodg.j«vd;14J()it«rg.Mdcdp.h日¥trv«r_n#wiad#.同曰Mtaad.u曾七啦河麗d£fMmzlwdfi.jawa:1145)atorg.iipjche.hadoup.hdfs.server_nwMiDde.NjHcNade.HoinCNanesModa-.皿曷:11*11/1(/1423'$5；M*制ITTMMhiJ*5G,*+*+

32、*SHIfTDiDHN_Ji5(ShuttingdownNdneNadeatubuntu/127.II-1.1。力LWbufit*，ha小即，h白強帕“弧20iIflliTaskTracker436£邳串2SB1DatHodeIIS'SSccondBryH-aneNode3404F4arwHode-322?Jbir*cker:/h;adD”fh詁必叩白.?言.后三-g#6.瀏覽NameNode和JobTracker的網(wǎng)絡(luò)接口，地址默認為:NameNode-http:/localhost:50070/JobTracker-http:/localhost:50030/回電Sno&

33、lt;ttj.<o'八出O巴/度博客_、事:1口8-ijEepiiE4*OiDCaa»Gt：5OCI7i4ir5£lNU：tLj£pNameNode'localhostOOO'Slartcd;MonOrt網(wǎng)Z253X35C3Tan1VersloniDJQJffllAriCmpikd：WWMlay407:S7:EDPDT獨11bymieUpfrjidvs；Th&rearewupMJe*5Inprogress,帆Nl./忤MmNamenodeL陵ClusterSummary7finesandddredoorle1RiloH：k5-

34、8tatdl.HeapSizeIs45.19MB/BSa.94ME*5，JCQrrfl|!ndCapadlyDF5Ustd&J01KBNanDF寫UtfedDFSMmainM1sMsGEDFSUsd得DF5RtrrudnlngliSk171NiLlVfeNddti1gadHMe0&eccHnnilsslonlngNM電;sDMurnher仆TIFnder-ReplicatedBlacks0NimtNdeStara£«:Gt時力薩DlrecuryTJP=SLjIvGliKalhasJEHadoapMjp/Red.NIbcillMU：Hd>«白p

35、Mdbp/tt+Wuq，AdmfaahkfMldA-MnalUdiFlr-r力工6，度宿號had口口口xfnenaduq/Xklinkv,舊3»H：E0(n岬btmckH.KlocalhostHadoopMap/ReduceAdministrationSEaE«:RUNNlMG$tartW;MonOrt2422用;5315T割a1Version:D2DJM3.0,r1M5333C&nnpllfrd!WedMuy4M:寫7加PDTMil均camTdiinlllfltr：11iO24i22-flBClusterSummary(HeapSize依45J9MB/888.94

36、MB)HuiilrllfigMa>pRiin電力工刖mhTasksTOEllSubmlHiQrwNW”cevpbtdU叩SkrteOccijiplBdSlotsRtctrvtdMapSkrts.Rft±i-rv>i4lRtdimSlotsMapTalkCapacityRtdiittTalkCapacityTarts/NMnBlacfcHilUidGriylliudN曲EExdud>edD0D1000d224.CDa00SchedulingInformationQueueNameStateSclKdulInglAfarmatlonrunning建FltCir(Jobl

37、d.PrlarirU$5rNjmi)Iidwnpi-iF.'uvirhiiidi3IDO1aviilfilarhy'faiiifh'dulynWhur&aitad2nd'IUKTinJIIkIdaRunningJobs|rwe|.下面嘗試在單機偽分布模式下運行wordcount樣例程序?qū)⑤斎胛募截惖椒植际轿募到y(tǒng)bin/hadoopfs-puthome/lee桌面testfileinput運行Hadoop自帶的wordcount例子bin/hadoopjarhadoop-examples-.jarwordcountinputoutp

38、ut篁忤（D都MBMMAtl相式的衰帶回IJiQ口皿朝由讓1拼，E，”府式56兒L”，I1U1|卜恰-卜.二E7«T-*-072-a-s-iftlla:blG®irDDtubuntu:/hai*ciip/had«op.2D.ZDXQkaotluburitml/hddoop/hddkjQp-9.29.2<I3.QN1bkfl/llMgpfstljMroGtuburktu:/hadoDp/hadoofi-6.6#1cbuild.jchIC*+,CHGE&.txtE«nrccntrlb“加1面il-THbllbr依耐

39、Ml/LICENSE,txt即NOTICE,tx.tREADME,tx.tcrcraotmbuntu:/hadoDfi/hadaDp-9.20.2413.D*bin/hadoopjiarhadaop-E-xaHples-fl.ZH-.ZBlljdriMrd£GUi>ttnputnutputn/lfl/24Hi門期731/36/2411/1B/24ll/lff/2431/16/2-111/IB/2431/30/24J1/1D/Z4：S7：Hihut.FlllhtutF«rtT飆號IInputgEH時興：1T*。MRrX,gXlli電哂；nunntnsjU：憶2MH呢嵋:

40、57;加JhfQndprtd.DDbCllent:ri叩隗reduce戰(zhàn):S7-43IHFQnapred.JobCltent:nap1MKreduce-BK；5-7；S5IHF4napred.3abclient;napJ.M常rcd«eIM鼻1H*<I四廂red,丸btlle,”Job£Qnpl«Tt;J115Z1Z2*l_0Wl；M；ETi其珅LCwJRtersi2箏TNFCImprtd.3Db£ltint:JoftGoyt/：ss：eiINFOnapred.JobCHenitiLaun?riedreduceIHFQnapred.Jobclien

41、trSLOTSHILLISJIAPS-lZfiMJiwanapred.3nbCllentrT&tnlTtwapentbyallfedkicvswallT*tal,片他七匕Y011色詁p品ualt同tlm«rt«ril«tifls=6券回口群.出一節(jié)點為ncdtL廿節(jié)點為Mritw,nodfiO）HOd.'RKti>：nanImaImIIDEO33-：-«鴕|im將輸出文件從分布式文件系統(tǒng)拷貝到本地文件系統(tǒng)查看bin/hadoopfs-getoutput/home/leeft檔cat/home/lee儀檔/output/*注*二件優(yōu)端

42、輯舊型霰0蟀璃P附的出|*IP主女片共圈身計算機3|imng為卻用ngN引png硼l.png3<5J.pogXflp«g王文件或&下員用司Bkpnq3|5ngMlfl-pnfl311nLpng.IZJpog以1®.pngRS11/IB/2122川號21白門嶺red.3*bCliFlit-IfiijitCduftCtri-22：5«；51Jhfflflaprrt,時ntatf=2151/1H/241加注飆.HFCRdprtd.JDbCllHiit!內(nèi)孫fbidugFrdH-iiUDrli：ll/lfl/24321EIXH1h=-3napred.3Db&

43、#163;llent:ReduceInputgroups-531/1D/Z422；S9：01工HFdniapred.JobCllent:Hdpoutputnote-rialteedbytes-57Z2；53：01IWOnapred.SeibelienitrCwiblne-ciutputrecrwds-S11/ie/M12j54：«1IKfl四4fr«LMbtlt電向t，Keptfipvcrec«fds=i»：54：«1T*9Rd4UC*thuFflitty*tVSE$751/IB/3422.!SS：»1h=!lnjpred.JnbCL

44、Lent:Reduceoutputrecords-531/1D/Z422：SS:01IHFOnapritd.JobCltcnt:SpilledRecord5-1011/10/142Zl54；Vl1W»C1ndpred.口口版1lent;Hapoutputbytes-411.1/Wi42割腕；機工府力h4rxLMWU電曲”l岫Utf欽函兆二$11/W/14»；»：«!I*。曬、Hspoutputrtcerdts=$31/IB/2422J59!»1THFCInjpred.DabCltenlE!lPLIir_flAli_ByJElBie231/1D/

45、Z422：SS：01INFOnapred.JobCltent:Reduceinputriecords-S電檔4rootubvntu:/hackiQp/hadcap-G.2D.Zl.0roDtiubuFitiii/hadtiGip/hDdop"9.20sZfll.Offblnyhadnopfs-q«toutput/hore/lce/JIfS10占匕酬3吊工山，h占現(xiàn)舜，MdogP.如.戶I/Nanr/lite/SE檔/MEMj*ct:M鉗ele*f文檔，翎是一個目取,151just1t3H1tilts1r:fhiado”fhai必叩?白.j；中日，.安裝部署多機Hadoop-

46、HDFS這個部署多機的實驗在實驗室沒有做完，最后實在宿舍完成的，節(jié)點為兩個，使用系統(tǒng)為wubi安裝的Ubuntu,所以不需要橋接方式（因為有固定IP）,以下步驟為我做為Master時的實驗截圖：本機名為node1,另外一臺機器為node2,兩臺機器均建立一個名為hadoop的管理員用戶用于統(tǒng)一部署安裝hadoopo編輯文件/etc/hostsubuntu替換為nodel片口眈3-«edrt041電ubuntu2g刊It/«tC/lM3£tXCredit:fl?S4):Gtk-HAHNIhKi*:ftttwiptinqtostar

47、ecJhangesinto,/root/.local/sharp/recently-usedbel'.j.butf-ntled;Mtroctyloc.sl/sh.»re/r«willy-used>xb的，叫川加“失»!；祝有那個文件或目錄Itaedlt:GtkMAfiNING*:Atteflptlr»gtasetthe»olin且.%*.皿工“EPzmtlYsEmt»t打工dT設(shè)有那中文/宜忤所口度1-W11低國工切克杼期M保M3F.上修,DhosisKhi7.e.a.iUcAllwst127.e.1.1rvadelrT

48、Mrollxl咐lliwsar«deitirableferIPmedibletfBS：ffnz：:白frui；zrreistJlg-18Mh8ttg-lgpgc*Ipfr->localnettpfi-Kastpreflxip4-dllrHdesIg一日llrouterstPi-alitestsiFtB呈上皿口卜刃文樣rsAHrirl碎:Inod»LJfilultInodgJiniMdtZiDmuIts-:«vm編輯/etc/hostname刪除該文件的所有內(nèi)容，添加node1*匕5醛片Ctk'IMnNING*£AttenpttngWre/r

49、KEntl.y"Used.Tbitl,Bbutfailed:阻哨：Gtk-HAHNIWi»*:ftttwipttrwtoecentlj-usfdKbel',.shutffltlcd:創(chuàng)St文性門州比“失RS;思右那個文件琥目最=g*dtt/-Btc/hdS-tnAM-hgLmniEM*巾卜匕5，七刃；HARMINGrH：rt/rtcfrntl-u-sed.*6*1,汕t門110才：創(chuàng)題文怦i.CiMiirStlK：境與印干文忤fitU承(9edtt:5!120);Gtk-HARNING*:ftttwiptingt/3tiare/rfr£!&fttl

50、y-yitd.Kbe,l,Bbutr迫運行命令hostnamenode1a)指定IP地址編輯文件/etc/network/interfacesautoeth0ifaceeth0inetstatic“、,，r：,'''''*'-：.04由h(»tiw«E口gdK其件第uiui：EjsvivF也霹他jimri理指州曜昉£地開版岸/ae的m.address96netmask直牛£D。ErptSubuBiuc/aiitoatlhaIfaccethaLn&ta

51、dldrcss90Em12-13日同為M忤日號:金netnaskZ55kZlS.Z35-.c|DEJUlia®HU*曲整世8.造麗計輕卡互膝heIfI血Inodfljnl5時|im重啟網(wǎng)絡(luò)®務(wù)sudo/etc/init.d/networkingrestart查看IP地址ifconfigb）修改/etc/hosts文件node1中:96node13node2node2中：96node13node2埋交性CD中.2上白ubu舊，E3卜i小；7Ji必傅1T小修”T

52、MtubrijntU-/*匚事t/'ttc/fiftMirk/lnttrfKeiautoethBtfflcee-th®inetstaticddkJresa-115.Z4.Z».19«netMlkUS.»S.2S3i.»卜8t卓uMrvtUE”EftMt,Running/ctc/Lnit.d/nctwarlkiingrestartis.deprecatedbeouseitnaynotendhLe-ag«tnsaneinterf-acesR«configuringnetworkinterfaces.u.RTMElLBil

53、K白釀H；flit/FilledEaMingUprootubuntu|r-Mltl電|nod?J_m|MxltZ3吃叫|r»ibLjyj|xdfIIngdr2jolacdel1HB5/9西lit11195沼|imc)配置從nodel無需輸入密碼ssh登錄nodel、node2,命令如下:首先在nodel、node2下建立home/hadoop/.ssh文件夾在nodel(master)上：$ssh-keygen-trsa$cat.ssh/id_rsa.pub>>.ssh/authorized_keys這兩行命令生成密鑰并將公鑰導(dǎo)出到authorized_keys中，然后將

54、此文件拷貝到node2(slaves)中的相同文件夾下，因為通過網(wǎng)絡(luò)命令拷貝比較困難所以我在這里用U盤拷貝的，之后就可以成功無密碼登錄node2d）編輯/conf/core-site.xml此處為了方便查找我更改了hadoop.tmp.dir的文件目錄，改為/home/hadoop/tmp<configuration><property><name></name><value>hdfs:/node1:9000</value></property><property>&l

55、t;name>hadoop.tmp.dir</name><value>/home/hadoop/tmp</value></property></configuration>e）編輯/opt/hadoop/conf/hdfs-site.xml,其中/這里我們將dfs.data.dir改為/home/hadoop/data/將.dir的默認文件夾改為/home/hadoop/name<configuration><property><name>.dir</na

56、me><value>/home/hadoop/name</value></property><property><name>dfs.data.dir</name><value>/home/hadoop/data</value></property><property><name>dfs.replication</name><value>2</value></property></configuration

57、>f）編輯/opt/hadoop/conf/mapred-site.xml<configuration><property><name>mapred.job.tracker</name><value>node1:9001</value></property></configuration>g）酉己置jobtrackermasters文件配置如下：node1slaves文件配置如下：node2在主節(jié)點編輯hadoop的配置之后，將其拷貝到node2的對應(yīng)目錄（U盤）h）格式化HDFSbin/ha

58、doopnamenode-formatI勤演g"日EM4HQSi國vm啟動Hadoop啟動后可使用命令jps查看namenod琲口datanode上已啟動進程，觀察是否namenodedatanode已啟動，關(guān)閉所有服務(wù)，使用命令stop-all.sh,再分別按順序在nodel和node2上開啟namenode(node1)>datanode(node2)、jobtracker(nodel)、tasktracker(node2),注意我在此處只能分別開啟，如果使用start-all.sh全部自動開啟的話啟動所有節(jié)點正常但是無活節(jié)點可用，可能與啟動順序有關(guān)，成功之后中截圖如下:也疊Qhi.biflididCDG”_痂，快用玨kM口EdE屯T”.步gEn0"r日氏，疣，日仁士口匕場互三且f二用與生活同看比«AJtSfi-fttX網(wǎng)捌