實(shí)驗(yàn)2-熟悉常用的HDFS操作

實(shí)驗(yàn)2熟悉常用的HDFS操作1實(shí)驗(yàn)?zāi)康睦斫釮DFS在Hadoop體系結(jié)構(gòu)中的角色；熟練使用HDFS操作常用的Shell命令；熟悉HDFS操作常用的JavaAPI。實(shí)驗(yàn)平臺(tái)操作系統(tǒng)：LinuxHadoop版本：或以上版本JDK版本：或以上版本JavaIDE：Eclipse實(shí)驗(yàn)內(nèi)容和要求編程實(shí)現(xiàn)以下指定功能，并利用Hadoop提供的Shell命令完成相同任務(wù)：向HDFS中上傳任意文本文件，如果指定的文件在HDFS中已經(jīng)存在，由用戶指定是追加到原有文件末尾還是覆蓋原有的文件；wen^ubuntu:/usr/local/^a^°叩/bin$Jhdfsdfs-test-etext.txtwenQubuivtii:/u5r/local/liadao|?/bln$echo$?127Men^ubuntui/usr/local/hada叩/btin$Jhdfsdlfs-appendloFilelocaLt?<ttext.txtappendloFile:/usr/local/hadoop/bin/local-txt(膾有那個(gè)文件或目錄)Nengubunto;/usr/local/hadaap/bln$Jhdfsdlfs-cp-ffile:///hone/hadacp/local.txttext*txtcp:'file:///hone/hadoop/localitxt':Nosuchfileordirectory二Problems&Javadoc隆DeclarationOConsalt黑<refminated>HDFSApi[JavaAppUkation]/usi/lib/jvrri/java-7-openjdk-amd64/bir/java(Dec15,20164:54:28AM)log]：warnngappenderscouldbefoundforloqger(org.Epache.hadoop.con-1.configuratior.deprecat:log4j:WARNPleaseinitializethelog4jsystemproperly,log4j:warnseeh;tp：///Locf4j/1.2/faq.litniL#noconfigformoreinfo./user/hadoop/text.txt已存在，/hm^/hadDDp/tert.txt己追if]至/user/hadDDp/text.txt從HDFS中下載指定文件，如果本地文件與要下載的文件名稱相同，則自動(dòng)對(duì)下載的文件重命名；hadoop@ubuntur/usr/loca'L/ha(lcop/btn$./hdfsdfs-test-efile:///home/hadoop/text*txthadoopsubuntu：/usr/iocal/hadODp/btnSJihdfsdfs-copyroiLocalr^textz^txhadaop^ubuntu:/usr/local./hadGDp/bln$*/hdfsdfs-copyToLocdltext*txt,/text>txt也Problems所Javadoc凰Declaration&Console戲^terminated>HDFSApi[JavaApplication]/u5r/lib/jvm/java-7-openjdk-amd&4/bin/java(Dec15r20log4j:WARNPleaseinitializetheLog4jsystenproperly.Iog4j:WARNSeehttp://loggingPap^che.arg/log4j/]..2/faq.html#noconfi，ifarworeinti/home/hadoop/te^t.txt已存在.將里新奇吾為：/home/hadoop/text.txt_0下ii完成將HDFS中指定文件的內(nèi)容輸出到終端中;hadoop^ubuntu:/usr/local^h^dDOp/bln$./hdfsdfs-cattext*txthallohadoop貿(mào)Problems@Javadoc凰Declaration5Console口<tEiminated>HDFSApi[JavaApplkaLion]/usi/[ib/jvE/jag-7-cipenidk-aEd&4/birVjava(Dec15,20154;58?Log4j:WARNPLeaseinitializetheLog4js/steniproperly.讀取文件：/user/hadoop/text.txtlog4j:WARNSeehttp：///log4j/I.2/fac.html^noconfigformoreinfo.hel'.ohadoop謨邱坑成顯示HDFS中指定的文件的讀寫權(quán)限、大小、創(chuàng)建時(shí)間、路徑等信息；h^doap^ubuntu:/usr/locaL/hadoop/btn$./hdfsdfs-Is-htext.txt-rw-r-r--1hadoopsupergroup132016^12-1504:54text*txtlag4j:WARNNaappenderscouLdbefoundforLoggerlorg.apache.hadoop.conf.Configuration.(lag4j:WARNPleaseinitializethelag4jsystemDroperly.Iag4j:WARNSeehttp：///log4j/l.2/faq.html#nocon'figformoreinfo.MB:hdis：//localhost：9000/user/hadoop/text.txtrv-r--r--大?。?3時(shí)1司：2016-12-1504:54:37i鬼取完成給定HDFS中某一個(gè)目錄，輸出該目錄下的所有文件的讀寫權(quán)限、大小、創(chuàng)建時(shí)間、路徑等信息，如果該文件是目錄，則遞歸輸出該目錄下所有文件相關(guān)信息；Jhadoapsubuntu:/usr/iDcai/hadcup/btriS*/hdfsdfs-Is-R-h/user/hddo&pI-rw-r--r--1hadoopsupergroup122016-12-1504：S4/user/hadoop/text.tIxt<r?npinar?ci>HdFSApi口巳戲Application]/usr/libZjvE/java^Fpwnjdk~dEciS4/birVjeva(。氏15,2Q165:02:40AMlog^j:WARNNdeppenderscouldbefoundforlogger(□rg.apsene.hadoop.conf.Configuration.deprflog^j:WARNPleaseinjti.alijetheLog4jsystempnoperllog4j:WARNSeehttp；//logging,/log4j71.2/faq.ht[il#naconfigformoreinfo.t通歸)諼耿目錄下所有文件的信息；/user/hadoopS3鋰：tidfs://localhost：9009/u5er/hadoop/text.txt極限：rw-r--r-大?。?3時(shí)間：2016121504:54:37i賣取完吏提供一個(gè)HDFS內(nèi)的文件的路徑，對(duì)該文件進(jìn)行創(chuàng)建和刪除操作。如果文件所在目錄不存在，則自動(dòng)創(chuàng)建目錄；

jhadoopQubuntu:/usr/local/hadocp/btn$*/hdfsdfs-rndir1/dir2/filenanejrm:'dirl/dir2/ffLename':Nosuchfileordirectory勤Problems4?Javadoc甌Dec:la(adon曰consoleK<t&rminatedHDFSApi[JavaApplicaLi&n]/usi/lib/jvm/jjava-7-&perjdk-amd64/bin/java(Dec15r2016S:06:2!log^：:WAftNNdappendlerscouldbefoundforlogger(org.apache.badoop.conf.ConfigLration.cl口g勺：warmPleaseLnitialirethelog4jsystenproperly.lag^;:WARNSeehttp：//lagging.apache.arg/lag^j/1.2/faq.fitml#rocantigformoreinfo.創(chuàng)廷支件去：/user/hadoop/input創(chuàng)理蹈徑:/user/hadoop/1nput/text.in提供一個(gè)HDFS的目錄的路徑，對(duì)該目錄進(jìn)行創(chuàng)建和刪除操作。創(chuàng)建目錄時(shí)，如果目錄文件所在目錄不存在則自動(dòng)創(chuàng)建相應(yīng)目錄;刪除目錄時(shí)，由用戶指定當(dāng)該目錄不為空時(shí)是否還刪除該目錄；lhadoop@jbuntu:/usr/local/hadoop/btn$*/hdfsdfs-mkdir-pdirl/dirZhadaopQul>untu:/usr/locil/hadooi)/binS./hdfsdfs-rndirdirl/dirZjhadoop^ubuntu:/usr/local/had?op/bin$./hdfsdfs-rn-Rdirl/dir2|「mNosuchfile折5「自￡!：9「v氐Problems?Javadoc⑥D(zhuǎn)eclaraticn國Console堂<terminaLed>HDFSApi[JavaApplicdtion]/u5r/lib/jvm/jflva-7-apenjdk-anid64/birvjava(DecII5rZ0165Log4j:WARNNoapperderscouldbefoundforlogger(org.apache.hadoop.conf.Configurat:Log4j:WARNPleaseinitializethelog4jsystemproperly.Log4j:WARNSeehttp：///log4j/1.2/faq.html￥noconfigformoreinfo.目錄不為空「不刪除：/user/hadnop/input向HDFS中指定的文件追加內(nèi)容，由用戶指定內(nèi)容追加到原有文件的開頭或結(jié)尾；hadcopOubuntiiz/usr/lQCdl/hadoop/btnS./hdfsdfs-e叩endToFllelocaltxttext+txtappendToFile:/usr/local/hadoop/bin/local*txt(Nosuchfileordirectory)hddoopQubuntu;/usr/Lk^I/hadoop/bin$+/hdf5dfs-gettext.txtget:'text.tKt1:FileexistshadcopOubuntiiz/usr/local/hadoop/binS+/catt：日Kt.txt》〉localtxtbash:,/cat:Nosuchfileardirectoryhadoopflubuntuj/usr/local/hadcop/binS./hdfsdfs'CopvFronLocal-ftext*tsttext*txt?...Probiprns@Javadoc隱Declaratian耳Console或、-；terminated*HDFSApi[JavaApplication]/usr/lil>/jvm/java-7-<jpenjdk-amd6^/bin/ja^a(Dec15r2016l0g4j:WARNNoappenderscouldbefoundforlogger(org.apache.badoop.conf.ConfiguraLog4j:warnPleaseinitializethelog4jsystemproperly.Log4j:WARNSeehttp：//logging,/log4j/l.2/faq.Iitml^nocon-igformarein-o.已追加向容到攵件末尾Zuser/hadlooF/text.ti<t刪除HDFS中指定的文件；IhadoDp@ubijntu:/usr/local/had0op/bin$*/hdfsdfs-rntext*txt16/12/1505:14:51INFOfs^TrashPolicyDefault:Nanenodetrashconfiguratian:DelettooInterval=Bpilnuxes,EnptlerInterval=enlnutesnDeletedtext.txt由于已經(jīng)刪除，所以會(huì)顯示操作失敗?！觯籧&rmindcea>HDr^ApiL.JavaAppucauorij/usryiiD/jvm/java-f-openjoK-amg^/DiQ/javaiD&cib,log4j:WARNNoappenderscouldbefoundforlogger(org.apache.hedaop.can't.Configurelog4j:WARNPleaseinitializethelog4jsystemproperly.Iog4]:WARNSeehttp://logging.apache.Drg/log4j/l.2/faq.htnl^noconfigformorein~o.新失敗(丈件不件在或刑昧失畋)刪除HDFS中指定的目錄，由用戶指定目錄中如果存在文件時(shí)是否刪除目錄；由于前面已經(jīng)刪除，所以顯示無該文件.

Ihadoapgubuntu:/L>sr/lccal/hadoop/bin$./hdfsdfs-rmdi.「dirl/dir2jrndir:'dtrl/dtr21hosuchfileordirectoryjhadoopgubuntu:/Lisr/lccal/hadocp/bin$./hdfsdfs-rn-Rdirl/dir2|「n:EL「1國nosuchfiledrJL「眈to「v&Problems@Javadoc底Declaration可ConsoleS3II、jerminated^-HDFSApipavaAppliczMion]/us「/lib/jfe*目"T-曰penjdk-尋ed64/biMwe(Dec15,20165:19;2UgQJ:*/arnN。叩penrierscauldbefoundforloggei<org.apacherhadooprconf.configuration-Logo-]:warmPleaseinitializeThelog<l-jsystemproperly.Log4]:WARNSeehttp：//logging.apache.Drg/log4j/l.2/faq.htnl^naconfigformoreinfo.W錄不為空「不刪除在HDFS中，將文件從源路徑移動(dòng)到目的路徑。hadoop^ubuntu:/usr/Local/hadoop/bin$*/hdfsdfs-nvtext*txttext2.txt移動(dòng)成功：就Ermifiated、HDFSApiApplication.對(duì)LibfjvmJjaua~7-Qpenjdk~amd64/bin/j洲a(E>k15,20165:2&:11AMJl四4」：warnNoappendsrscouldbefoundforlogger(arg.apache.hadoop.conf.configuration.deprecation].Iog4j:warnPleaseinitializethelog4jsystapraperly.Icig4j:warnSeeh?Tp：//logging,apache^org/log^j/].2/faq.htiTUnoconfigfornoreinfo.將知牛hdfs：///user/ii3do叩ftext.txthdfs:///user/hadQcip/new.txt編程實(shí)現(xiàn)一個(gè)類“MyFSDataInputStream”，該類繼承“，要求如下：實(shí)現(xiàn)按行讀取HDFS中指定文件的方法“readLine()”，如果讀到文件末尾，則返回空，否則返回文件一行的文本。進(jìn)行文檔讀取如下：SProblems?JavadocDeclaration國Console淀、<terminaced>MyF5DataInpurstream[Ja\;aApplication]/usr/lib/jvm/java-7-openjdk-amd64/birVjavaLog4j:WARNNoappendcrscouldbefoundforlogger[org.apach