空間大數(shù)據(jù)和無(wú)線網(wǎng)絡(luò)

空間大數(shù)據(jù)和無(wú)線網(wǎng)絡(luò)14120439大數(shù)據(jù)（bigdata）宏觀上大數(shù)據(jù)即指海量的數(shù)據(jù);從微觀上定義,大數(shù)據(jù)是指需要通過(guò)快速獲取、處理、分析以從中提取有價(jià)值的海量、多樣化的交易數(shù)據(jù)、交互數(shù)據(jù)與傳感數(shù)據(jù)等數(shù)據(jù),

主要特征包括：海量(volume)、類型繁多(variaty)、增長(zhǎng)迅速(velocity)、價(jià)值巨大(value)等移動(dòng)大數(shù)據(jù)的主要特征海量數(shù)據(jù)思科預(yù)測(cè)全球數(shù)據(jù)流量截止到2013年底,智能手機(jī)的數(shù)量?jī)H占全球在用手機(jī)總量的27%,但其產(chǎn)生的流量占全球手機(jī)總流量的95%(每部智能手機(jī)的移動(dòng)數(shù)據(jù)流量為529MB/月).業(yè)務(wù)類型演進(jìn)

傳統(tǒng)無(wú)線通信網(wǎng)在通信機(jī)制、互聯(lián)互通規(guī)則等方面與互聯(lián)網(wǎng)有完全不同的設(shè)計(jì)理念,難以適應(yīng)新業(yè)務(wù)的需求.數(shù)據(jù)多樣化

移動(dòng)數(shù)據(jù)類型更加繁多,包括結(jié)構(gòu)化數(shù)據(jù)、半結(jié)構(gòu)化數(shù)據(jù)和非結(jié)構(gòu)化數(shù)據(jù)結(jié)構(gòu)化數(shù)據(jù)，簡(jiǎn)單來(lái)說(shuō)就是數(shù)據(jù)庫(kù)。結(jié)合到典型場(chǎng)景中更容易理解，比如企業(yè)ERP、財(cái)務(wù)系統(tǒng)；醫(yī)療HIS數(shù)據(jù)庫(kù)；教育一卡通；政府行政審批；其他核心數(shù)據(jù)庫(kù)等。這些應(yīng)用需要哪些存儲(chǔ)方案呢？基本包括高速存儲(chǔ)應(yīng)用需求、數(shù)據(jù)備份需求、數(shù)據(jù)共享需求以及數(shù)據(jù)容災(zāi)需求。

非結(jié)構(gòu)化數(shù)據(jù)，包括視頻、音頻、圖片、圖像、文檔、文本等形式。具體到典型案例中，像是醫(yī)療影像系統(tǒng)、教育視頻點(diǎn)播、視頻監(jiān)控、國(guó)土GIS、設(shè)計(jì)院、文件服務(wù)器（PDM/FTP）、媒體資源管理等具體應(yīng)用，這些行業(yè)對(duì)于存儲(chǔ)需求包括數(shù)據(jù)存儲(chǔ)、數(shù)據(jù)備份以及數(shù)據(jù)共享等。

半結(jié)構(gòu)化數(shù)據(jù)，包括郵件、HTML、報(bào)表、資源庫(kù)等等，典型場(chǎng)景如郵件系統(tǒng)、WEB集群、教學(xué)資源庫(kù)、數(shù)據(jù)挖掘系統(tǒng)、檔案系統(tǒng)等等。這些應(yīng)用對(duì)于數(shù)據(jù)存儲(chǔ)、數(shù)據(jù)備份、數(shù)據(jù)共享以及數(shù)據(jù)歸檔

等基本存儲(chǔ)需求。隨著網(wǎng)絡(luò)技術(shù)的發(fā)展，特別是Internet和Intranet技術(shù)的飛快發(fā)展，使得非結(jié)構(gòu)化數(shù)據(jù)的數(shù)量日趨增大。這時(shí)，主要用于管理結(jié)構(gòu)化數(shù)據(jù)的關(guān)系數(shù)據(jù)庫(kù)的局限性暴露地越來(lái)越明顯。因而，數(shù)據(jù)庫(kù)技術(shù)相應(yīng)地進(jìn)入了“后關(guān)系數(shù)據(jù)庫(kù)時(shí)代”，發(fā)展進(jìn)入基于網(wǎng)絡(luò)應(yīng)用的非結(jié)構(gòu)化數(shù)據(jù)庫(kù)時(shí)代。所謂非結(jié)構(gòu)化數(shù)據(jù)庫(kù)，是指數(shù)據(jù)庫(kù)的變長(zhǎng)紀(jì)錄由若干不可重復(fù)和可重復(fù)的字段組成，而每個(gè)字段又可由若干不可重復(fù)和可重復(fù)的子字段組成。簡(jiǎn)單地說(shuō)，非結(jié)構(gòu)化數(shù)據(jù)庫(kù)就是字段可變的數(shù)據(jù)庫(kù)。數(shù)據(jù)的空-時(shí)域大動(dòng)態(tài)變化傳統(tǒng)大數(shù)據(jù)處理關(guān)系型數(shù)據(jù)進(jìn)行關(guān)系發(fā)現(xiàn)、分類、預(yù)測(cè)等基于線性時(shí)間空間大數(shù)據(jù)處理

cubictime移動(dòng)大數(shù)據(jù)對(duì)無(wú)線網(wǎng)絡(luò)的挑戰(zhàn)網(wǎng)絡(luò)部署移動(dòng)數(shù)據(jù)流量和信令均呈非線性高速增長(zhǎng),給無(wú)線接入網(wǎng)絡(luò)帶來(lái)巨大壓力.smallcell作為一種全新的網(wǎng)絡(luò)架構(gòu)體系，其最主要的特點(diǎn)就是將無(wú)線接入部分包括射頻、基帶部分、高層協(xié)議處理全部小型化，為運(yùn)營(yíng)商的熱點(diǎn)覆蓋、盲區(qū)覆蓋提供了一種極為靈活的方式，這也順應(yīng)了LTE寬帶高吞吐量的精細(xì)覆蓋要求。嚴(yán)重的的信號(hào)號(hào)衰落落及信信號(hào)傳傳播通通常在在視距距范圍圍內(nèi),使得室內(nèi)小小區(qū)形形狀呈呈現(xiàn)““準(zhǔn)””確定定性和和小區(qū)區(qū)邊緣緣銳利化化的特征征,這對(duì)室室內(nèi)網(wǎng)網(wǎng)絡(luò)覆覆蓋規(guī)規(guī)劃引引入了了難題題.室內(nèi)節(jié)節(jié)點(diǎn)的密集集程度度主要要受建建筑物物布局局的影影響,若以最小化室內(nèi)內(nèi)無(wú)線線節(jié)點(diǎn)點(diǎn)為網(wǎng)網(wǎng)絡(luò)規(guī)規(guī)劃目目標(biāo),銳利的的小小區(qū)邊邊緣會(huì)使問(wèn)問(wèn)題建建模異異常復(fù)復(fù)雜.資源管控?cái)?shù)據(jù)多多樣化化和流量空空-時(shí)域大大動(dòng)態(tài)態(tài)變化化給傳傳統(tǒng)資資源管管理體體系帶帶來(lái)了了新的難題題.數(shù)據(jù)安安全性用戶隱私安全數(shù)據(jù)的訪問(wèn)問(wèn)控制制和可信度高能耗圖4網(wǎng)絡(luò)側(cè)能耗耗分布圖thisarticleisstructuredasfollowsWegiveadetailedoverviewofthreeapplicationdomainsforbigdataprocessinginwirelessnetworkswebelievetobemostrelevantforthecomingdecadeWethendiscussthealgorithmicchallengesrelatedtotheseapplications,andtherelatedimplementationissues.Next,keyissuesforfutureresearcharediscussedbeforedrawingtheconclusions.ApplicationScenariosforSpatialBigDataApplicationScenario1:ManagementofWirelessNetworksUsingBigDataApplicationScenario2:MassiveSensingandIoTApplicationsApplicationScenario3:SmartCitiesandNoisySpatialDataApplicationScenario1:ManagementofWirelessNetworksUsingBigDataoptimizenetworks,increasesecuritythroughanomalydetection,andprovideandenhanceapplicationbehavior,amongotherthings.howtofusethisdatainanoptimalmannerSpitaldataDataminingtheactualcoveragespatialestimatesbasedon10,000spatialestimatesbasedon27000Exampleofatypicalspatialestimationtaskinmanagementofwirelessnetworks.CurrentlyapproachesDetailedpropagationsimulationsDrivetestRoughestimateExpensiveinaccuraciesalternativeMobileterminalsApplicationScenario2:MassiveSensingandIoTApplicationsExistingworkonWSNhaveastrongprototypingfocusSmartphones,large-scaleIoTinfrastructurescanresultinmillionsorevenbillionsofsensorreadingswithtime-spaceindexingwithinasinglecity.EmergingdataprocessingchallengesApplicationScenario3:SmartCitiesandNoisySpatialDatafirstchallengeisthatofrobustenoughprocessingofinformationinasufficientlyrapidmannertoenablerealtimecontrolapplications.Forexample:reroutingoftrafficflows,publictransportationscheduling,pollutionestimates,andnoiselevelsApplicationScenario3:SmartCitiesandNoisySpatialDataThesecondchallengesistheneedtocross-correlatenumerousdatasourcesforobtainingthebestandmostrobustestimates.UsingtheRightAlgorithmsOptimalityandRobustnessSelf-AwarenessofErrorsMatchwithComputationalInfrastructuresExtendibilitywithPriorandSideInformationControlledApproximationsScalableComputationalSolutions空間插值值——空間插值值常用于于將離散散點(diǎn)的測(cè)測(cè)量數(shù)據(jù)據(jù)轉(zhuǎn)換為為連續(xù)的的數(shù)據(jù)曲曲面，以以便與其其它空間間現(xiàn)象的的分布模模式進(jìn)行行比較，，它包括括了空間間內(nèi)插和和外推兩兩種算法法?？臻g內(nèi)插算法：通通過(guò)已知知點(diǎn)的數(shù)數(shù)據(jù)推求求同一區(qū)區(qū)域未知知點(diǎn)數(shù)據(jù)據(jù)?？臻g外推算法法：通過(guò)過(guò)已知區(qū)區(qū)域的數(shù)數(shù)據(jù)，推推求其它它區(qū)域數(shù)數(shù)據(jù)。Kriging算法satisfiesthepropertiesdiscussedabove;thatis,providingoptimalandrobustspatialestimates,Kriging算法克里金法法假定采采樣點(diǎn)之之間的距距離或方方向可以以反映可可用于說(shuō)說(shuō)明表面面變化的的空間相相關(guān)性。?？死锝鸾鸱üぞ呔呖蓪?shù)學(xué)函函數(shù)與指指定數(shù)量量的點(diǎn)或或指定半半徑內(nèi)的的所有點(diǎn)點(diǎn)進(jìn)行擬擬合以確確定每個(gè)個(gè)位置的的輸出值值?？死锝鸱ㄊ且粋€(gè)多步過(guò)過(guò)程；它包括括數(shù)據(jù)的探索索性統(tǒng)計(jì)分析析、變異函數(shù)數(shù)建模和創(chuàng)創(chuàng)建表面，還還包括研究方方差表面。該方法通常用在在土壤科學(xué)和和地質(zhì)中。hadoop是一個(gè)由Apache基金會(huì)所開(kāi)發(fā)發(fā)的分布式系統(tǒng)基礎(chǔ)架構(gòu)。實(shí)現(xiàn)了一個(gè)分布式文件系系統(tǒng)（HadoopDistributedFileSystem），簡(jiǎn)稱HDFS。HDFS有高容錯(cuò)性的特點(diǎn)，并且且設(shè)計(jì)用來(lái)部部署在低廉的的（low-cost）硬件上；而而且它提供高高吞吐量（highthroughput）來(lái)訪問(wèn)應(yīng)用程序的數(shù)據(jù)，適合合那些有著超超大數(shù)據(jù)集（（largedataset）的應(yīng)用程序序。mapreduceMapReduce本身就是用于于并行處理大數(shù)據(jù)集的軟件框架。MapReduce的根源是函數(shù)數(shù)性編程中的的map和reduce函數(shù)。它由兩兩個(gè)可能包含含有許多實(shí)例例（許多Map和Reduce）的操作組成成。Map函數(shù)接受一組組數(shù)據(jù)并將其其轉(zhuǎn)換為一個(gè)個(gè)鍵/值對(duì)列表，輸輸入域中的每每個(gè)元素對(duì)應(yīng)應(yīng)一個(gè)鍵/值對(duì)。Reduce函數(shù)接受Map函數(shù)生成的列列表，然后根根據(jù)它們的鍵鍵（為每個(gè)鍵鍵生成一個(gè)鍵鍵/值對(duì)）縮小鍵鍵/值對(duì)列表。ScalableComputationalSolutionsobjectivespecificallyahighlyscalableparallelizableimplementationofkrigingbasedontheHadoop,HBase,andMapReducecomputationalframeworkProvidingaflexibleframeworkonwhichdifferentapplicationsforprocessingofdataobtainedfromWSNsandothermeasurementsourcescanbeimplementedScalableComputationalSolutionstablesforstoringindividualsensorreadingsestimatesofaccuraciesofthesensorsusedunderlyingresourceHbaseHbaseestimationofvariousspatialstatisticsofinterestScalableComputationalSolutionsThekeycomputationsestimationofacorrelationmeasurecalledthesemivariogramconstructacorrelationmatrixcomputingoptimalspatialpredictionScalableComputationalSolutionsKD-treek-d樹(shù)（k-dimensional樹(shù)的簡(jiǎn)稱），，是一種分割k維數(shù)據(jù)空間的的數(shù)據(jù)結(jié)構(gòu)。。主要應(yīng)用于于多維空間關(guān)關(guān)鍵數(shù)據(jù)的搜索KD-treessignificantlyspeedingupthissearchandenablingahighdegreeofparallelismintheimplementationenabledbytheMapReducecomputationalmodelThemodulelaunchesseveralmappersreadingthedatafromthedatasources,convertingthedataformatsasneeded,andwritingthemintoaspecificHBase.aconstructorandasearchfunctionaMapReduceprogramcomputingtheexperimentalsemivariogramThefittingisdoneusinganimplementationofanonlinearconjugategradientoptimizerrealizedintheAnalyticalSemivariogramMmodule.TheSpatialpredictionMmoduleisfinallyresponsibleforcomputingthespatialestimate.ItisalsoaMapReduceprogramcomputationalburdenofthesestepsthreeHadoopserversResearchChallengesFoundationsmorescalablealgorithmsareneededincludinganomalydetection,systemidentificationandsimilarproblems,especiallyforspatio-temporalbigdata.DeploymentPlanningandHierarchicalNetworkDimensioninglarge-scaleWSNandIoTdeploymentscanresultinextremelylargedatastreamsHierarchicalcommunicationanddataprocessingsolutionsareneededtodealwithsuchvastquantitiesofdata,andplanningsuchdeploymentsjointlywiththedataprocessingframeworkisachallengingtask.Forexample,addingnewdatafusionandprocessingcentersevenwhennotstrictlyneededintermsofcomputationalpowercanincreasedeploymentcosts,butsimultaneouslysignificantlyimprove