課程資源云計(jì)算cclecture3商業(yè)平臺(tái)和開源

App Amazon(AmazonAlibaba(Aliyun234的 FileTheChubbylockserviceforloosely-coupleddistributed5 6 7提供數(shù)據(jù)結(jié)構(gòu)化MapReduce數(shù)據(jù)源

對(duì)BigTable中數(shù)使用BigTable 記錄MapReduce其它鎖服務(wù)和信 8

9 AmazonMachineImage ElasticBlock

AmazonAWS微軟WindowsSQL 云端操作系統(tǒng)Windows WindowsAzure WindowsAzure SQL 數(shù)據(jù)中心AuthorityContainer

SQLSQLSQL SQLAzure數(shù)據(jù)庫(kù)

WindowsAzure ?IOE+Greenplum+Hadoop+

愿景

))( (

Linux集群 混

M 混 文件 ，單集群>

FuxiMasterHA，AppMaster

..

Fuxi ..

..

…… ……

Met

飛天集群

100（目前單日平均 HadoopCommon:ThecommonutilitiesthatsupporttheotherHadoopmodules.HadoopDistributedFileSystem(HDFS?):Adistributedfilesystemthatprovideshigh-throughputaccesstoapplicationdata.HadoopYARN:Aframeworkforjobschedulingandclusterresourcemanagement.HadoopMapReduce:AYARN-basedsystemforparallelprocessingoflargedatasets.HadoopOzone:AnobjectstoreforHadoop.HadoopSubmarine:Alearningenginefor

Ambari:Aweb-basedtoolforprovisioning,managing,andmonitoringApacheHadoopclusterswhichincludessupportforHadoopHDFS,HadoopMapReduce,Hive,HCatalog,HBase,ZooKeeper,Oozie,PigandSqoop.AmbarialsoprovidesadashboardforviewingclusterhealthsuchasheatmapsandabilitytoviewMapReduce,PigandHiveapplicationsvisuallyalongwithfeaturestodiagnosetheirperformancecharacteristicsinauser-friendlymanner.Avro:AdataserializationCassandra:Ascalablemulti-masterdatabasewithnosinglepointsoffailure.Chukwa:AdatacollectionsystemformanagingdistributedHBase:Ascalable,distributeddatabasethatsupportsstructureddatastorageforlargetables.Hive:AdatawarehouseinfrastructurethatprovidessummarizationandadhocMahout:AScalablemachinelearninganddataminingPig:Ahigh-leveldata-flowlanguageandexecutionframeworkforparallelcomputation.Spark:AfastandgeneralcomputeengineforHadoopdata.Sparkprovidesasimpleandexpressiveprogrammingmodelthatsupportsawiderangeofapplications,includingETL,machinelearning,streamprocessing,andgraphcomputation.Tez:Ageneralizeddata-flowprogrammingframework,builtonHadoopYARN,whichprovidesapowerfulandflexibleenginetoexecuteanarbitraryDAGoftaskstoprocessdataforbothbatchandin ctiveuse-cases.TezisbeingadoptedbyHive?,Pig?andotherframeworksintheHadoopecosystem,andalsobyothercommercialsoftware(e.g.ETLtools),tore ceHadoopMapReduceastheunderlyingexecutionengine.ZooKeeper:Ahigh-performancecoordinationservicefordistributedapplications.

與機(jī)器學(xué)習(xí)等需要迭代的mapreduce的算法 WalrusWEucalypt