微軟并行數(shù)據(jù)倉庫技術(shù)概述在大數(shù)據(jù)存儲(chǔ)中的應(yīng)用

1、Microsoft Parallel DataWarehouse微軟并行數(shù)據(jù)倉庫2Data sources數(shù)據(jù)源3sourcesData Non-Relational Data數(shù)據(jù)源 非關(guān)系型數(shù)據(jù)Data sources 數(shù)據(jù)源Non-relational data非關(guān)系型數(shù)據(jù)Gain knowledge of all your existing data 通過任何數(shù)據(jù)獲得知識Enrich and optimize your data from non-traditional sources 通過非關(guān)系型數(shù)據(jù)源富集并優(yōu)化數(shù)據(jù)Keep legacyinvestment保持原有投資Buy new

2、tier onehardware appliance購買第一層硬件設(shè)備Acquire big datasolution獲取大數(shù)據(jù)解決方案Acquire businessintelligence獲取商業(yè)智能Roadblocks to evolving to a modern data warehouse現(xiàn)代化數(shù)據(jù)倉庫面臨的發(fā)展障礙Limitedscalability & ability tohandle new data types擴(kuò)展性有限，無法處理新型數(shù)據(jù)Significant training & stillsiloed需要大量培訓(xùn)，不同系統(tǒng)相互隔離High acquisi

3、tion/migrationCosts購買/遷移成本高Complex with lowadoption過于復(fù)雜，接受度低Microsoft Parallel Data Warehouse 微軟并行數(shù)據(jù)倉庫The turnkey modern data warehouse appliance 現(xiàn)代化數(shù)據(jù)倉庫設(shè)備一體機(jī)“New “新”數(shù)據(jù)源 sources” dataMove HDFS into the warehouse before analysis分析前需將 HDFS 遷移至倉庫ETLLearn new skills/學(xué)習(xí)新技能TSQLBuildIntegrateManageMaintain

4、Support構(gòu)建集成式管理的維護(hù)支持Hadoop alone is not the answer to all big data challengesHadoop 自身并非所有大數(shù)據(jù)問題的終極答案Steep learning curve, slow and inefficientHadoop ecosystem陡峭的學(xué)習(xí)曲線，緩慢低效的 Hadoop 生態(tài)系統(tǒng)“New” data sources“新”數(shù)據(jù)源“New” data sourcesSingle T-SQL query model for PDW and Hadoop with richfeatures of T-SQL inclu

5、ding joins without ETLPDW 與 Hadoop 可使用通用的 T-SQL 查詢模型，并具備豐富的T-SQL 功能，包括無需 ETL 的連接Leverages the power of MPP to enhance query executionperformance利用大規(guī)模并行處理的強(qiáng)大運(yùn)算能力改善查詢性能Supports Windows Azure HDInsight to enable new hybrid cloudscenarios支持 Windows Azure HDInsight，實(shí)現(xiàn)全新混合云場景Query non-Microsoft Hadoop dis

6、tributions such as Hortonworksand Cloudera查詢非微軟 Hadoop 發(fā)行版，例如 Hortonworks 與 ClouderaSQL ServerParallel DataWarehouseSQL Server 并行數(shù)據(jù)倉庫PolyBaseMicrosoftHDInsightWindows AzureHDInsightClouderaHortonworks LinuxSelect選擇Result set結(jié)果集Connecting islands of data with PolyBase 使用 PolyBase 連接信息孤島Bringing Hadoo

7、p point solutions and the data warehouse together for users and IT將 Hadoop 單點(diǎn)解決方案與數(shù)據(jù)倉庫打包供用戶與 IT 使用HortonworksWindows ServerComing Soon:PDW delivers enterprise-ready Hadoop with HDInsight即將發(fā)布：PDW 面向企業(yè)提供帶有 HDInsight 的 HadoopManageable, secured and highly available Hadoop integrated into the appliance可

8、管理，安全，高可用的 Hadoop 直接集成于一體機(jī)Accessible insights foreveryone with Microsoft BItools所有人可通過微軟 BI 工具訪問洞察力End-user authentication withActive Directory最終用戶使用 Active Directory實(shí)現(xiàn)身份驗(yàn)證Managed and monitoredusing System Center使用 System Center 進(jìn)行管理與監(jiān)控High performance tunedwithin the appliance裝置進(jìn)行高性能優(yōu)化100% Apache H

9、adoopSQL ServerParallel DataWarehouseSQL Server并行數(shù)據(jù)倉庫PolyBaseMicrosoftHDInsightC1C2C3C4R1R1R1R1R2R2R2R2R3R3R3R3R4R4R4R4R5R5R5R5R6R6R6R6Performance limitations and scale with a traditional data warehouse傳統(tǒng)數(shù)據(jù)倉庫的性能與擴(kuò)展性局限Scale up縱向擴(kuò)展Rowstore行存儲(chǔ)Sub-optimal performance for many datawarehouse queries很多數(shù)據(jù)倉庫

10、查詢無法獲得最優(yōu)性能Data數(shù)據(jù)Page 1Page 2Page 3Querying data by row 逐行查詢數(shù)據(jù)ForkliftForkliftDiminishing scale as requirements grow按需逐層擴(kuò)展Scale-out橫向擴(kuò)展技術(shù)Multiple nodes with dedicated CPU, memory, andstorage使用專用 CPU、內(nèi)存及存儲(chǔ)的多個(gè)節(jié)點(diǎn)Ability to incrementally add hardware for near-linearscale to multiple petabytes可逐漸添加硬件，用近乎線

11、性的方式擴(kuò)展至數(shù) PB 級別Ability to handle query complexity and concurrency atscale可并行處理大規(guī)模復(fù)雜查詢No “forklift” of prior warehouse to increase capacity倉庫擴(kuò)容無需提前投入較高成本Ability to scale out HDInsight and PDWHDInsight 與 PDW 可橫向擴(kuò)展Scale-out technologies in the Parallel Data Warehouse 并行數(shù)據(jù)倉庫中的橫向擴(kuò)展技術(shù)140TB6PBPDW /HDInsight

12、PDW /HDInsightPDW /HDInsightPDWPDW /HDInsightPDW /HDInsightPDW /HDInsightStore data in columnar format for massive compression 用列格式存儲(chǔ)數(shù)據(jù)，實(shí)現(xiàn)大規(guī)模壓縮Load data into or out of memory for next-generationperformance 通過內(nèi)存讀寫數(shù)據(jù)，提供下一代性能Updateable and clustered for real-time trickle loading 可更新可群集，實(shí)現(xiàn)實(shí)時(shí)涓流加載15Up to1

13、00 xfaster queries查詢速度最高提速 100 倍Updateable clustered columnstore vs. table with customary indexing可更新的群集列存儲(chǔ)相比使用傳統(tǒng)索引的表Up to15xmore compression壓縮率最高提升 15 倍Columnstore index representation列存儲(chǔ)索引的呈現(xiàn)Parallel query execution執(zhí)行并行查詢Query查詢Results結(jié)果Thousands/千PDW provides the industrys lowest DW appliance pri

14、ce/TBPDW 可提供業(yè)內(nèi)每 TB 成本最低的數(shù)據(jù)倉庫一體機(jī)High performance using commodity hardware 使用市售硬件獲得更高性能Significantlylower priceper TB than the closestcompetitor相比競爭對手大幅降低每 TB 價(jià)格Oracle EMC IBM Teradata MicrosoftNOTE: Orangelineindicatesaverageprice perTB.注意：橙色橫線代表每 TB 平均價(jià)格Price per terabyte for leading vendors主要供應(yīng)商每 T

15、B 成本Price per TB User-AvailableStorage(Compressed)每 TB 用戶可用存儲(chǔ)空間的價(jià)格（壓縮后）$30$25$20$15$10$5$0Lower storage costswith Windows Server 2012Storage Spaces通過 Windows Server 2012 存儲(chǔ)空間大幅降低存儲(chǔ)成本Microsoft Parallel Data Warehouse微軟并行數(shù)據(jù)倉庫The no-compromise modern data warehouse solution不妥協(xié)的現(xiàn)代化數(shù)據(jù)倉庫解決方案Meeting todays big dataanalytics requirements認(rèn)識當(dāng)今的大數(shù)據(jù)分析需求Enterprise-ready H