數(shù)據(jù)挖掘[6-5]Cluster-HierMeth (16)

1、Data Mining (Spring 2012), Tsinghua University0Data Model Review the basic concepts of database What is a data warehouse? A multi-dimensional data model Data warehouse architecture Data warehouse implementation From data warehousing to data miningData Mining (Spring 2012), Tsinghua University1數(shù)據(jù)立方體數(shù)

2、據(jù)立方體(1) 數(shù)據(jù)倉庫基于一個多維度數(shù)據(jù)模型，以數(shù)據(jù)立方體的方式看待數(shù)據(jù)數(shù)據(jù)倉庫基于一個多維度數(shù)據(jù)模型，以數(shù)據(jù)立方體的方式看待數(shù)據(jù) 一個數(shù)據(jù)立方體，例如銷售量，允許數(shù)據(jù)進(jìn)行建模，并在多個層面觀看一個數(shù)據(jù)立方體，例如銷售量，允許數(shù)據(jù)進(jìn)行建模，并在多個層面觀看 Dimension tables（維表）（維表）, 例如項目例如項目(項目名字項目名字, 品牌品牌, 類型類型), 或者時間或者時間(天天, 周周,月月,季度季度,年年) Fact table （事實表）包含相關(guān)維度表的層組（例如銷售額）和鍵（事實表）包含相關(guān)維度表的層組（例如銷售額）和鍵 在數(shù)據(jù)倉庫文獻(xiàn)中在數(shù)據(jù)倉庫文獻(xiàn)中,一個一個n維基

3、本立方體被稱為基本方體。維基本立方體被稱為基本方體。 擁有最高級匯總擁有最高級匯總的最上層的的最上層的0維方體維方體, 被稱為頂端立方體。長方體晶格形成了一個數(shù)據(jù)立方被稱為頂端立方體。長方體晶格形成了一個數(shù)據(jù)立方體。體。Data Mining (Spring 2012), Tsinghua University2數(shù)據(jù)立方體數(shù)據(jù)立方體 (2) 維度和維度表維度和維度表 維度維度: 是一個組織要保留的觀點或?qū)嶓w是一個組織要保留的觀點或?qū)嶓w. 維度表維度表: 是進(jìn)一步描述一個維度的一組屬性是進(jìn)一步描述一個維度的一組屬性. 每個維度有可能有與之相聯(lián)系的一個維度表每個維度有可能有與之相聯(lián)系的一個維度表.

4、時間，項目，地點，供應(yīng)者時間，項目，地點，供應(yīng)者 事實事實 和事實表和事實表 事實事實: 衡量一個主題衡量一個主題 事實表事實表: 事實的表現(xiàn)事實的表現(xiàn). 它包含每個相關(guān)維表的事實和鍵名。事實是數(shù)值，銷售金額Data Mining (Spring 2012), Tsinghua University3Data Cube (3) 數(shù)據(jù)立方的維度數(shù)量數(shù)據(jù)立方的維度數(shù)量觀察到的維度數(shù)量觀察到的維度數(shù)量. Sales（item time location dollars_sold ）基本方體基本方體: 包含所有在數(shù)據(jù)倉庫中可以被觀察到的維度的立方體包含所有在數(shù)據(jù)倉庫中可以被觀察到的維度的立方體.頂端立

5、方體頂端立方體: 不包含維度的立方體不包含維度的立方體.數(shù)據(jù)立方數(shù)據(jù)立方: 一個多維度數(shù)據(jù)模型中的所有立方體一個多維度數(shù)據(jù)模型中的所有立方體.Data Mining (Spring 2012), Tsinghua University4數(shù)據(jù)立方數(shù)據(jù)立方 One Example(1)ALLElectronics sales維度：維度：時間，項目，地點，品牌時間，項目，地點，品牌維度表：維度表：time(time_key day day_of_week month quarter year)item(item_key item_name brand type supplier_key)fact t

6、able：(time_key item_key brand_key location_key dollars_sold units_sold)Data Mining (Spring 2012), Tsinghua University5數(shù)據(jù)立方數(shù)據(jù)立方 One Example(2)2維數(shù)據(jù)立方：維數(shù)據(jù)立方：location”Vancouver” item(type)Time(quarter)entertainment computer security Q1 605 825 400 Q2 680 920 512 Q3 781 1026 501 Q4 824 1120 580Data Minin

7、g (Spring 2012), Tsinghua University6數(shù)據(jù)立方數(shù)據(jù)立方 One Example(3) 4維數(shù)據(jù)立方維數(shù)據(jù)立方家庭娛樂計算機電話安全Q1Q2Q3Q4VanciuerTorontoNew YorkChicagoitemtimeLocationsupplierData Mining (Spring 2012), Tsinghua University7Cube: A Lattice of Cuboidsalltimeitemlocationsuppliertime,locationtime,supplieritem,locationitem,supplierloc

8、ation,suppliertime,item,suppliertime,location,supplieritem,location,supplier0-D(apex) cuboid1-D cuboids2-D cuboids3-D cuboids4-D(base) cuboidData Mining (Spring 2012), Tsinghua University8數(shù)據(jù)倉庫的概念模型數(shù)據(jù)倉庫模型數(shù)據(jù)倉庫模型: 維度維度&層組層組 星型模式星型模式: 中間的事實表和一組維度表相連中間的事實表和一組維度表相連 雪花模式雪花模式: 是星型模式的改進(jìn)，一些維度層級標(biāo)準(zhǔn)化成一組更小的維度是星型模

9、式的改進(jìn)，一些維度層級標(biāo)準(zhǔn)化成一組更小的維度表，形成類似雪花的形狀表，形成類似雪花的形狀 Fact constellations（事實星座）（事實星座）:多個事實表共享維度表，看起來像星星的集合，因此被稱為星系模式或事實星座Data Mining (Spring 2012), Tsinghua University9Example of Star Schema time_keydayday_of_the_weekmonthquarteryeartimelocation_keystreetcitystate_or_provincecountrylocationSales Fact Table t

10、ime_key item_key branch_key location_key units_sold dollars_sold avg_salesMeasuresitem_keyitem_namebrandtypesupplier_typeitembranch_keybranch_namebranch_typebranchData Mining (Spring 2012), Tsinghua University10Example of Snowflake Schematime_keydayday_of_the_weekmonthquarteryeartimelocation_keystre

11、etcity_keylocationSales Fact Table time_key item_key branch_key location_key units_sold dollars_sold avg_salesMeasuresitem_keyitem_namebrandtypesupplier_keyitembranch_keybranch_namebranch_typebranchsupplier_keysupplier_typesuppliercity_keycitystate_or_provincecountrycityData Mining (Spring 2012), Ts

12、inghua University11Example of Fact Constellations Schematime_keydayday_of_the_weekmonthquarteryeartimelocation_keystreetcityprovince_or_statecountrylocationSales Fact Tabletime_key item_key branch_key location_key units_sold dollars_sold avg_salesMeasuresitem_keyitem_namebrandtypesupplier_typeitembr

13、anch_keybranch_namebranch_typebranchShipping Fact Tabletime_key item_key shipper_key from_location to_location dollars_cost units_shippedshipper_keyshipper_namelocation_keyshipper_typeshipperData Mining (Spring 2012), Tsinghua University12在在DMQL的多維數(shù)據(jù)定義語法的多維數(shù)據(jù)定義語法 多維數(shù)據(jù)集定義(事實表事實表)define cube : 維度定義維度定

14、義(維度表維度表)define dimension as () 特例特例(共享維度表共享維度表) First time as “cube definition” define dimension as in cube Data Mining (Spring 2012), Tsinghua University13在在DMQL定義星型模式定義星型模式define cube sales_star time, item, branch, location:dollars_sold = sum(sales_in_dollars), avg_sales = avg(sales_in_dollars),

15、units_sold = count(*)define dimension time as (time_key, day, day_of_week, month, quarter, year)define dimension item as (item_key, item_name, brand, type, supplier_type)define dimension branch as (branch_key, branch_name, branch_type)define dimension location as (location_key, street, city, provinc

16、e_or_state, country)Data Mining (Spring 2012), Tsinghua University14在在DMQL定義雪花模式定義雪花模式define cube sales_snowflake time, item, branch, location:dollars_sold = sum(sales_in_dollars), avg_sales = avg(sales_in_dollars), units_sold = count(*)define dimension time as (time_key, day, day_of_week, month, qu

17、arter, year)define dimension item as (item_key, item_name, brand, type, supplier(supplier_key, supplier_type)define dimension branch as (branch_key, branch_name, branch_type)define dimension location as (location_key, street, city(city_key, province_or_state, country)Data Mining (Spring 2012), Tsing

18、hua University15在在DMQL定義事實星座定義事實星座define cube sales time, item, branch, location:dollars_sold = sum(sales_in_dollars), avg_sales = avg(sales_in_dollars), units_sold = count(*)define dimension time as (time_key, day, day_of_week, month, quarter, year)define dimension item as (item_key, item_name, bra

19、nd, type, supplier_type)define dimension branch as (branch_key, branch_name, branch_type)define dimension location as (location_key, street, city, province_or_state, country)define cube shipping time, item, shipper, from_location, to_location:dollar_cost = sum(cost_in_dollars), unit_shipped = count(

20、*)define dimension time as time in cube salesdefine dimension item as item in cube salesdefine dimension shipper as (shipper_key, shipper_name, location as location in cube sales, shipper_type)define dimension from_location as location in cube salesdefine dimension to_location as location in cube sa

21、lesData Mining (Spring 2012), Tsinghua University16一個概念層級一個概念層級: 維度維度 一個概念層級定義了從一套更低級別的概念到更高、更一般的概念的映射一個概念層級定義了從一套更低級別的概念到更高、更一般的概念的映射序列。序列。類別類別:- 屬性的層級屬性的層級: 地點地點,省省,村村- 屬性值的層級或分組屬性值的層級或分組 對于一個給定的維度，或許會有不止一個概念層級對于一個給定的維度，或許會有不止一個概念層級.Data Mining (Spring 2012), Tsinghua University17一個概念層級一個概念層級: 維度維

22、度(地點地點)allEuropeNorth_AmericaMexicoCanadaSpainGermanyVancouverM. WindL. Chan.全部地區(qū)辦公室國家TorontoFrankfurt城市Data Mining (Spring 2012), Tsinghua University18多維度數(shù)據(jù)多維度數(shù)據(jù) 作為產(chǎn)品，月份和地區(qū)的一個函數(shù)的銷售量作為產(chǎn)品，月份和地區(qū)的一個函數(shù)的銷售量ProductRegionMonth維度維度:產(chǎn)品產(chǎn)品,地點地點,時間時間層級匯總路徑層級匯總路徑Industry Region YearCategory Country QuarterProduc

23、t City Month Week Office DayData Mining (Spring 2012), Tsinghua University19一個數(shù)據(jù)立方樣本一個數(shù)據(jù)立方樣本Total annual salesof TV in U.S.A.DateProductCountrysumsum TVVCRPC1Qtr2Qtr3Qtr4QtrU.S.ACanadaMexicosumData Mining (Spring 2012), Tsinghua University20Cuboids Corresponding to the Cubeallproductdatecountryprodu

24、ct,dateproduct,countrydate, countryproduct, date, country0-D(apex) cuboid1-D cuboids2-D cuboids3-D(base) cuboidData Mining (Spring 2012), Tsinghua University21數(shù)據(jù)倉庫和層級結(jié)構(gòu)的觀察數(shù)據(jù)倉庫和層級結(jié)構(gòu)的觀察 可視化可視化 OLAP功能功能 交互操作交互操作Data Mining (Spring 2012), Tsinghua University22典型的典型的OLAP 操作操作 上卷上卷(上鉆上鉆): 匯總數(shù)據(jù)匯總數(shù)據(jù)通過爬升到更高的

25、層級或者是減少維度通過爬升到更高的層級或者是減少維度 下鉆下鉆(下卷下卷): 與上卷相反與上卷相反從更高層級的匯總到更低層級的匯總或者使數(shù)據(jù)詳細(xì)化，或者引進(jìn)新的維度從更高層級的匯總到更低層級的匯總或者使數(shù)據(jù)詳細(xì)化，或者引進(jìn)新的維度 切片和切塊切片和切塊: 在一個或更多的維度上投射或選擇在一個或更多的維度上投射或選擇 旋轉(zhuǎn)旋轉(zhuǎn) (rotate): 重新定位立方體重新定位立方體, 可視化可視化, 3D到一系列的到一系列的2D平面平面 其他操作其他操作交叉探查交叉探查: 涉及不止一個事實表涉及不止一個事實表鉆取鉆取: 從立方體的最底層到它后端的相關(guān)表從立方體的最底層到它后端的相關(guān)表(用用SQL)Da

26、ta Mining (Spring 2012), Tsinghua University23典型的典型的OLAP 操作操作(1) Roll up 上卷上卷 (drill-up上鉆上鉆): 匯總數(shù)據(jù)匯總數(shù)據(jù)通過爬升到更高的層級通過爬升到更高的層級 (減少維度減少維度) roll-up on location from cities to countiesQ1Q2Q3Q4 Chicago New York Vancouver TV CD PC710 820 402471 605Q1Q2Q3Q4USACanada1181605 TV CD PCData Mining (Spring 2012), T

27、singhua University24典型的典型的OLAP 操作操作(2) Roll down下卷下卷 (Drill down下鉆下鉆): 與上卷相反與上卷相反從更高層級的匯總到更低層級的匯總或者使數(shù)據(jù)詳細(xì)化，或者引進(jìn)新的維度從更高層級的匯總到更低層級的匯總或者使數(shù)據(jù)詳細(xì)化，或者引進(jìn)新的維度 drill-down on time from quarters to monthsQ1Q2Q3Q4 Chicago New York Vancouver TV CD PC710 820 402471 605 TV CD PC New York Vancouver ChicagoJanFeb MarAprMayJunJulAugSepOctNovDec 102 150 150Data Mining (Spring 2012), Tsinghua University25典型的典型的OLAP 操作操作(3) Slice(切片切片) and dice