元數(shù)據(jù)管理-實(shí)用

1、元數(shù)據(jù)管理冷鵬 （2006年編寫(xiě)）2n數(shù)據(jù)集成技術(shù)數(shù)據(jù)集成技術(shù)n 數(shù)據(jù)展現(xiàn)分析數(shù)據(jù)展現(xiàn)分析n 數(shù)據(jù)倉(cāng)庫(kù)建模數(shù)據(jù)倉(cāng)庫(kù)建模n 廠商解決方案廠商解決方案n 談元數(shù)據(jù)管理談元數(shù)據(jù)管理n 數(shù)據(jù)倉(cāng)庫(kù)基礎(chǔ)數(shù)據(jù)倉(cāng)庫(kù)基礎(chǔ)n 項(xiàng)目開(kāi)發(fā)管理項(xiàng)目開(kāi)發(fā)管理3元數(shù)據(jù)管理工具元數(shù)據(jù)管理工具n Informatica MetaManagern DAG MetaCentern IBM MetaStagen Oracle OWBn CA Decisionbase Transformern 4元數(shù)據(jù)元數(shù)據(jù)MDC的開(kāi)放信息模型的開(kāi)放信息模型分析與設(shè)計(jì)（Analysis and Design） 對(duì)象與組件（Object and Com

2、ponent）數(shù)據(jù)庫(kù)與數(shù)據(jù)倉(cāng)庫(kù)（Database and Warehousing） 業(yè)務(wù)工程（Business Engineering） 知識(shí)管理（Knowledge Management） OMG組織的組織的CWM模型模型UML MOF（元對(duì)象設(shè)施）XMI（XML元數(shù)據(jù)交換）5元數(shù)據(jù)的重要性元數(shù)據(jù)的重要性及時(shí)發(fā)現(xiàn)系統(tǒng)變化及時(shí)發(fā)現(xiàn)系統(tǒng)變化找出需要調(diào)整的找出需要調(diào)整的ETL程序程序給出倉(cāng)庫(kù)中數(shù)據(jù)和指標(biāo)的來(lái)源及規(guī)則給出倉(cāng)庫(kù)中數(shù)據(jù)和指標(biāo)的來(lái)源及規(guī)則調(diào)整倉(cāng)庫(kù)的數(shù)據(jù)模型調(diào)整倉(cāng)庫(kù)的數(shù)據(jù)模型促進(jìn)項(xiàng)目組成員的溝通促進(jìn)項(xiàng)目組成員的溝通監(jiān)控?cái)?shù)據(jù)質(zhì)量監(jiān)控?cái)?shù)據(jù)質(zhì)量采用元數(shù)據(jù)跟用戶確認(rèn)新的需求采用元數(shù)據(jù)跟用戶確認(rèn)新的需

3、求6技術(shù)元數(shù)據(jù)技術(shù)元數(shù)據(jù)7業(yè)務(wù)元數(shù)據(jù)業(yè)務(wù)元數(shù)據(jù)8業(yè)務(wù)專家業(yè)務(wù)專家元數(shù)據(jù)元數(shù)據(jù)開(kāi)發(fā)人員開(kāi)發(fā)人員項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰(shuí)需要元數(shù)據(jù)誰(shuí)需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理分析人員分析人員業(yè)務(wù)定義業(yè)務(wù)定義如：某銷售指標(biāo)不正確，找出銷售指標(biāo)的計(jì)算方法及計(jì)算項(xiàng)來(lái)源業(yè)務(wù)專家業(yè)務(wù)專家9分析人員分析人員開(kāi)發(fā)人員開(kāi)發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰(shuí)需要元數(shù)據(jù)誰(shuí)需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理元數(shù)據(jù)元數(shù)據(jù)需求分析定義需求分析定義如：分析指標(biāo)定義、報(bào)表如：分析指標(biāo)定義、報(bào)表定義、數(shù)據(jù)來(lái)源定義定義、數(shù)據(jù)來(lái)源定義10開(kāi)發(fā)人員開(kāi)發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰(shuí)需要元數(shù)據(jù)誰(shuí)需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)

4、理元數(shù)據(jù)元數(shù)據(jù)不同開(kāi)發(fā)小組之間的交流不同開(kāi)發(fā)小組之間的交流如：DBA工程師與ETL工程師 DBA工程師與BI工程師 . 分析人員分析人員11元數(shù)據(jù)元數(shù)據(jù)開(kāi)發(fā)人員開(kāi)發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰(shuí)需要元數(shù)據(jù)誰(shuí)需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理業(yè)務(wù)管理與信息消費(fèi)業(yè)務(wù)管理與信息消費(fèi)如：數(shù)據(jù)分析、業(yè)務(wù)決策、給前系統(tǒng)提出改進(jìn)建議，對(duì)數(shù)據(jù)倉(cāng)庫(kù)系統(tǒng)二期、三期提出更多的業(yè)務(wù)需求分析人員分析人員12開(kāi)發(fā)人員開(kāi)發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰(shuí)需要元數(shù)據(jù)誰(shuí)需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理元數(shù)據(jù)元數(shù)據(jù)數(shù)據(jù)倉(cāng)庫(kù)項(xiàng)目管理控制數(shù)據(jù)倉(cāng)庫(kù)項(xiàng)目管理控制如：定義項(xiàng)目范圍、控制項(xiàng)目風(fēng)險(xiǎn)、評(píng)估不同系統(tǒng)集成時(shí)產(chǎn)

5、生的影響（影響分析）分析人員分析人員13元數(shù)據(jù)元數(shù)據(jù)開(kāi)發(fā)人員開(kāi)發(fā)人員業(yè)務(wù)人員業(yè)務(wù)人員項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰(shuí)需要元數(shù)據(jù)誰(shuí)需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理項(xiàng)目架構(gòu)設(shè)計(jì)項(xiàng)目架構(gòu)設(shè)計(jì)如：了解多個(gè)業(yè)務(wù)系統(tǒng)的結(jié)構(gòu)、關(guān)聯(lián)關(guān)系，數(shù)據(jù)轉(zhuǎn)換規(guī)則設(shè)計(jì).分析人員分析人員14元數(shù)據(jù)的存儲(chǔ)區(qū)元數(shù)據(jù)的存儲(chǔ)區(qū)15數(shù)據(jù)倉(cāng)庫(kù)元數(shù)據(jù)結(jié)構(gòu)圖數(shù)據(jù)倉(cāng)庫(kù)元數(shù)據(jù)結(jié)構(gòu)圖16元數(shù)據(jù)管理示意圖元數(shù)據(jù)管理示意圖源數(shù)據(jù)源數(shù)據(jù)EDW（ODS+DW+DM）前端工具前端工具connections暫存區(qū)暫存區(qū)元數(shù)據(jù)元數(shù)據(jù)數(shù)據(jù)庫(kù)數(shù)據(jù)庫(kù)17Informatica Metadata MangerMetadata Object Browser and Graphi

6、cal ReportsPowerCenter Integration ServerSuperGlue WarehouseMetadata SourcesSG Integration RepositoryCWM Class MetaModelPresentation ServerMetadata XConnectsSG ConsoleRole based dashboards and Metadata ReportsWin NT, 2000, XPWarehouse- Oracle 8.1.7, 9.2.0.1 and above, 10g- SQL Server 2000 SP3- DB2 U

7、DB 8.x EE/EEEIntegration Repo- Oracle 8.1.7, 9.2.0.1 and above, 10g- SQL Server 2000 SP3- DB2 UDB 7.x EE/EEE and aboveREPOSITORIES18PowerCenter Integration ServerInformatica Metadata ManagerMetadata Object Browser and Graphical ReportsSuperGlue WarehouseSG Integration RepositoryCWM Class MetaModelPr

8、esentation ServerMetadata XConnectsSG ConsoleRole based dashboards and Metadata ReportsWin NT, 2000, XPSERVERSPC 7.1.1 Integration ServerPC 7.1.1 on : NT, SOLARIS HP Unix, IBM AIX(PC supported versions)PA 4.1.1 Apps ServerBrowser IE 6.0 using : iPlanet 6.0, IIS 6.0 OR Apache 1.3BEA WL 6.1 SP4 on :-

9、NT 4.0 SP5, NT 2000 SP2, Win 2003- SOLARIS 7.0, 8.0- HP Unix 11.0, 11i- IBM AIX 4.3, 5.1Websphere 5.1 on :- NT 4.0 SP5, NT 2000 SP2, Win 2003- SOLARIS 7.0, 8.0- IBM AIX 4.3, 5.1Metadata Sources19Informatica Metadata ManagerMetadata Object Browser and Graphical ReportsPowerCenter Integration ServerSu

10、perGlue WarehouseSG Integration RepositoryCWM Class MetaModelPresentation ServerMetadata XConnectsSG ConsoleRole based dashboards and Metadata ReportsWin NT, 2000, XPETL- PowerCenter 5.x, 6.x, 7.x*- PowerMart 5.x, 6.x, 7.x*BI- PowerAnalyzer 4.x*- BOBJ 5.0, 5.1.x, 6.x- COGN Impromptu 7.x- COGN Report

11、Net 1.x- MSTR Designer 7.x- DB2 CubeViewsXCONNECTS* (on Oracle 8.1.7 and above, SQL Server 2000 SP3 and DB2 7.x EE/EEE and above)Databases- Oracle 8.1.7, 9i, 10g- SQL Server 2000- Sybase 12.x- TERADATA V2 R5 - DB2 UDB 7.1,7.2,8.1 EE/EEE- IBM Informix 9.2Design Tools- ERwin 3.0 to 3.5.2, 4.0 SP1 to 4

12、.1- Oracle Designer 1.3.2, 2.1.2, 6.0, 6i, 9i - Sybase PowerDesigner PDM 6.1.x, 7.5 to 9.5- IBM Rational ER 98(I) to 2000,2000e to 2002- Microsoft VISIO Database (ERX)- Embarcadero ERStudio 5.1 to 6.0 Metadata Sources20Source definition 源系統(tǒng)地址源系統(tǒng)地址, 網(wǎng)絡(luò)連接網(wǎng)絡(luò)連接, 訪問(wèn)方式訪問(wèn)方式Location, network connection, acc

13、ess methods計(jì)算機(jī)系統(tǒng)計(jì)算機(jī)系統(tǒng), 操作系統(tǒng)操作系統(tǒng)Computer platform definition, operating system definition數(shù)據(jù)庫(kù)服務(wù)器名數(shù)據(jù)庫(kù)服務(wù)器名,訪問(wèn)方式訪問(wèn)方式,數(shù)據(jù)結(jié)構(gòu)數(shù)據(jù)結(jié)構(gòu),有關(guān)的表有關(guān)的表,列列Database server name, access methods, schema, table, elements of interests源數(shù)據(jù)庫(kù)鏈接說(shuō)明源數(shù)據(jù)庫(kù)鏈接說(shuō)明 Source database links逆工程與源系統(tǒng)設(shè)計(jì)獲取定義說(shuō)明逆工程與源系統(tǒng)設(shè)計(jì)獲取定義說(shuō)明Reverse engineering and so

14、urce design capture specification元數(shù)據(jù)交換說(shuō)明元數(shù)據(jù)交換說(shuō)明Metadata exchange definition 21Staging area definition 源至?xí)捍鎱^(qū)映像（Source-staging mapping）暫存區(qū)至目標(biāo)區(qū)映像Staging-target mapping中間文件的傳輸說(shuō)明File transport specification中間文件的說(shuō)明及使用File specification and usage中間文件的前后處理(清理與壓縮)說(shuō)明File pre/post event processing specificatio

15、n暫存區(qū)數(shù)據(jù)庫(kù)的前后處理及中間處理說(shuō)明及使用Staging database usage and pre/post process specification 22Target modeling definition 星結(jié)構(gòu)Star schema事實(shí)表定義說(shuō)明Fact table definition度量Measurement數(shù)字鍵Numeric key交替鍵Foreign key索引Index definition定義數(shù)據(jù)（如數(shù)據(jù)類型，數(shù)值大小和省缺值）有效性制約Validation constraint數(shù)據(jù)域值定義Data range validation分區(qū)定義說(shuō)明Partition s

16、pecification存儲(chǔ)說(shuō)明Storage specification維表定義與說(shuō)明Dimension table definition主鍵Primary key緩慢變化的維Slowly changed dimension維的等級(jí)/層次Dimension hierarchy維的共享Dimension sharing維的版號(hào)化Dimension versioning數(shù)據(jù)庫(kù)完整性限定Database integrity constraints23Target modeling definition 維表定義與說(shuō)明Dimension table definition主鍵Primary key緩慢

17、變化的維Slowly changed dimension維的等級(jí)/層次Dimension hierarchy維的共享Dimension sharing維的版號(hào)化Dimension versioning數(shù)據(jù)庫(kù)完整性限定Database integrity constraints24Mapping definition 源至目標(biāo)映像Source-target mapping基本數(shù)據(jù)到綜合數(shù)據(jù)的映像Basic data-summary mapping標(biāo)準(zhǔn)字典Standard dictionary 轉(zhuǎn)換字典Conversion dictionary參照表Lookup tables映像表Mapping

18、tables 25ETL definition 轉(zhuǎn)換規(guī)則和方法Conversion rule and method數(shù)據(jù)清洗說(shuō)明Data cleaning specifications批作業(yè)說(shuō)明Batch job specification加載頻率Loading frequency累加時(shí)間片控制說(shuō)明Incremental time slice control specification回退說(shuō)明與準(zhǔn)則Rollback specification and policy剝離與修改數(shù)據(jù)準(zhǔn)則Strip and modification in bound data policy 轉(zhuǎn)換函數(shù)庫(kù)Conversio

19、n function library傳輸定義與說(shuō)明Transportation definition加載定義與說(shuō)明Loading definition26ETL definition 會(huì)期/對(duì)話管理定義與說(shuō)明Session management definition預(yù)定作業(yè)排時(shí)Scheduling批作業(yè)前后事件說(shuō)明及處理方法Pre-batch and post-batch job events批作業(yè)日志Batch logging會(huì)期/對(duì)話日志Session log拒收日志Rejected log壞數(shù)據(jù)日志Bad data log 27Summary management definition

20、事實(shí)表/維表至總計(jì)方陣映像Fact/dimension-summary cube mapping總計(jì)至總計(jì)映像Summary-summary mapping總計(jì)方陣設(shè)計(jì)Summary cube layout聚合定義說(shuō)明Aggregate definition實(shí)體化視圖/方陣定義說(shuō)明Materialized view/cube definition方陣間關(guān)系Relationships between cubes嵌入式實(shí)體化視圖/方陣之間的依賴性Dependency of nested materialized views/cubes方陣組 Grouping of cubes28Summary m

21、anagement definition 刷新過(guò)程的定義與說(shuō)明Refreshing definition刷新前后事件處理說(shuō)明Pre-refreshing and post refreshing events刷新方法Refreshing methods刷新組Refreshing groups刷新執(zhí)行順序Refreshing sequence刷新頻率Refreshing frequency刷新過(guò)程處理軟件Refreshing procedures刷新作業(yè)的監(jiān)控檢查Refreshing monitoring29OLAP definition 查詢機(jī)制Query engine虛擬方陣/視圖定義Virtual cube/view definition訪問(wèn)權(quán)限管理說(shuō)明Privilege management specification用戶接口定義與說(shuō)明User interface definition訪問(wèn)方式Access method結(jié)果數(shù)據(jù)提交方法Delivery method and definition結(jié)果數(shù)據(jù)傳輸定義與說(shuō)明Result set transportation definition用戶登入/登出User log in/log off用戶活動(dòng)日志以及出錯(cuò)處理User logging and error han