DataStage基礎(chǔ)培訓(xùn)教程

1、DataStageDataStage基礎(chǔ)培訓(xùn)基礎(chǔ)培訓(xùn) JerryJerry2議程議程 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAd

2、ministrator & Manager Routine & ControlRoutine & Control3演示：演示：Hello WorldExtractTransformLoan4 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameters & TablesDefine Parameters & Tables Hash FileHash File、TransformerTransformer、AggregatorAggregator Direc

3、tor & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control議程議程5DataStage ArchitectureTarget(Database or File)ODBC/NativeDataStage Connect APIDataStage Server (WinNT, Win2000 or UNIX)ODBC/NativeData Sources (Database or Fi

4、le)DataStage Connect APIDataStage Connect APIDataStage Connect APIData flowData flow6DataStage ComponentsManager DesignerDirectorMetadata collection and management Design process flowRun jobs, check logs and set schedulesDataStageAdministratorCreate, Edit projects7 Hello WorldHello World DataStageDa

5、taStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control議程議程8全局變量

6、與全局變量與Job變量變量 全局變量全局變量- - 生命周期：整個(gè)生命周期：整個(gè)ProjectProject- - 在在AdministratorAdministrator中定義中定義 JobJob變量變量- - 生命周期：一個(gè)生命周期：一個(gè)JobJob- - 在在DesignerDesigner、ManagerManager中定義中定義9演示：定義一個(gè)演示：定義一個(gè)Job變量變量在在DesignerDesigner中定義參數(shù)中定義參數(shù)10Meta data definition 元數(shù)據(jù)管理的重要組成部分元數(shù)據(jù)管理的重要組成部分 在在MangerManger或或DesignDesign中中定義

7、定義 演示：演示：- import from a flat file in .txt format- import from a flat file in .txt format- import from an DBMS table- import from an DBMS table11演示演示Table Definition在在ManageManage中定義中定義TableTable12 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Para

8、meter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control議程議程13演示：生成事實(shí)表演示：生成事實(shí)表明細(xì)表事實(shí)表關(guān)聯(lián)聚合14Hash File 用途：用途：- - 左連接時(shí)用作副表左連接時(shí)用作副表- - 多次被

9、訪問(wèn)的數(shù)據(jù)集多次被訪問(wèn)的數(shù)據(jù)集- - 存儲(chǔ)其他臨時(shí)數(shù)據(jù)存儲(chǔ)其他臨時(shí)數(shù)據(jù) 關(guān)鍵點(diǎn)：關(guān)鍵點(diǎn)：- - 必須指定必須指定keykey- output- output的的positionposition必須與必須與inputinput一致一致15Transformer 用途：用途：- - 提供豐富的運(yùn)算符和函數(shù)提供豐富的運(yùn)算符和函數(shù)- - 數(shù)據(jù)清洗、轉(zhuǎn)換數(shù)據(jù)清洗、轉(zhuǎn)換- - 關(guān)聯(lián)多個(gè)數(shù)據(jù)源關(guān)聯(lián)多個(gè)數(shù)據(jù)源 關(guān)鍵點(diǎn)：關(guān)鍵點(diǎn)：- - 副表的副表的keykey必須被主表的某個(gè)字段關(guān)聯(lián)必須被主表的某個(gè)字段關(guān)聯(lián)- - 盡量避免兩個(gè)盡量避免兩個(gè)TransformerTransformer直接相連直接相連16Aggre

10、gator 用途：用途：- - Sum, Max, Min, AverageSum, Max, Min, Average等聚合函數(shù)等聚合函數(shù)- - 一般用于生成事實(shí)表一般用于生成事實(shí)表17 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & Monitor

11、Director & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control議程議程18Debug and TuningView Status and LogsView Status and Logs- status, log, detail- status, log, detail等等多種視圖多種視圖- - 配合配合MonitorMonitor來(lái)查錯(cuò)、來(lái)查錯(cuò)、調(diào)優(yōu)調(diào)優(yōu)19Job Status Not CompiledNot Com

12、piled CompiledCompiled ResetReset RunningRunning FinishedFinished Finished (with warning)Finished (with warning) AbortAbort20ScheduleJob Job Add to ScheduleAdd to Schedule21 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash F

13、ileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control議程議程22Administrator Add a new projectAdd a new project Modify project propertiesModify project proper

14、ties- - 字符集字符集- - 日志保留天數(shù)日志保留天數(shù)- hash file and write catch - hash file and write catch Define environment viableDefine environment viable23Manager Import and export projects or jobsImport and export projects or jobs- - 兩種文件格式：兩種文件格式：.dsx .xml.dsx .xml- - 整個(gè)整個(gè)project, project, 根據(jù)根據(jù)categorycategory Tab

15、le definitionTable definition Manage RoutineManage Routine24演示：備份演示：備份project25 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control議程議程26Routine 一種自定義函數(shù)，使用一種自定義函數(shù)，使用VBVB