互聯(lián)網(wǎng)金融數(shù)據(jù)平臺架構(gòu)

1、互聯(lián)網(wǎng)金融數(shù)據(jù)平臺架構(gòu)技術(shù)創(chuàng)新，變革未來智慧IT2017Dianrong History201320122011201420152016Founded in Shanghai, China with technologyplatform from Lending ClubWe are Six years oldVC investment from Northern Light Venture Capital, Tiger Global, Standard Chartered Private Equity, China Internet Fintech Fund, Bohai Financial

2、, and Max GiantOver 2,600 employees and 28 offices across ChinaLaunched Chinas First Block chain platformwith FoxconnAs of end of Apr 2017, total lender volume reached 29 billion RMB, accumulated interest for lenders exceeded 1 billion RMB.數(shù)據(jù)庫設(shè)計(jì)目標(biāo)是事務(wù)處理，OLTP關(guān)系模型，面向應(yīng)用數(shù)據(jù)倉庫設(shè)計(jì)目標(biāo)是決策支持，OLAP多維模型，面向主題，反映歷史變化

3、數(shù)據(jù)湖泊存儲結(jié)構(gòu)化的、半結(jié)構(gòu)化的原始數(shù)據(jù)-應(yīng)用數(shù)據(jù)庫應(yīng)用日志流NoSQL數(shù)據(jù)庫機(jī) 器 學(xué) 習(xí) 平 臺任 務(wù) 調(diào) 度 平 臺數(shù)據(jù)湖泊數(shù)據(jù)集市 數(shù)據(jù)中間層HDFS (SparkSQL + Hive)數(shù)據(jù)原始層外部文本文件數(shù)據(jù)查詢引擎數(shù)據(jù)可視化平臺數(shù)據(jù)產(chǎn)品對外數(shù)據(jù)接口OLAP數(shù)據(jù)ELT平臺數(shù)據(jù)抽取轉(zhuǎn)換：Sqoop + Hive / SparkSQL作業(yè)調(diào)度： 定制化 Azkaban數(shù)據(jù)查詢平臺定制化 Spark Thrift Server + ZeppelinHue數(shù)據(jù)可視化展現(xiàn)平臺：TableauPentaho實(shí)時(shí)數(shù)據(jù)處理：Kafka + Spark Streaming機(jī)器學(xué)習(xí)與建模分析：Spark MLlib / GraphxOLAP引擎：待實(shí)施數(shù)據(jù)治理元數(shù)據(jù)管理數(shù)據(jù)重要性分級表級別血緣關(guān)系管理列級別血緣關(guān)系管理數(shù)據(jù)質(zhì)量告警數(shù)據(jù)SLA告警數(shù)據(jù)增量閾值告警數(shù)據(jù)脫敏用戶PII數(shù)據(jù)脫敏數(shù)據(jù)權(quán)限分級根據(jù)數(shù)據(jù)敏感級別進(jìn)行分級數(shù)