GB/T 44216-2024信息技術(shù)大數(shù)據(jù)批流融合計(jì)算技術(shù)要求

ICS

35.240

CCS

L67

中華人民共和國(guó)國(guó)家標(biāo)準(zhǔn)

GB/T44216—2024

信息技術(shù)大數(shù)據(jù)

批流融合計(jì)算技術(shù)要求

Informationtechnology—Bigdata—Technicalrequirementsforintegratedbatch

andstreamingcomputing

2024-07-24發(fā)布2025-02-01實(shí)施

國(guó)家市場(chǎng)監(jiān)督管理總局發(fā)布

國(guó)家標(biāo)準(zhǔn)化管理委員會(huì)

GB/T44216—2024

目次

前言

·····································································································

Ⅲ

引言

·····································································································

Ⅳ

1

范圍

··································································································

1

2

規(guī)范性引用文件

······················································································

1

3

術(shù)語(yǔ)和定義

···························································································

1

4

縮略語(yǔ)

································································································

2

5

系統(tǒng)架構(gòu)

······························································································

2

6

技術(shù)要求

······························································································

3

6.1

統(tǒng)一資源管理

····················································································

3

6.2

統(tǒng)一計(jì)算框架

····················································································

4

6.3

統(tǒng)一SQL接口

···················································································

4

6.4

統(tǒng)一API

·························································································

5

6.5

統(tǒng)一作業(yè)管理

····················································································

5

6.6

統(tǒng)一權(quán)限管理

····················································································

6

7

擴(kuò)展性要求

···························································································

6

8

兼容性要求

···························································································

7

9

性能指標(biāo)

······························································································

7

附錄A（資料性）

批流融合計(jì)算應(yīng)用場(chǎng)景

····························································

8

A.1

金融行業(yè)

························································································

8

A.2

智能制造行業(yè)

···················································································

8

A.3

物聯(lián)網(wǎng)

···························································································

8

A.4

航空航天

························································································

8

附錄B（資料性）

字符類型及操作中英文對(duì)照表

·····················································

9

參考文獻(xiàn)

································································································

11

Ⅰ

GB/T44216—2024

前言

本文件按照GB/T1.1—2020《標(biāo)準(zhǔn)化工作導(dǎo)則第1部分：標(biāo)準(zhǔn)化文件的結(jié)構(gòu)和起草規(guī)則》的規(guī)

定起草。

請(qǐng)注意本文件的某些內(nèi)容可能涉及專利。本文件的發(fā)布機(jī)構(gòu)不承擔(dān)識(shí)別專利的責(zé)任。

本文件由全國(guó)信息技術(shù)標(biāo)準(zhǔn)化技術(shù)委員會(huì)（SAC/TC28）提出并歸口。

本文件起草單位：阿里云計(jì)算有限公司、中國(guó)電子技術(shù)標(biāo)準(zhǔn)化研究院、浪潮電子信息產(chǎn)業(yè)股份有限

公司、華為技術(shù)有限公司、浙江中煙工業(yè)有限責(zé)任公司、螞蟻科技集團(tuán)股份有限公司、騰訊云計(jì)算

（北京）有限責(zé)任公司、杭州?？低晹?shù)字技術(shù)股份有限公司、北京易華錄信息技術(shù)股份有限公司、

上海寶信軟件股份有限公司、北京柏睿數(shù)據(jù)技術(shù)股份有限公司、山東浪潮數(shù)據(jù)庫(kù)技術(shù)有限公司、上海計(jì)

算機(jī)軟件技術(shù)開發(fā)中心、中國(guó)鐵道科學(xué)研究院集團(tuán)有限公司、中國(guó)電子系統(tǒng)技術(shù)有限公司、浪潮軟件科

技有限公司、中通服咨詢?cè)O(shè)計(jì)研究院有限公司、網(wǎng)易（杭州）網(wǎng)絡(luò)有限公司、中國(guó)南方電網(wǎng)有限責(zé)任公

司、北京百分點(diǎn)科技集團(tuán)股份有限公司、廣東電網(wǎng)有限責(zé)任公司、北京華勝天成科技股份有限公司、

浙江邦盛科技股份有限公司、中國(guó)石油化工集團(tuán)有限公司、普元信息技術(shù)股份有限公司、方正國(guó)際軟件

（北京）有限公司、浙江創(chuàng)鄰科技有限公司、四川發(fā)展數(shù)字金沙科技有限公司、北京庚頓數(shù)據(jù)科技有限

公司、陜西省信息化工程研究院、成都數(shù)據(jù)集團(tuán)股份有限公司、國(guó)能信息技術(shù)有限公司、成都數(shù)之聯(lián)科

技股份有限公司、云上貴州大數(shù)據(jù)產(chǎn)業(yè)發(fā)展有限公司、藥融云數(shù)字科技（成都）有限公司、浙江智臾科

技有限公司、深圳市中絲貿(mào)科技有限公司、深圳億維銳創(chuàng)科技股份有限公司、河南省新星科技有限公

司、浙江方信標(biāo)準(zhǔn)技術(shù)有限公司、海南電網(wǎng)有限責(zé)任公司、中國(guó)電信集團(tuán)財(cái)務(wù)有限公司、復(fù)旦大學(xué)、上海交通

大學(xué)。

本文件主要起草人：朱松、陳守元、許潔、王峰、吳濤、劉健、楊銳、金泳、郭智慧、劉海濤、

蔣楠、黃超、陳小龍、張延生、王凌、黃明、趙菁華、樸晟宏、高陽(yáng)、張黎明、陳敏剛、劉陳宇、

吳艷華、劉國(guó)棟、楊旭、王金超、張永良、馬進(jìn)、王宇靜、陳彬、楊秋勇、錢正浩、梁盈威、孫偉、

梁鋼、王新根、臧一超、張煜、周研、劉宇峰、王晉暉、張勇、李傲鐵、李正、段智琛、傅彥、

周俊臨、黃明峰、劉軍、王軍剛、張暉、李明、王中健、周小華、丘琳、許宏安、皮志新、董衛(wèi)魏、

郭威、張磊、馮曹沖、陳勇錦、王毅、李爽、鄭忠斌。

Ⅲ

GB/T44216—2024

引言

隨著數(shù)據(jù)量的增長(zhǎng)，分布式計(jì)算模式逐漸成為大數(shù)據(jù)處理和計(jì)算的主流架構(gòu)。為了滿足程序的健壯

性和實(shí)時(shí)性要求，設(shè)計(jì)出了許多分布式計(jì)算框架，以屏蔽底層復(fù)雜的任務(wù)劃分和集群調(diào)度細(xì)節(jié)，其中最

常用的兩種是批計(jì)算技術(shù)和流式計(jì)算技術(shù)，二者有著迥異的編程模型和編程接口，適用于不同的計(jì)算場(chǎng)

景。批計(jì)算技術(shù)適合需訪問全套記錄才能完成的計(jì)算工作，流計(jì)算技術(shù)很適合用來(lái)處理需對(duì)變動(dòng)或峰值

做出響應(yīng)，并且關(guān)注一段時(shí)間內(nèi)變化趨勢(shì)的數(shù)據(jù)。

在實(shí)際應(yīng)用中，經(jīng)常會(huì)遇到兩種計(jì)算技術(shù)共同工作的情況。將兩種計(jì)算框架進(jìn)行簡(jiǎn)單的疊加，則需

要在兩個(gè)不同的引擎上實(shí)現(xiàn)相同的執(zhí)行邏輯，還需要手工合并不同引擎的輸出結(jié)果。如果需要更改查詢

邏輯，兩個(gè)系統(tǒng)也需要同時(shí)進(jìn)行改動(dòng)。這會(huì)極大地增加工程的開發(fā)和維護(hù)成本。因此，統(tǒng)一的批流融合

計(jì)算技術(shù)成為了大數(shù)據(jù)領(lǐng)域的重要發(fā)展趨勢(shì)。

Ⅳ

GB/T44216—2024

2信息技術(shù)大數(shù)據(jù)

批流融合計(jì)算技術(shù)要求

1范圍

本文件規(guī)定了大數(shù)據(jù)批流融合計(jì)算技術(shù)要求，包括：技術(shù)要求、擴(kuò)展性要求、兼容性要求、性能指

標(biāo)等。

本文件適用于批流融合計(jì)算系統(tǒng)的設(shè)計(jì)、開發(fā)和部署。用戶理解、采用、建設(shè)批流融合計(jì)算技術(shù)和

批流融合產(chǎn)品與服務(wù)評(píng)估參照?qǐng)?zhí)行。

2規(guī)范性引用文件

下列文件中的內(nèi)容通過文中的規(guī)范性引用而構(gòu)成本文件必不可少的條款。其中，注日期的引用文

件，僅該日期對(duì)應(yīng)的版本適用于本文件；不注日期的引用文件，其最新版本（包括所有的修改單）適用

于本文件。

GB/T35295—2017信息技術(shù)大數(shù)據(jù)術(shù)語(yǔ)

3術(shù)語(yǔ)和定義

GB/T35295—2017界定