2024寒武紀(jì)TensorFlow用戶(hù)手冊(cè)

版本2021年05月06 插圖目 表格目 2.1版本記 CambriconTensorFlow架構(gòu)簡(jiǎn) 原生模式 融合模式 暫不支持的TensorFlow模 TensorFlow 使用 使用 安裝 安裝寒武紀(jì)Neuware依賴(lài) 獲取CambriconTensorFlow源代 使用 獲取MLU設(shè)備個(gè) 獲取詳細(xì)的MLU相關(guān)信 MLU內(nèi)存使用策 在多MLU系統(tǒng)中使用單 使用多MLU設(shè) MLU設(shè)備可見(jiàn) 使用 基于LowAPI的CambriconTensorFlow教 基于Keras的CambriconTensorFlow教 Horovod分布 增加自定義MLU算 MLU內(nèi)存越界檢 tensordump功能以及誤差計(jì)算工 TensorDump功

插圖目錄插圖目錄

CambriconTensorFlow在寒武紀(jì)軟件棧中的位

CambriconTensorFlow架 表格目錄表格目錄

11（（或保證本文件所且寒武紀(jì)不承擔(dān)因應(yīng)用或使用任何產(chǎn)品或服務(wù)而產(chǎn)生的任何責(zé)任。寒武紀(jì)不應(yīng)對(duì)因下列原因產(chǎn)生的任何12客戶(hù)產(chǎn)品設(shè)計(jì)。本指南列出的性能測(cè)試和等級(jí)要使用特定芯片或計(jì)算機(jī)系統(tǒng)或組件來(lái)測(cè)量。經(jīng)該等測(cè)試，本指南所示結(jié)果反映了寒武紀(jì)產(chǎn)品的大概性能。系統(tǒng)硬件或軟件設(shè)計(jì)或配置的任何不同會(huì)影響實(shí)際性能。如上所述，寒武紀(jì)不代表、擔(dān)保或保證本指南所述產(chǎn)品將適用于任何特定用途。寒武紀(jì)不代表或擔(dān)保測(cè)試每種產(chǎn)品以期避免應(yīng)用程序或產(chǎn)品的默認(rèn)情況?？蛻?hù)產(chǎn)品設(shè)計(jì)的脆弱性會(huì)影響寒武紀(jì)產(chǎn)品的質(zhì)量和可靠性并導(dǎo)致超出本指南范圍的額外或不同的情況和/或要求。 ?2021222.1:寒武紀(jì)TensorFlowV20210506更新時(shí)間：20210201移除融合模式相關(guān)內(nèi)容。融合模式的使用說(shuō)明，參見(jiàn)《寒武紀(jì)TensorFlow增加快速入門(mén)33TensorFlow是一款基于數(shù)據(jù)流的編程框架，廣泛應(yīng)用于各類(lèi)機(jī)器學(xué)習(xí)、深度神經(jīng)網(wǎng)絡(luò)，具有很高的可為支持寒武紀(jì)MLU（MachineLearningUnit，機(jī)器學(xué)習(xí)處理器），寒武紀(jì)擴(kuò)展了開(kāi)源深度學(xué)習(xí)編程框架TensorFlow（以下簡(jiǎn)稱(chēng)CambriconTensorFlow），在原生TensorFlow中增加對(duì)MLU的支持，實(shí)現(xiàn)已有TensorFlow模型在MLU關(guān)于原生TensorFlow的更多內(nèi)容，參見(jiàn)TensorFlow官網(wǎng)相關(guān)文檔資料1。本手冊(cè)主要介紹基于寒武紀(jì)MLU的CambriconTensorFlow目前CambriconTensorFlowTensorFlowv1.15.4寒武紀(jì)軟件棧具有端云一體的特點(diǎn)，既適用于終端智能芯片，也適用于云端智能板卡。CambriconTensorFlow是支持寒武紀(jì)MLU習(xí)編程框架，在寒武紀(jì)軟件棧所處位置如下圖所示。1\h.CAMBRICONTENSORFLOW3.1:CambriconTensorFlow基于原生TensorFlow，CambriconTensorFlow集成了寒武紀(jì)軟件棧、擴(kuò)展了TensorFlow設(shè)備層，并增加了對(duì)寒武紀(jì)MLU的支持。CambriconTensorFlow屏蔽硬件的相關(guān)屬性，允許用戶(hù)使用原生TensorFlowAPI進(jìn)行開(kāi)發(fā)，并且適配了部分KerasAPI。CambriconTensorFlow3.2:CambriconTensorFlow以下對(duì)CambriconTensorFlowTensorFlow的graphexecutorTensorFlowCPU、GPU該模式下，計(jì)算圖經(jīng)過(guò)圖優(yōu)化后，graphexecutorTensorFlow軟件棧層面，用戶(hù)可以通過(guò)CNNL（CambriconNeuwareNeuralNetworkLibrary，寒武紀(jì)神經(jīng)網(wǎng)絡(luò)計(jì)算庫(kù)、CNPlugin（CambriconNeuwarePlugin，寒武紀(jì)自定義算子庫(kù)）以及CNCC（CambriconNeuwareCompilerCollection，寒武紀(jì)BANGC語(yǔ)言編譯器）接入寒武紀(jì)深度優(yōu)化的通用算子或者靈關(guān)于TensorFlow算子的定義，參見(jiàn)Createanop2用戶(hù)也可以通過(guò)寒武紀(jì)BANG語(yǔ)言靈活添加自定義算子。更多內(nèi)容，參見(jiàn)添加自定義算子2\h.暫不支持的TENSORFLOWCambriconTensorFlow內(nèi)建了基于CNML（CambriconNeuwareMachineLearningLibrary，寒武紀(jì)機(jī)器學(xué)習(xí)庫(kù)）融合模式。融合模式下，CambriconTensorFlow會(huì)對(duì)用戶(hù)的計(jì)算圖進(jìn)行全圖優(yōu)化，將多個(gè)小算子融合成一個(gè)大算子，消除不必要的中間結(jié)果，生成更高效率、更具硬件親和性的MLU模式的更多內(nèi)容，參見(jiàn)《寒武紀(jì)TensorFlowexport目前Cambriconexport關(guān)于運(yùn)行融合模式的更多內(nèi)容，參見(jiàn)《寒武紀(jì)TensorFlow暫不支持的TensorFlow（just?in?time）編譯機(jī)制、AOT（ahead?of?time）有關(guān)TensorFlowLite的信息，參見(jiàn)TensorFlow33\h44CambriconTensorflow運(yùn)行時(shí)依賴(lài)CNToolkit以及計(jì)算加速庫(kù)CNML、CNNL、CNCLCNPlugin本章節(jié)主要介紹如何安裝CambriconTensorFlow。有關(guān)如何安裝CambriconNeuwareSDK您也可以使用CambrionensorFlw發(fā)布的Docer容器，其中已經(jīng)安裝了CambrionensorFlw所依賴(lài)的組件。使用CambrionensorFlw發(fā)布包中包含使用manylinux構(gòu)建的wheel包。用戶(hù)可以選擇對(duì)應(yīng)ython本的wheel包進(jìn)行安裝。下面以Python2virtualenvvirtualenv--system-site-packages-ppython2.7sourcesourcepythonpython-c"importtensorflowastf;.使用使用在使用已安裝CambriconTensorFlow運(yùn)行環(huán)境的Docker在主機(jī)上安裝MLU驅(qū)動(dòng)。關(guān)于MLU安裝Docker。關(guān)于DockerDocker4導(dǎo)入TensorFlowDocker運(yùn)行以下命令將TensorFlowdockerimage倉(cāng)庫(kù)。根據(jù)實(shí)際需求載入develimagereleaseimage。本節(jié)以發(fā)布的Ubuntu16.04##導(dǎo)入devel#導(dǎo)入release導(dǎo)入完成后，可執(zhí)行dockerimagels啟用CambriconTensorFlowCambriconTensorFlow鏡像已預(yù)裝CNToolkit、Python2.7或Python3.7通過(guò)以下命令啟動(dòng)TensorFlowdockerdockerrun[-it][--rm]--privileged[-phostPort:containerPort][-v?→TENSORFLOW_IMAGE_NAME:TAGdockerdockerrun-it--privileged-v/path/to/your/models:/path/to/your/models-v關(guān)于DockerDocker5使用releaseimage或develimage使用releaseimage檢測(cè)TensorFlowpythonpython-c"importtensorflowastf; 2)也可參見(jiàn)快速入門(mén)章節(jié)，在Docker內(nèi)運(yùn)行demo，驗(yàn)證TensorFlow使用develimage編譯安裝TensorFlowcdcd##編譯成功后自動(dòng)在當(dāng)前目錄創(chuàng)建虛擬環(huán)境virtualenv_mlu并安裝sourcepython-c"importtensorflowastf;從源碼編譯TensorFlow，首先要滿(mǎn)足TensorFlow社區(qū)對(duì)開(kāi)發(fā)環(huán)境的要求，具體參見(jiàn)從源代碼構(gòu)建6您可以使用CambriconTensorFlow發(fā)布的帶有devel標(biāo)識(shí)的開(kāi)發(fā)版dockerdocker器中，編譯所依賴(lài)的構(gòu)建工具、工具鏈以及依賴(lài)庫(kù)已經(jīng)安裝就緒?？梢灾苯舆M(jìn)入TensorFlow源碼目錄編譯TensorFlow源碼要求GCC7.3。推薦使用Ubuntu18（安裝TensorFlow使用Bazel##下載0.24.1版本Bazelwget/bazelbuild/bazel/releases/download/0.24.1/bazel-0.24.1-installer-bashbazel-0.24.1-installer-linux-x86_64.sh--安裝完成后，可執(zhí)行以下命令驗(yàn)證Bazel6\hbazelbazelBuildBuildlabel:Buildtarget:bazel-out/k8-Buildtime:TueApr216:29:262019Buildtimestamp:Buildtimestampasint:安裝以下寒武紀(jì)NeuwareCNToolkit（需要先安裝CNMLCNNLCNCLCNPlugin。具體安裝步驟，參見(jiàn)《寒武紀(jì)CNToolkit包版本要求，參見(jiàn)源碼包的cambricon_tensorflow/tensorflow/perty文件。獲取發(fā)布的CambriconTensorFlow源碼包并解壓。tartar-zxvf編譯過(guò)程中編譯器會(huì)到系統(tǒng)默認(rèn)路徑搜索寒武紀(jì)Neuware軟件包的頭文件和動(dòng)態(tài)鏈接庫(kù)。如果您的寒武紀(jì)Neuware軟件包的安裝路徑不在系統(tǒng)默認(rèn)路徑，可以通過(guò)設(shè)置環(huán)境變量exportNEUWARE_HOME=/path_to_neuware_home來(lái)指定搜索Neuware軟件包的路徑。 可以使用源碼包中的build_tensorflow_mlu.sh腳本來(lái)編譯CambriconTensorFlowcdcd#Neuware#export${TENSORFLOW_HOME}/virtualenv_mlu目錄下會(huì)生成TensorFlowwheel動(dòng)在virtualenv_mluwheel原生TensorFlow社區(qū)版本分發(fā)的TensorFlowpip、dockerCPUTensorFlowIItensorflow/core/platform/cpu_feature_guard.cc:142]YourCPUsupportsinstructionsthatCambrionensorFlw發(fā)布的pip、docer為了保證開(kāi)箱即用的性能，選擇了相對(duì)比較激進(jìn)的策略，默認(rèn)打開(kāi)了一些在x64平臺(tái)比較通用的指令集優(yōu)化。對(duì)應(yīng)的GCC編譯選項(xiàng)：----copt=-mfma--copt=-mavx2--copt=-mavx--copt=-msse4.1copt=-aws:AmazonWebaws:AmazonWebServicegcp:GoogleCloudPlatform對(duì)應(yīng)的bazel--config--config=noaws--CambrionensorFlw默認(rèn)沒(méi)有使能VX512F的支持。如果您需要開(kāi)啟編譯器的VX512F指令優(yōu)化請(qǐng)修改編譯腳本，加入--copt=-mavx512f編譯選項(xiàng)以使能該功能。 如果通過(guò)pi、docer分發(fā)的CambrionensorFlw因?yàn)镮SA兼容性無(wú)法在您的CPU上運(yùn)行，或您需要AS、GCP相關(guān)的服務(wù)特性，參見(jiàn)從源碼編譯自行修改編譯配置，重新編譯ensorFl。55使用本節(jié)介紹在CambrionTensorFlow中使用MLU進(jìn)行計(jì)算加速的基本用法。通常情況下，使用MLU的方式跟使用GPU關(guān)于使用GPU的更多信息，參見(jiàn)UsingGPUs7對(duì)于TensorFlowtf.kerasMLU獲取MLUimportimporttensorflowasprint("NumMLUsAvailable:",4卡MLU獲取詳細(xì)的MLUimportimporttensorflowas4MLU機(jī)器上將打印如下日志和輸出（此處省略時(shí)間戳、文件名、行號(hào)等信息：##TensorFlowgitversion（from#ICurrentMLU_Visible_DevicesCount:.使用#CambriconTensorFlow的版本號(hào)、編譯時(shí)間以及編譯時(shí)使用的NeuwarelibrariesICambriconTensorFlowversion:1.1.0wascompiledat2020-12-1617:04:50withToolkit:?→CNNL:1.1.1,CNML:7.9.2,CNPLUGIN:1.11.1,CNCL:Itfgitversion:v0.9.2-21252-#ICurrentlibraryversions:DRIVER:407,CNRT:40803,CNNL:1102,CNML:#ICreatedTensorFlowdevice(/job:localhost/replica:0/task:0/device:MLU:0with16384?→memory)->physicalMLU(device:0,name:ICreatedTensorFlowdevice(/job:localhost/replica:0/task:0/device:MLU:1with16384?→memory)->physicalMLU(device:1,name:ICreatedTensorFlowdevice(/job:localhost/replica:0/task:0/device:MLU:2with16384?→memory)->physicalMLU(device:2,name:ICreatedTensorFlowdevice(/job:localhost/replica:0/task:0/device:MLU:3with16384?→memory)->physicalMLU(device:3,name:MLU270)['/job:localhost/replica:0/task:0/device:CPU:0','/job:localhost/replica:0/task:0/device:MLU:0?→','/job:localhost/replica:0/task:0/device:MLU:1',將sessionoptions中l(wèi)og_device_placement設(shè)置為T(mén)rueimportimporttensorflowas#Createsaa=tf.constant([1.0,2.0,3.0,4.0,5.0,6.0],shape=[2,3],b=tf.constant([1.0,2.0,3.0,4.0,5.0,6.0],shape=[3,2],c=tf.matmul(a,#Createsasessionwithlog_device_placementsettosess=#RunstheIIDevice/job:localhost/replica:0/task:0/device:MLU:0->device:0,name:/job:localhost/replica:0/task:0/device:MLU:1->device:1,name:/job:localhost/replica:0/task:0/device:MLU:2->device:2,name:(下頁(yè)繼續(xù)/job:localhost/replica:0/task:0/device:MLU:3->device:3,name:>>>#Runsthe...MatMul:(MatMul):/job:localhost/replica:0/task:0/device:MLU:0a:(Const):/job:localhost/replica:0/task:0/device:MLU:0b:(Const):[[22.[49.CambriconTensorFlow的算子支持CPU設(shè)備和MLU"/cpu:0":"/cpu:0":機(jī)器的CPU"/device:MLU:0":MLU"/device:MLU:1":MLUMU設(shè)備比CPUMUCambrion會(huì)將該算子優(yōu)先分配到MU設(shè)備上，實(shí)現(xiàn)計(jì)算加速。如果希望某個(gè)算子在特定設(shè)備上運(yùn)行，而不是在CambriconTensorFlowwithtf.deviceimportimporttensorflowas#Createsawitha=tf.constant([1.0,2.0,3.0,4.0,5.0,6.0],shape=[2,3],b=tf.constant([1.0,2.0,3.0,4.0,5.0,6.0],shape=[3,2],c=tf.matmul(a,#Createsasessionwithlog_device_placementsettosess=#RunstheIIDevice/job:localhost/replica:0/task:0/device:MLU:0->device:0,name:/job:localhost/replica:0/task:0/device:MLU:1->device:1,name:/job:localhost/replica:0/task:0/device:MLU:2->device:2,name:/job:localhost/replica:0/task:0/device:MLU:3->device:3,name:(下頁(yè)繼續(xù)>>>#Runsthe...MatMul:(MatMul):/job:localhost/replica:0/task:0/device:MLU:0a:(Const):/job:localhost/replica:0/task:0/device:cpu:0b:(Const):[[22.[49.MLUTensorFlow對(duì)GPU內(nèi)存的使用策略是：默認(rèn)情況下，在啟動(dòng)階段申請(qǐng)全部的GPUmemory。用戶(hù)可以申請(qǐng)內(nèi)存。無(wú)論使用哪種策略，設(shè)備側(cè)內(nèi)存被TensorFlow詳細(xì)說(shuō)明，參見(jiàn)AllowingGPUmemorygrowth8CambriconTensorFlow對(duì)MLU內(nèi)存的使用策略與GPU類(lèi)似。兩者主要區(qū)別在于TensorFlowMLU啟動(dòng)階段不會(huì)預(yù)分配全部MLU內(nèi)存，而是在運(yùn)行階段按需分配。與GPU相同，TensorFlow申請(qǐng)走的MLUexportexport關(guān)閉MLUTensorFlow初始化階段，所有MLU在多MLU系統(tǒng)中使用單使用方式與GPU相同。更多信息，參見(jiàn)UsingasingleGPUonamulti?GPUsystem989MLU使用方式與GPU相同。更多信息，參見(jiàn)UsingmultipleGPUs10MLU默認(rèn)情況下，當(dāng)前系統(tǒng)的所有MLU設(shè)備對(duì)TensorFlow均可見(jiàn)。如果要控制TensorFlow進(jìn)程使用某一個(gè)或者某幾個(gè)MLU設(shè)備，可以借助寒武紀(jì)驅(qū)動(dòng)提供的MLU_VISIBLE_DEVICES環(huán)境變量來(lái)控制MLU設(shè)備需要注意的是，TensorFlow0Itensorflow/core/common_runtime/direct_session.cc:362]DeviceItensorflow/core/common_runtime/direct_session.cc:362]Device/job:localhost/replica:0/task:0/device:MLU:0->device:0,name:/job:localhost/replica:0/task:0/device:MLU:1->device:1,name:/job:localhost/replica:0/task:0/device:MLU:2->device:2,name:/job:localhost/replica:0/task:0/device:MLU:3->device:3,name:/job:localhost/replica:0/task:0/device:MLU:4->device:4,name:/job:localhost/replica:0/task:0/device:MLU:5->device:5,name:/job:localhost/replica:0/task:0/device:MLU:6->device:6,name:/job:localhost/replica:0/task:0/device:MLU:7->device:7,name:exportexport/job:localhost/replica:0/task:0/device:MLU:0->device:/job:localhost/replica:0/task:0/device:MLU:0->device:0,name:exportexport可以在啟動(dòng)日志中看到，對(duì)應(yīng)設(shè)備在TensorFlow0、1、2、3/job:localhost/replica:0/task:0/device:MLU:0->device:0,name:/job:localhost/replica:0/task:0/device:MLU:1->device:1,name:10

.基于LOWAPICAMBRICONTENSORFLOW/job:localhost/replica:0/task:0/device:MLU:2->device:2,name:/job:localhost/replica:0/task:0/device:MLU:3->device:3,name:CNMIXCambrionensorFlw和CambrionPoc在MLU200CambriconTensorFlowCNMIX基于LowAPI的CambriconTensorFlow確認(rèn)CambriconTensorFlow更多安裝信息，參見(jiàn)安裝章節(jié)。導(dǎo)入TensorFlowimportimportcnmiximporttensorflowasimporttensorflow.keras.datasets.mnistas(x_train,(x_train,y_train),(x_test,y_test)=mnist.load_data()batch_size,data_num=50,len(x_train)defdefinitial=tf.truncated_normal(shape,return(下頁(yè)繼續(xù)definitial=tf.constant(0.1,returndefconv2d(x,returntf.nn.conv2d(x,W,strides=[1,1,1,1],defreturntf.nn.max_pool(x,ksize=[1,2,2,strides=[1,2,2,1],xx=tf.placeholder(tf.float32,[None,28,y_=tf.placeholder("float",[None,W_conv1=weight_variable([5,5,1,b_conv1=x_image=tf.reshape(x,[-h_conv1=tf.nn.relu(conv2d(x_image,W_conv1)+h_pool1=W_conv2=weight_variable([5,5,32,b_conv2=h_conv2=tf.nn.relu(conv2d(h_pool1,W_conv2)+h_pool2=W_fc1=weight_variable([7*7*64,b_fc1=h_pool2_flat=tf.reshape(h_pool2,[-1,h_fc1=tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1)+keep_prob=h_fc1_drop=tf.nn.dropout(h_fc1,W_fc2=weight_variable([1024,b_fc2=(下頁(yè)繼續(xù)y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop,W_fc2)+cross_entropycross_entropy=-train_step=tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction=tf.equal(tf.argmax(y_conv,1),tf.argmax(y_,1))accuracy=tf.reduce_mean(tf.cast(correct_prediction,"float"))基于CNMIX構(gòu)建sessionsession_configsession_config=cnmix.mixed_precision_graph_rewrite_config(batch_size,data_num)sess=tf.Session(config=session_config)EPOCH,EPOCH,iters=10,int(data_num/forepochinforiinstart,end=batch_size*i,batch_size*train_x,train_y=x_train[start:end],np.eye(10)[y_train[start:end]]sess.run(train_step,feed_dict={x:train_x,y_:train_y,keep_prob:0.5})test_loss,test_accuracy=sess.run([cross_entropy,accuracy],feed_dict={x:x_test,y_:np.eye(10)[y_test],keep_prob:1.0})print("epoch%d,TestLoss:%.10f,TestAccuracy:%.5f"%(epoch+1,?→test),WARNING:tensorflow:FromWARNING:tensorflow:Froma.py:12:Thenametf.enable_eager_executionisdeprecated.Instructionsforupdating:Usetf.wherein2.0,whichhasthesamebroadcastruleasnp.whereepoch1,TestLoss:0.7170808105,TestAccuracy:0.95410epoch2,TestLoss:0.5037763672,TestAccuracy:epoch3,TestLoss:0.4459250977,TestAccuracy:epoch4,TestLoss:0.3748795898,TestAccuracy:epoch5,TestLoss:0.3611684570,TestAccuracy:epoch6,TestLoss:0.3054287842,TestAccuracy:epoch7,TestLoss:0.2697987305,TestAccuracy:.基于KERAS的CAMBRICONTENSORFLOWepoch8,TestLoss:0.2209449219,TestAccuracy:epoch9,TestLoss:0.2141630127,TestAccuracy:epoch10,TestLoss:0.2400469238,TestAccuracy:98%基于的確認(rèn)CambriconTensorFlow更多安裝信息，參見(jiàn)安裝章節(jié)。導(dǎo)入TensorFlow、CNMIX##importosimporttensorflowas載入并準(zhǔn)備好MNISTmnistmnist=tf.keras.datasets.mnist(x_train,y_train),(x_test,y_test)=mnist.load_data()x_train,x_test=x_train/255.0,x_test/255.0將模型各層堆疊以搭建tf.keras.Sequentialmodelmodel=model.add(tf.keras.layers.Flatten(input_shape=(28,28)))model.add(tf.keras.layers.Dense(128,activation='relu'))model.add(tf.keras.layers.Dense(10,activation='softmax'))基于CNMIXsessionconfigsession_configsession_config=cnmix.mixed_precision_graph_rewrite_config(batch_size,data_num,model.model.fit(x_train,y_train,epochs=5,model.evaluate(x_test,y_test,60000/60000[==============================]-7s118us/sample-loss:0.3359-acc:60000/60000[==============================]-7s117us/sample-loss:0.1624-acc:60000/60000[==============================]-6s93us/sample-loss:0.1224-acc:60000/60000[==============================]-6s98us/sample-loss:0.1002-acc:60000/60000[==============================]-6s96us/sample-loss:0.0858-acc:10000/10000-1s-loss:0.0768-acc:在網(wǎng)絡(luò)訓(xùn)練中，網(wǎng)絡(luò)代碼模型常會(huì)基于CPU、GPU等設(shè)備進(jìn)行構(gòu)建。若要基于MLU設(shè)備訓(xùn)練網(wǎng)絡(luò)，則在GPUwithwithtf.device("/GPU:0"):res=tf.add(1.,2.)在MLUwithwithtf.device("/MLU:0"):res=tf.add(1.,2.)配置CNMIX和session導(dǎo)入CNMIX模塊，通過(guò)CNMIX設(shè)置ensorFlw執(zhí)行GPU相關(guān)sessionconfigconfigconfig=執(zhí)行MLU相關(guān)sessionconfigimportimport#CNMIXconfigdata_numbatch_sizeconfig=global_batch_size=global_batch_size,data_num=data_num,opt_level=devices=['/device:MLU:'devices=['/device:MLU:'+str(mlu_id)formlu_idinrange(num_mlus)]distribution=tf.distribute.MirroredStrategy(devices=devices,CambriconTensorFlow只支持KerasAPI的MirroredStrategyHorovod#GPU#GPUconfig.gpu_options.visible_device_list=#config.mlu_options.visible_device_list=1166增加自定義MLU本章節(jié)以添加均方誤差（Mean?SquareError，MSE）算子為例，說(shuō)明如何在框架內(nèi)完整添加一個(gè)自定義MLU注冊(cè)MSE算子對(duì)應(yīng)的OpDef在tensorflow/core/user_ops/mlu_special_ops.ccREGITER_OP的宏注冊(cè)Mse子。.Input("a:.Input("b:.Output("output:.Attr("T:{half,在REGISTER_OP了MseShapeFnStatusStatusMseShapeFn(shape_inference::InferenceContext*c)ShapeHandleShapeHandleinput_b;ShapeHandleinput;std::vector<DimensionHandle>dims;for(inti=0;i<c->Rank(input)-1;i++){c->set_output(0,c-return(下頁(yè)繼續(xù).增加自定義MLU添加完上面兩個(gè)函數(shù)之后，REGISTER_OP宏就完成了MSE算子OpDef的定義。除此之外，該宏還自動(dòng)在PythonAPI中添加了MSE注冊(cè)MSE算子的OpKerneltensorflow/core/kernels/mlu_kernels文件夾下，創(chuàng)建新文件mse_op.cc。在該文件中，利用REGISTER_KERNEL_BUILDER定義了MSE算子的兩個(gè)后端：DEVICE_CPU和DEVICE_MLU。使用該宏時(shí)，要保證Name中的字符串和第1節(jié)中REGISTER_OP中的算子名稱(chēng)一致。通過(guò)調(diào)用TF_CALL_MLU_FLOAT_TYPES，向REGISTER_MLU分別傳入half和floatmse_op.cc文件中的代碼如下（完整代碼：#include#includeclassMseOp:publicOpKernel{explicitMseOp(OpKernelConstruction*context):OpKernel(context)voidCompute(OpKernelContext*ctx)overrideclassMLUMseOp:publicOpKernelexplicitMLUMseOp(OpKernelConstruction*ctx):OpKernel(ctx)voidCompute(OpKernelContext*ctx)overrideif(!ctx->ValidateInputsAreSameShape(this))return;autoinput=ctx->input(0);Tensor*output=nullptr;TensorShapefor(inti=0;i<input.dims()-1;i++){ctx->allocate_output(0,output_shape,&output));OP_REQUIRES_OK(ctx,ComputeInternal(ctx));StatusComputeInternal(OpKernelContext*ctx)autoinput_a=ctx->input(0);autoinput_b=ctx->input(1);intline_width=input_a.dim_size(input_a.dims()-1);cnrtQueue_tqueueCHECK_NOTNULL(reinterpret_cast<cnrtQueue*>(ctx----returnMseOp<CPUDevice,float>);MseOp<CPUDevice,Eigen::half>);#define #undef}//namespace在MseOp的實(shí)現(xiàn)中，實(shí)現(xiàn)了一個(gè)簡(jiǎn)易的CPU版MSE算子。在MLUMseOp的ComputeInternal函數(shù)中，準(zhǔn)備了MseKernel函數(shù)的入?yún)?，后調(diào)用CustomedOps中定義的MseKernel函數(shù)，從而調(diào)用BANGC版的MSE在CustomedOps中實(shí)現(xiàn)MseKernel所有自定義BANGC實(shí)現(xiàn)的kernel文件全部位于tensorflow/core/kernels/mlu_kernels/customed_ops目錄。為了添加MseKernel函數(shù)，需要添加新文件mse_kernel.mlu，并修改兩個(gè)文件bangc_kernels.h和BUILD。在mse_kernel.mlu文件中，利用BANGC編寫(xiě)MSE算子相關(guān)的計(jì)算邏輯，并封裝一個(gè)MseKernelbangc_kernels.h文件聲明了所有的BANGC：#define#include<stdint.h>#include"cnrt.h"namespacemlustaticvoidMishKernel(cnrtQueue_tvoid*input,void*output,intinput_size,intdata_type,intstaticvoidMseKernel(cnrtQueue_tvoid*input_a,void*input_b,void*output,intdata_nums,intline_width,int}//namespace#endif//mse_kernel.mlu文件如下(完整代碼)：#include"bangc_kernels.h"#defineBLOCK_SIZE12800#defineALIGN_UP(a,b)(((a)+(b)-1)/(b)*mlu_funcvoidmseKernelInter(TT*input_b,T*output,intintline_width)mlu_entryvoidvoid*input_a,void*input_b,void*output,intdata_nums,intline_width,intdata_type)if(data_type==6)mseKernelInter((float*)input_a,(float*)input_b,(float*)output,data_nums,}elseif(data_type==3)mseKernelInter((half*)input_a,(half*)input_b,(half*)output,data_nums,void*input_a,void*input_b,void*output,intdata_nums,intline_width,intdata_type)cnrtDim3_tdim={4,1,

cnrtFunctionType_tfunc_type=mseKernel<<<dim,func_type,關(guān)于BUILD文件的修改，參見(jiàn)修改BUILD文件為了測(cè)試新增算子的功能和精度，需要同時(shí)添加算子的單測(cè)文件。參考本章節(jié)添加算子后，在PythonAPIC++API中都能直接調(diào)用該算子。因此，需添加兩個(gè)算子單測(cè)文件：tensorflow/core/kernels/mlu_kernels/mse_op_mlu_test.cc和tensorflow/python/kernel_tests/mlu_kernel_tests/mse_op_test.py。在C++API和PythonAPI中，可以分別通過(guò)調(diào)用Mse和tensorflow.mse接口來(lái)調(diào)用該算子。修改BUILD在添加新算子過(guò)程中，只需要修改tensorflow/core/kernels/mlu_kernels/BUILD該文件主要完成以下內(nèi)容：tf_kernel_librarymse_op.cc、mse_op_mlu.hmse_kernel編譯成新的tar?get:mse_op。該target同樣可以被其他target利用tf_cc_test_mlu函數(shù)，將mse_op_mlu_test.cc單測(cè)文件編譯成一個(gè)binarymlu_core_kernelsmse_opTensorFlow在生成mlu_core_kernels目標(biāo)函數(shù)的deps域中，添加一行":mse_op"以下為該文件的代碼示例：name=srcs=["mse_op.cc"],hdrs=["mse_op_mlu.h"],deps=["@local_config_mlu//:mlu",size="medium",(下頁(yè)繼續(xù)srcs=linkstatic=1,deps=[#Publicsupportname="mlu_core_kernels",deps=[...//77本章主要講述CambriconTensorFlowMLU中。內(nèi)存越界的bug（出現(xiàn)的時(shí)機(jī)和出錯(cuò)的現(xiàn)象都是隨機(jī)的，很難定位。別插入header和footer，在deallocate時(shí)拷回到host端檢查之前插入的header和master遇到日志中有[cnrtError]、mluunfinished這些關(guān)鍵字時(shí)，優(yōu)先打開(kāi)內(nèi)存越界檢查：exportTF_MLU_DEBUG_MEMORY_OVERSTEP=truecoredump中有Footermaskhasbeenoverwritten或者Headermaskhasbeenoverwritten，說(shuō)明存在內(nèi)存debugTen?sorFlow（./build_tensorflow_mlu.sh?g），后使能內(nèi)存申請(qǐng)調(diào)用棧記錄.MLU以自定義的Overstep算子為例。在OverstepCompute函數(shù)中模擬了內(nèi)存越界的行為（直接往output拷貝一組大于output.NumElements()。假設(shè)網(wǎng)絡(luò)中調(diào)用了Oversteplog20212021-01-1810:50:48.701641:[cnrtError][186985][Card:0]Erroroccurredin2021-01-1810:50:48.701678:[cnrtError][186985][Card:0]Returnvalueis230,?→CHECKfailed.Function:MLUCnrtMemcpyAsync@line:221returnedcode632046:Unknownerror.?→errordetected,forceterminating...Aborted(coredumped)log[cnrtError]exportTF_MLU_DEBUG_MEMORY_OVERSTEP=true打開(kāi)內(nèi)存泄漏檢20212021-01-1811:15:05.364044:E?→cc:141][230845]i=0mask=0xcdcdcdcdcdcdcdcd2021-01-1811:15:05.364053:E?→cc:141][230845]i=1mask=0xcdcdcdcdcdcdcdcd2021-01-1811:15:05.364059:E?→cc:229][230845]Footermaskhasbeenoverwritten,ptr=2021-01-1811:15:05.364067:E?→cc:235][230845]wasallocated2021-01-1811:15:05.364238:I?→cc:101][230845]call可以看出，新生成的logFootermaskhasbeenoverwritten?→python2.7/site-:tensorflow::MLUDebugAllocator::AllocateRaw(unsignedlong,unsigned:tensorflow::Allocator::AllocateRaw(unsignedlong,unsigned?→tensorflow::AllocationAttributes(下頁(yè)繼續(xù).MLU:float*tensorflow::TypedAllocator::Allocate<float>(tensorflow::Allocator*,unsigned?→tensorflow::AllocationAttributes?→python2.7/site-:tensorflow::Tensor::Tensor(tensorflow::Allocator*,?→tensorflow::TensorShapeconst&,tensorflow::AllocationAttributes:?→tensorflow::AllocationAttributes::tensorflow::OpKernelContext::allocate_output(int,tensorflow::TensorShape?→tensorflow::Tensor**,:tensorflow::OpKernelContext::allocate_output(int,tensorflow::TensorShape::tensorflow::BaseMLUDevice::Compute(tensorflow::OpKernel*,?→python2.7/site-?→python2.7/site-?→python2.7/site-:std::function<void()>::operator()()::Eigen::ThreadPoolTempl<tensorflow::thread::EigenEnvironment>::ThreadPoolTempl(int,?→tensorflow::thread::EigenEnvironment)::{lambda()#1}::operator()():std::_Function_handler<void(),?→<tensorflow::thread::EigenEnvironment>::ThreadPoolTempl(int,?→tensorflow::thread::EigenEnvironment)::{lambda()#1}>::_M_invoke(std::_Any_data:std::function<void()>::operator()():tensorflow::thread::EigenEnvironment::CreateThread(std::function<void?→::operator()():std::_Function_handler<void?→tensorflow::thread::EigenEnvironment::CreateThread(std::function<void?→M_invoke(std::_Any_data .TENSORDUMP:std::function<void()>::operator()():voidstd::_Bind_simple<std::function<void()>:std::_Bind_simple<std::function<void()>:std::thread::_Impl<std::_Bind_simple<std::function<void()>()>:/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba):/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)可以看到，[10]中出現(xiàn)OverstepOp的Compute，說(shuō)明OverstepOp的Compute中存在內(nèi)存越界。tensordump通過(guò)設(shè)置環(huán)境變量將網(wǎng)絡(luò)中各節(jié)點(diǎn)輸出tensordumpexportexportexport //運(yùn)行結(jié)束后網(wǎng)絡(luò)中間節(jié)點(diǎn)的輸出tensor會(huì)被dump到dump_dir文件夾。以上方法可將device為CPU和MLU的節(jié)點(diǎn)輸出全部dump到指定文件夾。dump出來(lái)的文件命名規(guī)則為${node_name}_output_${output_port}_${execution_order}。例如，node_name：節(jié)點(diǎn)名稱(chēng)。此例中為resnet50.conv_1.conv2d。原本名稱(chēng)應(yīng)為“/output_port：此文件是這個(gè)node0execution_order：此tensor被dump出來(lái)的順序，該順序與node10最好只進(jìn)行一次推理，比如只運(yùn)行一個(gè)batch使用tensordump在進(jìn)行dump之前，需要清空dump_dircdcd--truth_value_dir=/path/to/truth_value_dump_dir--test_value_dir=/path/to/test_value_dump_dir5%兩個(gè)輸入文件夾中的數(shù)據(jù)是使用tensordump功能dump出來(lái)的。誤差計(jì)算工具會(huì)根據(jù)命名規(guī)則進(jìn)兩個(gè)輸入文件夾中的數(shù)據(jù)不是使用ensordump功能dump出來(lái)的，但對(duì)應(yīng)文件的名稱(chēng)完全一致。誤差計(jì)算工具會(huì)根據(jù)文件的讀取順序計(jì)算誤差輸出報(bào)告。文件內(nèi)容的格式必須是每行只含有一個(gè)數(shù)字。output_right.txt：只包含誤差小于等于??thresholdoutput_abnormal：包含由于某些異常導(dǎo)致無(wú)法計(jì)算誤差的文件的報(bào)告。異常包括含有nan值、含有inf值、兩個(gè)文件中數(shù)據(jù)量不同、無(wú)法解析文件中的數(shù)據(jù)（比如亂碼）等。truth_value_file:import.Preprocessor.mul_output_0_1503test_value_file:import.Preprocessor.mul_output_0_1507tensor_name:import/Preprocessor/mul:0iteration:error_rate:mse:is_same_size:truth_value_file：命令中??truth_value_dirtest_value_file：命令中??test_value_dirtensor_name：網(wǎng)絡(luò)中的tensor名稱(chēng)。iteration：該tensordump出來(lái)。在一次推理中如果一個(gè)tensordump出來(lái)，則說(shuō)error_rate：相對(duì)誤差，命令中??thresholdmse：truth_value_file:import.Preprocessor.mul_output_0_1503tensor_name:import/Preprocessor/mul:0iteration:1error_rate:Nonemse:Noneis_same_size:False當(dāng)兩個(gè)文件中的數(shù)據(jù)量不同時(shí)，則不進(jìn)行誤差計(jì)算，并把報(bào)告寫(xiě)入到output_abnormal.txt原因?yàn)閕s_same_size為False。文件中包含nan或者truth_value_file:(下頁(yè)繼續(xù)test_value_file:iteration:1error_rate:Nonemse:Noneis_same_size:Truehasnan/-nan:truth_value_file:test_value_file:hasinf/-inf:test_value_file:Truen