【全球軟件開發(fā)大會(huì)】AIGC浪潮下WebNN的演進(jìn)與實(shí)踐

演講人：付俊偉胡寧馨,

英特爾首席工程師,

W3C

Web

Neural

Network

(WebNN)

標(biāo)準(zhǔn)的起草和主

要編輯者,

Chromium

committer

and

Chromium

WebNN

組件的主要擁有者張敏,

IntelWebNN

團(tuán)隊(duì)的技術(shù)經(jīng)理,

Chromium

and

ONNX

Runtime

WebNN

EP

的開發(fā)者,WebNN

developer

preview的作者付俊偉,

英特爾高級(jí)軟件工程師,

Chromium

committer

and

Chromium

WebNN的

基礎(chǔ)架構(gòu)設(shè)計(jì)和Chromium

Shape

Detection

API

主要開發(fā)者

03

如何使用WebNN

02

WebNN的架構(gòu)設(shè)計(jì)

04

WebNN的性能對比

01

WebNN出現(xiàn)的背景目

錄WebNN

ExecutionProviderofONNX

RuntimeWebwithGPUaccelerationfrom

DirectML.

Runningon

Intel?CoreUltra7

processor

155Hwith

integratedArcGPU.https://microsoft.github.io/webnn-developer-preview/WebNNOperationmatMulgathersigmoidsoftmaxDirectMLGEMMGATHERLOGISTICSOFTMAXTFLiteBATH_MATMULGATHERACTIVATION_SIGMOIDACTIVATION_SOFTMAXCoreMLmatmulgather_along_axissigmoidsoftmaxStableDiffusioni

Image

DecoderImageGenerationText

EncoderAcat

under

the

snowUnetStep1UnetStep4UnetStep2UnetStep3框架WebAPIAPIextensionsWebGPUWebNNWeb引擎Web

Browser(e.g.,Chrome/Edge)JavaScriptRuntime(e.g.,

Electron/Node.js)運(yùn)用場景系統(tǒng)MLAPIs硬件Transformers.jsTensorFlow.jsMediaPipeWebONNXRuntime

WebWebAssemblyBackground

SegmentationGPUNPUImageClassificationCPUNoiseSuppressionOtherML

OSAPIsNatural

LanguageObject

DetectionWindowsStudio

EffectsTFLiteCoreMLDirectMLComputational

Graph(Web)

inputoutput

WebNN

為Web帶來了神經(jīng)網(wǎng)絡(luò)的統(tǒng)一抽象CompiledGraph

(Native)device

type:

cpu/gpu/npupowerpreference:high-perf/low-powerOutputBuffers

(CPU/GPU)InputBuffers

(CPU/GPU)MLGraphMLContextMLGraphBuildertmptmpWebNN

API

二

Other

Web

APIFused

conv2doutputbiasfilterinput↓

createbuild

↓compute

Dataflow

Call

flowcompileconv2daddreluRendererProcessGPU/Utility

ProcessApps/FrameworksNativeMLAPIsOS

DriversHardwareWeb

ApplicationWebNNMojo

ClientWebNNMojo

ServerJS

ML

FrameworksMLGraphBuilderMLGraphMLContext

↓

BNNS/MPSDirectMLMCDMDirectMLBackendCoreMLCoreMLBackendTFLiteBackendTFLiteXNNPACK/

DelegateNPUCPUGPUmacOS

WindowsAndroid/ChromeOS/LinuxChromiumIPCWasmKernelsinputWebNN

GraphIntermediateWasmKernelsBrowserswithWebNN

supportWeb

ApplicationTensorFlowLiteWebONNXRuntime

WebNPUGPUCPUNative

CPUKernelsNativeNPUKernelsNative

GPUKernels

母

ONNX

1.18

release

WasmKernelsWebGL

KernelsWebGPU

KernelsWebNN

GraphPost-ProcessingConv2dMatMulTensorF2Pre-processingIntegrationStatusPrototype

Availableintermediateintermediateweightsweightsbiasweb

Assemblybackend

web

NN

backendWebNN

ExecutionProviderofONNX

RuntimeWebwithGPUaccelerationfrom

DirectML.

Runningon

Intel?CoreUltra7

processor

155Hwith

integratedArcGPU.https://microsoft.github.io/webnn-developer-preview/VanillaJS(plainJavaScript)useofWebNNAPI,with

NPUaccelerationfrom

DirectML.Runningon

Intel?CoreUltra7

processor

155Hwith

integrated

Intel?AI

Boost

NPU.4.4.53.03.1

3.31.2

3.03.23.03.23.03.12.

.03.12.

.03.12.8

2.93.12.92.52.52.72.32.21.81.01.01.01.01.01.01.01.01.01.01.01.01.01.01.0The

average

performanceof

listed

15

models

onwebNN

Oncpu

isabout

93%of

nativeXNN

packMediaPipeModelsInferencePerformance(Normalized

/HigherisBetter)100.0%90.0%

80.0%

70.0%

60.0%

50.0%

40.0%

30.0%

20.0%

10.0%

0.0%?Browser:

Chrome

Canary

118.0.5943.0?DUT:Dell/Linux/i7-1260P,

singlep-core?Workloads:

MediaPipe

solutionmodels

(FP32,batch=1)

WebNNXNNPackNative

XNNPackWebNNvsNative5.04.5

4.03.53.02.5

2.0

1.5

1.0

0.5

0.0WebNNvs.

Nat

ive

RatioInferenceSpeedup3.2

3.3

WasmSIMD?CPU:Intel(R)Core(TM)Ultra7

155H

3.80

GHz?

GPU:

Intel(R)

Arc(TM)

Graphics

WebNN

GPU

Native

DirectML

WebNN

GPU

vs.

Native

DirectML?GPUDriver:3512weoNN

Direct

ML

is

about83%

of

native

DML

On87.9

87.3

87.4

88.882.6

82.471.4100001000100101120.0100.080.060.040.020.00.095.089.5

87.676.5?

Browser:Chrome

Canary126.0.6459.0?

OS:Windows11Pro23H2?

DUT:

Asus

Zenbook88.178.672.0

71.591.4

93.286.779.0WebNN

DirectMLvs.

Native

DirectMLInferenceT

ime(ms)(Logscale)95.673.0Percentage(%)91.585.381.575.881.586.1%73.4%62.7%?

Browser:Chrome

Canary126.0.6459.0?

OS:Windows11Pro23H2?

DUT:

Asus

Zenbook?CPU:Intel(R)Core(TM)Ultra7

155H

3.80

GHz?

NPU:Intel(R)AI

Boost?

NPU

Driver:381The

averageperformance

oflisted

4models

onWebNNDirectMLis

about

80%

of

native

DML

onMTL

NPUMobileNetV2SqueezeNet

1.0ResNet50v1EffiecientNet

Lite4 WebNN

DirectML

NPUNative

NPUWebNN

NPUvs

Native100.0%90.0%80.0%

70.0%

60.0%

50.0%

40.0%

30.0%

20.0%

10.0%

0.0%8.007.006.005.004.003.002.001.000.00WebNNDirectMLvs

Nativeon

MTLNPUWebNNvs

Nat

ive

(%)InferenceT

ime

(ms)95.8%SpeechtoText

PoC

Demofor

KhanAcademy

Khanmigo.WebNN

ExecutionProviderofONNX

RuntimeWebwith

NPUaccelerationfrom

DirectML.

Runningon

Intel?CoreUltra7

processor

155Hwith

integrated

Intel?AI

Boost

NPU.?

在Microsoft

Edge

和

Google

chrome

開發(fā)版本上試用

weoNN?

打開about://flags

網(wǎng)址

和

搜索Enables

web

NN

APPI"

、

weoNN

?

web

NN

標(biāo)