大模型深度定制以及在芯片設(shè)計(jì)應(yīng)用中的探索（英文版）

ChipNeMo-DomainAdapted

LLMsforChipDesign

劉鳴杰MingjieLiu

SlidescontentsfromHaoxingRenandMiguelMartinez

Outline

LLMIntroduction

HowcanLLMhelpchipdesignprocess

Domain-adaptionLLMtechniquesforchipdesign

LLMApplicationsToday

NaturalLanguageTasks

Generation:writeemails

Open/ClosedQA:answerquestionswith/withoutcontext

Brainstorming:generateideasChat

Rewrite:languagetranslationSummarization

Classification:sentimentanalysisReasoning:math/logic

Applications

ChatBot,APIs

CodingTasks

Understand/explaincode,errormessagesDocument,commentcode

Generatecodesnippetsfromspecs

TranslatecodetoadifferentlanguageEdit,debugcode

ToolUse/Agents

Applications

CoPilot

PromptEngineering

Zeroshot

Prompt:

Classifythetextintoneutral,negativeorpositive.

Text:Ithinkthevacationisokay.

Sentiment:

Response:

Neutral

Fewshot

Prompt:

Thisisawesome!//NegativeThisisbad!//Positive

Wowthatmoviewasrad!//PositiveWhatahorribleshow!//

Response:

Negative

mptingguide.ai/

ChainofThoughts(CoT)

Prompt:

Iwenttothemarketandbought10apples.Igave2applestotheneighborand2applestotherepairman.Ithenwentandbought5

moreapplesandate1.HowmanyapplesdidIremainwith?

Let'sthinkstepbystepandcalculateeachstep.

Response:

Youhave10apples.

Yougive2applestotheneighborand2to

therepairman.

Youhave6applesremaining.Youbuy5moreapples.

Youhave11apples.Youeat1apple.

Youhave10applesremaining.

RetrievalAugmentedGeneration

AddContexttoGroundLLMonUnseenFacts

5

4

LLM

Response

“TellmeaboutSM”

1

4

VectorDBsupportingsimilaritysearch

RetrievalModel

Embedding

Chunk

xxxx

AGPUcontainstwoormore

StreamingMultiprocessors(SM)dependingupon…

2

EmbeddingVector

3

HowcanLLMHelpChipDesignProcess?

Know-howAssistance+CodingAssistance

Know-howAssistanceGeneratinginsights,knowledge,ideas

Designknow-howQ&A:questionsaboutdesigns,infrastructures,tools,flows,HWdomains,etc.

AnalysisandReport:summarization,checkrule

violations,writetestplans,visualizationofdesignandrelateddata,etc.

Triageadesignproblem:debugaregressionproblem,howtofixabug,etc.

CodingAssistanceGeneratingcode(software,RTL,testbenches,EDAscripts,toolsscripts,andconfigs)

Generatecodeforauxiliarydesigntaskssuchasassertions,comments,etc.

Generatelower-levelprogramsfromhigher-leveldescriptions

Generatescriptsforspecifictasks(VLSI,Verification)Transformcodeformoreefficientimplementation

HWTeamLLMApplicationSurvey

(~100proposals)

CodeGenuQ&A

aTriage

Analysis&Report

15%

46%

17%

21%

UseCasesEvaluated

EngineeringAssistant

Chatbot

Designknow-howQ&A

EDAScriptsGenerationBugSummaryandAnalysis

CodeGenAnalysisandReport

UseGeneral-PurposeLLMsforChipDesign

Challengesofgeneral-purposeLLMsforChipDesign

Lackofspecificcodinglanguage/toolsknowledgeLackofdesignknowledge

Lackofdesigntasks-specificskillsReference/Accuracyrequirement

Solutions

PromptEngineering

RetrievalAugmentedGeneration(RAG)

Additionalchallenges

RetrievalaccuracyContextlimitationComplexquestionsCodingquestions

Canwedobetter?

TypicalLLMTrainingFlow

Humanfeedback

RLHF

Trainascalarscorefor(prompt,response)

LowqualitydataHighqualitydata

Comparisondata

Prompt

maximizescorefromrewardmodel

Text

e.g.Internetdata

Demonstrationdata

PredictnexttokenGivenprompt

Classification

ReinforcementLearning

predictresponse

Pretraining

Supervisedfinetuning

Rewardmodel

Finalmodel

FoundationLLM

SFTmodel

Scale>1trilliontokens10K-1Mexamples100k-1Mcomparisons10k-100kprompts

Basedon

/2023/05/02/rlhf.html

andNeurIPStutorial(AndrewNg)

ChipNeMo:Domain-AdaptionofLLMforChipDesign

GPUHours(A100)

1000000

100000

10000

1000

100

10

10000000

1

7B13B70B

PretrainingDAPTSFT

/abs/2311.00176

DomainAdaptationTechniques

TraincustomizedLLMfordomain

Customtokenizationimproveinferenceefficiency

Trainingdatarebalanceimprovetrainingdataquality

Domain-adaptedpretraininglearncoding/tools/designknowledge

General/Domain-specificinstructionalignmentlearntofollowgeneralanddomain-specificinstructions

Domain-adaptedretrievalaugmentedgeneration(RAG)improveretrievalaccuracy

Tokenization

Customizedtokenizerhelpstokenizationefficiencyandperformance

TrainingDataRebalance

Collected24Bdatatokensfrominternaldocumentsandcode,including2BtokensofGitHubandwikidata

AdjusttrainingweightstobalancecodeandtexttokensRemovemostlymachine-generatedcode

text

text

code

code

CollectedDataTokens(24B)TrainingTokens(24B)

AutoEvalForChipNeMoFoundationModels

Multiplechoicequestions(humangenerated)toevaluatemodelperformance

DesignKnowledge(94)

WhatdoesCGAstandfor?

A:CooperativeGridArray

B:Co-dependentGridArrayC:CUDAGridArray

D:CooperativeGPUArray

MMLU(14.6K)

LetGdenotedthesetofallnxnnon-

singularmatriceswithrationalnumbersasentries.ThenundermultiplicationGisa/an?

A:subgroup

B:finiteabeliangroup

C:infinite,nonabeliangroupD:infinite,abelian

EDAScripts(74)

HowdoIgettheobjectoftheABCnetinVIVID?

A:get_net("ABC")B:get_nets("ABC")C:get_cell("ABC")D:get_pins("ABC")

BugAnalysis(70)

WhatisthebugmodulethatdealswithMATHSarchitecture?

A:DFX-MATHS-ArchitectureB:DFX-MATHS-Access

C:DFX-Architecture-MATHSD:DFTMATHSLink

OpenDomainCircuitDesign

(227)

WhichVerilogsystemtaskemitsa

formattedstringwithacarriagereturn?A:"$display"

B:"$write"C:"$probe"D:"$finish“

WhatisaG-elementinHSPICE?

A:voltage-controlledcurrentsourceB:voltage-controlledvoltagesourceC:current-controlledvoltagesourceD:current-controlledcurrentsource

Domain-AdaptivePretraining

FoundationModelPerformanceComparison

Performanceimproveswithbasemodelsize

ChipNeModomain-adaptivepre-trainingprovidessignificantperformanceimprovementsoverthebasemodel

BestChipNeMomodelhasbetterperformancethanGPT-3.5*onallbenchmarksandGPT-4ondesignknow-howandBugsbenchmarks

LLaMA2-7BLLaMA2-13BLLaMA2-70BGPT-3.5

ChipNeMo-7BChipNeMo-13BChipNeMo-70BGPT-4

100

80

70

60

50

40

90

30

DesignScriptingBugsCircuitsMMLU

RetrievalAugmentedGeneration

Customretrievalmodelimprovesretrievalaccuracy

Additionalcontext-Retrieval-Augmented-Generation(RAG)orOracle-helpsalot

RAGresultssignificantlydependonretrievalaccuracy

Fine-tunedretrievalmodelwithdomaindata

improvesretrievalmodelaccuracySamplepassagesfromthedocstore

Generatequeriesfromselectedpassages

Generatepositive/negativeresponsesforeachquery

Fine-tuneretrievalmodelwith3Kquery/responsepairs

ImproveretrievalaccuracyoverE5andbetterthanSOTAsentencetransformer

Integratedsearchengineforbetterretrieval

10.90.80.70.60.50.40.30.20.10

E5SentenceTransformerDomain-AdaptedE5

SpecsTestbenchBuildTOTAL

RetrievalModelAccuracy

BetterAlignmentTechnique

AttributeConditionedSFT(SteerLM)

IssuewithSFT

Responsesareterse

TrainingexamplesarenotexactlycleanToxicity

SteerLM:RLHFreplacement

SFTwithscoresonhelpfulness,verbosity,etc.foreachexample’spromptcontext

Prompt,Helpfulness:4,Correctness:4,Verbosity:2,…Response

TrainedeasierthanRLHF

ChipNeMo-70B-SteerLMmodeloutperformsGPT-4by20%,evenwithRAG

Humanevaluation(1-7)88designquestionswithRAG

SteerLM:AttributeConditionedSFTasan(User-Steerable)AlternativetoRLHF,Arxiv2023

EDAScriptGeneration

IntegratedwithVLSItoolsandeditor

Caneditandexecutegeneratedcodeonrealdesigns

SFTdatacollectionwithmodel-generatedcomments

RetrieverelevantAPIsascontextfromadatabase

ChipNeMomodelsperformmuchbetteronsimpleproblems

Basemodelperformanceimportancefor