NVIDIA 面向大語言模型的全棧 AI 解決方案

NVIDIA面向大語言模型的全棧AI解決方案大語言模型生態(tài)系統(tǒng)快速演進導致優(yōu)化異常挑戰(zhàn)?新的算子和定制技術使得優(yōu)化成為持續(xù)變化的目標需要全棧AI解決方案助力模型開發(fā)和應用部署LLaMaLLaMaGPTFalconStarcoderChatGLMMPT…ImagefromMooler0410/LLMsPracticalGuideASurveyonChatGPTandBeyond.arXiv[Cs.CL].Retrievedfrom/abs/2304.13712FP32FP16BF16FP8(E5M2)FP8(E4M3)FP8相較于FP16/BF162倍的吞吐signmultiplyaccumulateintoTCRangePrecisionexponentsignmultiplyaccumulateintoTCRangePrecisionexponentmantissaDD——s—D——DD—— bias/act/… convertFP32|FP16|BF16|FP8TransformerEngineFP8混合精度訓練其它操作仍在高精度參數(shù)更新 TransformerEngine算子融合混合精度訓練thehistoryandJax?Hyperparameter:GBS2048_SEQ4096_MBS2_TP2_PP1_DP128?Hyperparameter:GBS1024_SEQ4096_MBS4_TP4_PP1_DP32HellaSwagWinoGrandeHellaSwagWinoGrande數(shù)據(jù)中心級別的計算機由網(wǎng)絡定義數(shù)據(jù)中心AIEthernetFabric?TraditionalEthernetnetworkcansuffice?TraditionalEthernetnetworkforNorth-Southtraffic?NVIDIASpectrum-XEthernetforAINVIDIASpectrum-X-世界上第一款為AI而生的以太網(wǎng)平臺融合標準以太網(wǎng)連接和專業(yè)的高性能架構PerfectEffectiveStandardEthernetSpectrum-4Spectrum-4EthernetSwitchsupportedsoftwarethatspeedsdeveloperexecutionofeverydeveloper’sjobssupportedsoftwarethatspeedsdeveloperexecutionofeverydeveloper’sjobsthousandsAccelerateend-to-performanceNVIDIAAIEnterpriseAIFrameworksNVIDIAAIWorkflowManagementandMLOpsJobScheduling&OrchestrationPROVISIONINGMONITORINGMANAGINGClusterManagementPROVISIONINGMONITORINGMANAGINGCLUSTERINGCLUSTERINGNETWORKIOSTORAGEIOIOMANAGEMENTNetwork/StorageNETWORKIOSTORAGEIOIOMANAGEMENTIN-NETWORKCOMPUTEIN-NETWORKCOMPUTEOperatingSystemNVIDIANeMo大語言模型應用落地全流程Acceleratedcitedresponses數(shù)據(jù)處理提升模型性能NeMoDataCuratorsteps:?Classifier-basedfilteringInternetscaledatasetsInternetscaledatasets.△.口△口口△口!!extracttextTextre- NVIDIAMegatron-CorePerformanceatScalePerformanceatScaleMemory,Compute,andCommunicationOptimizationParallelismTechniquesMemorySavingTechniquesSelectiveActivationRecomputeDistributedOptimizersPrecisionawareoptimizers(BF16,FP8)withFusedAdam●●●●●●●●●●●architectureFormalizedProductFormalizedProductSupportLatestresearchandNVIDIANeMo對大語言模型定制的套件根據(jù)場景定制大語言模型的不同方式Accuracyforspecificuse-casesPROMPTENGINEERINGPROMPTLEARNINGPARAMETEREFFICIENTFINE-TUNINGFINETUNINGPROMPTENGINEERING?LeastexpertiseBenefits??SFTNVIDIATriton推理服務器快速可擴展簡單的推理服務開源軟件(Jetson)Kubernetes,KServe,PlatformsLatencyLLM生產(chǎn)部署的極致性能TensorRT-LLMisanopen-sourcelibrarytooptimizeinferenceperformanceonthelatestLargeLanguageModelsforNVIDIAGPUs.ItisSoTAPerformancePerformanceTCOEaseExtension#defineanewactivationdefsilu(input:Tensor)→Tensor:#implementmodelslikeinDLFWsclassLlamaModel(Module)def__init__(…)self.layers=ModuleList([…])forlayerinself.layers:LLMBatchingwithTritonStaticInflightBatchingCost為RAG帶來最佳的準確度和延時Acceleratedvectorsearch★ApplyforEarlyAccessStateoftheArtAccuracy?Latesttechniquesforchunkingandmetadata?Alldatasources,suchasunstructured(vectordatabases)andstructured(SQL,Tabular)LowestLatencyandHighestThroughputProductionReadywithEnterpriseSupport?ProductionreadyservicenterprisesupportNVIDIANeMoGuardrails保持LLM安全TopicalGuardrailsTopicalGuardrailsSafetyGuardrailsSecurityGuardrailsLLMs聽取ChinaAIDay任一演講，即可獲取DL會后您將收到優(yōu)惠碼專屬郵件任選右側(cè)一門公開課使用優(yōu)惠碼（僅限一門）開啟LLM實戰(zhàn)之旅構建基于大語言模型（LLM）的應用利用大模型開源生態(tài)系統(tǒng)快速開發(fā)基于預訓練大語言模型的應用基礎基礎高效定制大語言模型（LLM）學習提示工程和各類高效參數(shù)微調(diào)方法對預訓練LLM模型進行定制以適應特定應用進階進階大語言模型（LLM）、生成式AI系列近期公開