vmware電信運(yùn)營(yíng)商nfv解決方案v

VMware電信運(yùn)營(yíng)商N(yùn)FV解決方案劉承罡2016.4版本更新CONFIDENTIAL2V0.1根據(jù)VMworld2014的內(nèi)容和中國(guó)移動(dòng)、中國(guó)電信三個(gè)案例編寫初稿V0.2增加兩種運(yùn)營(yíng)商N(yùn)FV的實(shí)現(xiàn)方式比較V0.3細(xì)化浙江電信iVPN案例V0.4學(xué)習(xí)BrucePPT，增加PCRF的網(wǎng)元介紹V0.5增加浙江電信當(dāng)時(shí)處理高cpu占用率下丟包的經(jīng)驗(yàn)分享V0.6刪除廣東移動(dòng)智能網(wǎng)案例、增加一部分VIO/VIC的內(nèi)容Agenda1什么是NFV2VMware的NFV之路3NFV的技術(shù)難點(diǎn)和解決方案4VMwareNFV案例CONFIDENTIAL3NFV概述Source:EuropeanTelecommunicationsStandardsInstitute(ETSI)NetworkFunctionsVirtualizationApproach4NFV對(duì)運(yùn)營(yíng)商的優(yōu)勢(shì)將服務(wù)和硬件完全解耦E.g.3Gservicesand4Gservicesusesamephysicalinfrastructure彈性的容量Withuniformpoolofresources,applythemtoservicesthatneedthemLongandshort-termdemandchanges新業(yè)務(wù)的快速部署SWinstall/upgradevs.physicalinstall&cable業(yè)務(wù)高度定制化E.g.deployuniqueservicechainsforeachcustomerorclassofcustomer5vCloudNFVPlatform6ETSINFVArchitectureReferencemodelService,VNF&InfrastructureDescriptionSampletextComputeHardwareStorageHardwareNetworkHardwarevSphereNSXVSANVirtualizationLayerEMS1VNF1EMS2VNF2VNF3NFVOEMS3VNFMNFVIVNFVIMvCloudDirectorvROPsvRLogInsightSiteRecoveryMgrFCAPSOpenStackAModularETSIConformingPlatformforSeamlessIntegration.OperationsandBusinessSupportSystems(OSS/BSS)運(yùn)營(yíng)商兩種NFV實(shí)現(xiàn)架構(gòu)CONFIDENTIAL7NFV&IT統(tǒng)一基礎(chǔ)架構(gòu)平臺(tái)ComputeStorageNetwork計(jì)算池存儲(chǔ)池網(wǎng)絡(luò)池OPENSTACK/VCVNFVNFVNFVNFNFV平臺(tái)-NEP1硬件-NEP1資源池-NEP1NFVM&O-NEP1VNFVNFNFV平臺(tái)-NEPN硬件-NEP1資源池-NEP1NFVM&O-NEP1VNFVNFIT云平臺(tái)硬件-NEP1資源池-NEP1NFVM&O-NEP1VNFVNF+……NFV以NEP廠商為單位的分散平臺(tái)vsAgenda1什么是NFV2VMware的NFV之路3NFV的技術(shù)難點(diǎn)和解決方案4VMwareNFV案例CONFIDENTIAL8VMwareApproachtoNFV統(tǒng)一的資源池為NFV優(yōu)化的HypervisorE.g.Lowlatency/jitterpacketdelivertoVMs通過(guò)網(wǎng)絡(luò)虛擬化為融合網(wǎng)絡(luò)提供網(wǎng)絡(luò)自定義(L2-L7)NSX!=NFVNFV不僅僅就是VirtualComputerAgilityofnetworkingrequiredforNFVMulti-tenancyandisolationDecouplenetworkservicesfromphysicalinfrastructureDynamicservicechaining網(wǎng)元虛擬化優(yōu)化Insomecases,partnerwithNEPs9CONFIDENTIALContinuedenhancementsbasedonuserdemand投資保護(hù):VMwareIntegratedOpenstackforNFV10ProductionServicesvCloudDirectorAPIOpenStackMaturationProofsofConcept,TestDeploymentsCommercial@ScaleTelcoGradeLiveTrialServicesOpenContainerAPIvCloudNFVPlatformvRealizeTMOperationsVirtualSANNSXTMvSphereCloudFoundryAPIvCloudAPIVIOAPIContainersVMwareIntegratedOpenStack整體架構(gòu)Horizon(webportal)CLITools/SDKsvCAC/ITBMCostvisibility,governance,etc.vCenterNova

(compute)Neutron(network)Cinder(blockstorage)Glance

(images)vCenterDatastores3rd-party/VirtualSANHeat,Trove,Ceilometer,etc.(Additionalhigherlayerservicesinfuturereleases)Keystone(identity)LocalDB&LDAPSSOLogInsightLogcollection,O/SContentpackNSXvCOPsOpenStackmgmtpacks基于“OpenStackIcehouse”發(fā)行版VMware提供Openstack開(kāi)源軟件打包、優(yōu)化、支持一體化服務(wù)體系充分利用VMwareSDDC現(xiàn)有組件，和Openstack取長(zhǎng)補(bǔ)短Swift

(objectstore)BasicopensourcevSphereWebClientInstall,ConfigureandTroubleshootEMCViPRVIORoadmapSummaryVIO1.0ProductionReadyArchitectureAutomatedDeployment&ConfigurationAutomatedPatchingNovawithvCenterDriverGlanceandCinderwithVMDKdriversNSXvNeutronpluginL2/L3/securitygroups/floatingIPsAuthenticationw/ActiveDirectoryEncryptedPasswordsinconfigurationfilesFeaturesIcehouse-basedNova,Neutron,Cinder,Glance,Horizon,Keystone,HeatOpenStack

ReleaseLife-CycleMgmt.ofOpenStackServicesMulti-vCenterwithAZs&RegionsAutomatedUpgradesBackup&RecoveryProceduresSupportforvSphereReservations/LimitsSupportforguidedplacementofVMsLBaaSKeystone+ActiveDirectoryImprovementsHeatAuto-scalingwithCeilometerBrandingofHorizonPortalKiloBased+CeilometerVIO2.012

SDN技術(shù)方向的正確性是持續(xù)成功的重要保障！Overlay技術(shù)是服務(wù)器虛擬化的“絕佳伴侶”和必然方向OpenflowcontrolcontrolOverlayvsVirtualizationlayercontrol13通過(guò)虛擬化的方式來(lái)實(shí)現(xiàn)overlay以及相關(guān)服務(wù)是順勢(shì)而為!1.DecouplePhysicalVirtual2.Reproduce3.AutomateNetwork

OperationsCloud

Operations硬件無(wú)關(guān)性通過(guò)虛擬化帶來(lái)的運(yùn)維和自動(dòng)化優(yōu)勢(shì)不改變現(xiàn)有網(wǎng)絡(luò)VirtualPhysicalAgenda1什么是NFV2VMware的NFV實(shí)現(xiàn)之道3NFV的技術(shù)難點(diǎn)和解決方案4VMwareNFV案例CONFIDENTIAL15為NFV優(yōu)化的Hypervisor部分NFV應(yīng)用和傳統(tǒng)企業(yè)應(yīng)用的兩大不同點(diǎn)Sensitivitytolatency&jitterNeedforhighthroughputatsmallpacketsize需要Hypervisor的針對(duì)性性能優(yōu)化必要時(shí)候必須作出一些妥協(xié)Hardtojointlyoptimizeforlatency,throughputandefficiencyHardtomaintainalltheoperationalbenefitsofvirtualizationwhileoptimizingperformance16優(yōu)化ESXi的性能Manydifferentworkloadcharacteristics:

Latencysensitive,Lowjitter,Highthroughput,HighpacketratesESXi網(wǎng)絡(luò)架構(gòu)圖18LogicalViewPhysicalViewInterconnectESXi網(wǎng)絡(luò)數(shù)據(jù)路徑MessagecopyfromapplicationtoGOS(kernel)GOS(networkstack)+vNICdriverqueuespacketforvNICVMexittoVMM/HypervisorvNICimplementationemulatesDMAfromVM,sendstovSwitchvSwitchqueuespacketforpNICpNICDMAspacketandtransmitsonthewireNetworkSwitchVMsVirtualSwitchNICServerManagementAgentsBackgroundTasksESXiHypervisor19ESXi虛擬交換機(jī)和直連模式的數(shù)據(jù)路徑20VMwareESXivSwitchVirtualMachineGuestOSKernelApplicationIXGBEVirtualMachineGuestOS

KernelApplicationIXGBEVFDirectPathI/OSR-IOVVirtualMachineGuestOSKernelApplicationVMXNET3UplinkPhysicalNICLatencyscale21Latencyisameasureoftimedelayexperiencedinasystem,theprecisedefinitionofwhichdependsonthesystemandthetimebeingmeasured.(Wikipedia)Source:ESXi5.5對(duì)于延遲和抖動(dòng)的優(yōu)化新的虛擬機(jī)屬性:“Latencysensitivity”High=>lowestlatencyMedium=>lowlatency將物理CPU排他性的指定給虛擬CPU，“LatencySensitivity=High”VMsPhysicalCPUsnotusedforschedulingotherVMsorESXitasks當(dāng)GuestOS發(fā)生空閑的時(shí)候（idle），在VMM層面進(jìn)入空閑狀態(tài)LowerslatencytowakeuptheidleGuestOS,comparedtoidlinginESXivmkernel可開(kāi)關(guān)的vNICinterruptcoalescing在DirectPathI/O狀態(tài)下,優(yōu)化中斷的交付路徑優(yōu)化ESXivmkernel，實(shí)現(xiàn)更多的預(yù)先搶占（pre-emptible），更好的處理并發(fā)和中斷Reducesjitterduetolong-runningkernelcode22ESXi5.5網(wǎng)絡(luò)延遲和抖動(dòng)23Singe2-vCPUVMtoNative,RHEL6.2,RTTfrom‘ping–i0.001’IntelXeonE5-2640@2.50GHz,Intel82599EBPNICMedianreducedby15usoverVMXNET3,6usoverSR-IOV99.99thpercentilelowerthan50usPerformancegaptonativeisbetween2us-10usESXi5.5在InfiniBandDirectPathI/O下的超低延遲24HPProliantDL380Gen8,IntelE5-2667v2:2x8@3.0Ghz2x64GBMemoryMellanoxConnectX-3FDRInfiniBand/RoCEadapterMellanoxOFED(OpenFabricsEnterpriseDistribution)ver2.2ESXi實(shí)時(shí)虛擬化性能ISRLatencyInterruptperiodISRISRJittercyclictest–p99–a1–m–n–D10m–qESXi5.0ESXi5.1ESXiRTPrototypeMin5μsMax1676μsAvg13μsMin4μsMax1536μsAvg7μsMin2μsMax23μsAvg4μs99%-ile56μs99%-ile11μs99.99%-ile118μs99.99%-ile17μs2-socket8cores/socketIntelXeonE5-2650(SB)CPUs@2GHz,32GBRAM2vCPU,2GB64-bitLinuxVM,3.2.13-rt23kernel25ESXi5.5單主機(jī)對(duì)外的吞吐率測(cè)試(80Gbps)268PNICsover

8vSwitches16LinuxVMs

Apache4xIntelE5-465032cores,16threadsvSphere5.5IXIAXT80-V2

TrafficGeneratorHTTPGet75+Gbps4.1MillionPPS2.2K左右包大小HTTPPOST75+Gbps7.3MillionPPS1.2K左右包大小ESXi5.5vSphere內(nèi)部吞吐率(IPv4andIPv6）27Dirtying48GBRAMIntelXeon2.9GHzE5-266734.5Gbps

IPv432.5Gbps

IPv6Intel?DataPlaneDevelopmentKitLibrariesfornetworkapplicationdevelopmentonIntelPlatformsSpeedsupnetworkingfunctionsEnablesuserspaceapplicationdevelopmentFacilitatesbothrun-to-completionandpipelinemodelsFree,Open-sourced,BSDLicensed

Git:ScalesfromIntelAtomtomulti-socketIntelXeonarchitectureplatformsAbouttwodozenpre-builtexampleapplicationsEALMALLOCMBUFMEMPOOLRINGTIMERCoreLibrariesKNIPOWERIVSHMEMPlatformLPMEXACTMATCHACLClassifyE1000IXGBEVMXNET3IGBI40eVIRTIOPacketAccess(PMD–Native&Virtual)ETHDEVXENVIRTPCAPRINGMETERSCHEDQoSIntel?DPDKSampleApplicationsLinuxKernelUserSpaceKNIIGB_UIOCustomerApplicationsISVEco-SystemApplications3rdPartyNIC28SlideCourtesyIntelDPDKinvSphereESXiEnvironment29VMwareESXivSwitchVirtualMachineLinuxKernelDPDKAppIXGBEVMwareESXivSwitchVirtualMachineLinuxKernelUnmodifiedAppVMXNET3VMwareESXivSwitchVirtualMachineLinuxKernelDPDKAppVMXNET3VirtualMachineLinux

KernelDPDKAppIXGBEVFDirectPathI/OSR-IOVESXi5.5packet-rateswithDPDK30L3ForwardingPerformanceonVMwareESXi5.5CanoePassRomleyEP(IVTL1@2.3GHz)-8pCPUs/socketIntel?ServerBoardS2600CPIntel?Niantic10GbEEthernetControllerIntelVT-dIOTLBLimitationClosegapinfuturevSpherereleasewithVT-dSuperPagesupport(IntelIvyBridgeEPandnewer)FocusareaforfuturevSpherereleaseTypicalforTelcoCoreNetworksTypicalforEnterpriseDatacenterNetworks性能調(diào)優(yōu)通用指南vSphereESXi默認(rèn)是為整體的高性能和公平性設(shè)計(jì)MaximizesoverallperformanceofallVMswithoutunfairlypenalizinganyVMDefaultsarecarefullytunedforhighthroughput在一個(gè)測(cè)試環(huán)境部署生產(chǎn)環(huán)境之前，需要對(duì)可優(yōu)化參數(shù)做完整測(cè)試必要時(shí)按主機(jī)優(yōu)化，研究性能影響Maintaingoodchangecontrolpractices在一定情況下，對(duì)低延遲的調(diào)節(jié)可能會(huì)影響吞吐率和整機(jī)效率，因此需要考慮TradeoffConsiderisolatinglatency-sensitiveVMsondedicatedhostsDRShostgroupscanbeusedtomanagegroupsofhostssupportinglatency-sensitiveVMs31CONFIDENTIALPerformanceRelatedReferencesBestPracticesforPerformanceTuningofLatency-SensitiveWorkloadsinvSphereVMs

Intel?DataPlaneDevelopmentKit(Intel?DPDK)withVMwarevSphere

DeployingExtremelyLatency-SensitiveApplicationsinVMwarevSphere5.5

TheCPUSchedulerinVMwarevSphere5.1

RDMAPerformanceinVirtualMachinesusingQDRInfiniBandonVMwarevSphere5

32CONFIDENTIALCONFIDENTIAL33In2016:VMwareIntegratedContainers體系架構(gòu)BonnevilleDockerDaemonESXi服務(wù)器傳統(tǒng)虛擬機(jī)Vanilla

Docker

ClientDockerRESTAPIContainerContainerContainerMountImageDockerImagesPullImageDockerHub容器虛擬機(jī)

模板(運(yùn)行態(tài))容器成為虛擬化環(huán)境的頭等艙用戶不再需要管理Linux容器主機(jī)，ESX自身就是容器主機(jī)虛擬化帶來(lái)諸多好處：安全、隔離和多操作系統(tǒng)支持，遷移及高可用等Agenda1什么是NFV2VMware的NFV實(shí)現(xiàn)之道3NFV的技術(shù)難點(diǎn)和解決方案4VMwareNFV案例（國(guó)內(nèi)）CONFIDENTIAL34中國(guó)電信&中國(guó)移動(dòng)省公司部署NFV的前提網(wǎng)絡(luò)部已經(jīng)擁有云平臺(tái)，承載IT業(yè)務(wù)、網(wǎng)管業(yè)務(wù)和VAS業(yè)務(wù)運(yùn)營(yíng)商已經(jīng)完成全I(xiàn)P化改造對(duì)于CAPS有一個(gè)合理的預(yù)期，至少前置機(jī)具備橫向擴(kuò)展能力CONFIDENTIAL35浙江電信—iVPNNFV（企業(yè)一家）浙江電信業(yè)務(wù)平臺(tái)云資源池37浙江電信業(yè)務(wù)云計(jì)算資源池有兩個(gè)節(jié)點(diǎn)：紹興輕紡城節(jié)點(diǎn)，金華城南節(jié)點(diǎn)。計(jì)算資源：400余臺(tái)X86服務(wù)器；存儲(chǔ)資源：1PB；網(wǎng)絡(luò)資源：各類網(wǎng)絡(luò)設(shè)備40余臺(tái)。iVPN業(yè)務(wù)NFV案例說(shuō)明企業(yè)一家業(yè)務(wù)是中國(guó)電信以及中國(guó)聯(lián)通面向集團(tuán)用戶推出的、具有企業(yè)個(gè)性化特征的語(yǔ)音通信業(yè)務(wù)。集團(tuán)用戶的手機(jī)與集團(tuán)的固定分機(jī)實(shí)施綁定，形成統(tǒng)一的群，我們稱為：企業(yè)一家。手機(jī)與固定分機(jī)、手機(jī)與手機(jī)之間可以通過(guò)短號(hào)進(jìn)行群內(nèi)互相撥打，以提高企業(yè)通信效率，降低溝通成本主要NFV的網(wǎng)元包括：東信IPS200,32*2M(信令接入點(diǎn)）iSCP&SDP(運(yùn)行scp應(yīng)用）SMP(運(yùn)行smp應(yīng)用）CDR(運(yùn)行數(shù)據(jù)處理業(yè)務(wù)）INES運(yùn)行連接sdp的soap接口程序，web運(yùn)行webserver數(shù)據(jù)庫(kù)是Informix雙機(jī)熱備38iVPN云化部署架構(gòu)&SDP節(jié)點(diǎn)部署說(shuō)明本次iVPN數(shù)據(jù)庫(kù)輕量化改造配置7個(gè)SDP節(jié)點(diǎn)，任意兩個(gè)SDP節(jié)點(diǎn)都采用ER方式同步，也就是說(shuō)SCP連接到這7個(gè)SDP節(jié)點(diǎn)中的任意一個(gè)，更新的數(shù)據(jù)都會(huì)被同步到其他6個(gè)SDP點(diǎn)。通過(guò)SDP數(shù)據(jù)庫(kù)集群的建立，與SCP形成多點(diǎn)分散接入，一方面可以保住SDP整體處理能力維持在較高水準(zhǔn)，另一方面起到了較好的容災(zāi)效果，任意1個(gè)或幾個(gè)SDP節(jié)點(diǎn)的故障，可以比較方便的把故障SDP節(jié)點(diǎn)對(duì)應(yīng)的SCP的數(shù)據(jù)庫(kù)操作請(qǐng)求轉(zhuǎn)移到其他SDP節(jié)點(diǎn)中，考慮到整體安全性，把SDP的7個(gè)節(jié)點(diǎn)分散在云資源池兩個(gè)節(jié)點(diǎn)中，金華部署4個(gè)SDP節(jié)點(diǎn)，紹興部署3個(gè)SDP節(jié)點(diǎn)，由于informixER模式的異步同步特性，對(duì)網(wǎng)絡(luò)延遲不敏感，經(jīng)測(cè)試，完全滿足業(yè)務(wù)運(yùn)行需求。iVPN業(yè)務(wù)NFV之前存在的主要問(wèn)題擴(kuò)展性弱：傳統(tǒng)E1中繼的信令接入方式，效率低、成本高、擴(kuò)展性較差；服務(wù)器主備雙機(jī)的結(jié)構(gòu)，在完成提高平臺(tái)安全性的同時(shí)，也給擴(kuò)容帶來(lái)了較大的限制，必須以2N的方式增加服務(wù)器設(shè)備；靈活較低:業(yè)務(wù)處理服務(wù)器的Unix架構(gòu)，由于其封閉性，在業(yè)務(wù)開(kāi)發(fā)方面，響應(yīng)效率較慢，不能滿足前端快速靈活開(kāi)展新業(yè)務(wù)的需求；施工周期長(zhǎng):無(wú)論是硬件的到貨和裝配，還是業(yè)務(wù)的加載與調(diào)測(cè)，均需要較長(zhǎng)的時(shí)間周期；建設(shè)成本較高:前置信令接入硬件交換機(jī)，Unix小型機(jī)的價(jià)格比較昂貴；機(jī)房空間:機(jī)架數(shù)量、供配電等配套資源占用很大，不符合節(jié)能減排的趨勢(shì)。40vSphere5.5對(duì)于延遲敏感性的語(yǔ)音業(yè)務(wù)的進(jìn)一步支持：支持vMotionCONFIDENTIALVPN業(yè)務(wù)NFV案例說(shuō)明NFV之前的部署架構(gòu)IBMAIX