數(shù)據(jù)中心液冷技術(shù) - 液冷需求白皮書

IFOREWARDInChinaandtheUnitedStates,datacentersaregrowinginnumber,size,andintensity,andconsumealotofenergy.ChinaandtheUnitedStatesarebothcommittedtoimproveenergyefficiency,buthavealsodevelopedinconsistentstandards,specifications,andtestingprotocols.InordertopromotetheinfluentialInternetandITcompaniesinChinaandtheUnitedStatestoexchangeandcoordinatedatacenterstandardsandspecifications,togivemanufacturersclearmarketsignals,therebypromotingtechnologycompatibilityandcostreduction,achievinghighefficiencyandlowemissionofdatacenters.InitiatedbyLawrenceBerkeleyNationalLaboratory(LBNL),LBNLandChinaInstituteofElectronics(CIE)jointlylaunchedtheU.S.-ChinaGreenDataCenterBilateralWorkingGrouptogetherwithGoogle,Facebook,Microsoft,Intel，Baidu,Tencent,Alibaba,andJDCloud.TheworkinggroupissupportedbytheU.S.DepartmentofEnergyandtheChineseMinistryofIndustryandInformationTechnology.數(shù)量、規(guī)模、強(qiáng)度均不斷增長，消。中國和美國均致力于提高能源利用效率，但同時也形標(biāo)準(zhǔn)、規(guī)格和測試協(xié)議。為推動中美兩國有影響力的互LiquidCoolingRequirementsWhitePaper液冷需求白皮書驗室(LBNL)會同中國電子學(xué)會(CIE)聯(lián)合百度(Baidu)、阿工作組。該項工作得到了美國能源部和中國工業(yè)和信Liquidcoolingisthecurrentfocusofthebilateralworkinggroup.Giventhedevelopmentofeachliquidcoolingtechnologyscheme,itisnotyetpossibletoprovethatthesolutionisoptimal.Thetechnicalsolutionspreferredbythemembersoftheworkinggroupsaredifferent.Themethodsoftheworkinggrouphavebeencontinuouslyevolved.Thefinaldeterminedmethodis:toapproachfromtheuser'sperspectiveandinaccordancewiththebusinessdevelopmentplan,toexploretheneedforliquidcooling.Lastly,theformationandjointreleaseofthe"LiquidCoolingRequirementsWhitePaper."Inthelaterstage,accordingtothetechnicalsolutionsoftheirrespectiveinterests,theInternetcompaniesinterestedintheprogramwilljointlyworkwiththetechnicalsolutionprovidertoformaworkinggrouptodiscussthespecifictechnicalstandardsaroundthe“LiquidCoolingRequirementsWhitePaper”.Thespecificworkframeworkisasfollows.U.S.-ChinaGreenDataCenterBilateralWorkingGroup作組液體冷卻是聯(lián)合工作組目前的工作重點。鑒于各個液冷技術(shù)方展階段，尚無法證明那種方案是最優(yōu)解。且各工作組成員方案有所不同。工作組的工作方法經(jīng)過不斷演化，最終確形成并聯(lián)合發(fā)布“液冷需求白皮書”。后期根據(jù)各自所感解決方案，由對該方案感興趣的互聯(lián)網(wǎng)企業(yè)聯(lián)合該技術(shù)解paperwasreviewedrevisedandconfirmedbythebilateralworkinggroupmembersandnowitwillbereleasedpublicly.LiquidCoolingRequirementsWhitePaper液冷需求白皮書經(jīng)過全體聯(lián)合工作組成員的討論、修改、確認(rèn)，聯(lián)合發(fā)布本液TeamLeader(召集人):rtorLBNLChineseConvener(中方召集人)：郭豐(GuoFeng)(CIE)王娟(WangJuan)(CIE)TeamMember(成員)：沈波(BoShen)(LBNL)exanderGoogle宋甲英(SongJiaying)(CIE)唐虎(TangHu)(Baidu)sMaloneGoogleImwalleGoogleusudanIyengarGoogleeilFarshchianGoogleVeerendraMulay(Facebook)hujaIntelelPattersonIntelSprengerIntelyWinkelIntel夏宇陽(XiaYuyang)(Intel)李孝眾(LiXiaozhong)(Baidu)rayBaiduoBaiduyLiuAlibaba周天宇(ZhouYangfan)(Alibaba)朱華(ZhuHua)(Tencent)ent梅方義(MeiFangyi)(Tencent)黃軼彪(HuangYibiao)(JDCloud)魏莉莉(WeiLili)(JDCloud)RubensteinMicrosoftDrafter(起草人)：郭豐(GuoFeng)(CIE)Translator(翻譯)：葉嶸(YeRong)ContcntsFOREWARDTEXTPartI.Use-CaseScenarioforLiquidCooling -1-第一部分：液冷應(yīng)用場景 -1-PartII.OperatingSpecificationforLiquidCoolingSystem -6-第二部分：液冷系統(tǒng)運行參數(shù) -6-A.GeneralSpecifications -6- 1.OverallRequirements -6-1、散熱總需求 -6-2.SpecificationsforInfrastructure -8-2、基礎(chǔ)設(shè)施要求 -8-3.SpecificationforOperation&Maintenance -12-3、運行與維護(hù)要求 -12-4.Others -14-4、其他 -14-B.GeneralPrinciplesforUseofEquipment -15- Contactinformation -15- 15--1-TEXTPartI.Use-CaseScenarioforLiquidCooling部分：液冷應(yīng)用場景Itisexpectedthatovertimetheeverincreasingdatavolumewilldemandprocessingchipswithhigherpowerdensity;fasterdatatransmission,andeasiness-to-userequirementswoulddrivetheadoptionofmachineswithincreasinglyhigherenergydensity.Naturallyitwillbemorechallengingforheatmanagementindatacenters.的發(fā)展趨勢是，對數(shù)據(jù)處理能力需求的快速增長將提高。由此會帶來更高的散熱需求。Astheheatloadincreases,conventionalaircoolingmethodsareunderincreasingpressure.Datacentersneedmorepowerfulfanswithlargerdiameter,largerspacetoallowheatdissipation,moreefforttomaintainacceptableoperatingenvironment,etc.AllthesewillresultinLiquidCoolingRequirementsWhitePaper液冷需求白皮書-2-highairnoise,thermalimpacttotheenvironment,andwillbetranslatedintoincreasedCAPEXandOPEX.配備更高轉(zhuǎn)速及更大直徑的風(fēng)扇、更大體積的Onthecontrary,liquidcoolingtechnologyisabletohandlehigherheatloadwithminimalnoiseandloweroperatingcost.ItalsogivesmoreleewaytoembracethecomingtechnologyadvancementsforITequipment,andallowsforheatrecoveryandreusesoastoachievehigherenergyefficiencyandlowerenvironmentalimpact.ButinsomecasesliquidcoolingcomeswithhigherCAPXinvestment,whichmightnotbeanattractivesolutionforlowerpowerdensityscenarios.熱量回收及綜合利用以提高能源利用效率及減少對環(huán)境的影響。情況下建設(shè)成本相對較高，對于低功率密度的應(yīng)用場景不具有Inlightoftheaboveconsiderations,webelievethatonceheatloadperstandardunitinthe19inchrackreachesaround500W,liquidcoolingwouldbecomepopular.Liquidcoolingwouldbecomeawidelyacceptedcoolingmethodwithconsiderablemarketshare.AndonceheatU.S.-ChinaGreenDataCenterBilateralWorkingGroup作組-3-loadperstandardunitequipmentreachesaround1000W,liquidcoolingwillbecometheprimarycoolingmethodoreventhesolemethod.一種能夠被廣泛接受并占有相當(dāng)份額的散熱形式。當(dāng)機(jī)柜內(nèi)(Note1:SpecificconsensusisyettobereachedontheevolutionarypathfortherelationbetweenaliquidcoolingapplicationandaverageperingleITequipmentandortheevolutionarypathfortherelationwithsinglechippowerdensity.Asliquidcoolingistobeusedinsuchcases,relativecontentinthisWhitepapercanbeusedasreference.)于液冷應(yīng)用與單個IT設(shè)備的平均每標(biāo)準(zhǔn)UNIT演進(jìn)路徑，以及與單芯片功率密度之間關(guān)系演進(jìn)路徑，(Note2:StandardUNITisaunitofthicknessforservers.ItisputforwardbyEIAandhasbeenwidelyadoptedbytheindustry.ItcanbeabbreviatedinshortasU,where1U=4.445cm.)(補(bǔ)充說明2：標(biāo)準(zhǔn)UNIT即美國電子工業(yè)協(xié)會(EIA)所提出并LiquidCoolingRequirementsWhitePaper液冷需求白皮書-4-Thereforewebelieveitisnecessaryforthedatacenterdesignersandcoolingsolutionproviderstogiveenoughattentiontothistrend,andtoinvolvemoreorganizationstothedevelopmentofliquidcoolingtechnologies.情況下，我們認(rèn)為，有必要引起數(shù)據(jù)中心設(shè)計方及散熱方對此種趨勢的足夠重視，并吸引更多機(jī)構(gòu)參與到液冷技術(shù)的本。基于以上考慮，我們聯(lián)合發(fā)布本液冷需求白皮書，作為Asliquidcoolingtechnologiescontinuetogrow,thefirststepforadatacenteroperatoristodefinealiquidcoolingapplicationscenariointhedatacenter,andthenidentifyconcernsandoperatingspecificationrequirementsforthecoolingsystem.Thecoolingsolutionproviderscandevelopdesignsbasedonthosespecificrequirements.的用戶，我們將首先設(shè)定一個液冷散熱型數(shù)據(jù)中心的應(yīng)后以數(shù)據(jù)中心用戶角度向液冷散熱型數(shù)據(jù)中心的設(shè)計總包方提出基于該場景下的對于液冷系統(tǒng)的關(guān)注點和相關(guān)的運行參數(shù)要該要求進(jìn)行相應(yīng)的開發(fā)。Wepicturetheusecasescenariotobeageneral-purposedatacenter，engagedinmultipleregularservicessuchascloudcomputing,videoprocessing,searchengine,etc.,whoseheatloadonaperstandardU.S.-ChinaGreenDataCenterBilateralWorkingGroup作組-5-unitequipmentina19inchrackisnolessthan500W,ormightbeashighas00Wormore,undercontinuesandstableoperation.Thefollowingcontentsinthiswhitepaperareoperatingspecificationsofacoolingsystemunderthisdefinedusecase.LiquidCoolingRequirementsWhitePaper液冷需求白皮書-6-PartII.OperatingSpecificationforLiquidCoolingSystem部分：液冷系統(tǒng)運行參數(shù)A.GeneralSpecificationsNo.nReferenceValueacity1.Theminimumracklevelcoolingcapacityis500Wperstandardunitin19inchrackunderstableoperation;thesupportedcoolingcapacitycanbe1000Worabove.ForcertaindatacenterITequipmentwithspecialcoolingneeds,thecoolingcapacityrequiredbyequipmentvendorsshouldbesatisfied.3.ForcertainITequipmentchipwithhigherpowerdensitieswhosevendorrecommendsliquidcooling,thecoolingcapacityrequiredbythevendorsshouldbesatisfied.U.S.-ChinaGreenDataCenterBilateralWorkingGroup作組-7-No.nReferenceValuenmentforITIT設(shè)備運行環(huán)境條件1.Nointerventionisrequiredonambienttemperatureorhumidity,unlessthedatacenterislocatedataplacewithspecialclimaticconditions,and/orinterventionisnecessarytoensurepersonalsafetyortomeetspecialrequiredoperatingconditionsforthecoolingsystem.2.Dustremovaland/orstaticproofisrequired.行維護(hù)人員人身安全或液冷系統(tǒng)運行所必需LiquidCoolingRequirementsWhitePaper液冷需求白皮書-8-onsforInfrastructureNo.nReferenceValueMinimum600mm.Incasewiderrackisneeded,therackwidthshouldbeincreasedby100mm.最小600mm，如果有必要增加機(jī)架寬度，需要按1.Theliquidpipingdesigninadatacentershouldbeconsistentandbasedononespecification.2.Dueconsiderationsshouldbemadeonsystemsecurityandserviceability,withsufficientvalvesandpumpsinplace.3.Dueconsiderationsshouldbemadeoncoolantrefillandreplacement.1、同一數(shù)據(jù)中心內(nèi)液體管路的設(shè)計需遵循同一neeringnsshouldbemadeon1.Theweightofthecoolingsystem.2.Possiblevibrationduringoperationoftheliquidcoolingsystem.3.Coolantleakage,anditspotentialdamagestobuildingstructuresandpotentialpersonalinjuries.U.S.-ChinaGreenDataCenterBilateralWorkingGroup作組-9-No.nReferenceValue3、應(yīng)充分考慮冷卻液泄漏可能造成的對建筑結(jié)構(gòu)的損害以及對數(shù)據(jù)中心運行維護(hù)人員的人Thecoolantusedshouldbenon-hazardoustohumans,environmentallyfriendly,fire-safe;anditspropertiessuchasoxicitycorrosivenessoxidizabilityvolatilityGreenhouseeffect/flammabilityshouldbecompliantwithpertinenthealthandsafetylawsandregulationsofthecountrywherethecoolingsystemisoperated.Thecoolantusedshouldhavestablechemicalandphysicalpropertiesthroughoutservicelife.Orchangesinchemicalandphysicalpropertieshavenoeffectoncoolingapplications.nconvenienceofoperationandCAPEX+OPEX.4.Itcanberecycledanddisposedsafely.可燃性等必須滿足數(shù)據(jù)中心所在國法律法規(guī)學(xué)與物理性質(zhì)的變化對其冷卻應(yīng)用不產(chǎn)生影LiquidCoolingRequirementsWhitePaper液冷需求白皮書-10-No.nReferenceValuesRequirementsarethesameasthecoolant.Inaddition,othermaterialsshouldmeetthellowings1.Dueconsiderationismadeoneasydisassembly.2.Recycledmaterialsandrecyclablematerialsareencouraged.delemetryAtmultiplesubsystemlevel,monitoringofoperatingstatusissupported;Centralizeddisplayandcontrolinterface(dashboard)issupported.2.Itsupportsremotecontrolofvalvesandpumpsatcriticalnodes;meanwhile,manualstart/stopfunctionremainsenabled.3.Distributeddisplaysandcontrolinterfaces,voicecommunicationssystemoneachsubsystemshouldbeadequatelyprovided,accordingtooperationandmaintenanceneeds.U.S.-ChinaGreenDataCenterBilateralWorkingGroup作組-11-No.nReferenceValue4.Dueconsiderationsaretobemadeonthesafety,reliabilityandfool-proofdesignforthecontrolandtelemetrysystem.5.Dueconsiderationsaretobemadeonautomatedcontrolfunctions.1、需具備對各級液冷分系統(tǒng)的運行情況進(jìn)行監(jiān)2、需要具備對各關(guān)鍵節(jié)點的閥門與泵的遠(yuǎn)程控3、應(yīng)結(jié)合運行維護(hù)需求針對各分系統(tǒng)設(shè)置足夠AnalysisreportingonliquidcoolingsystemfailuresisrequiredContingencyplanisrequired.2.Criticalcomponentredundancyorbackuptisrequired3.Sufficientsparepartsisrequired,particularlyeones1、需提供液冷系統(tǒng)故障模式分析報告及應(yīng)急方LiquidCoolingRequirementsWhitePaper液冷需求白皮書-12-orOperationMaintenanceNo.nReferenceValueSystemLevelServiceyearsFortheindependentliquidcoolingunitcomingwithoftheITequipment,theservicelifecanbeconsistentwiththeservicelifeoftheIT液冷單元，其使用壽命可與IT設(shè)備使用壽命一1.Coolingsystemasawholeshouldnevercrashunderanycircumstance.2.Singlepointoffailureintheliquidcoolingsystemand/orcomponentfailure,includingcoolantleakageshouldnotimpairnormaloperationofthedatacenterasawhole.1、任何情況下都不能出現(xiàn)液冷系統(tǒng)全系統(tǒng)功能的部件故障，包括冷卻液泄漏均不能影響數(shù)eabilityOnsiterepairorreplac