新浪云計算實踐

1、OpenStack in Sina新浪云計算實踐AgendaOpenStack OverviewArchitecture AnalysisIntegration ChangesSina ContributionsAWS模式的巨大成功構(gòu)建了完整的云計算生態(tài)系統(tǒng)通過Web Service(API)管理一切服務(wù)*完全面向服務(wù)架構(gòu)SOA(Service-Oriented Architecture)*事實上的IaaS 標(biāo)準(zhǔn)成功的商業(yè)模式*https:/112678702228711889851/posts/eVeouesvaVX*/articles/5701.htmlMore Detail: http:

2、/products/OpenStack橫空出世目標(biāo)：AWS開源實現(xiàn)Rackspace & NASA聯(lián)合成立SwiftNovaOpenStack CompaniesMore detail: /community/companies/來源：OpenStack, OpenNebula，Eucalyptus，CloudStack社區(qū)活躍度比較/?p=1856Open Source Apache 2.0 license, NO enterprise version Open Design Open Design Summit Open Development Anyone can involve dev

3、elopment process Open development management via Launchpad & Github Open Community OpenStack Foundation in 2012 OpenStack MissionTo produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implem

4、ent and massively scalable.OpenStack ProjectsCore ProjectsOpenStack Compute(Nova)OpenStack Object Storage(Swift)Image Service (Glance)Identity (Keystone)Dashboard (Horizon)Network Connectivity (Quantum)Community ProjectsMelangeAltas-LBCrowbarJujuRedDwarfBurrowAWSOpenStackEC2novaS3swiftEBSnova-volume

5、ELBAtlas-LBSQSBurrowConsoleDashboardIAMKeystoneVPCQuantumRDSRedDwarfArchitecture OverviewDetail OverviewWhere to Get Started?Ubuntu 12.04 server 集成OpenStackT申請測試賬號devstack.sh一鍵安裝OpenStack DevelopmentBug trackingAuthorization(group membership)Feature planning(Blueprints)Hosting code &formal docsMaili

6、ng listsUser support(Answers)WikiContinuousintegrationInformal docsNova Key FeaturesReST-base APIAsynchronous communicationHorizontally scalableShared nothing architecture*Distribute everythingTest everything100% Python Based* http:/wiki/Shared_nothing_architecture* /BasicDesignTenetsOpenStack Compu

7、te: Novanova-apiCompute API ServerOpenStack API, EC2 compatibility APInova-computeCompute workerManage compute host and VMsLibvirt(QEMU,KVM,LXR), XenServer and XCP, ESX(i)*nova-networkNetwork controllerManage network resources: IPAM, VLAN, NAT*/HypervisorSupportMatrixOpenStack Compute: Nova(cont.)no

8、va-schedulerDetermines the placement of new resourcesnova-volumeBlock storage, remote attach a LVM volume using iSCISI protocolLike Amazon EBS, but far way from matureRabbitMQMessage QueueCast and RPC Call for services Keystone: ConceptUser/TenantAuthentication/AuthorizationTokenService/EndpointRole

9、Keystone: User CaseNova NetworkL2FLAT, FLATDHCP, VLANL3IPAM(IP Address Management)Fixed IP, Floating IPGateway, NAT, VPNQuantumQuantum BasicsNova: virtual serverQuantum: virtual networkQuantum :Expose a API for creating virtual networks and attaching instances(e.g.,novaservers) to those networksMana

10、ge switches(virtual or physical) in the data center to implement connectivity described via APIProvide a“plugin” architecture to leverage support using different back-end technologiesQuantum: available pluginsOpen vSwitch - Builds isolated networks with OVS and L2-in-L3 tunnelCisco UCS - Isolation b

11、ased on VLAN and net-profiles applied to Cisco UCS converged network adaptersLinux Bridge - Build isolated networks with VLAN interfaces and linux bridges - Works with every Linux DistroNTT-Data Ryu - Acts as a proxy for the NTT Ryu platformNicira NVP - Acts as a proxy for the Nicira NVP platformSwi

12、ft: Storage TypesTypesProtocolApplicationBlock StorageSATA, SCISI, iSCISISAN, NAS, EBSFile StorageExt3/4, XFS, NTFSPC, Servers, NFSObject StorageHTTP, RESTAmazon S3, Google Cloud Storage, Rackspace Cloud FilesSpecific StorageSpecific protocol based on tcpMySQL, MongoDB, HDFSWe want a Object Storage

13、like Amazon S3.Swift vs Amazon S3FeaturesSwiftAmazon S3object/bucket CRUDaccount/bucket/object ACL object metadatelarge objectrate limitexpiring objectstatic webREST APIAccount supportXAccount metadataXBucket metadataXBucket sync across clusterXObject versioningXLog to bucketXNotificationXReduced Re

14、dundancy StorageXSOAP APIXServer Side EncryptionXBitTorrent protocolXSwift EvaluationExtremely Durable and Highly AvailableSuperior ScalabilityLinear Growth of PerformanceSymmetric Architecture No Single-failureSimple & ReliableSwift ComponentsThe Ring: Mapping of names to entities (accounts,contain

15、ers, objects) on disk.Stores data based on zones, devices, partitions, and replicasWeights can be used to balance the distribution of partitionsUsed by the Proxy Server for many background processesProxy Server: Request routing, exposes the public APIReplication: Keep the system consistent, handle f

16、ailuresUpdaters: Process failed or queued updatesAuditors: Verify integrity of objects, containers, and accountSwift ArchitectureLoad BalancerProxy ServerObject ServerContainer ServerAccount ServerZone1Proxy ServerObject ServerContainer ServerAccount ServerZone2Proxy ServerObject ServerContainer Ser

17、verAccount ServerZone3Proxy ServerObject ServerContainer ServerAccount ServerZone4Proxy ServerObject ServerContainer ServerAccount ServerZone5PUT abc.pngGET abc.png1 Zone = 1 Physical Server with 12x2T diskWrite/Read applies quorum protocol31Swift InstallationPhysical Deploymentdisk1disk2disk3disk4s

18、dasdbsdcdisk5sdddisk12sdkStorage NodesOS installationSwift packagesProxy ServerAccount ServerContainer ServerObject Serverraid 1Conclusion核心功能基本可用，但穩(wěn)定性需要加強云服務(wù)(web service)比較豐富起步雖晚，但發(fā)展飛快，OpenStack生態(tài)系統(tǒng)正在形成邏輯結(jié)構(gòu)清晰、文檔豐富、源碼規(guī)范易懂，便于二次開發(fā)Open Source | Desgin | Development | CommunityIntegration ChallengesBest

19、 Network TopologySecurity EnhancementLoad BalancerCDN ServicesMetering & BillingInfrastructure & PlatformPhysical ServersTraditional OperationVirtualization Platform(IaaS)VM Management System(VMMS) Sina Web Service(SWS)VMMS is private solution developed in-houseSWS is based on OpenStackApplication P

20、latform(PaaS)Virtual Host Sina App Engine(SAE)SAE provides both Public and Private Service.Proved to be Efficient and Robust新浪云計算Nova NetworkNetworking is the biggest challenges for IaaSNetwork Topology: VLANFlatDHCPFlatDHCP & MultihostNetwork Topology (VLAN)Drawback:Pre-allocate network for future

21、projectsHard-limit of vlan 4096Traffic bottleneck in the gateway/NATCapability:Accessibility of VMs within one tenantIsolation of VMs from different tenantsVM is able to access public networkVM can be accessible from public networkIsolation between virtual network and internal networkNetwork Topolog

22、y(Flat)Capability:Accessibility of all VMs in the fixed IP rangeVM is able to access public networkVM can be accessible from public networkFull isolation between virtual network and internal networkBonus:Do not need pre-allocate for new projectsEliminating bottleneck between tenantsDrawback:Tenant i

23、solation has goneTraffic bottleneck still exists in NATNetwork Topology(Flat & Multihost)Capability:Accessibility of all VMs in the fixed IP rangeVM is able to access public networkVM can be accessible from public networkBonus:Totally distributed architecture avoid single-point failure.Multiple gate

24、way eliminates NAT bottleneckHigh speed between OS regionsDrawback:Tenant isolation lessensNeed security facility(SWS-filter) to protect intranetIf security problems were solved, this would be our best choice!Security in OpenStackStatic filters - L2 FilterMAC, IP, and ARP spoofing protectionNot conf

25、igurableDefined in /etc/libvirt/nwfilter/*.xmlImplemented by ebtablesebtables -t nat -listSecurity Group - L3 FilterRole-based firewallOne security group is a RoleIngress filteringTarget is the instanceSource can be CIDR or another groupImplemented by iptablesSee details: iptables -t filter -n -LWhi

26、telist mechanism(ACCEPT rules)Security EnhancementSWS Filter Prevent Intranet PenetrationIntranet is the internal network outside of OpenStackEgress filteringTarget is internal networkSource is instances in OpenStackImplementationWhitelist mechanism(ACCEPT rules)On the top of nova-filter-top Forward

27、 ChainRationalSWS filter is managed by cloud manager Only explicit authorized packets can reach Internal network C Packet should be controlled within Compute NodeSecurity EnhancementSecurity Group VS SWS Filter Load BalancerGoalsLoad Balance Dispatch requestSupport multiple routing algorithmHealth c

28、heckAccelerationReality: narrow bandwidth between ISPsBuilding fiber channels from ISPs to pivotGiven the same endpoint within users ISPIPv4 ShortageReality: dozens of public IPs support hundreds of VMsIPv4 has been exhaustedIPv6 is not realistic yet in ChinaUnicomOthers ISPMobileTelecomPivotSmart DNSDNS