大型分布式網(wǎng)絡(luò)課程第11周作業(yè)

Heartbeat項(xiàng)目是Linux-HA工程的一個(gè)組成部分，它實(shí)現(xiàn)了一個(gè)高可用集群系統(tǒng)。心跳服務(wù)和集群通信是高可用集群的兩個(gè)關(guān)鍵組件，在Heartbeat項(xiàng)目里，由heartbeat模塊實(shí)現(xiàn)了這兩個(gè)功能。heartbeat最的包括兩個(gè)動(dòng)資源接管模塊來接管運(yùn)行在對方主機(jī)上的資源或者服務(wù)。[的配置文件來定義信息傳遞的方式和協(xié)議等。它是一個(gè)新興的，2008年推出，但其實(shí)它并不是一個(gè)真正意義上的新，在2002年的時(shí)候有一個(gè)項(xiàng)目Corosync60%Openais.Corosync個(gè)完整的HA功能，但是要實(shí)現(xiàn)，更復(fù)雜的功能，那就需要使用Openaishb_guiHAcorosync+pacemaker+mysql+drbd實(shí)現(xiàn)y三、安裝四、編譯安裝五、安裝六、my與drbdyq數(shù)據(jù)的鏡像七、利用sh配置yql的高可用 部 以附件的形式上傳 的源碼 ，網(wǎng)上很容易望各位博友帶著下面的疑問去實(shí)現(xiàn)corosync+pacemaker+mysql+drbdmysql1、corosync是什么？pacemaker是什么？corosync與pacemaker的關(guān)系2、mysqldrbd3 corosync、pacemaker、y、drbd之間的關(guān)系立關(guān)系？各個(gè)節(jié)點(diǎn)會(huì)不會(huì)搶占資源使其出現(xiàn)腦裂(split-brain)？出現(xiàn)腦裂可以有fence設(shè)備自行解決corosynccorosyncOpenaisOpenais的一個(gè)子 ，相比Heartbeat這款很老很 corosync與HeartbeatcorosyncHeartbeatpacemaker：是眾多集群資源管理器(ClusterResourceManager) 個(gè)，其主要功能是管理來著集群信息層發(fā)送來的信息。Pacemaker是集群的,它管理邏輯和集群信息.集群信息更新通過Corosync通知到各個(gè)節(jié)點(diǎn).常見 有heartbeatv1-->haresourceshearbeatv2--->hearbeatv3--->pacemaker 如下圖corosync與pacemaker之間的關(guān)系mysql：drbd：DRBD:(distributedreplicationblockdevice)即分布式塊設(shè)行，，但是主從DRBD服務(wù)器之間是可以進(jìn)行調(diào)換的。、hosts##把主機(jī)名改[root@jie2~]#sed-is/`grepHOSTNAME/etc/sysconfig/network|awk-'=''{print/g#把主機(jī)名改[root@jie3~]#sed-is/`grepHOSTNAME/etc/sysconfig/network|awk-'=''{print/g[root@jie2~]#cat>>/etc/hosts<<[root@jie3~]#cat>>/etc/hosts<<、ssh互[root@jie2[root@jie2~]#ssh-keygen-trsa-P[root@jie2~]#ssh-copy-id-i.ssh/id_rsa.pub[root@jie3~]#ssh-keygen-trsa-P[root@jie3~]#ssh-copy-id-i.ssh/id_rsa.pub、關(guān)閉[root@jie2[root@jie2~]#chkconfig--delNetworkManager[root@jie2~]#chkconfigNetworkManageroff[root@jie2~]#serviceNetworkManagerstop[root@jie3~]#chkconfig--delNetworkManager[root@jie3~]#chkconfigNetworkManager[root@jie3~]#serviceNetworkManager、時(shí)間同步(博主用的是自己的ntp[root@jie2[root@jie2~]#ntpdate[root@jie3~]#ntpdate三、安裝corosync+pacemaker1）、安裝corosync+pacemaker[root@jie2~]#yum-yinstallcorosync[root@jie2~]#yum-y--nogpgchecksh-[root@jie3~]#yum-yinstallcorosync[root@jie3~]#yum-y--nogpgchecksh-4.el6.x86_64.rpmpssh-2.3.1-#節(jié)#節(jié)的操[root@jie2~]#cd[root@jie2corosync]#mvcorosync.conf.example[root@jie2corosync]#vim#Pleasereadthecorosync.conf.5manualcompatibility:totem#心跳信息傳遞version:secauth:#認(rèn)證信息threads:interface#定義心跳信息傳遞的接ringnumber:bindnetaddr: #綁定的網(wǎng)絡(luò)地址，寫網(wǎng)絡(luò)地mcastaddr: #多播地mcastport:ttl:#多播的端#生存周}}loggingfileline:to_stderr:to_logfile:#是否輸出在屏幕#定義自己的日to_syslog:#是否由syslog記錄日logfile: #日志文件的存放路debug:timestamp:#時(shí)間戳是否關(guān) AMF}}amfmode:}servicever:name: #pacemakercorosync}aisexecuser:root}[root@jie2corosync]#scpcorosync.conf##節(jié)的操[root@jie2corosync]#corosync-CorosyncClusterEngineAuthenticationkeygenerator.Gathering1024bitsforkeyfrom/dev/random.Presskeysonyourkeyboardtogenerateentropy(bits=#遇到這個(gè)情況，表示電腦的隨機(jī)數(shù)不夠，各位朋友可以不停的隨盤，或者也可以生成隨機(jī)[root@jie2corosync]#scpauthkey#把認(rèn)證文件到主機(jī)[root@jie2~]#servicecorosyncStartingCorosyncClusterEngine[OK[root@jie2Lastupdated:ThuAug814:43:13Lastchange:SunSep116:41:182013_attributeStack:classicopenais(withCurrent-partitionwithVersion:..-7.el6-2Nodesconfigured,2expectedOnline:][root@jie3~]#servicecorosyncStartingCorosyncClusterEngine[OK[root@jie3Lastupdated:ThuAug814:43:13Lastchange:SunSep116:41:182013_attributeStack:classicopenais(withCurrent-partitionwithVersion:..-7.el6-2Nodesconfigured,2expectedOnline:]四、編譯安裝yl(##節(jié)的操#1)、解壓編譯安[root@jie2~]#tarxfmysql-PlatformDevelopment"[root@jie2~]#cdmysql-[root@jie2mysql-5.5.33]#yum-yinstallcmake[root@jie2mysql-5.5.33]#cmake.-DCMAKE_INSTALL_PREFIX=/usr/local/mysql\-DMYSQL_DATADIR=/mydata/data-DSYSCONFDIR=/etc-DWITH_INNOBASE_STORAGE_ENGINE=1-DWITH_ARCHIVE_STORAGE_ENGINE=1-DWITH_BLACKHOLE_STORAGE_ENGINE=1-DWITH_READLINE=1-\-DWITH_ZLIB=system-DWITH_LIBWRAP=0-\[root@jie2mysql-5.5.33]#make&&makeinstall [root@jie2mysql-5.5.33]#cp/usr/local/mysql/support-files/my- [root@jie2mysql-5.5.33]#cp/usr/local/mysql/support-files/mysql.server/etc/rc.d/init.d/mysqld[root@jie2mysql-5.5.33]#cd/usr/local/mysql/[root@jie2mysql]#useradd-r-u306mysql[root@jie2mysql]chownRroot:mysql*[root@jie2"PATH=/usr/local/mysql/bin:$PATH">/etc/profile.d/mysqld.sh[root@jie2mysql]#source/etc/profile.d/mysqld.sh[root@jie2mysql]#echo"/usr/local/mysql/lib"[root@jie2mysql]#ldconfig-v|grep[root@jie2[root@jie2mysql]#ln-sv/usr/local/mysql/include/先別初始化數(shù)據(jù)庫，安裝drbd把drbd掛載到 據(jù)庫的數(shù)據(jù)存放到drbd掛載的 安裝rpm包的drbd必須保證找相同內(nèi)核版本的drbd-d、先劃分一個(gè)分區(qū)，此分區(qū)做成drbd鏡像(RHEL6.x的重新格式化一個(gè)新[root@jie2~]#fdiskCommand(mforhelp):nCommandaction primarypartition(1-pPartitionnumber(1-4):cylinder(7859-15665,default7859):Usingdefaultvalue7859Lastcylinder,+cylindersor+size{K,M,G}(7859-15665,15665):Command(mforhelp): [root@jie3~]#fdisk/dev/sdaCommand(mforhelp):nCommandaction primarypartition(1-pPartitionnumber(1-4):Partitionnumber(1-4):cylinder(7859-15665,default7859):Usingdefaultvalue7859Lastcylinder,+cylindersor+size{K,M,G}(7859-15665,15665):Command(mforhelp):、安裝drbd和修改配置文#1)#1)、安裝#節(jié)的操[root@jie2~]#rpm-ivhdrbd-kmdl-2.6.32-358.el6-warning:drbd-kmdl-2.6.32-358.el6-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY#################################[root@jie2~]#rpm-ivhdrbd-8.4.3-warning:drbd-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY###################################節(jié)的操[root@jie3~]#rpm-ivhdrbd-kmdl-2.6.32-358.el6-warning:drbd-kmdl-2.6.32-358.el6-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY#################################[root@jie3~]#rpm-ivhdrbd-8.4.3-warning:drbd-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY###################################2)、修改drbd的配置文#節(jié) 的操[root@jie2~]#cd[root@jie2drbd.d]# mon.conf#全局配置文globalusage-count#minor-countdialog-refreshdisable-ip-}common

handlers{pri-on-incon-degr"/usr/lib/drbd/notify-pri-on-degr.sh;/usr/lib/drbd/notify-emergency-reboot.sh;echob/proc/sysrq-trigger;reboot-after-sb.sh;/usr/lib/drbd/notify-emergency-reboot.sh;echob>/proc/sysrq-trigger;reboot-lclio-error"/usr/lib/drbd/notify-io-/usr/lib/drbd/notify-emergency-shutdown.sh;echoo>/proc/sysrq-trigger;halt-f";#fence-peer -fence-#split-brain"/usr/lib/drbd/notify-split-

#out-of-sync"/usr/lib/drbd/notify-out-of-sync.sh#before-resync- -lvm.sh-p15---c#after- -}startup#wfc-timeout#degr-wfc-timeout}disk}net}

on-io-error#fencingresource-cram-hmac-alg"sha1";syncer}}

rate[root@jie2drbd.d]#catmydata.res#資源配置文resource{ meta-disk} meta-diskmeta-disk}}copy[root@jie2drbd.d]#mon.conf3）drbd##節(jié)的操drbd資[root@jie2~]#drbdadmcreate-mdWritingmetadata...initializingactivitylogNOTinitializingbitmaplk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:NosuchfileNewdrbdmetadatablocksuccessfullycreated.#提示已經(jīng)創(chuàng)建成功lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:Nosuchfileor#啟動(dòng)服[root@jie2~]#servicedrbdStartingDRBDresources:[createres:drbdpreparedisk:drbdadjustdisk:adjustnet:]#節(jié)的操drbd[root@jie3~]#drbdadmcreate-mdWritingmetadata...initializingactivitylogNOTinitializingbitmaplk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:NosuchfileNewdrbdmetadatablocksuccessfullycreated.#提示已經(jīng)創(chuàng)建成功lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:Nosuchfileor[root@jie3~]#servicedrbdStartingDRBDresources:[createres:drbdpreparedisk:adjustdisk:drbdadjustnet:drbd]、設(shè)置一個(gè)主節(jié)點(diǎn)，然后同 的數(shù)據(jù)(此步驟只需在一個(gè)節(jié)點(diǎn)上操作##設(shè)drbd的主節(jié)[root@jie2~]#drbdadmprimary--force[root@jie2~]#cat#查看同步進(jìn)version:8.4.3(api:1/proto:86-GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515buildbygardner@,2013-05-2704:30:210:cs:SyncSourcero:Primary/Secondaryds:UpToDate/InconsistentCr--ns:1897624ns:1897624nr:0dw:0dr:1901216al:0bm:115lo:0pe:3ua:3ep:1wo:f[=================>..]sync'ed:90.3%finish:0:00:07speed:26,792(27,076)[root@jie2~]#watchn1'cat/proc/drbd'[root@jie2~]#cat/proc/drbdversion:8.4.3(api:1/proto:86-101)GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515buildbygardner@,2013-05-2704:30:210:cs:Connectedro:Primary/Secondaryds:UpToDate/UpToDateCr----ns:120nr:354dw:435dr:5805al:6bm:9lo:0pe:0ua:0ap:0wo:fdrdb分區(qū)([root@jie2[root@jie2~]#mke2fs-text4六、myql與drbd實(shí)現(xiàn)yql數(shù)據(jù)的鏡、在drbd的主節(jié)點(diǎn)上，掛載drbd的分區(qū)，然后初始化數(shù)據(jù)[root@jie2[root@jie2~]#mkdir/mydata#創(chuàng)建用于掛drbd[root@jie2~]#mount/dev/drbd0[root@jie2~]#mkdir[root@jie2~]#chownRmysql.mysql/mydata#把文件的屬主和屬組改成[root@jie2 #修改mysql的配置datadir=innodb_file_per_table[root@jie2~]#/usr/local/mysql/scripts/mysql_install_db----datadir=/mydata/data/--#初始化數(shù)據(jù)[root@jie2~]#servicemysqldStartingMySQLStartingMySQL [ 、驗(yàn)證drbd是否鏡##節(jié)的操#1)、先在drbd的主節(jié)點(diǎn)上面創(chuàng)建一個(gè)數(shù)據(jù)[root@jie2~]#mysql>show++||++|information_schema|||performance_schema||++4rowsinset(0.00mysql>createdatabaseQueryOK,1rowaffected(0.01sec)mysql>showdatabases;++||++|information_schema|||||performance_schema||++5rowsinset(0.00#2)、停mysql服務(wù)，卸載drbd掛載[root@jie2~]#servicemysqld[root@jie2~]#umount/dev/drbd0#卸drbd的掛載[root@jie2~]drbdadmsecondarymydata#把此節(jié)點(diǎn)改為drbd#節(jié) 的操#3）、 變?yōu)閐rbd的主節(jié)[root@jie3~]#drbdadmprimary #把此節(jié)點(diǎn)改為drbd的主節(jié)[root@jie3~]#mkdir[root@jie3~]#mount/dev/drdb0/mydata[root@jie3~]#vim datadir=/mydata/datainnodb_file_per_table=1[root@jie3~]#servicemysqldstart#此節(jié)點(diǎn)上不用初始化數(shù)據(jù)庫，直接StartingMySQL [ [root@jie3~]#mysql>show #可以看見jie2數(shù)據(jù) | |information_schema| | |performance_schema| 55rowsinset(0.00七、利用sh配置yq的高可需要定義集群資源而yq、drbd都是集群的資源，由集群管理的資源開機(jī)是、關(guān)閉drbd的服務(wù)和關(guān)閉mysql的服[root@jie2[root@jie2~]#servicemysqld[root@jie2~]#servicedrbd[root@jie3~]#servicemysqld[root@jie3~]#umount/dev/drbd0#之drbd已經(jīng)掛載節(jié)點(diǎn)了[root@jie3~]#servicedrdb定義drbd的資源(提供drbd的資源RA由OCF類別中的linbit提供[root@jie2~][root@jie2~]#(live)configure#propertystonith-enabled=false(live)configure#propertyno-quorum-policy=ignore(live)configure#primitivemysqldrbdocf:linbit:drbdparamsdrbd_resource=mydataopmonitorrole=Masterinterval=10timeout=20opmonitorrole=Slaveinterval=20timeout=20opstarttimeout=240opstoptimeout=100#定義drbd的主從資(live)configure(live)configure#msms_mysqldrbdmysqldrbdmetamaster-max=1master-node-max=1clone-max=2clone-node-max=1notify=true(live)configure#(live)configure(live)configure#primitivemystoreocf:heartbeat:Filesystemparamsdevice="/dev/drbd0"directory="/mydata"fstype="ext4"opmonitorinterval=40timeout=40opstarttimeout=60opstoptimeout=60(live)configure#(live)configure#colocationmystore_with_ms_mysqldrbdinf:mystore(live)configure#orderms_mysqldrbd_before_mystoremandatory:ms_mysqldrbd:promotemystore:start(live)configure#定義vip資源、mysql服務(wù)的資源約束關(guān)(live)configure(live)configure#primitivemyvipocf:heartbeat:IPaddrparamsip="172.16.22.100"opmonitorinterval=20timeout=20on-fail=restart(live)configure#primitivemyserverlsb:mysqldopmonitorinterval=20timeout=20on-fail=restart(live)configure#(live)configure#colocationmyserver_with_mystoreinf:myserver(live)configure#ordermystore_before_myservermandatory:mystore:startmyserver:start(live)configure#(live)configure#colocationmyvip_with_myserverinf:myvip(live)configure(live)configure#ordermyvip_before_myservermandatory:myvip(live)configure#(live)configure#\attributes\attributesprimitivemyserverlsb:mysqldopmonitorinterval="20"timeout="20"on-fail="restart"primitivemysqldrbdocf:linbit:drbd\paramsdrbd_resource="mydata"opmonitorrole="Master"interval="10"timeout="20"\opmonitorrole="Slave"interval="20"timeout="20"\opstarttimeout="240"interval="0"\opstoptimeout="100"interval="0"primitivemystoreocf:heartbeat:Filesystem\paramsdevice="/dev/drbd0"directory="/mydata"\opmonitorinterval="40"timeout="40"\opstarttimeout="60"interval="0"\opstoptimeout="60"interval="0"primitivemyvipocf:heartbeat:IPaddr\paramsip="172.16.22.100"\opmonitorinterval="20"timeout="20"on-fail="restart" -msms_mysqldrbdmysqldrbdmsms_mysqldrbdmysqldrbdmetamaster-max="1"master-node-max="1"clone-max="2"clone-node-max="1"notify="true"colocationmyserver_with_mystoreinf:myservercolocationmystore_with_ms_mysqldrbdinf:mystorems_mysqldrbd:Mastercolocationmyvip_with_myserverinf:myvipmyserverorderms_mysqldrbd_before_mystoreinf:ms_mysqldrbd:promoteordermystore_before_myserverinf:mystore:startmyserver:startordermyvip_before_myserverinf:myvipmyserverproperty$id="cib-bootstrap-options"dc-version="1.1.8-7.el6-394e906"cluster-infrastructure="classicopenais(withplugin)"\expected-quorum-votes="2"\stonith-enabled="false"no-quorum-查看資源運(yùn)行的狀態(tài)運(yùn)行 [root@jie2[root@jie2Lastupdated:ThuAug817:55:30Lastchange:SunSep116:41:182013via _attributeonStack:classicopenais(withplugin)Current-partitionwithVersion:1.1.8-7.el6-2Nodesconfigured,2expected5ResourcesOnline:]Master/SlaveSet:ms_mysqldrbdMasters:]Slaves:Slaves:]myvip [root@jie3 node #把此節(jié)點(diǎn)設(shè)置為備用節(jié)[root@jie3 Lastupdated:MonSep201:45:07Lastchange:MonSep201:44:592013via _attributeonStack:classicopenais(withplugin)CurrentDC: -partitionwithquorumVersion:1.1.8-7.el6-394e9062Nodesconfigured,2expected5Resourcesconfigured. :standbyOnline:[ Master/SlaveSet:ms_mysqldrbdMasters: #資源已然轉(zhuǎn) 上Stopped:[mysqldrbd:1 myvip(ocf::heartbeat:IPaddr): 由于定義了drbd的資源約束，Masters運(yùn)行在那個(gè)節(jié)點(diǎn)，則此節(jié)點(diǎn)不可能成drbd[root@jie3[root@jie3~]#catversion:8.4.3(api:1/proto:86-GIT-hash:GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515buildbygardner@,2013-05-2704:30:210:cs:Connectedro:Primary/Secondaryds:UpToDate/UpToDateCr-----ns:426nr:354dw:741dr:6528al:8bm:9lo:0pe:0ua:0ap:0wo:f[root@jie3~]#drbdadmsecondary0:Statechangefailed:(-12)DeviceisheldopenbysomeoneCommand'drbdsetupsecondary0'terminatedwithexitcode手動(dòng)的停掉myvip還是會(huì)啟動(dòng)（因?yàn)槎x資源是指on-[root@jie2[root@jie2~]#ifconfig|grepLinkinetHWaddrinet6addr:fe80::20c:29ff:fe1f:74cf/64Scope:LinkUPBROADCASTRUNNINGMULTICASTMTU:1500RXpackets:216