廣東移動(dòng)rvc系統(tǒng)sdu3數(shù)據(jù)庫(kù)集群vcs健康檢查報(bào)告

1、Business Critical Servi移動(dòng) RVC 系統(tǒng) sdu3 數(shù)據(jù)庫(kù)VCS 健康檢查Prepared By:Parco Liu Last Updated:2014-3-28目錄1.介紹3對(duì)象3目的3范圍3術(shù)語(yǔ)3結(jié)果31.11.21.31.41.52.客戶環(huán)境43.VCS 健康檢查項(xiàng)目5License Key 檢查5GAB port 檢查5VCS 心跳網(wǎng)絡(luò)檢查6VCS 日志檢查6VCS Cluster 節(jié)點(diǎn)及 Service Group 運(yùn)行狀態(tài)檢查6VCS 進(jìn)程檢查7VCS 配置文件的一致性檢查7文件系統(tǒng)使用狀況檢查103.13.23.33.43.53.63.73.84.結(jié)論10

2、1.介紹1.1對(duì)象本文檔是用來(lái)輔助 Symantec Business Critical Servi戶。的顧問(wèn)和 VCS/NBU 的用1.2目的本文檔用于評(píng)估移動(dòng) RVC 系統(tǒng) sdu3 數(shù)據(jù)庫(kù) VCS 集群的配置和運(yùn)行狀態(tài)是否良好。1.3范圍本文檔的范圍限制在檢查 VCS 產(chǎn)品.1.4術(shù)語(yǔ)HACluster VCS VXVM VXFS CVM CFSSFRACHigh AvailabilityA logical grouof servers (nodes) providing failover capabilityVERITAS Cluster Server VERITAS Volume M

3、anager VERITAS File System Cluster Volume Manager Cluster File SystemVERITAS Storage Foundation for RAC (包括 VCS 和 VXVM,支持 Oracle RAC)NetBackupNBU1.5結(jié)果通過(guò)系列健康檢查項(xiàng)目的成功完成，驗(yàn)證 VCS 當(dāng)前配置和運(yùn)行狀態(tài)是否良好。2.客戶環(huán)境移動(dòng) RVC 系統(tǒng) sdu3 的 VCS 集群由 2 臺(tái) LINUX 服務(wù)器組成，系統(tǒng)環(huán)境如下：系統(tǒng)健康檢查時(shí)間：2014 年 3 月 28 日symantec 工程師:用戶工程師:參加劉扳浩主機(jī)名操作系統(tǒng)版本管理

4、ORACLE 版本sdu3aSUSE10 SP2sdu3bSUSE10 SP23.VCS 健康檢查項(xiàng)目通過(guò)以下 8 個(gè)檢查項(xiàng)目，確認(rèn) sdu3a 和 sdu3b 兩主機(jī)上的 VCS 進(jìn)程、配置、運(yùn)行狀況以及性能等都處于良好的狀態(tài)。3.1 License Key 檢查需要確認(rèn)兩節(jié)點(diǎn)上的 License Key 都為L(zhǎng)icense。非License Key 過(guò)期后，一旦系統(tǒng)重啟， VCS 進(jìn)程將無(wú)法正常啟動(dòng)。在生產(chǎn)系統(tǒng)上，要避免使用Demo License.使用 vxlicrep 命令可以檢查 License 的類型.以下為L(zhǎng)icenses 的輸出：的檢查主機(jī)檢查結(jié)果：VERITAS Cluste

5、r Server 的License key 為。3.2 GAB port 檢查VCS 的正常運(yùn)行，有賴于各包含模塊的正常運(yùn)行。使用主機(jī)上運(yùn)行了那些模塊，以下是gabconfig a 輸出：gabconfiga 命令可檢查檢查結(jié)果： GAB Port 特性均正常。sdu3a: # gabconfig -a GAB Port Memberships=Port a gen5c0401 membership 01 Port h gen5c0407 membership 01Product Name= VERITAS File SystemLicense Type= PERMANENTProduct N

6、ame= VERITAS Cluster ServerLicense Type= PERMANENTProduct Name= VERITAS Volume ManagerLicense Type= PERMANENTProduct Name= VERITAS Storage Foundation Standard HALicense Type= PERMANENTProduct Name= VERITAS Volume ManagerLicense Type= PERMANENTProduct Name= VERITAS File SystemLicense Type= PERMANENTP

7、roduct Name= VERITAS Database Edition for OracleLicense Type= PERMANENTProduct Name= VERITAS SANPoControlLicense Type= PERMANENT3.3 VCS 心跳網(wǎng)絡(luò)檢查VCS 通過(guò)心跳網(wǎng)絡(luò)傳遞 cluster 狀態(tài)信息，當(dāng)前的每個(gè) VCS 節(jié)點(diǎn)都配置了兩條心跳線。VCS 心跳網(wǎng)絡(luò)運(yùn)行 LLT/GAB 協(xié)議，跳網(wǎng)絡(luò)是否正常, 檢查輸出如下:可以通過(guò)檢查 LLT 的狀態(tài)來(lái)確認(rèn)心檢查結(jié)果：節(jié)點(diǎn)的心跳網(wǎng)絡(luò)是健康的。3.4 VCS 日志檢查檢查 VCS 的日志文件， 察看近期是否有異常信息

8、。VCS 日志文件是/var/VRTSvcs/log/engine_A.log ，可以在 database 的 VRTSexplorer 的包中來(lái)查看 log 文件的修改時(shí)間和察看最近有沒(méi)有異常狀況。檢查結(jié)果：VCS 日誌近期無(wú)異常信息。3.5 VCS Cluster 節(jié)點(diǎn)及 Service Group 運(yùn)行狀態(tài)檢查檢查 2 個(gè)節(jié)點(diǎn)的 VCS 運(yùn)行狀態(tài)以及所有的Service Group 運(yùn)行狀態(tài)，如下：sdu3a: # hasus -summary- SYSTEM SE- SystemSeFrozenA sdu3aRUNNING0A sdu3bRUNNING0- GROUP SE2014/0

9、3/28 14:39:55 VCS INFO V-16-2-13716 (sdu3a) Resource(vgora): Output of the completed operation (monitor)=Using logical volume(s) ond line=2014/03/28 14:39:56 VCS INFO V-16-2-13716 (sdu3a) Resource(vgopt): Output of the completed operation (monitor)=Using logical volume(s) ond line=sdu3a: # llts-nvv

10、LLT node information:NodeSeLink Sus Address* 0 sdu3aOPENeth10UP80:FB:06:35:B8:BD eth9UP80:FB:06:35:B8:BF eth8UP80:FB:06:35:B8:BE1 sdu3bOPENeth10UP80:FB:06:35:B8:21 eth9UP80:FB:06:35:B8:23 eth8UP80:FB:06:35:B8:22從以上輸出可以看出，sdu3a、 sdu3b 都在一個(gè) Cluster 集群內(nèi)，運(yùn)行狀態(tài)正常，都為 RUNNING 狀態(tài)；各個(gè)服務(wù)組運(yùn)行正常。檢查結(jié)果：運(yùn)行正常。3.6 VCS

11、進(jìn)程檢查主要檢查 VCS進(jìn)程had 和hashadow 是否存在，參數(shù)是否正確。檢查結(jié)果：had 和hadshadow 進(jìn)程正常。3.7 VCS 配置文件的一致性檢查檢查 VCS 配置文件/etc/VRTSvcs/conf/config/main.cf 的一致性，結(jié)果應(yīng)無(wú)錯(cuò)誤信息。檢查輸出如下：檢查結(jié)果：VCS 的配置文件main.cf 通過(guò)一致性檢查。main.cf 配置文件內(nèi)容如下：sdu3a: # cat /etc/VRTSvcs/conf/config/main.cf include OracleASMTypes.cfinclude ENIPTypes.cfinclude types.

12、cfinclude BMEApacheTypes.cf include Db2udbTypes.cf include OracleTypes.cf include SybaseTypes.cfcluster sdu3Cluster (UserNames = admin = aLMeLGlIMhMMkUMgLJ ClusterAddress = 192.168.130.115 Administrators = admin )system sdu3a (ShutdownTimeout = 300)sdu3a: # hacf -verify /etc/VRTSvcs/conf/configroot1

13、28191 0 2012 ?00:23:32 /opt/VRTSvcs/bin/hadroot128211 0 2012 ?00:00:00 /opt/VRTSvcs/bin/hashadow- GroupSystemProbedAutoDisabledSeB ENIPStandbyGroup sdu3aYNOFFLINEB ENIPStandbyGroup sdu3bYNONLINEBENIPWithOraclePrimaryGroup sdu3aYN ONLINEBENIPWithOraclePrimaryGroup sdu3bYN OFFLINEB FaultSpyGrpsdu3aYNO

14、NLINEB FaultSpyGrpsdu3bYNOFFLINEsystem sdu3b (ShutdownTimeout = 300)groupENIPStandbyGroup (SystemList = sdu3b = 0, sdu3a = AutoStartList = sdu3b, sdu3a )1ENIPApp ENIPStandbyApp (Critical = 0 ENIPAppUser = see HotBackup = 1ActiveOrStandby = 0)/ resource dependency tree/group ENIPStandbyGroupENIPApp E

15、NIPStandbyAppgroupENIPWithOraclePrimaryGroup ( SystemList = sdu3a = 0, sdu3b = AutoStartList = sdu3a, sdu3b )1ATAELVMVolumeGroup vgopt (VolumeGroups = vgopt User = oracle)ATAELVMVolumeGroup vgora (VolumeGroups= vgora RawDevi= 15 = /dev/mapper/vgora-lvctl3,14 = /dev/mapper/vgora-lvctl2,13 = /dev/mapp

16、er/vgora-lvctl1,12 = /dev/mapper/vgora-lvredo6,11 = /dev/mapper/vgora-lvredo5,10 = /dev/mapper/vgora-lvredo4,987654321=/dev/mapper/vgora-lvredo3, /dev/mapper/vgora-lvredo2, /dev/mapper/vgora-lvredo1, /dev/mapper/vgora-lvuser, /dev/mapper/vgora-lvtemp, /dev/mapper/vgora-lvundo, /dev/mapper/vgora-lvsy

17、saux, /dev/mapper/vgora-lvsystem, /dev/mapper/vgora-lvspfile User = oracle)ATAELVMVolumeGroup vgphydb ( VolumeGroups = vgphydb RawDevi= 16 = /dev/mapper/vgphydb-lvdata,17 = /dev/mapper/vgphydb-lvindex User = oracle)ENIPApp ENIPPripp (ENIPAppUser = see HotBackup = 1)IP FloatIPservice (Device = bond1A

18、ddress = 192.168.130.115NetMask = 255.255.255.0Options = broadcast 192.168.130.255 OnlineRetryLimit = 3)Mount lvarchive (MountPo= /home/oracle/archive BlockDevice = /dev/mapper/vgopt-lvarchive FSType = reiserfsFsckOpt = -y)NIC NICservice (Device = bond1)OracleDB OracleDB (User = oracle RunMode = COL

19、D DBName = ora11gLISTENER_NAME = LISTENER ORACLE_SID = ora11gORA_CRS_HOME = /oracle/crs LISTENER_REP_NAME = LISTENER)ENIPPripp requires FloatIPserviceENIPPripp requires OracleDBFloatIPservice requires NICservice OracleDB requires lvarchive OracleDB requires vgoraOracleDB requires vgphydb lvarchive r

20、equires vgopt/resource dependency treegroup ENIPWithOraclePrimaryGroupENIPApp ENIPPrippIP FloatIPserviceNIC NICserviceOracleDB OracleDBMount lvarchiveATAELVMVolumeGroup vgoptATAELVMVolumeGroup vgora ATAELVMVolumeGroup vgphydbgroup FaultSpyGrp (SystemList = sdu3a = 0, sdu3b = 1 AutoStartList = sdu3a, sdu3b )SNE_FaultSpy FaultSpy (RestartResGrps = ENIPStandbyGroup 3.8 文件系統(tǒng)使用狀況檢查使用 df 檢查文件系統(tǒng)使用的狀況，輸出如下：檢查結(jié)果：可以看到各個(gè)文件系統(tǒng)空間使用率良好。在日常中需要注意避免使用率達(dá)到 100%的情況。4.結(jié)論通過(guò)本次對(duì)移動(dòng) RVC 系統(tǒng) sdu3 數(shù)