oracle 10g RAC linux vmware workstation FREENAS共享存儲.docx

RAC安裝文檔一 、規(guī)劃軟件規(guī)劃軟件名稱軟件版本備注信息系統(tǒng)rhel-server-5.5-x86_64系統(tǒng)內核2.6.18-194.el5-2.0.5-1.el5.x86_64Oracle cluster10201_clusterware_linux_x86_64Oracle database10201_database_linux_x86_64Oracle ASMoracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64oracleasmlib-2.0.4-1.el5.x86_64oracleasm-support-2.1.7-1.el5.x86_64FreeNASFreeNAS-i386-LiveCD-543.isoIp地址規(guī)劃Rac1Rac2Priv10/2411/24Public/24/24Vip53/2454/24Oracle_sidOradb1Oradb2二、安裝freeNAS，作為共享存儲FreeNAS是一套免費的NAS服務器，它能將一部普通PC變成網絡存儲服務器。該軟件基于FreeBSD，Samba 及PHP，支持CIFS (samba), FTP, NFS protocols, Software RAID (0,1,5) 及 web 界面的設定工具。用戶可通過Windows、Macs、FTP、SSH 及網絡文件系統(tǒng) (NFS) 來訪問存儲服務器；FreeNAS可被安裝于硬盤或移動介質USB Flash Disk上，所占空間不足 16MB。1、freeNAS軟件安裝將鏡像文件放入光驅，啟動選擇9安裝，回車選擇3，安裝完全系統(tǒng)在HDD+DATA+SWAP partition 將會創(chuàng)建三個分區(qū)1.OS 2.DATA 3.SWAP 選擇安裝介質 選擇安裝分區(qū) ad0 OS分區(qū)大小默認380M 是否創(chuàng)建是swap分區(qū)，yesSwap分區(qū)默認大小256M 完成安裝，重新啟動系統(tǒng)啟動完成后，設置ip地址：53/24設置完成重置web登錄密碼，選擇3重置web界面密碼為：freenas 帳號是：admin登錄：修改語言環(huán)境-修改為簡體中文：Systemgeneral: save 保存。2、添加磁盤，模擬存儲添加三塊磁盤做raid5 ，給rac做共享存儲-OK重新啟動NAS服務器，登錄管理界面找到磁盤管理點擊最右側下方的+號圖標，添加磁盤選項：已格式化的文件系統(tǒng)選擇 softraid，其他默認將三塊磁盤依次添加完成添加完成后-應用更改硬盤-軟raid選擇raid5 ，并點擊右側+號，添加Raid名稱為rac，選擇三塊硬盤勾選“建立并初始raid”-“添加”應用更改：初始化raid，中間需要幾分鐘時間；完成：3、配置ISCSI配置ISCSI目標：打開后，首先設置端口，默認端口是3260“應用更改”配置ISCSI服務發(fā)起端定義“應用更改”配置ISCSI服務目標端定義-“應用更改”打開“設置”勾選右側啟動保存并重啟服務4、主機連接freeNAS首先在服務器上安裝安裝iscsi，iscsi-initiator-utils.x86_64 0:71-0.16.el5，并開啟服務，使用iscsiadm連接freeNASrootrac1 # yum install *iscsi*iscsi-initiator-utils.x86_64 0:71-0.16.el5rootrac2 # /etc/init.d/iscsi startiscsid is stopped OK iSCSI daemon: OK OK Setting up iSCSI targets: iscsiadm: No records found! OK rootrac1 # iscsiadm -m discovery -t sendtargets -p 5353:3260,1 iqn.2007-09.jp.ne.peach.istgt:disk0rootrac1 # iscsiadm -m node -T iqn.2007-09.jp.ne.peach.istgt:disk0 -p 53 -lLogging in to iface: default, target: iqn.2007-09.jp.ne.peach.istgt:disk0, portal: 53,3260Login to iface: default, target: iqn.2007-09.jp.ne.peach.istgt:disk0, portal: 53,3260: successful使用fdisk -l查看共享存儲連接情況，兩臺機器均已連接rootrac1 # fdisk -lDisk /dev/hda: 21.4 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System/dev/hda1 * 1 13 104391 83 Linux/dev/hda2 14 778 6144862+ 83 Linux/dev/hda3 779 816 305235 82 Linux swap / Solaris/dev/hda4 817 2610 14410305 5 Extended/dev/hda5 817 2610 14410273+ 83 LinuxDisk /dev/sda: 42.9 GB, 42949410816 bytes64 heads, 32 sectors/track, 40959 cylindersUnits = cylinders of 2048 * 512 = 1048576 bytes查看rac節(jié)點rootrac2 # fdisk -lDisk /dev/hda: 21.4 GB, 21474836480 bytes255 heads, 63 sectors/track, 2610 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System/dev/hda1 * 1 13 104391 83 Linux/dev/hda2 14 778 6144862+ 83 Linux/dev/hda3 779 816 305235 82 Linux swap / Solaris/dev/hda4 817 2610 14410305 5 Extended/dev/hda5 817 2610 14410273+ 83 LinuxDisk /dev/sda: 42.9 GB, 42949410816 bytes64 heads, 32 sectors/track, 40959 cylindersUnits = cylinders of 2048 * 512 = 1048576 bytesDisk /dev/sda doesnt contain a valid partition table重啟iscsi服務，查看存儲掛載情況rootrac1 # /etc/init.d/iscsi restartLogging out of session sid: 1, target: iqn.2007-09.jp.ne.peach.istgt:disk0, portal: 53,3260Logout of sid: 1, target: iqn.2007-09.jp.ne.peach.istgt:disk0, portal: 53,3260: successfulStopping iSCSI daemon: iscsid dead but pid file exists OK iSCSI daemon: OK OK Setting up iSCSI targets: Logging in to iface: default, target: iqn.2007-09.jp.ne.peach.istgt:disk0, portal: 53,3260Login to iface: default, target: iqn.2007-09.jp.ne.peach.istgt:disk0, portal: 53,3260: successful OK 至此共享存儲配置完成；共享存儲劃分分區(qū)，因為存儲是做的raid5，存儲容量大約在40G左右，所以劃分三個主分區(qū)，作為ASM磁盤組使用，另外劃分若干擴展分區(qū)，使用作為安裝OCR和spfile；這里只需要在一個節(jié)點上做就可以了，因為存儲是共享的。rootrac1 # fdisk /dev/sdaDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel. Changes will remain in memory only,until you decide to write them. After that, of course, the previouscontent wont be recoverable.The number of cylinders for this disk is set to 40959.There is nothing wrong with that, but this is larger than 1024,and could in certain setups cause problems with:1) software that runs at boot time (e.g., old versions of LILO)2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK)Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-40959, default 1): Using default value 1Last cylinder or +size or +sizeM or +sizeK (1-40959, default 40959): +10000mCommand (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 2First cylinder (9539-40959, default 9539): Using default value 9539Last cylinder or +size or +sizeM or +sizeK (9539-40959, default 40959): +10000mCommand (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 3First cylinder (19077-40959, default 19077): Using default value 19077Last cylinder or +size or +sizeM or +sizeK (19077-40959, default 40959): +10000mCommand (m for help): nCommand action e extended p primary partition (1-4)eSelected partition 4First cylinder (28615-40959, default 28615): Using default value 28615Last cylinder or +size or +sizeM or +sizeK (28615-40959, default 40959): +12000mCommand (m for help): nFirst cylinder (28615-40059, default 28615): Using default value 28615Last cylinder or +size or +sizeM or +sizeK (28615-40059, default 40059): +1024mCommand (m for help): nFirst cylinder (29593-40059, default 29593): Using default value 29593Last cylinder or +size or +sizeM or +sizeK (29593-40059, default 40059): +1024mCommand (m for help): nFirst cylinder (30571-40059, default 30571): Using default value 30571Last cylinder or +size or +sizeM or +sizeK (30571-40059, default 40059): +1024mCommand (m for help): nFirst cylinder (31549-40059, default 31549): Using default value 31549Last cylinder or +size or +sizeM or +sizeK (31549-40059, default 40059): +4000mCommand (m for help): nFirst cylinder (35365-40059, default 35365): Using default value 35365Last cylinder or +size or +sizeM or +sizeK (35365-40059, default 40059): +4000mCommand (m for help): wqThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.分區(qū)劃分完成后，檢查分區(qū)結果，兩個節(jié)點分別做檢查。rootrac1 # fdisk -lDisk /dev/hda: 22.5 GB, 22548578304 bytes255 heads, 63 sectors/track, 2741 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System/dev/hda1 * 1 25 200781 83 Linux/dev/hda2 26 286 2096482+ 82 Linux swap / Solaris/dev/hda3 287 2741 19719787+ 83 LinuxDisk /dev/sda: 42.9 GB, 42949410816 bytes64 heads, 32 sectors/track, 40959 cylindersUnits = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System/dev/sda1 1 9538 9766896 83 Linux/dev/sda2 9539 19076 9766912 83 Linux/dev/sda3 19077 28614 9766912 83 Linux/dev/sda4 28615 40059 11719680 5 Extended/dev/sda5 28615 29592 1001456 83 Linux/dev/sda6 29593 30570 1001456 83 Linux/dev/sda7 30571 31548 1001456 83 Linux/dev/sda8 31549 35364 3907568 83 Linux/dev/sda9 35365 39180 3907568 83 Linuxrootrac2 # fdisk -lDisk /dev/hda: 22.5 GB, 22548578304 bytes255 heads, 63 sectors/track, 2741 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System/dev/hda1 * 1 25 200781 83 Linux/dev/hda2 26 286 2096482+ 82 Linux swap / Solaris/dev/hda3 287 2741 19719787+ 83 LinuxDisk /dev/sda: 42.9 GB, 42949410816 bytes64 heads, 32 sectors/track, 40959 cylindersUnits = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System/dev/sda1 1 9538 9766896 83 Linux/dev/sda2 9539 19076 9766912 83 Linux/dev/sda3 19077 28614 9766912 83 Linux/dev/sda4 28615 40059 11719680 5 Extended/dev/sda5 28615 29592 1001456 83 Linux/dev/sda6 29593 30570 1001456 83 Linux/dev/sda7 30571 31548 1001456 83 Linux/dev/sda8 31549 35364 3907568 83 Linux/dev/sda9 35365 39180 3907568 83 Linux5、映射裸設備映射裸設備，分別在兩個節(jié)點上配置，在這里映射的是擴展分區(qū)，而不是主分區(qū)，主分區(qū)是用來做ASM磁盤組安裝數據庫的。rootrac1 # vi /etc/sysconfig/rawdevices/dev/raw/raw1 /dev/sda5/dev/raw/raw2 /dev/sda6/dev/raw/raw3 /dev/sda7修改權限rootrac1 # vi /etc/udev/permissions.d/50-udev.permissionsraw/*:oracle:oinstall:0660啟動并驗證，兩臺機器rootrac1 # /etc/init.d/rawdevices startAssigning devices: /dev/raw/raw1 - /dev/sda5/dev/raw/raw1: bound to major 8, minor 5 /dev/raw/raw2 - /dev/sda6/dev/raw/raw2: bound to major 8, minor 6 /dev/raw/raw3 - /dev/sda7/dev/raw/raw3: bound to major 8, minor 7done 在rac2節(jié)點上啟動并驗證rootrac2 # /etc/init.d/rawdevices startAssigning devices: /dev/raw/raw1 - /dev/sda5/dev/raw/raw1: bound to major 8, minor 5 /dev/raw/raw2 - /dev/sda6/dev/raw/raw2: bound to major 8, minor 6 /dev/raw/raw3 - /dev/sda7/dev/raw/raw3: bound to major 8, minor 7done6、安裝ASM按照順序安裝以下三個包，兩個節(jié)點都要安裝。首先oracleasm是要和系統(tǒng)內核版本一致的，查看linux內核版本。rootrac1 # uname -aLinux rac1 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux軟件包oracleasm的版本必須和內核版本一致：oracleasm-support-2.1.7-1.el5.x86_64.rpmoracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpmoracleasmlib-2.0.4-1.el5.x86_64.rpm按順序安裝三個軟件包rootrac2 # rpm -ivh oracleasm-support-2.1.7-1.el5.x86_64.rpm warning: oracleasm-support-2.1.7-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159Preparing. # 100% 1:oracleasm-support # 100%rootrac2 # rpm -ivh oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm warning: oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159Preparing. # 100% 1:oracleasm-2.6.18-194.el# 100%rootrac2 # rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159Preparing. # 100% 1:oracleasmlib # 100%創(chuàng)建asm磁盤，首先創(chuàng)建oracle用戶和 oinstall組，兩個節(jié)點都做，并且要保證兩臺節(jié)點的用戶和組的id是一致的rootrac1 # groupadd oinstallrootrac1 # groupadd dbarootrac1 # useradd -g oinstall -G dba oraclerootrac1 # passwd oracleChanging password for user oracle.New UNIX password: BAD PASSWORD: it is too simplistic/systematicRetype new UNIX password: passwd: all authentication tokens updated successfully.查看用戶和組id是否一致，或者在創(chuàng)建用戶和組的時候直接指定用戶和組的id。例如：Useradd -u（指定uid）。rootrac1 cluster# id oracleuid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(dba)rootrac2 cluster# id oracleuid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(dba)創(chuàng)建ASM，需要兩個節(jié)點都要做rootrac1 # /etc/init.d/oracleasm configureConfiguring the Oracle ASM library driver.This will configure the on-boot properties of the Oracle ASM librarydriver. The following questions will determine whether the driver isloaded on boot and what permissions it will have. The current valueswill be shown in brackets (). Hitting without typing ananswer will keep that current value. Ctrl-C will abort.Default user to own the driver interface oracle: oracleDefault group to own the driver interface oinstall: oinstallStart Oracle ASM library driver on boot (y/n) y: yScan for Oracle ASM disks on boot (y/n) y: yWriting Oracle ASM library driver configuration: doneInitializing the Oracle ASMLib driver: OK Scanning the system for Oracle ASMLib disks: OK 接下來告訴 ASMLib 驅動程序要使用哪些磁盤。注意，這些磁盤是不包含任何內容。命令格式：/etc/init.d/oracleasm createdisk DISK_NAME device_name這里只在其中一個節(jié)點中配置即可，另外一個基點能夠自動檢測到配置rootrac1 # /etc/init.d/oracleasm createdisk VOL1 /dev/sda1Marking disk VOL1 as an ASM disk: OK rootrac1 # /etc/init.d/oracleasm createdisk VOL2 /dev/sda2Marking disk VOL2 as an ASM disk: OK rootrac1 # /etc/init.d/oracleasm createdisk VOL3 /dev/sda3Marking disk VOL3 as an ASM disk: OK rootrac1 # /etc/init.d/oracleasm enableWriting Oracle ASM library driver configuration: doneInitializing the Oracle ASMLib driver: OK Scanning the system for Oracle ASMLib disks: OK rootrac2 # /etc/init.d/oracleasm enableWriting Oracle ASM library driver configuration: doneInitializing the Oracle ASMLib driver: OK Scanning the system for Oracle ASMLib disks: OK 在兩個節(jié)點上掃描ASM磁盤rootrac1 # /etc/init.d/oracleasm listdisksVOL1VOL2VOL3rootrac2 # /etc/init.d/oracleasm listdisksVOL1VOL2VOL3二 、安裝oracle cluster1、安裝依賴包安裝oracle集群所需要的軟件包，兩個節(jié)點都要安裝。Rpm qa |grep compatcompat-boost-1331-1.33.1-5.0.1.el4compat-libgcc-296-2.96-132.7.2compat-openldap-2.1.30-12.el4compat-dapl-2.0.15-1.el4compat-gcc-32-c+-3.2.3-47.3compat-libstdc+-33-3.2.3-47.3compat-db-4.1.25-9compat-libcom_err-1.0-5compat-libstdc+-296-2.96-132.7.2compat-gcc-32-3.2.3-47.3openmotifbinutils-.6-5.el5control-center-2.16.0-14.el5gcc-4.1.2-14.el5gcc-c+-4.1.2-14.el5glibc-2.5-18libstdc+libstdc+-develmake-3.81-1.12、修改內核參數修改linux內核參數，兩個節(jié)點都要修改rootrac1 # cat /etc/sysctl.conf /etc/security/limits.conf /etc/pam.d/login /etc/rc.d/rc.local EOFmodprobe hangcheck-timer hangcheck_tick=30 hangcheck_margin=180EOF6、添加環(huán)境變量Oracle軟件環(huán)境配置創(chuàng)建安裝目錄,兩個節(jié)點都要創(chuàng)建。mkdir -p /oracle/app/oracle/10g/crs_1mkdir -p /oracle/app/oracle/10g/db_1chown -R oracle:oinstall /oracle/app/oracle/10g/crs_1 chown -R oracle:oinstall /oracle/app/oracle/10g/db_1chmod -R 775 /oracle/app/oracle/10g/crs_1 chmod -R 775 /oracle/app/oracle/10g/db_1oracle 用戶的環(huán)境變量在oracle用戶下，編輯/home/oracle/.bash_profile添加如下內容：在rac1機器上修改export ORACLE_SID=oradb1export ORACLE_BASE= /oracle/app/oracle/10gexport CRS_HOME=$ORACLE_BASE/product/db_1export ORACLE_HOME= /oracle/app/oracle/10g/crs_1export LD_L