Goodus 기술노트 [25 회 ] Vmware RAC 구축방법 Author 반채헌 Creation Date 2007-11-19 Last Updated 2007-11-19 Version 1.0 Copyright(C) 2004 Goodus Inc. All Rights Reserved Version 변경일자 변경자 ( 작성자 ) 주요내용 1 2007-11-19 반채헌과장 문서최초작성 2 3 1
Contents 1. 운영체제환경의개요... 3 2. 게스트운영체제환경의개요... 4 3. 가상디스크레이아웃의개요... 4 4. RAC 데이터베이스환경의개요... 4 4.1 첫번째가상머신구성... 5 4.2 첫번째가상머신에 Enterprise Linux 설치하기... 9 4.3 오라클사용자환경파일의생성... 21 5. 두번째가상머신의생성및설정... 33 5.1 네트워크설정변경.... 33 5.2 Oracle Automatic Storage Management (ASM) 설정... 42 6. O2CB 드라이버설정... 47 7. Oracle Clusterware 설치... 53 7.1 데이터베이스설정의확인... 101 8.2 테이블스페이스생성 TEST 과정... 107 8.3 플래시복구영역의공간사용현황점검... 109 2
VMware Server 를이용하여 Oracle Enterprise Linux 기반 Oracle RAC 10g 설치하기 ( 저자 : Vincent chan 참고자료변경수정함 ) 가상머신테크놀로지를이용하여 Oracle Enterprise Linux 운영체제테스트용 Oracle Real Application Cluster 10g Release 2 를설치하는방법을설명. 전혀비용을들이지않고도 VMware Server(VMware 의무료가상화소프트웨어 ) 를이용하 여일반가정용컴퓨터에 Red Hat Advanced Server 기반 Oracle Real Application Clusters (RAC) 10g 환경을구현함. VMware Server는하나의물리적머신에서여러개의운영체제를동시에실행할수있게합니다. 각각의가상머신은별도의운영체제와디스크, 프로세서, 메모리등의가상컴포넌트를갖습니다. 가상화테크놀로지는하나의물리적서버에서다른데이터또는소프트웨어에영향을주지않고개발 / 테스트작업을수행하기위한용도로활용되고있습니다. VMware 소프트웨어는특히개발 / 테스트과정의총소유비용을절감하고애플리케이션개발기간을단축할수있는솔루션으로각광받고있습니다. 본가이드에서는 Enterprise Linux와 VMware Server를기반으로실행되는 2 노드구성의 Oracle RAC 10g Release 2 환경을설치하고구성하는방법을설명하고있습니다. 이가이드는교육 / 테스트용으로만활용할수있으며, 오라클및다른어떤벤더도이러한구성에대한기술지원을제공하지않고있음을참고. 1. 운영체제환경의개요 호스트 네임 OS 프로세서메모리디스크네트워크카드 samsung- Windows XP Intel(R) 2 GB DDR2 120 GB, Realtek RTL8139 cba1f14 Professional Pentium(R) SDRAM, Ultra Family PCI Fast Service Pack 2 MProcessor 797 MHz ATA/133, Ethernet NIC (32-bit) 노트북 2.00GHz 7200 RPM SensR-50 3
2. 게스트운영체제환경의개요 호스트네임 OS 프로세서메모리 rac1 Oracle Enterprise Linux 4 (5upgrade) (32-bit) 1 700 MB rac2 Oracle Enterprise Linux 4 (5upgrade) (32-bit) 1 700 MB 3. 가상디스크레이아웃의개요 호스트가상디스크 게스트 가상 디스크 가상 디바이스 노드 사이즈 (MB) 설명 d:\vm\rac\localdisk.vmdk /dev/sda1 /dev/sda2 /dev/sda3 SCSI 0:0 20 / 마운트포인트 스왑공간 오라클바이너리 d:\vm\rac\sharedstorage\ocfs2disk.vmdk /dev/sdb SCSI 1:0 512 OCFS2 디스크 d:\vm\rac\sharedstorage\asmdisk1.vmdk /dev/sdc SCSI 1:1 3072 ASM disk group 1 d:\vm\rac\sharedstorage\asmdisk2.vmdk /dev/sdd SCSI 1:2 3072 ASM disk group 1 d:\vm\rac\sharedstorage\asmdisk3.vmdk /dev/sde SCSI 1:3 2048 ASM 플래시복구영역 4. RAC 데이터베이스환경의개요 호스트네임 ASM 인스턴스 네임 RAC 인스턴스 네임 데이터베이스 네임 데이터베이스 파일스토리지 OCR & Voting Disk rac1 +ASM1 devdb1 devdb ASM OCFS2 rac2 +ASM2 devdb2 devdb ASM OCFS2 4
4.1 첫번째가상머신구성 C:\>mkdir vm\rac\rac1 C:\>mkdir vm\rac\rac2 C:\>mkdir vm\rac\sharedstorage 데스크탑의 VMware Server 아이콘을더블클릭하여애플리케이션을실행합니다 : 1. Ctrl-N 을눌러새로운가상머신을생성합니다. 2. New Virtual Machine Wizard: Next 를클릭합니다. 3. Select the Appropriate Configuration: a. Virtual machine configuration: Custom 을선택합니다. 4. Select a Guest Operating System: a. Guest operating system: Linux 를선택합니다. b. Version: Red Hat Enterprise Linux 4 를선택합니다. 5. Name the Virtual Machine: a. Virtual machine name: rac1 을입력합니다. b. Location: d:\vm\rac\rac1. 을입력합니다. 6. Set Access Rights: a. Access rights: Make this virtual machine private 를선택합니다. 7. Startup / Shutdown Options: a. Virtual machine account: User that powers on the virtual machine 을선택합니다. 8. Processor Configuration: a. Processors: One 을선택합니다. 9. Memory for the Virtual Machine: a. Memory: 700MB 를선택합니다. 10. Network Type: a. Network connection: Use bridged networking 을선택합니다. 11. Select I/O Adapter Types: a. I/O adapter types: LSI Logic 을선택합니다. 12. Select a Disk: a. Disk: Create a new virtual disk 를선택합니다. 13. Select a Disk Type: a. Virtual Disk Type: SCSI (Recommended) 를선택합니다. 14. Specify Disk Capacity: 5
a. Disk capacity: 20GB. 를입력합니다. b. Allocate all disk space now 의선택을해제합니다. 공간을절약할생각이라면, 굳이지금모든디스크공간을할당할필요가없습니다. 15. Specify Disk File: a. Disk file: localdisk.vmdk. 를입력합니다. b. Finish 를클릭합니다. 16-24 번과정을반복하여 4 개의가상 SCSI 하드디스크를생성합니다 - ocfs2disk.vmdk (512MB), asmdisk1.vmdk (3GB), asmdisk2.vmdk (3GB), asmdisk3.vmdk (2GB). 16. VMware Server Console: Edit virtual machine settings 를클릭합니다. 17. Virtual Machine Settings: Add 를클릭합니다. 18. Add Hardware Wizard: Next 를클릭합니다. 19. Hardware Type: a. Hardware types: Hard Disk 를선택합니다. 20. Select a Disk: a. Disk: Create a new virtual disk 을선택합니다. 21. Select a Disk Type: a. Virtual Disk Type: SCSI (Recommended) 을선택합니다. 22. Specify Disk Capacity: a. Disk capacity: 0.5GB. 을입력합니다. b. Allocate all disk space now 을선택합니다. 공간을절약할생각이라면, 굳이지금모든디스크공간을할당할필요가없습니다. 하지만성능문제를고려하여, 각각의가상공유디스크에디스크공간을미리할당해주는것이바람직합니다. 흔한일은아니지만오라클데이터베이스생성과정에서, 또는데이터베이스에 DML 작업이과도하게발생하여공유디스크의사이즈가급격히증가하는경우, 가상머신의동작이잠시멈추는상황이발생할수도있습니다. 23. Specify Disk File: a. Disk file: C:\vm\rac\sharedstorage\ocfs2disk.vmdk 을입력합니다. b. Advanced 를클릭합니다. 24. Add Hardware Wizard: a. Virtual device node: SCSI 0:1 을선택합니다. b. Mode: 모든공유디스크에대해 Independent, Persistent 를선택합니다. c. Finish 를클릭합니다. 6
마지막으로, 사설인터커넥트를위한가상네트워크카드를추가하고플로피드라이브를 제거합니다. 25. VMware Server Console: dit virtual machine settings 를클릭합니다. 26. Virtual Machine Settings: Add 를클릭합니다. 27. Add Hardware Wizard: Next 를클릭합니다. 28. Hardware Type: a. Hardware types: Ethernet Adapter. 29. Network Type: a. Host-only: A private network shared with the host b. Finish 를클릭합니다. 30. Virtual Machine Settings: a. Floppy 를선택하고 Remove 를클릭합니다. 31. Virtual Machine Settings: OK 를클릭합니다. 7
가상머신설정파일의수정 : 두대의가상 RAC 노드간에디스크를공유하기위해서는추 가로매개변수를설정해주어야합니다. C:\vm\rac\rac1\Red Hat Enterprise Linux 4.vmx 파일을열고아래굵은글씨체로표시된매개변수들을추가함. config.version = "8" virtualhw.version = "6" scsi0.present = "TRUE" scsi0.virtualdev = "lsilogic" memsize = "700" scsi0:0.present = "TRUE" scsi0:0.filename = "localdisk.vmdk" ide1:0.present = "TRUE" ide1:0.filename = "auto detect" ide1:0.devicetype = "cdrom-raw" floppy0.autodetect = "TRUE" ethernet0.present = "TRUE" ethernet0.wakeonpcktrcv = "FALSE" usb.present = "TRUE" ehci.present = "TRUE" sound.present = "TRUE" sound.filename = "-1" sound.autodetect = "TRUE" svga.autodetect = "TRUE" pcibridge0.present = "TRUE" mks.keyboardfilter = "allow" displayname = "rac1" guestos = "rhel4" nvram = "Red Hat Enterprise Linux 4.nvram" deploymentplatform = "windows" virtualhw.productcompatibility = "hosted" tools.upgrade.policy = "useglobal" disk.locking = "FALSE" disklib.datacachemaxsize = "0" scsi1.sharedbus = "virtual" 8
scsi0:1.present = "TRUE" scsi0:1.filename = "C:\vm\rac\sharedstorage\ocfs2disk.vmdk" scsi0:1.mode = "independent-persistent" scsi0:1.devicetype = "disk" scsi0:2.present = "TRUE" scsi0:2.filename = "C:\vm\rac\sharedstorage\asmdisk1.vmdk" scsi0:2.mode = "independent-persistent" scsi0:2.devicetype = "disk" scsi0:3.present = "TRUE" scsi0:3.filename = "C:\vm\rac\sharedstorage\asmdisk2.vmdk" scsi0:3.mode = "independent-persistent" scsi0:3.devicetype = "disk" scsi0:4.present = "TRUE" scsi0:4.filename = "C:\vm\rac\sharedstorage\asmdisk3.vmdk" scsi0:4.mode = "independent-persistent" scsi0:4.devicetype = "disk" floppy0.present = "FALSE" ethernet1.present = "TRUE" ethernet1.connectiontype = "hostonly" ethernet1.wakeonpcktrcv = "FALSE" 4.2 첫번째가상머신에 Enterprise Linux 설치하기 오라클웹사이트에서다운로드하고파일압축을풉니다 : Enterprise-R4-U5-i386-disc1.iso Enterprise-R4-U5-i386-disc2.iso Enterprise-R4-U5-i386-disc3.iso Enterprise-R4-U5-i386-disc4.iso 9
1. VMware Server 콘솔에서오른쪽패널의 CD-ROM 디바이스를더블클릭하고 disk 1 을위한 ISO 이미지 (Enterprise-R4-U5-i386-disc1.iso) 를선택합니다. 2. VMware Server console o Start this virtual machine 을클릭합니다. 3. Enter 를눌러그래픽모드에서설치하도록선택합니다. 4. 미디어테스트를생략하고설치를시작합니다. 5. Welcome to enterprise Linux: Next 를클릭합니다. 6. Language Selection: < 사용할언어를선택합니다.> 7. Keyboard Configuration: < 사용할키보드를선택합니다 > 8. Installation Type: Custom. 9. Disk Partitioning Setup: Manually partition with Disk Druid. o Warning: Yes 를클릭하여모든디바이스 (sda, sdb, sdc, sdd, sde) 를초기화합니다. 10. Disk Setup: /dev/sda 를더블클릭하여 sda 드라이브에마운트포인트 (/, /u01) 와스왑공간을위한디스크공간을할당합니다. OCFS2, ASM 을위한드라이브설정은뒷부분에서수행하게됩니다. o Add Partition: Mount Point: / File System Type: ext3 Start Cylinder: 1 End Cylinder: 910 File System Type: Swap Start Cylinder: 911 End Cylinder: 1170 Mount Point: /u01 File System Type: ext3 Start Cylinder: 1171 End Cylinder: 2610 10
11. Boot Loader Configuration: 디폴트 /dev/sda1 만을선택하고나머지는선택되지않은상태로둡니다. 12. Network Configuration: a. Network Devices eth0 을선택합니다. 1. Configure Using DHCP 의선택을해제합니다. 2. Activate on boot 을선택합니다. 3. IP Address: 192.168.10.171. 을입력합니다. 4. Netmask: 255.255.255.0. 을입력합니다. eth1 을선택합니다. 1. Configure Using DHCP 의선택을해제합니다. 2. Activate on boot 을선택합니다. 11
3. IP Address: 10.10.10.71. 을입력합니다. 4. Netmask: 255.255.255.0. 을입력합니다. b. Hostname manually 를선택하고 rac1.mycorpdomain.com. 을입력합니다. c. Miscellaneous Settings Gateway: 192.168.10.1. 을입력합니다. Primary DNS: < 옵션 > Secondary DNS: < 옵션 > 13. Firewall Configuration: a. No Firewall 을선택합니다. 방화벽이활성화된경우, ocfs2 파일시스템을마운트하는과정에서 mount.ocfs2: Transport endpoint is not connected while mounting 에러가발생할수있습니다. b. Enable SELinux?: Active. 14. Warning No Firewall: Proceed 를클릭합니다. 15. Additional Language Support: < 사용할언어를선택합니다 >. 16. Time Zone Selection: < 표준시간대를선택합니다 > 17. Set Root Password: <root 패스워드를입력합니다 > 18. Package Group Selection: a. X Window System 을선택합니다. b. GNOME Desktop Environment 를선택합니다. c. Editors 를선택합니다. Details 를클릭하고사용할텍스트편집기를선택합니다. d. Graphical Internet 을선택합니다. e. Text-based Internet 을선택합니다. f. Office/Productivity 를선택합니다. g. Sound and Video 를선택합니다. h. Graphics 를선택합니다. i. Server Configuration Tools 를선택합니다. j. FTP Server 를선택합니다. k. Legacy Network Server 를선택합니다. Details 를클릭합니다. 1. rsh-server 를선택합니다. 2. Sstrong>telnet-server 를선택합니다. l. Development Tools 를선택합니다. m. Legacy Software Development 를선택합니다. n. Administration Tools 를선택합니다. 12
o. System Tools 를선택합니다. Details 를클릭합니다. 디폴트로선택된패키지와별도로아래패키지를선택합니다. 1. ocfs-2-2.6.9-42.0.0.0.1el(up 커널을위한드라이버 ) 을선택하거나, 또는 ocfs-2-2.6.9-42.0.0.0.1elsmp(smp 커널을위한드라이버 ) 를선택합니다. 2. ocfs2-tools 를선택합니다. 3. ocfs2console 을선택합니다. 4. oracle oracleasm-2.6.9-42.0.0.0.1el(up 커널을위한드라이버 ) 를선택하거나, 또는 oracleasm-2.6.9-42.0.0.0.1elsmp(smp 커널을위한드라이버 ) 를선택합니다. 5. sysstat 을선택합니다. p. Printing Support 를선택합니다. 19. About to Install: Next 를클릭합니다. 20. Required Install Media: Continue 를클릭합니다. 21. Change CD-ROM: VMware Server 콘솔에서 Ctrl-D 를눌러 Virtual Machine Settings 를실행합니다. CD-ROM 디바이스를클릭하고 disk 2 를위한 ISO 이미지 (Enterprise-R4-U5-i386-disc2.iso) 를선택하고, 다시 disk3 을위한 ISO 이미지 (Enterprise-R4-U5-i386-disc3.iso) 를선택합니다. 22. 설치작업이종료된후, a. VMware Server 콘솔에서 Ctrl-D 를눌러 Virtual Machine Settings 를실행합니다. CD-ROM 디바이스를클릭하고 Use physical drive 를클릭합니다. b. Reboot 를클릭합니다. 23. Welcome: Next 를클릭합니다. 24. License Agreement: Yes, I agree to the License Agreement 를선택합니다. 25. Date and Time: 날짜와시간을설정합니다. 26. Display: < 원하는해상도를선택합니다 > 27. System User: 항목을빈칸으로남겨두고 Next 를클릭합니다. 28. Additional CDs: Next 를클릭합니다. 29. Finish Setup: Next 를클릭합니다. 13
이것으로 VMware Server 에 Enterprise Linux 를설치하는작업을완료하였습니다. 하지만위에서설치해야할 PKG 가다설치되지않았을겁니다. Enterprise Linux 소프트웨어패키지의설치. 오라클소프트웨어설치를위해추가해 주어야하는패키지가아래와같습니다. Enterprise Linux 의 64 비트버전을설치한경우에는, 인스톨러가이패키지를이미설치한 상태이어야합니다. libaio-0.3.105-2.i386.rpm openmotif21-2.1.30-11.rhel4.6.i386.rpm 14
------------------------------------------- [root@rac1 proc]# rpm -qa grep libaio* libaio-0.3.105-2 [root@rac1 proc]# rpm -qa grep openmotif21* openmotif21-2.1.30-11.rhel4.6 [root@rac1 proc]# cd [root@rac1 ~]# ls Desktop anaconda-ks.cfg install.log install.log.syslog [root@rac1 ~]# cd /u01 [root@rac1 u01]# ls lost+found pkg [root@rac1 u01]# cd pkg [root@rac1 pkg]# ls ocfs2-tools-1.2.2-1.i386.rpm oracleasm-2.6.9-55.elsmp-2.0.3-1.i686.rpm ocfs2console-1.2.2-1.i386.rpm oracleasm-support-2.0.3-1.i386.rpm oracleasm-2.6.9-55.el-2.0.3-1.i686.rpm oracleasmlib-2.0.2-1.i386.rpm [root@rac1 pkg]# ls -ltr 1528 -rw-r--r-- 1 root root 1036426 2?? 5 15:26 ocfs2-tools-1.2.2-1.i386.rpm -rw-r--r-- 1 root root 177275 2?? 5 15:26 ocfs2console-1.2.2-1.i386.rpm 15
-rw-r--r-- 1 root root 22662 7??26 15:58 oracleasm-support-2.0.3-1.i386.rpm -rw-r--r-- 1 root root 12948 7??26 15:58 oracleasmlib-2.0.2-1.i386.rpm -rw-r--r-- 1 root root 127517 7??31 10:25 oracleasm-2.6.9-55.el-2.0.3-1.i686.rpm -rw-r--r-- 1 root root 129361 7??31 10:32 oracleasm-2.6.9-55.elsmp-2.0.3-1.i686.rpm -rw-r--r-- 1 root root 2926535 Aug 3 10:47 ocfs2-2.6.9-55.el-1.2.5-2.i686.rpm 다음명령으로 pkg 설치한다. [root@rac1 pkg]# rpm -ivh ocfs2-tools-1.2.2-1.i386.rpm [100%] ########################################### 1:ocfs2-tools ########################################### [100%] [root@rac1 pkg]# LANG=C [root@rac1 pkg]# rpm -ivh ocfs2console-1.2.2-1.i386.rpm Preparing... ########################################### [100%] 1:ocfs2console ########################################### [100%] [root@rac1 pkg]# rpm -ivh oracleasm-support-2.0.3-1.i386.rpm Preparing... ########################################### [100%] 1:oracleasm-support ########################################### [100%] 16
[root@rac1 pkg]# rpm -ivh oracleasm-2.6.9-55.el-2.0.3-1.i686.rpm Preparing... ########################################### [100%] 1:oracleasm-2.6.9-55.EL ########################################### [100%] [root@rac1 pkg]# rpm -ivh oracleasmlib-2.0.2-1.i386.rpm Preparing... ########################################### [100%] 1:oracleasmlib ########################################### [100%] [root@rac1 pkg]# rpm -ivh ocfs2-2.6.9-55.el-1.2.5-2.i686.rpm Preparing... ########################################### [100%] 1:ocfs2-2.6.9-55.EL ########################################### [100%] ------------------------- 모든 pkg 설치완료함 ------------------------ [root@rac1 pkg]# 필요한 rpm 설치완료. 위의내용을전부 install 해야함.( 아래와같이모두완료되야한다 ) ------------------------------------------------------------------- 전부확인한다. 17
VMware Tools 설치. VMware Tools 는호스트머신과게스트머신간의시간을동기화하기 위해사용됩니다. VMware 콘솔에서 root 사용자로로그인합니다. 1. VM 을클릭한후 Install VMware Tools 를선택합니다. 2. rac1 Virtual Machine: Install 을클릭합니다. 3. 데스크탑의 VMware Tools 아이콘을더블클릭하여애플리케이션을실행합니다. 4. cdrom: VMwareTools-6.0.0-45-731.i386.rpm 을더블클릭합니다. 5. Completed System Preparation: Continue 를클릭합니다. 6. 터미널을열고 vmware-config-tools.pl 을실행합니다. o 원하는디스플레이사이즈를입력합니다 게스트 OS 와호스트 OS 의시간동기화. Oracle Clusterware 와 Oracle Database 소프트웨어를설치하는과정에서오라클인스톨러는먼저로컬노드에소프트웨어를설치한후원격노드에소프트웨어를카피하는작업을수행합니다. 양쪽 RAC 노드의날짜와시간이동기화되지않은경우아래와같은에러가발생할수도있습니다. 18
"/bin/tar:./inventory/components21/oracle.ordim.server/10.2.0.1.0: time stamp 2007-07-04 13:24:04 is 25 s in the future" 따라서 Oracle RAC 설치를수행하기전에, 가상머신과호스트머신의시간을동기화해주어야합니다. root 사용자로로그인하여아래작업을실행하여시간을동기화합니다. 1. vmware-toolbox 를실행하여 VMware Tools Properties 윈도우를엽니다. Options 탭에서 Time synchronization between the virtual machine and the host operating system 을선택합니다. tools.synctime = "TRUE" 매개변수가가상머신설정파일 (C:\vm\rac\rac1\Red Hat Enterprise Linux 4.vmx) 에추가된것을확인할수있을것입니다. 2. 2. "/boot/grub/grub.conf" 파일을편집하여 kernel /boot/ 라인에 clock=pit nosmp noapic nolapic 옵션을추가합니다. 이제양쪽커널에옵션을추가하는작업을완료했습니다. 변경작업은현재사용중인커널에대해서만수행해주면됩니다 [root@rac1 grub]# vi grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You do not have a /boot partition. This means that # all kernel and initrd paths are relative to /, eg. # root (hd0,0) # kernel /boot/vmlinuz-version ro root=/dev/sda1 # initrd /boot/initrd-version.img #boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/boot/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux ES (2.6.9-55.EL) root (hd0,0) kernel /boot/vmlinuz-2.6.9-55.el ro root=label=/ rhgb quiet clock=pit nosmp noapic nolapic initrd /boot/initrd-2.6.9-55.el.img 19
Rac1 리부팅한다. [root@rac1 ~]# reboot [root@rac1 ~]# df -kp Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda1 7194780 3305940 3523364 49% / none 353344 0 353344 0% /dev/shm /dev/sda3 11385160 65160 10741660 1% /u01 [root@rac1 ~]# fdisk -l Disk /dev/sda: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 910 7309543+ 83 Linux /dev/sda2 911 1170 2088450 82 Linux swap /dev/sda3 1171 2610 11566800 83 Linux Disk /dev/sdb: 536 MB, 536870912 bytes 64 heads, 32 sectors/track, 512 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System Disk /dev/sdc: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System Disk /dev/sdd: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes 20
Device Boot Start End Blocks Id System Disk /dev/sde: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System 오라클사용자의생성 root 사용자로로그인하여아래와같이실행 [root@rac1 ~]# groupadd oinstall [root@rac1 ~]# groupadd dba [root@rac1 ~]# mkdir -p /export/home/oracle /ocfs [root@rac1 ~]# useradd -d /export/home/oracle -g oinstall -G dba -s /bin/ksh oracle [root@rac1 ~]# chown oracle:dba /export/home/oracle /u01 [root@rac1 ~]# passwd oracle Changing password for user oracle. New UNIX password: BAD PASSWORD: it is based on a dictionary word Retype new UNIX password: passwd: all authentication tokens updated successfully. 4.3 오라클사용자환경파일의생성 [root@rac1 ~]# vi /export/home/oracle/.profile export PS1="`/bin/hostname -s`-> " export EDITOR=vi export ORACLE_SID=devdb1 export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1 export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1 21
export LD_LIBRARY_PATH=$ORACLE_HOME/lib export PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:/bin:/usr/bin:/usr/sbin:/ usr/local/bin:/usr/x11r6/bin umask 022 파일시스템디렉토리구조의생성. oracle 사용자로로그인하여아래와같이실행 [root@rac1 ~]# su - oracle rac1-> pwd /export/home/oracle rac1-> mkdir -p $ORACLE_BASE/admin rac1-> mkdir -p $ORACLE_HOME rac1-> mkdir -p $ORA_CRS_HOME rac1-> mkdir -p /u01/oradata/devdb oracle 사용자의 Shell Limit 설정. 텍스트편집기를사용하여아래라인을 /etc/security/limits.conf, /etc/pam.d/login, /etc/profile 에추가합니다. 보다상세한정보는제품문서에서확인하실수있습니다. /etc/security/limits.conf oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 /etc/pam.d/login session required /lib/security/pam_limits.so /etc/profile if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then 22
else fi fi ulimit -p 16384 ulimit -n 65536 ulimit -u 16384 -n 65536 Enterprise Linux 소프트웨어패키지의설치. 오라클소프트웨어설치를위해추가해주어야하는패키지가아래와같습니다. Enterprise Linux의 64 비트버전을설치한경우에는, 인스톨러가이패키지를이미설치한상태이어야합니다. 아닌경우아래처럼설치한다 커널매개변수설정. 텍스트편집기를사용하여 /etc/sysctl.conf 에아래라인을추가합니다. 변경사항을즉시적용하기위해 "/sbin/sysctl -p" 를실행합니다. [root@rac1 ~]# vi /etc/sysctl.conf # Controls the System Request debugging functionality of the kernel kernel.sysrq = 0 # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 kernel.shmall = 2097152 kernel.shmmax = 2147483648 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 65536 net.ipv4.ip_local_port_range = 1024 65000 net.core.rmem_default = 1048576 23
net.core.rmem_max = 1048576 net.core.wmem_default = 262144 net.core.wmem_max = 262144 root@rac1 pkg]# vi /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost 192.168.10.171 rac1.mycorpdomain.com rac1 #eth0 public 192.168.10.71 rac1-vip.mycorpdomain.com rac1-vip # 서비스 ip 10.10.10.71 rac1-priv.mycorpdomain.com rac1-priv #eth1 private 192.168.10.172 rac2.mycorpdomain.com rac2 #eth0 public 192.168.10.72 rac2-vip.mycorpdomain.com rac2-vip # 서비스 ip 10.10.10.72 rac2-priv.mycorpdomain.com rac2-priv #eth1 private Vi /etc/host 대한설명참조 127.0.0.1 localhost 192.168.10.171 rac1.mycorpdomain.com rac1 ---> eth0 192.168.10.71 rac1-vip.mycorpdomain.com rac1-vip --à eth0:1 10.10.10.71 rac1-priv.mycorpdomain.com rac1-priv ---> eth1 -- ->PRIVATE 사설 IP ( 서비스 (telnet) 접속안된다. device(eth1) 지만내부적으로 ping 가능함 ( 외부안됨 ) 192.168.10.172 rac2.mycorpdomain.com rac2 ---> eth0 192.168.10.72 rac2-vip.mycorpdomain.com rac2-vip --à eth0:1 10.10.10.72 rac2-priv.mycorpdomain.com rac2-priv -> eth1(inter- 24
connect --->PRIVATE 사설 IP ( 서비스 (telnet) 접속않된다.) device(eth1) 하지만내부적으로 ping 가능함 ( 외부안됨 ) PRIVATE 사설 IP 는? rac1 의 (eth1) 10.10.10.71 와 rac2 10.10.10.72(eth1) 는 rac1 ~ rac2 사이의내부적 으로는 ping 이가능하지만외부에서는 ping 이안될것이다. 다음은네크워크관계에서 ping 이가는지확인한다. [root@rac1 ~]# ifconfig -a eth0 Link encap:ethernet HWaddr 00:0C:29:BA:66:40 inet addr:192.168.10.171 Bcast:192.168.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:feba:6640/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2036 errors:0 dropped:0 overruns:0 frame:0 TX packets:1859 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:181137 (176.8 KiB) TX bytes:161413 (157.6 KiB) Interrupt:10 Base address:0x2024 eth0:1 Link encap:ethernet HWaddr 00:0C:29:BA:66:40 inet addr:192.168.10.71 Bcast:192.168.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:10 Base address:0x2024 eth1 Link encap:ethernet HWaddr 00:0C:29:BA:66:4A inet addr:10.10.10.71 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:feba:664a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10873 errors:0 dropped:0 overruns:0 frame:0 TX packets:13417 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:5497444 (5.2 MiB) TX bytes:10060292 (9.5 MiB) Interrupt:9 Base address:0x20a4 lo Link encap:local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 25
inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:11849 errors:0 dropped:0 overruns:0 frame:0 TX packets:11849 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5140424 (4.9 MiB) TX bytes:5140424 (4.9 MiB) sit0 Link encap:ipv6-in-ipv4 NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) [root@rac1 ~]# ping 192.168.10.171 PING 192.168.10.171 (192.168.10.171) 56(84) bytes of data. 64 bytes from 192.168.10.171: icmp_seq=0 ttl=64 time=0.928 ms 64 bytes from 192.168.10.171: icmp_seq=1 ttl=64 time=0.817 ms --- 192.168.10.171 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms rtt min/avg/max/mdev = 0.817/0.872/0.928/0.062 ms, pipe 2 [root@rac1 ~]# ping 192.168.10.172 PING 192.168.10.172 (192.168.10.172) 56(84) bytes of data. 64 bytes from 192.168.10.172: icmp_seq=0 ttl=64 time=40.7 ms 64 bytes from 192.168.10.172: icmp_seq=1 ttl=64 time=1.82 ms 64 bytes from 192.168.10.172: icmp_seq=2 ttl=64 time=6.05 ms 64 bytes from 192.168.10.172: icmp_seq=3 ttl=64 time=46.0 ms --- 192.168.10.172 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3006ms rtt min/avg/max/mdev = 1.827/23.654/46.003/19.859 ms, pipe 2 [root@rac1 ~]# ping 192.168.10.71 PING 192.168.10.71 (192.168.10.71) 56(84) bytes of data. 64 bytes from 192.168.10.71: icmp_seq=0 ttl=64 time=1.07 ms 64 bytes from 192.168.10.71: icmp_seq=1 ttl=64 time=1.00 ms 26
64 bytes from 192.168.10.71: icmp_seq=2 ttl=64 time=0.453 ms --- 192.168.10.71 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.453/0.845/1.079/0.278 ms, pipe 2 [root@rac1 ~]# ping 192.168.10.72 PING 192.168.10.72 (192.168.10.72) 56(84) bytes of data. 64 bytes from 192.168.10.72: icmp_seq=0 ttl=64 time=2.50 ms 64 bytes from 192.168.10.72: icmp_seq=1 ttl=64 time=2.59 ms 64 bytes from 192.168.10.72: icmp_seq=2 ttl=64 time=43.9 ms --- 192.168.10.72 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 2.506/16.341/43.927/19.506 ms, pipe 2 [root@rac1 ~]# ping 10.10.10.71 PING 10.10.10.71 (10.10.10.71) 56(84) bytes of data. 64 bytes from 10.10.10.71: icmp_seq=0 ttl=64 time=0.991 ms 64 bytes from 10.10.10.71: icmp_seq=1 ttl=64 time=0.927 ms 64 bytes from 10.10.10.71: icmp_seq=2 ttl=64 time=0.484 ms --- 10.10.10.71 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.484/0.800/0.991/0.227 ms, pipe 2 [root@rac1 ~]# ping 10.10.10.72 PING 10.10.10.72 (10.10.10.72) 56(84) bytes of data. 64 bytes from 10.10.10.72: icmp_seq=0 ttl=64 time=7.98 ms 64 bytes from 10.10.10.72: icmp_seq=1 ttl=64 time=44.9 ms 64 bytes from 10.10.10.72: icmp_seq=2 ttl=64 time=18.7 ms 64 bytes from 10.10.10.72: icmp_seq=3 ttl=64 time=10.2 ms --- 10.10.10.72 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3145ms rtt min/avg/max/mdev = 7.985/20.471/44.917/14.672 ms, pipe 2 정상적으로동작함 위의설정은 vmware 이므로설정된것임원래는 rac1-vip 과 rac2-vip ( 실서버사용 27
할때는서비스 IP 접속가능하도록 eth0:1 로물리적으로 device 추가해서사용한다. hangcheck-timer 커널모듈의설정. hangcheck timer 커널모듈은시스템의상태를모니터링하고장애가발생한 RAC 노드를재시작합니다. 노드의장애상황을파악하기위해사용되는두가지매개변수로 hangcheck_tick( 시스템모니터링빈도정의 ) 과 hangcheck_margin(rac 노드의리셋을수행하기위한최대지연허용시간 ) 이있습니다. /etc/modprobe.conf 파일에아래라인을추가하여 hangcheck 커널모듈매개변수를설정 해줍니다. [root@rac1 pkg]# vi /etc/modprobe.conf options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180 모듈을즉시로드하기위해 "modprobe -v hangcheck-timer" 명령을실행합니다 [root@rac1 pkg]# modprobe -v hangcheck-timer insmod/lib/modules/2.6.9-5.el/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=30 hangcheck_margin=180 실행됨. # ASM을위한디스크파티션생성. OCFS2, Oracle ASM 을위한디스크파티션생성. OCFS2(/dev/sdb) 와 Oracle ASM(/dev/sdc, /dev/sdd, /dev/sde) 을위한로우디스크를생성합니다 rac1 에서 root 사용자로로그인하여아래와같이실행. [root@rac1 pkg]# fdisk /dev/sdb 28
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-512, default 1): Using default value 1 Last cylinder or +size or +sizem or +sizek (1-512, default 512): Using default value 512 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@rac1 pkg]# fdisk /dev/sdc Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-391, default 1): Using default value 1 Last cylinder or +size or +sizem or +sizek (1-391, default 391): Using default value 391 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@rac1 pkg]# fdisk /dev/sdd 29
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-391, default 1): Using default value 1 Last cylinder or +size or +sizem or +sizek (1-391, default 391): Using default value 391 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@rac1 pkg]# fdisk /dev/sde Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-261, default 1): Using default value 1 Last cylinder or +size or +sizem or +sizek (1-261, default 261): Using default value 261 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@rac1 pkg]# fdisk -l 30
Disk /dev/sda: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 910 7309543+ 83 Linux /dev/sda2 911 1170 2088450 82 Linux swap /dev/sda3 1171 2610 11566800 83 Linux Disk /dev/sdb: 536 MB, 536870912 bytes 64 heads, 32 sectors/track, 512 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 512 524272 83 Linux Disk /dev/sdc: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdc1 1 391 3140676 83 Linux Disk /dev/sdd: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdd1 1 391 3140676 83 Linux Disk /dev/sde: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sde1 1 261 2096451 83 Linux 31
[root@rac1 ~]# vi /etc/sysconfig/rawdevices /dev/raw/raw1 /dev/sdc1 /dev/raw/raw2 /dev/sdd1 /dev/raw/raw3 /dev/sde1 ~ [root@rac1 ~]# /sbin/service rawdevices restart Assigning devices: /dev/raw/raw1 --> /dev/sdc1 /dev/raw/raw1: bound to major 8, minor 33 /dev/raw/raw2 --> /dev/sdd1 /dev/raw/raw2: bound to major 8, minor 49 /dev/raw/raw3 --> /dev/sde1 /dev/raw/raw3: bound to major 8, minor 65 done [root@rac1 ~]# chown oracle:dba /dev/raw/raw[1-3] [root@rac1 ~]# chmod 660 /dev/raw/raw[1-3] [root@rac1 ~]# ls -ltr /dev/raw/raw[1-3] crw-rw---- 1 oracle dba 162, 1 Jul 31 12:28 /dev/raw/raw1 crw-rw---- 1 oracle dba 162, 2 Jul 31 12:28 /dev/raw/raw2 crw-rw---- 1 oracle dba 162, 3 Jul 31 12:28 /dev/raw/raw3 [root@rac1 ~]# ln -sf /dev/raw/raw1 /u01/oradata/devdb/asmdisk1 [root@rac1 ~]# ln -sf /dev/raw/raw2 /u01/oradata/devdb/asmdisk2 [root@rac1 ~]# ln -sf /dev/raw/raw3 /u01/oradata/devdb/asmdisk3 /etc/udev/permissions.d/50-udev.permissions 수정 Raw-device는부팅시점에 re-mapping 됩니다. 디폴트설정에서로우디바이스의 owner-ship은부팅시점에 root 사용자에게귀속됩니다. 하지만 oracle 사용자에게 owner-ship이귀속되지않는경우, ASM이공유파티션에접근할때문제가발생할수있습니다. /etc/udev/permissions.d/50-udev.permissions 파일의 raw/*:root:disk:0660 라인을주석처리하고새로운라인 raw/*:oracle:dba:0660 을추가합니다. 32
[root@rac1 ~]# vi /etc/udev/permissions.d/50-udev.permissions ~~ # raw devices ram*:root:disk:0660 #raw/*:root:disk:0660 raw/*:oracle:dba:0660 ~~ oracleasmlib 패키지설치. OTN 에서 ASM 라이브러리를다운로드하고 root 사용자로 ASM RPM 을설치합니다 Cluster 을설치하기위해서아래의내용이없다면설치해준다. [root@rac1 pkg]# rpm -qa grep oracleasm oracleasm-support-2.0.3-1 oracleasmlib-2.0.2-1 oracleasm-2.6.9-22.el-2.0.3-1 [root@rac1 pkg]# rac1 기본 setting 완료함 5. 두번째가상머신의생성및설정 두번째가상머신을생성하기위해, 첫번째가상머신을셧다운하고, C:\vm\rac\rac1 의모든파일을 C:\vm\rac\rac2 로복사한다음몇가지설정을 변경해줍니다. 5.1 네트워크설정변경. 1. rac1 에서 root 사용자로로그인한후, # shutdown -h now 실행. 2. 호스트시스템에서 rac1 폴더의모든파일을 rac2 로복사합니다. C:\>copy C:\vm\rac\rac1 C:\vm\rac\rac2 33
3. VMware Server 콘솔에서 Ctrl-O 를눌러두번째가상머신, C:\rac\rac2\Red Hat Enterprise Linux 4.vmx 를엽니다. 4. VMware Server console: o o 가상머신의이름을 rac1 에서 rac2 로변경합니다. rac1 탭을마우스오른쪽버튼으로클릭하고 Settings 를선택합니다. Options 탭을선택합니다. 1. Virtual machine name: Virtual machine name: rac2 입력. Start this virtual machine 를클릭하여 (rac1 이셧다운된상태에서 ) rac2 를시작함. o rac2 Virtaul Machine: Create a new identifier 를클릭함. 34
root 사용자로로그인한상태에서 system-config-network 를실행하여네트워크설정을 변경함. 35
5. IP Address: 각각의이더넷디바이스를더블클릭하고아래표를참고하여변경 작업을수행합니다. 디바이스 IP 주소서브넷마스크디폴트게이트웨이주소 eth0 192.168.10.172 255.255.255.0 192.168.10.1 eth1 10.10.10.72 255.255.255.0 < 공란으로비워둠 > 6. MAC Address: Hardware Device 탭으로이동하여각이더넷디바이스의새로운 MAC 어드레스를조회합니다. 7. Hostname and DNS: 아래표를참고하여 DNS 탭의항목을변경하고 Ctrl-S 를눌러저장합니다. Hostname Primary DNS Secondary DNS DNS search path rac2.mycorpdomain.com DNS IP 주소를입력하거나공란으로비워둡니다. DNS IP 주소를입력하거나공란으로비워둡니다. 디폴트설정을승인하거나공란으로비워둡니다. 8. 마지막으로, 각이더넷디바이스를활성화합니다. /etc/hosts 파일수정. /etc/hosts 파일에아래항목을추가합니다. 127.0.0.1 localhost VIPCA 는 Oracle Clusterware 소프트웨어설치과정에서루프백주소를사용하려시도함. [root@rac2 ~]# vi /export/home/oracle/.profile ORACLE_SID 값을 devdb2 로변경 LANG=C export PS1="`/bin/hostname -s`-> " export EDITOR=vi export ORACLE_SID=devdb2 -à 변경함 export ORACLE_BASE=/u01/app/oracle 36
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1 export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1 export LD_LIBRARY_PATH=$ORACLE_HOME/lib export PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/ usr/x11r6/bin umask 022 SSH를이용한 user equivalence의설정. Cluster Ready Services(CRS) 및 RAC 설치과정에서, Oracle Universal Installer(OUI) 는 oracle 사용자로 ( 패스워드를별도로입력하지않고 ) 모든 RAC 노드에소프트웨어를복사할수있어야합니다. Oracle 10g에서는 rsh 대신 ssh를이용하여이작업을수행할수있습니다 oracle user 에서작업한다. 먼저 rac1 에서아래와같이작업합니다 Last login: Thu Jul 26 20:01:08 2007 [root@rac1 ~]# su - oracle rac1-> mkdir ~/.ssh rac1-> chmod 700 ~/.ssh rac1-> ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/export/home/oracle/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oracle/.ssh/id_rsa. Your public key has been saved in /export/home/oracle/.ssh/id_rsa.pub. The key fingerprint is: 53:5f:37:26:91:d8:6d:43:8c:ba:da:39:03:ba:2a:ca oracle@rac1.mycorpdomain.com rac1-> ssh-keygen -t dsa 37
Generating public/private dsa key pair. Enter file in which to save the key (/export/home/oracle/.ssh/id_dsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oracle/.ssh/id_dsa. Your public key has been saved in /export/home/oracle/.ssh/id_dsa.pub. The key fingerprint is: 93:73:15:37:cf:06:b1:e6:ff:00:e5:4a:70:af:a2:d7 oracle@rac1.mycorpdomain.com 다음으로 rac2 에서아래와같이작업합니다 Last login: Thu Jul 26 20:13:53 2007 [root@rac2 ~]# su - oracle rac2-> mkdir ~/.ssh rac2-> chmod 700 ~/.ssh rac2-> ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/export/home/oracle/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oracle/.ssh/id_rsa. Your public key has been saved in /export/home/oracle/.ssh/id_rsa.pub. The key fingerprint is: fa:a4:7f:a6:8e:eb:53:51:3d:fa:29:31:8d:42:42:a1 oracle@rac2.mycorpdomain.com rac2-> ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/export/home/oracle/.ssh/id_dsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oracle/.ssh/id_dsa. Your public key has been saved in /export/home/oracle/.ssh/id_dsa.pub. The key fingerprint is: 71:2f:2d:e3:1c:4d:cc:4c:00:cd:25:95:28:f8:c9:48 oracle@rac2.mycorpdomain.com 38
rac1 에서아래와같이작업합니다 rac1-> cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys rac1-> cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys rac1-> ssh rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys The authenticity of host 'rac2 (192.168.10.172)' can't be established. RSA key fingerprint is be:e5:a9:27:2d:5b:e5:48:be:10:f4:16:b6:a4:1a:3d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac2,192.168.10.172' (RSA) to the list of known hosts. oracle@rac2's password: oracle 엔터 rac1-> ssh rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys oracle@rac2's password: oracle 엔터 rac1-> scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys oracle@rac2's password: oracle 엔터 authorized_keys 100% 1712 1.7KB/s 00:00 rac1-> 사전 test 해봅니다. 각노드의연결을테스트합니다. 아래명령을두번째실행했을때패스워드를묻는프롬 프트가뜨지않음을확인 실전 > rac1-> ssh rac1 date The authenticity of host 'rac1 (192.168.10.171)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac1,192.168.10.171' (RSA) to the list of known hosts. Thu Aug 2 19:23:36 KST 2007 rac1-> ssh rac2 date 39
Thu Aug 2 19:24:13 KST 2007 rac1-> ssh rac1-priv date The authenticity of host 'rac1-priv (10.10.10.71)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac1-priv,10.10.10.71' (RSA) to the list of known hosts. Thu Aug 2 19:25:05 KST 2007 rac1-> ssh rac2-priv date The authenticity of host 'rac2-priv (10.10.10.72)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac2-priv,10.10.10.72' (RSA) to the list of known hosts. Thu Aug 2 19:24:31 KST 2007 rac1-> ssh rac1.mycorpdomain.com date The authenticity of host 'rac1.mycorpdomain.com (192.168.10.171)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac1.mycorpdomain.com' (RSA) to the list of known hosts. Thu Aug 2 19:25:26 KST 2007 rac1-> ssh rac2.mycorpdomain.com date The authenticity of host 'rac2.mycorpdomain.com (192.168.10.172)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac2.mycorpdomain.com' (RSA) to the list of known hosts. Thu Aug 2 19:24:48 KST 2007 40
rac1-> ssh rac1-priv.mycorpdomain.com date The authenticity of host 'rac1-priv.mycorpdomain.com (10.10.10.71)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac1-priv.mycorpdomain.com' (RSA) to the list of known hosts. Thu Aug 2 19:25:42 KST 2007 rac1-> ssh rac2-priv.mycorpdomain.com date The authenticity of host 'rac2-priv.mycorpdomain.com (10.10.10.72)' can't be established. RSA key fingerprint is 50:f8:cb:b0:12:12:a2:7b:cb:54:4f:d4:d0:89:15:6d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'rac2-priv.mycorpdomain.com' (RSA) to the list of known hosts. Thu Aug 2 19:25:01 KST 2007 rac1-> rac1-> 패스워드묻는문구안나옴을확인합니다 ( 정상확인 ) 다시실행 (test) 해본다. rac1-> ssh rac1 date Mon Aug 6 14:55:24 KST 2007 rac1-> ssh rac2 date Mon Aug 6 14:55:19 KST 2007 rac1-> ssh rac1-priv date Mon Aug 6 14:55:35 KST 2007 rac1-> ssh rac2-priv date Mon Aug 6 14:55:30 KST 2007 rac1-> ssh rac1.mycorpdomain.com date Mon Aug 6 14:55:47 KST 2007 rac1-> ssh rac2.mycorpdomain.com date 41
Mon Aug 6 14:55:41 KST 2007 rac1-> ssh rac1-priv.mycorpdomain.com date Mon Aug 6 14:55:59 KST 2007 rac1-> ssh rac2-priv.mycorpdomain.com date Mon Aug 6 14:55:52 KST 2007 양쪽 load 에설정 (rac1, rac2) 5.2 Oracle Automatic Storage Management (ASM) 설정 Oracle ASM은오라클데이터베이스와긴밀하게통합되어있으며오라클의데이터베이스관리툴과연동합니다. Oracle ASM은데이터베이스스토리지관리업무를단순화하고로우디스크 I/O 성능을개선하는효과를제공합니다. ASMLib 설정. root 사용자로로그인하여양쪽노드에서 ASMLib 을설정해줍니다. [root@rac1 ~]# /etc/init.d/oracleasm configure Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Start Oracle ASM library driver on boot (y/n) [n]: y Fix permissions of Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: [ OK ] Loading module "oracleasm": [ OK ] Mounting ASMlib driver filesystem: [ OK ] Scanning system for ASM disks: [ OK ] 42
rac1 설정성공함 ASM 디스크의생성. root 사용자로두노드중하나에서만 ASM 디스크를 생성합니다.( rac1 만생성 ) [root@rac1 ~]# [root@rac1 ~]# /etc/init.d/oracleasm createdisk VOL1 /dev/sdc1 Marking disk "/dev/sdc1" as an ASM disk: [ OK ] [root@rac1 ~]# /etc/init.d/oracleasm createdisk VOL2 /dev/sdd1 Marking disk "/dev/sdd1" as an ASM disk: [ OK ] [root@rac1 ~]# /etc/init.d/oracleasm createdisk VOL3 /dev/sde1 Marking disk "/dev/sde1" as an ASM disk: [ OK ] [root@rac1 ~]# /etc/init.d/oracleasm scandisks Scanning system for ASM disks: [ OK ] [root@rac1 ~]# /etc/init.d/oracleasm listdisks VOL1 VOL2 VOL3 성공함. [root@rac1 ~]# Rac2 에서도 oracleasm 설정함. [root@rac2 ~]# /etc/init.d/oracleasm configure Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: oracle 43
Default group to own the driver interface []: dba Start Oracle ASM library driver on boot (y/n) [n]: y Fix permissions of Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: [ OK ] Loading module "oracleasm": [ OK ] Mounting ASMlib driver filesystem: [ OK ] Scanning system for ASM disks: [ OK ] rac2 도성공함. Oracle Cluster File System(OCFS2) 설정 OCFS2는오라클이개발한범용클러스터파일시스템으로 Enterprise Linux 커널과통합되어있습니다. OCFS2는전체노드가클러스터파일시스템에동시접근하는것을가능하게하며, 로우디바이스관리의필요성을제거합니다. 본가이드에서는 OCFS2 파일시스템에 OCR과 Voting Disk를위치시키는방법을사용합니다. OCFS2에대한자세한정보는 OCFS2 사용자가이드에서확인하실수있습니다. Enterprise Linux 설치과정에서이미 OCFS2 RPM 이설치되었을것입니다. * 양쪽노드에 RPM 이선택되었는지확인합니다. [root@rac1 ~]# rpm -qa grep ocfs ocfs2-tools-1.2.2-1 ocfs2-2.6.9-55.el-1.2.5-2 ocfs2console-1.2.2-1 [root@rac2 ~]# rpm -qa grep ocfs ocfs2-tools-1.2.2-1 ocfs2-2.6.9-55.el-1.2.5-2 ocfs2console-1.2.2-1 OCFS2 설정파일의생성. root 사용자로로그인하여아래와같이실행 44
[root@rac2 ~]# ocfs2console OCFS2 Console: Cluster, Configure Nodes를선택합니다. The cluster stack has been started : Close를클릭합니다. Node Configuration: Add를클릭합니다. Add Node: 아래노드를추가한후 Apply를클릭합니다. Name: rac1 IP Address: 192.168.10.171 IP Port: 7777 Name: rac2 IP Address: 192.168.10.172 IP Port: 7777 5. 생성된설정파일의내용을확인함 45
[root@rac1 ~]# more /etc/ocfs2/cluster.conf node: ip_port = 7777 ip_address = 192.168.10.171 number = 0 name = rac1 cluster = ocfs2 node: ip_port = 7777 ip_address = 192.168.10.172 number = 1 name = rac2 cluster = ocfs2 cluster: node_count = 2 name = ocfs2 rac1 의내용그대로 copy 한다. [root@rac2 etc]# more /etc/ocfs2/cluster.conf node: ip_port = 7777 ip_address = 192.168.10.171 number = 0 name = rac1 cluster = ocfs2 node: ip_port = 7777 ip_address = 192.168.10.172 number = 1 46
name = rac2 cluster = ocfs2 cluster: node_count = 2 name = ocfs2 [root@rac2 etc]# 6. O2CB 드라이버설정. O2CB는노드와클러스터파일시스템간의커뮤니케이션을관리하는일련의클러스터링서비스로구성됩니다. 각서비스에대한설명이아래와같습니다 NM: Node Manager / cluster.conf에설정된모든노드의상태를추적. HB: Heartbeat 서비스 / 노드가클러스터에가입 / 탈퇴하는경우업 / 다운통보를전달 TCP: 노드간의커뮤니케이션을처리 DLM: Distributed Lock Manager / 락, 락의소유자및상태정보를추적 CONFIGFS: 사용자공간 (/config) 에마운트되는구성파일시스템 DLMFS: 커널스페이스 DLM을위한사용자공간인터페이스 * 양쪽노드에아래작업을수행하여부팅시 O2CB 가실행되도록설정해줍니다. heartbeat dead threshold를묻는프롬프트에서 7 이상의값을입력하여낮은성능의 IDE 디스크드라이브로인해 node crash 가발생하는것을방지해주어야합니다. heartbeat dead threshold는 fence time을계산하기위한변수로활용됩니다. Fence time (seconds) = (heartbeat dead threshold -1) * 2 본가이드의구성에서는 120 초의 fence time 이면충분합니다. heartbeat dead threshold 값은 * 양쪽 node 에동일하게설정되어야합니다. 47
root 사용자로로그인하여아래와같이실행 [root@rac1 ~]# /etc/init.d/o2cb unload Stopping O2CB cluster ocfs2: OK Unmounting ocfs2_dlmfs filesystem: OK Unloading module "ocfs2_dlmfs": OK Unmounting configfs filesystem: OK Unloading module "configfs": OK [root@rac1 ~]# /etc/init.d/o2cb configure Configuring the O2CB driver. This will configure the on-boot properties of the O2CB driver. The following questions will determine whether the driver is loaded on boot. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Load O2CB driver on boot (y/n) [n]: y Cluster to start on boot (Enter "none" to clear) [ocfs2]: Specify heartbeat dead threshold (>=7) [7]: 61 Writing O2CB configuration: OK Loading module "configfs": OK Mounting configfs filesystem at /config: OK Loading module "ocfs2_nodemanager": OK Loading module "ocfs2_dlm": OK Loading module "ocfs2_dlmfs": OK Mounting ocfs2_dlmfs filesystem at /dlm: OK Starting O2CB cluster ocfs2: OK [root@rac1 ~]# 48
rac1 설정성공함 --------------------------------------------------------- [root@rac2 ~]# /etc/init.d/o2cb unload 아무것도안나옴 [root@rac2 ~]# /etc/init.d/o2cb configure Configuring the O2CB driver. This will configure the on-boot properties of the O2CB driver. The following questions will determine whether the driver is loaded on boot. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Load O2CB driver on boot (y/n) [n]: y Cluster to start on boot (Enter "none" to clear) [ocfs2]: Specify heartbeat dead threshold (>=7) [7]: 61 Writing O2CB configuration: OK Loading module "configfs": OK Creating directory '/config': OK Mounting configfs filesystem at /config: OK Loading module "ocfs2_nodemanager": OK Loading module "ocfs2_dlm": OK Loading module "ocfs2_dlmfs": OK Creating directory '/dlm': OK Mounting ocfs2_dlmfs filesystem at /dlm: OK Starting O2CB cluster ocfs2: OK [root@rac2 ~]# rac2 설정성공 참고 : 파일시스템의포맷. 파일시스템의포맷, 마운팅을수행하기전에, OC2B 가양쪽 node 에서온라인상태인지확인합니다. O2CB heartbeat 는파일시스템이 mount 되기전까지활성화되지않습니다. 49
[root@rac1 ~]# /etc/init.d/o2cb status Module "configfs": Loaded Filesystem "configfs": Mounted Module "ocfs2_nodemanager": Loaded Module "ocfs2_dlm": Loaded Module "ocfs2_dlmfs": Loaded Filesystem "ocfs2_dlmfs": Mounted Checking O2CB cluster ocfs2: Online Checking O2CB heartbeat: Not active [root@rac1 ~]# --> 온라인상태확인 [root@rac2 ~]# /etc/init.d/o2cb status Module "configfs": Loaded Filesystem "configfs": Mounted Module "ocfs2_nodemanager": Loaded Module "ocfs2_dlm": Loaded Module "ocfs2_dlmfs": Loaded Filesystem "ocfs2_dlmfs": Mounted Checking O2CB cluster ocfs2: Online Checking O2CB heartbeat: Not active [root@rac2 ~]# --> 온라인상태확인 파일시스템포맷작업은두 node 중하나에서만수행합니다. root 사용자로 rac1 에로그인하여아래와같이실행합니다. [root@rac2 ~]# ocfs2console OCFS2 Console: Tasks, Format을선택합니다. Format: Available devices: /dev/sdb1 Volume label: oracle 50
Cluster size: Auto Number of node slots: 4 Block size: Auto OCFS2 Console: CTRL-Q를눌러종료합니다. 파일시스템의마운트. 파일시스템을마운트하기위해, * 양쪽 node 에서 아래와같이명령을실행합니다. [root@rac1 ~]# mount -t ocfs2 -o datavolume,nointr /dev/sdb1 /ocfs [root@rac2 ~]# mount -t ocfs2 -o datavolume,nointr /dev/sdb1 /ocfs 부팅시에파일시스템이 mount 되도록, 양쪽노드의 /etc/fstab 에아래라인을추가해 줍니다. [root@rac1 ~]# vi /etc/fstab # This file is edited by fstab-sync - see 'man fstab-sync' for details LABEL=/ / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 LABEL=/u01 /u01 ext3 defaults 1 2 LABEL=SWAP-sda2 swap swap defaults 0 0 # Beginning of the block added by the VMware software.host:/ /mnt/hgfs vmhgfs defaults,ttl=5 0 0 # End of the block added by the VMware software /dev/hdc /media/cdrecorder auto pamconsole,fscontext=system_u:object_r:removable_t,exec,noauto,managed 0 0 /dev/fd0 /media/floppy auto pamconsole,fscontext=system_u:object_r:removable_t,exec,noauto,managed 0 0 /dev/sdb1 /ocfs ocfs2 _netdev,datavolume,nointr 0 0 -->> 추가해줌 51
[root@rac2 ~]# vi /etc/fstab # This file is edited by fstab-sync - see 'man fstab-sync' for details LABEL=/ / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 LABEL=/u01 /u01 ext3 defaults 1 2 LABEL=SWAP-sda2 swap swap defaults 0 0 # Beginning of the block added by the VMware software.host:/ /mnt/hgfs vmhgfs defaults,ttl=5 0 0 # End of the block added by the VMware software /dev/hdc /media/cdrecorder auto pamconsole,fscontext=system_u:object_r:removable_t,exec,noauto,managed 0 0 /dev/fd0 /media/floppy auto pamconsole,fscontext=system_u:object_r:removable_t,exec,noauto,managed 0 0 /dev/sdb1 /ocfs ocfs2 _netdev,datavolume,nointr 0 0 ---> 추가해줌 Oracle Clusterware 디렉토리생성. OCFS2 파일시스템에 OCR, Voting Disk 가위치할 디렉토리를생성. rac1 에서아래와같이작업 [root@rac1 ~]# mkdir /ocfs/clusterware [root@rac1 ~]# chown -R oracle:dba /ocfs 이것으로 OCFS2 의셋업과정을완료하였습니다. * 양쪽 node 의공유클러스터파일시스템에서파일읽기 / 쓰기를수행할수있는지확인 함. 52
7. Oracle Clusterware 설치 다운로드를완료한후, rac1 에 oracle 사용자로로그인하여아래와같이실행 Xhost 및 xclock 이실행되도록함. 53
클러스터설치한다. 54
55
56
57
58
Oracle Clusterware 설치위의그림참고함 59
다운로드를완료한후, rac1 에 oracle 사용자로로그인하여아래와같이실행합니다 : rac1-> /u01/staging/clusterware/runinstaller Welcome: Next를클릭합니다. Specify Inventory directory and credentials: Enter the full path of the inventory directory: /u01/app/oracle/orainventory. Specify Operating System group name: oinstall. Specify Home Details: Name: OraCrs10g_home /u01/app/oracle/product/10.2.0/crs_1 Product-Specific Prerequisite Checks: 물리적메모리요구사항에관련한경고를무시합니다. Specify Cluster Configuration: Add를클릭합니다. Public Node Name: rac2.mycorpdomain.com Private Node Name: rac2-priv.mycorpdomain.com Virtual Host Name: rac2-vip.mycorpdomain.com Specify Network Interface Usage: Interface Name: eth0 Subnet: 192.168.10.0 Interface Type: Public Interface Name: eth1 Subnet: 10.10.10.0 Interface Type: Private Specify Oracle Cluster Registry (OCR) Location: External Redundancy를선택합니다. 구성의단순화를위해, 본가이드에서는 OCR을미러링하지않는것으로합니다. 실제운영환경에서는 OCR 디스크의가용성개선을위해멀티플렉싱을사용해야합니다. Specify OCR Location: /ocfs/clusterware/ocr Specify Voting Disk Location: External Redundancy를선택합니다. 구성의단순화를위해, 본가이드에서는 Voting Disk를미러링하지않는것으로합니다. Voting Disk Location: /ocfs/clusterware/votingdisk 위의마지막그림보면. Summary: Install 을클릭합니다. install 하다가 rac2 하다가 errors 난다. 60
계속 yes 누르고진행한다. 85% 정도에서 PRKC-1002 RAC2 Error 나면서창이뜬다. Error 가안나면계속진행한다. 계속진행하고 100% 가되면아래처럼진행한다. 먼저아래와같이 root 사용자로실행한다. 1>Execute -à /u01/app/oracle/orainventory/orainstroot.sh on rac1. Execute -à /u01/app/oracle/orainventory/orainstroot.sh on rac2. Rac1 에서실행. [root@rac1 orainventory]# pwd /u01/app/oracle/orainventory [root@rac1 orainventory]# ls Contents install.platform orainst.loc orainstroot.sh sessioncontext.ser ContentsXML logs orainstaller.properties oui [root@rac1 orainventory]# pwd /u01/app/oracle/orainventory [root@rac1 orainventory]# sh orainstroot.sh Changing permissions of /u01/app/oracle/orainventory to 770. 61
Changing groupname of /u01/app/oracle/orainventory to oinstall. The execution of the script is complete Rac2 에서실행 [root@rac2 orainventory]# ls Contents ContentsXML install.platform logs orainst.loc orainstroot.sh [root@rac2 orainventory]# sh orainstroot.sh Changing permissions of /u01/app/oracle/orainventory to 770. Changing groupname of /u01/app/oracle/orainventory to oinstall. The execution of the script is complete [root@rac2 orainventory]# 먼저아래와같이 root 사용자로실행한다. 2>Execute-à /u01/app/oracle/product/10.2.0/crs_1/root.sh on rac1. * Execute-à /u01/app/oracle/product/10.2.0/crs_1/root.sh on rac2 [root@rac1 crs_1]# pwd /u01/app/oracle/product/10.2.0/crs_1 [root@rac1 crs_1]# ls JRE cfgtoollogs cv install jlib log orainst.loc plsql relnotes srvm OPatch clone diagnostics install.platform jre network oracore precomp root.sh xdk bin crs evm inventory ldap nls oui racg slax cdata css has jdk lib opmn perl rdbms sqlplus [root@rac1 crs_1]# sh root.sh WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root WARNING: directory '/u01/app/oracle/product' is not owned by root WARNING: directory '/u01/app/oracle' is not owned by root WARNING: directory '/u01/app' is not owned by root 62
WARNING: directory '/u01' is not owned by root Checking to see if Oracle CRS stack is already configured /etc/oracle does not exist. Creating it now. Setting the permissions on OCR backup directory Setting up NS directories Oracle Cluster Registry configuration upgraded successfully WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root WARNING: directory '/u01/app/oracle/product' is not owned by root WARNING: directory '/u01/app/oracle' is not owned by root WARNING: directory '/u01/app' is not owned by root WARNING: directory '/u01' is not owned by root assigning default hostname rac1 for node 1. assigning default hostname rac2 for node 2. Successfully accumulated necessary OCR keys. Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897. node <nodenumber>: <nodename> <private interconnect name> <hostname> node 1: rac1 rac1-priv rac1 node 2: rac2 rac2-priv rac2 Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Now formatting voting device: /ocfs/clusterware/votingdisk Format of 1 voting devices complete. Startup will be queued to init within 90 seconds. Adding daemons to inittab Expecting the CRS daemons to be up within 600 seconds. CSS is active on these nodes. rac1 CSS is inactive on these nodes. rac2 Local node checking complete. Run root.sh on remaining nodes to start CRS daemons. 성공적으로끝남. [root@rac2 crs_1]# sh root.sh 63
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root WARNING: directory '/u01/app/oracle/product' is not owned by root WARNING: directory '/u01/app/oracle' is not owned by root WARNING: directory '/u01/app' is not owned by root WARNING: directory '/u01' is not owned by root Checking to see if Oracle CRS stack is already configured /etc/oracle does not exist. Creating it now. Setting the permissions on OCR backup directory Setting up NS directories Oracle Cluster Registry configuration upgraded successfully WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root WARNING: directory '/u01/app/oracle/product' is not owned by root WARNING: directory '/u01/app/oracle' is not owned by root WARNING: directory '/u01/app' is not owned by root WARNING: directory '/u01' is not owned by root clscfg: EXISTING configuration version 3 detected. clscfg: version 3 is 10G Release 2. assigning default hostname rac1 for node 1. assigning default hostname rac2 for node 2. Successfully accumulated necessary OCR keys. Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897. node <nodenumber>: <nodename> <private interconnect name> <hostname> node 1: rac1 rac1-priv rac1 node 2: rac2 rac2-priv rac2 clscfg: Arguments check out successfully. NO KEYS WERE WRITTEN. Supply -force parameter to override. -force is destructive and will destroy any previous cluster configuration. Oracle Cluster Registry for cluster has already been initialized Startup will be queued to init within 90 seconds. Adding daemons to inittab Expecting the CRS daemons to be up within 600 seconds. CSS is active on these nodes. rac1 64
rac2 CSS is active on all nodes. Waiting for the Oracle CRSD and EVMD to start Waiting for the Oracle CRSD and EVMD to start Oracle CRS stack installed and running under init(1m) Running vipca(silent) for configuring nodeapps 진행하다가다음과같은 message 발생함. 제공된인터페이스 "eth0" 가공용이아닙니다. 가상 ip 를구성하려면공용인터페이스를 사용해야합니다. < 참고 > rac2의 root.sh 스크립트는 VIPCA를자동으로호출하지만, "The given interface(s), "eth0" is not public. Public interfaces should be used to configure virtual IPs" 에러와함께실패합니다. public 인터페이스에사설 IP 주소 (192.168.10.72) 가사용되고있기때문에 Oracle Cluster Verification Utility(CVU) 가적절한 Public 인터페이스를찾을수없는것입니다. * 이문제를해결하려면 VIPCA 를수동으로실행해야합니다. root 사용자로로그인하여두번째 node ( rac2) 에서 VIPCA 를수동으로실행합니다. # /u01/app/oracle/product/10.2.0/crs_1/bin/vipca Ex> [root@rac2 bin]# pwd /u01/app/oracle/product/10.2.0/crs_1/bin [root@rac2 bin]#./vipca Welcome: Next를클릭합니다. Network Interfaces: eth0를선택합니다. Virtual IPs for cluster nodes: Node name: rac1 IP Alias Name: rac1-vip 65
IP address: 192.168.10.71 Subnet Mask: 255.255.255.0 Node name: rac2 IP Alias Name: rac2-vip IP address: 192.168.10.72 Subnet Mask: 255.255.255.0 Summary: Finish를클릭합니다. 나머지그림자료참조한다. 66
모두선택한다. 위의내용을입력한후다음클릭한다. 성공적으로설정되어완료를클릭한다. 67
설치를시작합니다. 설치가완료되어확인을클릭합니다. 68
종료를클릭합니다. 화면이사라지면 rac1 의 Execute Configuration 스크립트스크린으로아래로 돌아가 OK 를클릭합니다. 69
OK 클릭함. 70
성공적으로끝났음. cluster 잘설치됐는지확인한다. Configuration Assistants: 모든점검이성공적으로완료되었는지확인합니다. EXIT 아직하지말고 Clusterware 설치아래와같이해서성공적인확인후에한다. OUI 는 Clusterware 설치가완료된후사후점검작업을수행합니다. CVU 가실패한경우문제를해결하고다시 oracle 사용자로아래명령을재실행합니다 Rac1 에서실행한다. rac1->/u01/app/oracle/product/10.2.0/crs_1/bin/cluvfy stage -post crsinst -n rac1,rac2 Performing post-checks for cluster services setup Checking node reachability... Node reachability check passed from node "rac1". 71
Checking user equivalence... User equivalence check passed for user "oracle". Checking Cluster manager integrity... Checking CSS daemon... Daemon status check passed for "CSS daemon". Cluster manager integrity check passed. Checking cluster integrity... Cluster integrity check passed Checking OCR integrity... Checking the absence of a non-clustered configuration... All nodes free of non-clustered, local-only configurations. Uniqueness check for OCR device passed. Checking the version of OCR... OCR of correct Version "2" exists. Checking data integrity of OCR... Data integrity check for OCR passed. OCR integrity check passed. Checking CRS integrity... Checking daemon liveness... Liveness check passed for "CRS daemon". Checking daemon liveness... 72
Liveness check passed for "CSS daemon". Checking daemon liveness... Liveness check passed for "EVM daemon". Checking CRS health... CRS health check passed. CRS integrity check passed. Checking node application existence... Checking existence of VIP node application (required) Check passed. Checking existence of ONS node application (optional) Check passed. Checking existence of GSD node application (optional) Check passed. Post-check for cluster services setup was successful. 정상적으로확인되었음. EXIT 클릭하고종료한다. Oracle cluster services 성공적으로완료되었습니다. rac1-> crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 73
ora.rac2.gsd application ONLINE ONLINE rac2 ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip application ONLINE ONLINE rac2 rac1-> crsctl check crs CSS appears healthy CRS appears healthy EVM appears healthy. [root@rac2 ~]# su - oracle rac2-> crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora.rac2.gsd application ONLINE ONLINE rac2 ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip application ONLINE ONLINE rac2 rac2-> crsctl check crs CSS appears healthy CRS appears healthy EVM appears healthy 클러스터정상적으로설치됨. ( 완료 ) 74
Oracle Database 10g Release 2 설치 다운로드를완료한후, rac1 에 oracle 사용자로로그인하여아래와같이 실행함. rac1-> /u01/staging/database/runinstaller 1. Welcome: Next 를클릭합니다. 2. Select Installation Type: o Enterprise Edition 을선택합니다. 3. Specify Home Details: o Name: OraDb10g_home1 o Path: /u01/app/oracle/product/10.2.0/db_1 4. Specify Hardware Cluster Installation Mode: o Cluster Installation 을선택합니다. o Select All 을클릭합니다. 5. Product-Specific Prerequisite Checks: o 물리적메모리요구사항에관련한경고를무시합니다. 6. Select Configuration Option: o Create a Database 를선택합니다. 7. Select Database Configuration: o Advanced 를선택합니다. 8. Summary: Install 을클릭합니다. 9. Database Templates: o General Purpose 를선택합니다. 10. Database identification: o Global Database Name: devdb o SID Prefix: devdb 11. Management Options: o Configure the Database with Enterprise Manager 를선택합니다. 12. Database Credentials: 75
o Use the Same Password for All Accounts 를선택합니다. 13. Storage Options: o Automatic Storage Management (ASM) 을선택합니다. 14. Create ASM Instance: o SYS password: <SYS 패스워드를입력합니다 >. o Create initialization parameter file (IFILE) 을선택합니다. 15. ASM Disk Groups: o Create New 를클릭합니다. 16. Create Disk Group: 두개의디스크그룹 (DG1, RECOVERYDEST) 을생성합니다. o Disk Group Name: DG1 o Normal redundancy 를선택합니다. o Disk Path, ORCL:VOL1, ORCL:VOL2 를선택합니다. 표준 Linux I/O 를이용하여 ASM 디스크를설정한경우, /u01/oradata/devdb/asmdisk1 와 /u01/oradata/devdb/asmdisk2 를선택해주어야합니다. o OK 를클릭합니다. 76
77
78
79
92% 15 분정도조금오랜시간이걸린다. 80
81
Password : oracle 이라고입력한다. 82
아래의 ASM parameters 클릭해서아래의내용확인한다. 위의그림의 asm_diskstring 의 /dev/raw/raw* 을설정안해도상관없다. 아래처럼해도관계없다. 83
Sys password : oracle 로입력한다. Next 한다. 84
OK 클릭한다. 85
위의그림에서 create New 을클릭한다. 다음과같은화면이나온다. 86
OK 클릭한다. 87
Create New 을클릭한다. OK 을클릭한다. 88
ASM Disk Groups: Next 를클릭. 89
NEXT 클릭한다. Database File Locations: Use Oracle-Managed Files 를선택합니다. Database Area : +DG1 90
Recovery Configuration: Specify Flash Recovery Area 를선택 Flash Recovery Area: +RECOVERYDEST Flash Recovery Area Size: 1500M * Enable Archiving 를선택 Database Content : 샘플스키마를선택또는해제하고 NEXT 클릭. Database Services : Next 를클릭. 나중에 DBCA 또는 srvctl 을이용하여서비스를추가로생성, 변경할수있음 ( EX> TAF ) NEXT 클릭한다. 91
나머지는 default 설정함.( 경우에따라서다르게설정가능함 ) Initialization Parameters : Custom 를선택 Shared Memory Management : Automatic *SGA Size : 200MB *PGA Size : 25MB * 필요에따라나머지매개변수를변경해줍니다 NEXT 클릭함. 92
Finish 클릭함. 아래와같은화면나옴. OK 클릭하면아래화면나오면서 Database Configuration Assistant 화면시작. 93
아래의그림처럼 100% 완료되면 Exit 클릭한다. 아래와같은화면생성됨. 94
Start cluster Database 실행중이므로창을닫으면절대안됨. Vmware 라시간이상당히걸림.(10~20 분 ) 기다리면아래와같이 succeeded. 성공적으로끝남. 그리고다음화면이나옴. 주의할점은. Execute Configuration scripts : root 사용자로로그인하여아래스크립트를실행합니다. rac1 에서 /u01/app/oracle/product/10.2.0/db_1/root.sh 스크립트를실행함. rac2 에서 /u01/app/oracle/product/10.2.0/db_1/root.sh 스크립트를실행함. 그리고완료되면아래의화면에서 OK 클릭한다. 95
rac1 의 Execute Configuration 스크립트스크린으로돌아가 OK 를클릭함. EXIT 을클릭합니다. 모든 RAC2 구성과구축이끝났습니다. 직원여러분수고하셨습니다. 감사합니다. 96