【让学习oracle前进一步】
一、实验环境
1.1、IP地址规划
网卡名 | 节点名 | IP地址 | 网卡类型 | 别名 | |
Eth0 | ctpdb1 | 172.17.80.11 | private | ctpdb1-priv | vm2 |
Eth1 | ctpdb1 | 192.168.1.11 | public | ctpdb1 | Host-only |
无 | ctpdb1 | 192.168.1.111 | virtual | ctpdb1-vip | |
无 | ctpdb1 | 192.168.1.10 | 无 | rac-cluster | |
Eth0 | ctpdb2 | 172.17.80.12 | private | ctpdb2-priv | vm2 |
Eth1 | ctpdb2 | 192.168.1.12 | public | ctpdb2 | Host-only |
无 | ctpdb2 | 192.168.1.112 | virtual | ctpdb2-vip | |
#------------------------------------------------
1.2、主机信息
节点1：/etc/hosts
192.168.1.11ctpdb1
192.168.1.111 ctpdb1-vip
172.17.80.11 ctpdb1-priv
192.168.1.12ctpdb2
192.168.1.112 ctpdb2-vip
172.17.80.12 ctpdb2-priv
192.168.1.10 ctpdb-cluster
#------------------------------------------------
节点2：/etc/hosts
192.168.1.11ctpdb1
192.168.1.111 ctpdb1-vip
172.17.80.11 ctpdb1-priv
192.168.1.12ctpdb2
192.168.1.112 ctpdb2-vip
172.17.80.12 ctpdb2-priv
192.168.1.10 ctpdb-cluster
#------------------------------------------------
1.3、基础信息
systemOS：rhel 5.8x64kernel:2.6.18-308
media: p10404530_112030_Linux-x86-64_1of7.zip(oracle)
p10404530_112030_Linux-x86-64_2of7.zip(oracle)
p10404530_112030_Linux-x86-64_3of7.zip(grid)
version:Oracle 11.2.0.3
asm:oracleasm-2.6.18-308.el5-2.0.5-1.el5.x86_64.rpm
oracleasmlib-2.0.4-1.el5.x86_64.rpm
oracleasm-support-2.1.7-1.el5.x86_64.rpm
VM:VMware Workstation9.0
1.4、分区规划
1.4.1 节点1复制到节点2，来实现操作系统一致性。
1.4.2 共享磁盘：通过手动修改ctpdb1.vmx来实现。
1.4.3 磁盘信息
RAC1 hostname ctpdb1 共40G,mem 2G,SWAP4G, boot 100M ,/剩余
RAC2 hostname ctpdb2 共40G,mem 2G,SWAP4G ,boot 100M, /剩余
-----------------------------------------------------
二、安装虚拟机操作系统、创建共享磁盘
2.1 使用wmwareworkstation 9
2.2选择安装linux
节点1、节点2分别存放F盘下vm_ctpdb1/vm_ctpdb2
2.3磁盘格式，选择SCSI
2.4添加虚拟机共享硬盘，分区如下
2.4.1添加1/3OCR磁盘，大小为：1G
2.4.2修改磁盘节点名称为：ocr1.vmdk，节点为：SCSI1:0
2.5注：以上述方式分别添加和修改如下磁盘：
2.5.1添加2/3OCRGROUP,大小为:1G,名称为:ocr2.vmdk, 节点为:SCSI1:1;
2.5.2添加3/3 OCRGROUP,大小为:1G,名称为:ocr3.vmdk, 节点为:SCSI1:2;
2.5.3添加1/3DATAGROUP,大小为:10G, 名称为:data1.vmdk, 节点为:SCSI1:3;
2.5.4添加2/3DATAGROUP,大小为:10G, 名称为:data2.vmdk,节点为:SCSI1:4;
2.5.4添加3/3DATAGROUP,大小为:10G, 名称为:data3.vmdk,节点为:SCSI1:5;
2.5.5添加1/2ARCHGROUP,大小为:5G, 名称为:arch1.vmdk,节点为:SCSI1:6;
2.5.6添加2/2ARCHGROUP,大小为:5G, 名称为:arch2.vmdk,节点为:SCSI1:8;
注：SCSI1:7已经被占用;
------------------------------------------------
2.6RMAN和DUMP备份数据存于/u02下EXT3文件系统下。
---------------------------------------------------------------------
2.7磁盘组介绍
CRSGROUP （集群软件）
VOTE1 1G scsi1:0
VOTE2 1G scsi1:1
VOTE3 1G scsi1:2
----------------------------
DATAGROUP （数据文件）
DATA1 10G scsi1:3
DATA2 10G scsi1:4
DATA3 10G scsi1:5
----------------------------
ARCHGROUP （归档日志）
ARCH1 5G scsi1:6
ARCH2 5G scsi1:8 //1:7被占用
------------------------------------------
三、修改虚拟机操作系统基本参数
3.1在节点1安装linux 操作系统，安装完成后复制成节点2即可。
注：3.1.1操作系统怎么装，不在介绍。
3.1.2iptables和selinux 设置为关闭状态。
3.2在节点1刷掉不用的服务和无效的账号，从而提高系统的性能和安全。
以root用户执行以下命令：
chkconfig anacron off
chkconfig atd off
chkconfig autofs off
chkconfig cups off
chkconfig cups-config-daemon off
chkconfig firstboot off
chkconfig haldaemon off
chkconfig isdn off
chkconfig kudzu off
chkconfig netfs off
chkconfig nfslock off
chkconfig pcmcia off
chkconfig portmap off
chkconfig rpcgssd off
chkconfig rpcidmapd off
chkconfig sendmail off
chkconfig smartd off
chkconfig snmpd off
chkconfig avahi-daemon off
#--------------------------------------
userdel adm
userdel lp
userdel sync
userdel shutdown
userdel halt
userdel news
userdel uucp
userdel operator
userdel gopher
#--------------------------------------
3.3关闭NTP服务，使用oracle的ctssd服务器进行时间同步。
service ntpd stop
chkconfig ntpd off
mv /etc/ntp.conf /etc/ntp.conf.org
rm /var/run/ntpd.pid
--------------------------------------------
3.4开始在ctpdb1节点配置环境,以root用户
3.4.1修改hosts文件
192.168.1.11ctpdb1
192.168.1.111 ctpdb1-vip
172.17.80.11 ctpdb1-priv
192.168.1.12ctpdb2
192.168.1.112 ctpdb2-vip
172.17.80.12 ctpdb2-priv
192.168.1.10 ctpdb-cluster
--------------------------------------------
3.4.2 vi /etc/security/limits.conf
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
------------------------------------------------------------
3.4.5 vi /etc/sysctl.conf
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
sysctl -p 使其立即生效
------------------------------------------------------------
3.4.6 vi/etc/pam.d/login
session required /lib/security/pam_limits.so
5.5 vi /etc/profile
if [ $USER = "oracle" ]||[ $USER = "grid" ];then
if [ $SHELL = "/bin/ksh" ];then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
###################
---------------------------------------------------------------------四、创建用户和用户环境变量
4.1创建用户
groupadd oinstall
groupadd dba
groupadd oper
groupadd asmadmin
groupadd asmdba
groupadd asmoper
useradd -g oinstall -G dba,asmdba,oper -d/home/oracle oracle
useradd -g oinstall -Gdba,asmadmin,asmdba,asmoper -d /home/grid grid
------------------------------------------------------------
4.2创建oracle使用到的目录结构,并赋予相应的属性。
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/grid
mkdir -p/u01/app/oracle/product/11.2.0/db_1
mkdir -p /u02/rman
chown -R grid:oinstall /u01
chown -R oracle:oinstall /u01/app/oracle
chown -R oracle:oinstall /u02
chmod -R 775 /u01
chmod -R 775 /u02
------------------------------------------------------------
4.3创建oracle用户和grid用的环境变量,等节点2复制完后，需要做相应更改。
注：SID环境
oracle RAC SID：
ctpdb1
ctpdb2
集群实例名：
ctpdb
-------------------------------------------------------------
4.4 root用户执行
su - oracle
vi .bash_profile
export ORACLE_HOSTNAME= ctpdb1
ORACLE_SID= ctpdb1;export ORACLE_SID
ORACLE_BASE=/u01/app/oracle;exportORACLE_BASE
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1;exportORACLE_HOME
ORACLE_TERM=xterm;export ORACLE_TERM
NLS_DATA_FORMAT="DD-MON-YYYYHH24:MI:SS";export NLS_DATA_FORMAT
TNS_ADMIN=$ORACLE_HOME/network/admin;exportTNS_ADMIN
ORA_NLS11=$ORACLE_HOME/nls/data;exportORA_NLS11
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH
THREADS_FLAG=native;export THREADS_FLAG
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
--------------------------------------------------------------------
4.5 root用户登录
su - grid
vi .bash_profile
export ORACLE_HOSTNAME= ctpdb1
ORACLE_SID=+ASM1;export ORACLE_SID
ORACLE_BASE=/u01/app/grid;exportORACLE_BASE
ORACLE_HOME=/u01/app/11.2.0/grid;exportORACLE_HOME
ORACLE_TERM=xterm;export ORACLE_TERM
NLS_DATA_FORMAT="DD-MON-YYYYHH24:MI:SS";export NLS_DATA_FORMAT
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH
export CLASSPATH
THREADS_FLAG=native;export THREADS_FLAG
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
---------------------------------------------------------------------
4.6root用户检查rpm包是否符合要求（可通过get_rpm.sh脚本扫描,见附件）。
注：若不符合，则由系统安装iso文件，找到该rpm包，
执行：rpm -ivh 安装即可。如：
mkdir /cdrom
mount /dev/cdrom /cdrom
cd /cdrom/Server
rpm -ivh libaio-devel-0.3.106-5.*
rpm -ivh libstdc++-devel-4.1.2-52.el5.*
rpm -ivh sysstat-7.0.2-11.el5.x86_64.rpm
---------------------------------------------------------------------
五、复制虚拟机，通过修改虚拟机文件是共享磁盘生效
注：【重点】如果此项修改失败或有问题，将导致整个环境无法完成。
5.1复制节点1到新的节点2。
注：复制后，将节点1临时重命名，避免复制的节点2会读取节点1数据。
5.2文本编辑节点1下的ctpdb1.vmx文件改共享磁盘组
shutdown -h now 关掉节点1
新增：
注：修改前备份ctpdb1.vmx文件。
disk.locking = "false"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize ="0"
diskLib.DataCacheMinReadAheadSize ="0"
diskLib.dataCachePageSize ="4096"
diskLib.maxUnsyncedWrites = "0"
scsi1.sharedBus = "virtual"
scsi2.sharedBus = "virtual"
scsi0:0.mode ="independent-persistent"
----------------------------------------------------------------
5.3在复制后的 ctpdb2目录内，将所有ctpdb1的文件名改为 ctpdb2,并将目录改为vm_ctpdb2,
注：否则ctpdb 2会找ctpdb 1的文件运行虚拟机。
5.4修改节点2的ctpdb2.vmx虚拟机配置文件。
5.4.1将ctpdb2.vmx内，所有ctpdb1字符串改为ctpdb2。
5.5编辑网卡参数。（具体参考附件）
5.5.1将:ethernet0.generatedAddress= "00:0c:29:32:7b:c9"
改为:ethernet0.generatedAddress= "00:0c:29:32:b7:c9"
5.5.2将:ethernet1.generatedAddress= "00:0C:29:32:7B:D3"
改为:ethernet1.generatedAddress= "00:0c:29:32:b7:d3"
5.5.3将:uuid.location= "56 4d f0 6c 1a 13 d8 90-9c dd ca 11 59 32 7b c9"
改为:uuid.location ="56 4d 81 1b 7f cb f2 b1-0c 00 23 1f f2 bd b7 ff"
5.5.4将:uuid.bios= "56 4d f0 6c 1a 13 d8 90-9c dd ca 11 59 32 7b c9"
改为:uuid.bios = "56 4df0 6c 1a 13 d8 90-9c dd ca 11 59 32 b7 c9"
---------------------------------------------------------------------
5.6启动节点2，并修改虚拟机网卡配置文件的MAC地址
以root用户执行
vi/etc/sysconfig/network-scripts/ifcfg-eth0/eth1
5.6.1将:HWADDR=00:0C:29:32:7B:C9
改为:HWADDR=00:0C:29:32:B7:C9
5.6.2将:HWADDR=00:0C:29:32:7B:D3
改为:HWADDR=00:0C:29:32:B7:D3
5.7在节点2修改虚拟机网卡的IP地址
---------------------------------------------------------------------
5.7.1以root用户执行
vi /etc/sysconfig/network-scripts/ifcfg-eth0/eth1
将:172.17.80.11
改为:172.17.80.12
将：192.168.1.11
改为:192.168.1.12
--------------------------------------------------------------------
5.8在节点2修改主机名
以root用户执行
5.8.1vi /etc/sysconfig/network
HOSTNAME=ctpdb2
---------------------------------------------------------------------
六、设置ssh信任关系
6.1分别给节点1、2的oracle和grid创建密码。
passwd oracle
passwd grid
--------------------------------------------------------------------
6.2调通节点1、节点2的通信关系。
[root@ctpdb1 ~]# ping ctpdb1
64 bytes from ctpdb1 (192.168.1.11):icmp_seq=1 ttl=64 time=0.027 ms
[root@ctpdb1 ~]# ping ctpdb2
64 bytes from ctpdb2 (192.168.1.12):icmp_seq=1 ttl=64 time=1.73 ms
[root@ctpdb2 ~]# ping ctpdb1
64 bytes from ctpdb1 (192.168.1.11):icmp_seq=1 ttl=64 time=0.305 ms
[root@ctpdb2 ~]# ping ctpdb2
64 bytes from ctpdb2 (192.168.1.12):icmp_seq=1 ttl=64 time=0.026 ms
---------------------------------------------------------------------
6.3建SSH信任关系，oracle和grid用户
6.3.1节点1、2的oracle和grid用户都要执行
ssh-keygen -t rsa
---------------------------------------------------------------------
6.3.2节点1
cp id_rsa.pub authorized_keys
scp authorized_keys oracle@ctpdb2:~/.ssh
rm authorized_keys
6.3.3节点2
cat id_rsa.pub >> authorized_keys
scp authorized_keys oracle@ctpdb1:~/.ssh
---------------------------------------------------------------------
6.3.4在节点1、2分别在oracle和grid用户都要执行
ssh ctpdb1 date
ssh ctpdb1-priv date
ssh ctpdb2 date
ssh ctpdb2-priv date
---------------------------------------------------------------------
七、建立分区
注：对物理磁盘进行分区，但不做格式化
7.1以root用户，节点1
[root@ctpdb1 ~]# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 535 4192965 82 Linux swap / Solaris
/dev/sda3 536 5221 37640295 83 Linux
Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sdd: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sde: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sdf: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sdg: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sdh: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Disk /dev/sdi: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
---------------------------------------------------------------------
7.2root用户执行，只需要在节点1上执行。
fdisk -l|grep dev
Disk /dev/sdb: 1073 MB, 1073741824 bytes
Disk /dev/sdc: 1073 MB, 1073741824 bytes
Disk /dev/sdd: 1073 MB, 1073741824 bytes
Disk /dev/sde: 10.7 GB, 10737418240 bytes
Disk /dev/sdf: 10.7 GB, 10737418240 bytes
Disk /dev/sdg: 10.7 GB, 10737418240 bytes
Disk /dev/sdh: 5368 MB, 5368709120 bytes
Disk /dev/sdi: 5368 MB, 5368709120 bytes
---------------------------------------------------------------------
7.3root用户分别对以上磁盘执行：
[root@ctpdb1 ~]# fdisk /dev/sdb
Command (m for help): p
Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK(1-130, default 130):
Using default value 130
Command (m for help): p
Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 130 1044193+ 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition tablefailed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the nextreboot.
Syncing disks.
---------------------------------------------------------------------
7.4root用户分别在节点1、节点2执行分区检查。
[root@ctpdb1 ~]# fdisk -l >fdisk1.log
[root@ctpdb2 ~]# fdisk -l >fdisk2.log
[root@ctpdb1 ~]#scp fdisk1.log root@ctpdb2:~/
---------------------------------------------------------------------
7.5root用户在节点2上对比，确保无差异，
[root@ctpdb2 ~]# diff fdisk1.log fdisk2.log
即：虚拟机环境下的RAC环境基础已经完成大半。
---------------------------------------------------------------------
八、配置ASMLib和创建ASM磁盘
8.1 root用户分别在节点1、节点2执行安装oracleasm
[root@ctpdb1 oracleasm]# uname -rm
2.6.18-308.el5 x86_64
[root@ ctpdb1 oracleasm]# ls
oracleasm-2.6.18-308.el5-2.0.5-1.el5.x86_64.rpm
注：版本务必与kernel版本一致。
oracleasm-support-2.1.7-1.el5.x86_64.rpm
oracleasmlib-2.0.4-1.el5.x86_64.rpm
[root@ ctpdb1 oracleasm]# rpm -ivh *.rpm
---------------------------------------------------------------------
8.2 root用户分别在节点1、节点2执行
/etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot propertiesof the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it willhave. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface[grid]: grid
Default group to own the driver interface[asmadmin]:asmadmin
Start Oracle ASM library driver on boot(y/n) [y]: y
Scan for Oracle ASM disks on boot (y/n)[y]:y
Writing Oracle ASM library driverconfiguration: done
Initializing the Oracle ASMLib driver:[ OK ]
Scanning the system for Oracle ASMLibdisks: [ OK ]
---------------------------------------------------------------------
8.3建立ASM磁盘
8.3.1检查系统磁盘
[root@ctpdb1 ~]# fdisk -l |grep dev |grepLinux |grep -v sda
/dev/sdb1 1 130 1044193+ 83 Linux
/dev/sdc1 1 130 1044193+ 83 Linux
/dev/sdd1 1 130 1044193+ 83 Linux
/dev/sde1 1 1305 10482381 83 Linux
/dev/sdf1 1 1305 10482381 83 Linux
/dev/sdg1 1 1305 10482381 83 Linux
/dev/sdh1 1 652 5237158+ 83 Linux
/dev/sdi1 1 652 5237158+ 83 Linux
---------------------------------------------------------------------
8.3.2只需在节点1执行
[root@ctpdb1 ~]# oracleasm createdisk VOTE1/dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk VOTE2/dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk VOTE3/dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk DATA1/dev/sde1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk DATA2/dev/sdf1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk DATA3/dev/sdg1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk ARCH1/dev/sdh1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]# oracleasm createdisk ARCH2/dev/sdi1
Writing disk header: done
Instantiating disk: done
[root@ctpdb1 ~]#
---------------------------------------------------------------------
8.3.3root用户分别在节点1、节点2查看和扫描磁盘
oracleasm listdisks
oracleasm scandisks
注：节点2会在磁盘扫描后显示和节点1相同的磁盘信息，务必确保正确、一致。
[root@ctpdb2 oracleasm]# oracleasmlistdisks
[root@ctpdb2 oracleasm]# oracleasmscandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "VOTE1"
Instantiating disk "VOTE2"
Instantiating disk "VOTE3"
Instantiating disk "DATA1"
Instantiating disk "DATA2"
Instantiating disk "DATA3"
Instantiating disk "ARCH1"
Instantiating disk "ARCH2"
[root@ctpdb2 oracleasm]# oracleasmlistdisks
ARCH1
ARCH2
DATA1
DATA2
DATA3
VOTE1
VOTE2
VOTE3
[root@ctpdb2 oracleasm]#
--------------------------------------------------------
九、在节点1上执行安装grid
注、只需要节点1，节点2有oracle通过ssh自动同步。
9.1使用oracle用户上传以下介质到/u02下。
p10404530_112030_Linux-x86-64_1of7.zip
p10404530_112030_Linux-x86-64_2of7.zip
p10404530_112030_Linux-x86-64_3of7.zip
9.2配置Xmanager 图形界面连接ctpdb1的oracle用户和grid用户
关于：配置Xmanager 图形界面参考：
http://linuxxx.blog.51cto.com/1824808/1040165
9.3以grid用户登录节点1，cd /u02,执行
unzip p10404530_112030_Linux-x86-64_3of7.zip
cd grid
执行：./runInstaller启动图形化安装向导界面
9.4选择Skipsoftware updates
9.5选择for aCluster
9.6选择TypicalInstallation 典型安装
9.7点击add添加节点2的配置信息（图片有个cptdb2-vip错误,正确为ctpdb2-vip）
9.8击Identifynetwork interfaces 指定网络接口
注：配置Private 和Public地址，其他地址可设置为Do Not Use.
9.9点击SSHConnectivity 测试节点1到节点2的SSH功能。
输入grid的密码，点击test即可。
9.10选择ClusterRegistry Storage Type 类型为:Oracle Automatic StorageManagement,并设置sysasm password为：grid
注：无视此问题，点击yes（因为密码太简单的原因）
9.11设置Disk groupname为：CRSGROUP，Redundancy 为：Extrnal模式。
9.12选择InventoryDirectory为：/u01/app/oraInventory（一般默认即可）
9.13节点1安装环境信息检查。
9.13.1cvuqdisk-1.0.9-1,此rpm包在/u02/grid/rpm,使用root 用户
rpm -ivh cvuqdisk-1.0.9-1.rpm
9.13.2scp rpm -ivh cvuqdisk-1.0.9-1.rpm root@ctpdb2:~/
9.13.3root用户在节点2上，执行cvuqdisk-1.0.9-1.rpm即可。
注：忽略resolv.conf文件问题。选择Ignore all,点击Next, 点击yes跳过，点击install安装。
9.14以root用户在节点1、2分别执行
root@ctpdb1~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of/u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions forworld.
Changing groupname of /u01/app/oraInventoryto oinstall.
The execution of the script is complete.
[root@ctpdb1 ~]#/u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle11g
The following environment variables are setas:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bindirectory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratabfile as needed by
Database Configuration Assistant when adatabase is created
Finished running generic part of rootscript.
Now product-specific root actions will beperformed.
Using configuration parameter file:/u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites duringinstallation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pawallet
peer wallet keys
pawallet keys
peer cert request
pacert request
peer cert
pacert
peer root cert TP
profile reader root cert TP
paroot cert TP
peer pa cert TP
pa peercert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pauser cert
Adding Clusterware entries to inittab
CRS-2672: Attempting to start 'ora.mdnsd'on 'ctpdb1'
CRS-2676: Start of 'ora.mdnsd' on 'ctpdb1'succeeded
CRS-2672: Attempting to start 'ora.gpnpd'on 'ctpdb1'
CRS-2676: Start of 'ora.gpnpd' on 'ctpdb1'succeeded
CRS-2672: Attempting to start'ora.cssdmonitor' on 'ctpdb1'
CRS-2672: Attempting to start 'ora.gipcd'on 'ctpdb1'
CRS-2676: Start of 'ora.gipcd' on 'ctpdb1' succeeded
CRS-2676: Start of 'ora.cssdmonitor' on'ctpdb1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on'ctpdb1'
CRS-2672: Attempting to start 'ora.diskmon'on 'ctpdb1'
CRS-2676: Start of 'ora.diskmon' on'ctpdb1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ctpdb1'succeeded
ASM created and started successfully.
Disk Group CRSGROUP created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCRkeys.
Creating OCR keys for user 'root', privgrp'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk0b0cea08e4f14fc7bf5c53d50eba4303.
Successfully replaced voting disk groupwith +CRSGROUP.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfullyreplaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1.ONLINE 0b0cea08e4f14fc7bf5c53d50eba4303(ORCL:VOTE1) [CRSGROUP]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on'ctpdb1'
CRS-2676: Start of 'ora.asm' on 'ctpdb1'succeeded
CRS-2672: Attempting to start'ora.CRSGROUP.dg' on 'ctpdb1'
CRS-2676: Start of 'ora.CRSGROUP.dg' on'ctpdb1' succeeded
CRS-2672: Attempting to start'ora.registry.acfs' on 'ctpdb1'
CRS-2676: Start of 'ora.registry.acfs' on'ctpdb1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@ctpdb1 ~]#
务必确保节点1、2都能出现
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
9.15在节点1和节点2分别进行ping通ctpdb-cluster，如果可以ping通，则此错误可以忽略。点击ok,再点击close，至此grid安装完成。
[root@ctpdb1 ~]# ping ctpdb-cluster
PING ctpdb-cluster (192.168.1.10) 56(84)bytes of data.
64 bytes from ctpdb-cluster (192.168.1.10):icmp_seq=1 ttl=64 time=0.027 ms
[root@ctpdb2 ~]# ping ctpdb-cluster
PING ctpdb-cluster (192.168.1.10) 56(84)bytes of data.
64 bytes from ctpdb-cluster (192.168.1.10):icmp_seq=1 ttl=64 time=1.16 ms
---------------------------------------------------------------------
9.16图形界面
使用grid用户在节点1，执行asmca，创建DATAGROUP和ARCHGROUP磁盘组
9.17.1在Disk Groups下点击 Create，
Disk Groups Name 为：DATAGROUP，External(none)模式，并勾选DATA1/DATA2/DATA3，点击ok继续，再点击ok确定完成。
9.17.2在Disk Groups下点击 Create，
Disk Groups Name 为：ARCHGROUP，External(none)模式，并勾选ARCH1/ARCH2，点击ok继续,再点击ok确定完成。
最后点击Exit退出即可。
---------------------------------------------------------------------
十、图形界面以oracle用户登录,进入/u02/ 目录
10.1 unzip p10404530_112030_Linux-x86-64_1of7.zip
unzip p10404530_112030_Linux-x86-64_2of7.zip
10.2 cd database
执行./ ./runInstaller 启动图形化安装向导界面，取消选择
I wish to receive security updates via MyOracle Support.
next 继续
10.3 选择Skipsoftware updates ,next继续
10.4 选择Installdatabase software only,next 继续
10.5 选择OracleReal Application Cluster database installation,一定勾选所有nodename,为了顺利期间，点击SSH Connectivity ,输入oracle密码，点击test,做SSH信任连接测试。next继续
10.6选择oracle运行语言为English,next 继续。
10.7选择EnterpriceEdition (4.5G)企业版，next继续
10.8查看oracle安装路径,Oracle Baes为 /u01/app/oracle；
Software Location为/u01/app/oracle/product/11.2.0/db_1；next继续
10.9OSDBA为dba,OSOPER为oper；next继续
10.10各项检查通过后，next继续
此时为报：resolv.conf失败和SCAN警告，忽略即可。（可点击如下：more details查看更多信息），选择Ignore All,next继续，点击yes。
10.11点击install
10.12开始安装oracle软件，此过程时间较长，因为oracle本身需要将节点1数据信息通过ssh同步至节点2。
使用root用户，在连个节点1、节点2分别执行:
/u01/app/oracle/product/11.2.0/db_1/root.sh
点击ok, Close 关闭即可。
十一、DBCA 建库
11.1在节点1上，使用oracle用户，执行dbca
选择Oracle Real ApplicationCluster(RAC)database,next继续；
11.2选择Create aDatabase 创建一个数据库, next继续。
11.3选择CustomDatabase 自定义安装, next继续。
11.4Global Database Name ctpdb,SID Prefixctpdb,Select All 所有节点。next继续。
11.5在EnterpriseManager取消Configure Enterprise Manger 和AutomaticMaintenance Tasks 取消Enable sutomatic maintenance tasks。next继续。
11.6设置统一密码，点击yes,next继续。
11.7选择StorageType 为：ASM方式，Database Area为：+DATAGROUP，next继续。
11.8 取消flashrecovery area和Enable Archiving，next继续。
11.9全部取消，next继续。
11.10Memory使用默认大小，40%。Sizing BlockSize为8192，Process为300。
Character Sets为：如图所示。Connection Mode为Dedicated Server Mode独享模式。next继续，再次 next继续。
11.11点击Flinish,点击ok。开始创建数据库。
十一、数据库测试
11.1使用grid 用户检查数据库运行状态。
[grid@ctpdb1 ~]$ crs_stat -t
11.2使用grid 用户停止oracle，启动oracle。
[grid@ctpdb1 ~]$ srvctl status database -dctpdb
Instance ctpdb1 is running on node ctpdb1
Instance ctpdb2 is running on node ctpdb2
[grid@ctpdb1 ~]$
[grid@ctpdb1 ~]$ srvctl stop database -dctpdb
[grid@ctpdb1 ~]$
[grid@ctpdb1 ~]$ srvctl status database -dctpdb
Instance ctpdb1 is not running on nodectpdb1
Instance ctpdb2 is not running on nodectpdb2
[grid@ctpdb1 ~]$
[grid@ctpdb1 ~]$ srvctl start database -dctpdb
[grid@ctpdb1 ~]$ srvctl status database -dctpdb
Instance ctpdb1 is running on node ctpdb1
Instance ctpdb2 is running on node ctpdb2
[grid@ctpdb1 ~]$
srvctl命令：
[grid@ctpdb1 ~]$ srvctl
Usage: srvctl <command> <object>[<options>]
commands:enable|disable|start|stop|relocate|status|add|remove|modify|getenv|setenv|unsetenv|config|convert|upgrade
objects:database|instance|service|nodeapps|vip|network|asm|diskgroup|listener|srvpool|server|scan|scan_listener|oc4j|home|filesystem|gns|cvu
11.3使用grid 用户启动监听，做客户端连接测试。
[grid@ctpdb1 ~]$ lsnrctl status 检查监听状态
[grid@ctpdb1 ~]$ lsnrctl stop 停掉监听，
[grid@ctpdb1 ~]$ lsnrctl start 启动监听，OK 监听正常启动。
11.4
RAC版tnsnames.ora配置,含
# tnsnames.ora Network Configuration File: D:\oracle\product\10.2.0\db_1\network\admin\tnsnames.ora
# Generated by Oracle configuration tools.
CTPDB1=
(DESCRIPTION=
(ADDRESS=
(PROTOCOL=TCP)
(HOST=192.168.1.111)
(PORT=1521)
)
(CONNECT_DATA=
(FAILOVER_MODE=
(BACKUP=ctpdb2)
(TYPE=select)
(METHOD=preconnect)
)
(INSTANCE_NAME=ctpdb1)
(SERVICE_NAME=ctpdb)
)
)
CTPDB2=
(DESCRIPTION=
(ADDRESS=
(PROTOCOL=TCP)
(HOST=192.168.1.112)
(PORT=1521)
)
(CONNECT_DATA=
(FAILOVER_MODE=
(BACKUP=ctpdb1)
(TYPE=select)
(METHOD=preconnect)
)
(INSTANCE_NAME=ctpdb2)
(SERVICE_NAME=ctpdb)
)
)
11.5停止数据库节点1，查看数据库运行状态，做节点冗余切换测试。
[oracle@ctpdb1 ~]$ sqlplus sys/oracle assysdba
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>
根据图片能看到oracle已经无缝的从节点1切换到节点2。
SQL> startup; 启动节点1，客户端仍然正常的接到节点2上。
11.6停止数据库节点1/2，重启数据库服务器。//以下不再做演练操作
11.7启动数据库节点1/2做正常操作。
-----------------------------------------------------------------------------------------
最近寒枫不是一般的忙，随着系统逐渐扩大，公司安排核心系统最近上线，不过切换前后的平稳的过度，能在闲时也整理整理文档，也是一种惬意的享受。毕竟要上夜班了~
只是无论学习Oracle还是Linux等其他技术也好，均要徐徐渐进；十年磨一剑，此剑必须要能拿的出，用得上，打的赢 ；如果半途而非，最终将竹篮打水一场空，除非你的竹篮密密麻麻。
接下来的时间，等系统也稳定了，没有这么多琐事了，还需定心学习Oracle、Linux，同时也喜欢在业余时间和大家讨论技术，包括Oracle、Linux以及其他关于系统交易、程序化交易、金融市场，寒枫均有很大兴趣。
本文也没有统一的编写格式还需文学者指导。
本文主题为：【让学习oracle前进一步】
如果你对上文不感兴趣，那么请点击http://umiwi.diandian.com看看这个也到消遣以下。
-----------------------------------------------------------------------------------------
转载请注明出处:http://linuxxx.blog.51cto.com/1824808/1274890#576555
-----------------------------------------------------------------------------------------
参考文档：
1、CTP交易系统官方安装文档oracle 11.2.0.0 +rhel 5.5版
2、http://wangchunhai.blog.51cto.com/225186/42184/
3、http://www.itpub.net/thread-1009235-1-1.html
4、http://linuxxx.blog.51cto.com/1824808/1040165
附件：
1、get_rpm.sh 和get_rpm 用来获取rpm是否安装。
2、RAC版的tnsnames.ora
3、虚拟机修改的文件
-----------------------------------------------------------------------------------------