oracle 11G 11.2.0.4 RAC環境打補丁

一、準備工作

1，數據庫環境

操作系統版本 : RedHat 7.2 x64
數據庫版本 : Oracle 11.2.0.4 x64 RAC
Grid : 11.2.0.4
Oracle database: 11.2.0.4

2，準備內容

GI PSU : p215****75_112040_Linux-x86-64.zip
OPatch : p688****80_112000_Linux-x86-64.zip
DB PSU : p213****35_112040_Linux-x86-64.zip 僅單實例使用。

OPatch下載地址：
https://updates.oracle.com/download/688****80.html

注意：GI PSU包括DB PSU,CPU。

二、更新OPatch版本，必須要更新

說明：分別在兩個節點更新OPatch版本。

1. 更新OPatch版本，需要更新到11.2.0.3.11，OPatch直接壓縮替換就可以了。

(1) 上傳p688****80_112000_Linux-x86-64.zip到/u01/software目錄下, /u01/software給777權限，也可以採用/tmp目錄。
(2) root用戶下將原OPatch目錄改名。

mv /u01/app/11.2.0/grid/OPatch /u01/app/11.2.0/grid/OPatchbk
mv /u01/app/oracle/product/11.2.0/db_1/OPatch /u01/app/oracle/product/11.2.0/db_1/OPatchBK

2. grid目錄OPatch替換

(1) root用戶
cd /u01/software/;chown grid:oinstall p688****80_112000_Linux-x86-64.zip

(2) grid用戶

su - grid
cd /u01/software/
unzip p688****80_112000_Linux-x86-64.zip

(3) root用戶

cd /u01/software/
mv OPatch /u01/app/11.2.0/grid/

(4) grid用戶測試

su - grid
/u01/app/11.2.0/grid/OPatch/opatch version

OPatch Version: 11.2.0.3.11
OPatch succeeded.

3. oracle目錄OPatch替換

(1) root用戶

cd /u01/software/;chown oracle:oinstall p688****80_112000_Linux-x86-64.zip

(2) oracle用戶

su - oracle
cd /u01/software/
unzip p688****80_112000_Linux-x86-64.zip

(3) root用戶

cd /u01/software/
mv OPatch /u01/app/oracle/product/11.2.0/db_1/

(4) oracle用戶測試

su - oracle

/u01/app/oracle/product/11.2.0/db_1/OPatch/opatch version

OPatch Version: 11.2.0.3.11
OPatch succeeded.

三、開始升級GI PSU軟件包

1. 前提

(1) root用戶下打grid需要關閉grid(兩節點都執行)

cd /u01/app/11.2.0/grid/bin
./crsctl stop has

2. grid用戶下解決GI PSU補丁包

(1) root用戶操作

cd /u01/software/;chown grid:oinstall p215****75_112040_Linux-x86-64.zip

(2) grid用戶操作

su - grid
cd /u01/software/;unzip p215****75_112040_Linux-x86-64.zip

3. grid用戶下測試兼容性，針對多次打CPU的環境中。

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/215****75/213****49 -oh $ORACLE_HOME
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/215****75/213****42 -oh $ORACLE_HOME
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/215****75/213****35 -oh $ORACLE_HOME

----------------------------------------------------
執行示例如下：
Oracle Interim Patch Installer version 11.2.0.3.11
Copyright (c) 2015, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.0/grid/oraInst.loc
OPatch version : 11.2.0.3.11
OUI version : 11.2.0.4.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2015-12-18_16-21-46PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.

四、開始升級GRID

1. 依次節點1，節點2上執行。

第1步: 執行腳本, 一定要執行，否則無法成功，確保CRS是關閉狀態。

也可以先查看補丁包中的README文件。

[root@db1 bin]# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -unlock

Using configuration parameter file: /grid/grid_home/crs/install/crsconfig_params
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.
Successfully unlock /grid/grid_home

[root@db1 bin]#

第2步：下載補丁包（目前11.2.0.4.8 PSU），準備升級

su - grid

/u01/app/11.2.0/grid/OPatch/opatch napply -oh $ORACLE_HOME -local /u01/software/215****75/213****49
/u01/app/11.2.0/grid/OPatch/opatch napply -oh $ORACLE_HOME -local /u01/software/215****75/213****42
/u01/app/11.2.0/grid/OPatch/opatch napply -oh $ORACLE_HOME -local /u01/software/215****75/213****35

過程中按三個Y

2. 檢查升級情況

$ /u01/app/11.2.0/grid/OPatch/opatch lsinv

Patch 213****35 : applied on Fri Dec 18 17:38:24 CST 2015
Unique Patch ID: 192****62
Patch description: "Database Patch Set Update : 11.2.0.4.8 (213****35)"
Created on 1 Sep 2015, 07:49:44 hrs
Sub-patch 20760982; "Database Patch Set Update : 11.2.0.4.7 (20760982)"
Sub-patch 20299013; "Database Patch Set Update : 11.2.0.4.6 (20299013)"
Sub-patch 197****89; "Database Patch Set Update : 11.2.0.4.5 (197****89)"
Sub-patch 191****51; "Database Patch Set Update : 11.2.0.4.4 (191****51)"
Sub-patch 185****09; "Database Patch Set Update : 11.2.0.4.3 (185****09)"
Sub-patch 18031668; "Database Patch Set Update : 11.2.0.4.2 (18031668)"
Sub-patch 174****14; "Database Patch Set Update : 11.2.0.4.1 (174****14)"
Bugs fixed:
...

$ /u01/app/11.2.0/grid/OPatch/opatch lspatches

213****35;Database Patch Set Update : 11.2.0.4.8 (213****35)
213****42;ACFS Patch Set Update : 11.2.0.4.8 (213****42)
213****49;OCW Patch Set Update : 11.2.0.4.8 (213****49)

OPatch succeeded.

3. 兩節點啓動crs

說明：因爲是手動打PSU,需要手動進行關閉與啓動操作。
# cd /u01/app/11.2.0/grid/bin
# ./crsctl start has
CRS-4123: Oracle High Availability Services has been started.

五、打database PSU補丁

1. 準備工作

(1) Root用戶下，重新給PSU補丁包賦予權限

cd /u01/software/
rm -rf 215****75
chown oracle:oinstall p215****75_112040_Linux-x86-64.zip

(2) oracle用戶

su - oracle
cd /u01/software/;unzip p215****75_112040_Linux-x86-64.zip

(3) Oracle用戶下檢查兼容性,針對多次打CPU的環境中。

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/software/215****75/213****35 -oh $ORACLE_HOME
----------------------------------------------------
Oracle Interim Patch Installer version 11.2.0.3.11
Copyright (c) 2015, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version : 11.2.0.3.11
OUI version : 11.2.0.4.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch2015-12-21_09-25-57AM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
----------------------------------------------------

2. oracle database打補丁，還是用同一個補丁包

su - oracle

opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/app/25476126 -oh $ORACLE_HOME
./opatch napply -oh $ORACLE_HOME -local /home/app/25476126/
/u01/app/oracle/product/11.2.0/db_1/OPatch/opatch napply -oh $ORACLE_HOME -local /u01/software/215****75/213****35

查看打補丁情況：

/u01/app/oracle/product/11.2.0/db_1/OPatch/opatch lsinv
/u01/app/oracle/product/11.2.0/db_1/OPatch/opatch lspatches

六、升級數據庫數據字典（此步驟可選，如果還未創建數據庫可以不用做此步驟）

1. 數據庫啓動

(1) root用戶下打grid需要關閉grid(兩節點都執行)

cd /u01/app/11.2.0/grid/bin
./crsctl start has

或直接執行如下命令，同時關閉兩個節點。
./crs_start -all

ORA-01565: error in identifying file '+DATA/orcl/spfileorcl.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/orcl/spfileorcl.ora
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15040: diskgroup is incomplete
ORA-15040: diskgroup is incomplete
ORA-15040: diskgroup is incomplete

說明：無法啓動數據庫，經過查詢是因爲oracle權限被更改爲oracle:oinstall。通過如下方式解決。
在集羣的所有節點上執行如下命令修改：
# chown oracle:asmadmin /u01/app/oracle/product/11.2.0/db_1/bin/oracle
# chmod 6751 /u01/app/oracle/product/11.2.0/db_1/bin/oracle

SQL> startup
ORACLE instance started.

Total System Global Area 250533****80 bytes
Fixed Size 225****32 bytes
Variable Size 687****20 bytes
Database Buffers 179****12 bytes
Redo Buffers 20054016 bytes
Database mounted.
Database opened.

SQL> @?/rdbms/admin/catbundle.sql psu apply
中間等待過程。

執行完後我們可以查到PSU更新信息：
SQL>

set line 150
col ACTION_TIME for a30
col ACTION for a8
col NAMESPACE for a8
col VERSION for a10
col BUNDLE_SERIES for a5
col COMMENTS for a20
select * from dba_registry_history;

ACTION_TIME ACTION NAMESPAC VERSION ID BUNDL COMMENTS
------------------------------ -------- -------- ---------- ---------- ----- --------------------
24-AUG-13 12.03.45.119862 PM APPLY SERVER 11.2.0.4 0 PSU Patchset 11.2.0.2.0
22-DEC-15 03.53.27.330912 PM APPLY SERVER 11.2.0.4 0 PSU Patchset 11.2.0.2.0
24-DEC-15 04.58.29.381100 PM APPLY SERVER 11.2.0.4 8 PSU PSU 11.2.0.4.8

3 rows selected.

六、升級中碰到問題

在升級過程遇到一個問題

[grid@ygc26 OPatch]$ ./opatch napply -oh $ORACLE_HOME -local /home/app/25476126/
Oracle Interim Patch Installer version 11.2.0.3.15
Copyright (c) 2017, Oracle Corporation. All rights reserved.

Oracle Home : /home/app/11.2.0.4/grid
Central Inventory : /home/app/oraInventory
from : /home/app/11.2.0.4/grid/oraInst.loc
OPatch version : 11.2.0.3.15
OUI version : 11.2.0.4.0
Log file location : /home/app/11.2.0.4/grid/cfgtoollogs/opatch/opatch2017-04-27_21-04-32PM_1.log
Verifying environment and performing prerequisite checks...
Prerequisite check "CheckActiveFilesAndExecutables" failed.
The details are:

Following executables are active :
/home/app/11.2.0.4/grid/lib/libclntsh.so.11.1
Prerequisite check "CheckActiveFilesAndExecutables" failed.
The details are:

Following executables are active :
/home/app/11.2.0.4/grid/lib/libsqlplus.so
UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.Prerequisite check "CheckActiveFilesAndExecutables" failed.
Log file location: /home/app/11.2.0.4/grid/cfgtoollogs/opatch/opatch2017-04-27_21-04-32PM_1.log

OPatch failed with error code 73

檢查下那些文件又問題

[grid@ygc26 OPatch]$ ./opatch prereq CheckActiveFilesAndExecutables -phBaseDir /home/app/25476126/ -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.15
Copyright (c) 2017, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /home/app/11.2.0.4/grid
Central Inventory : /home/app/oraInventory
from : /home/app/11.2.0.4/grid/oraInst.loc
OPatch version : 11.2.0.3.15
OUI version : 11.2.0.4.0
Log file location : /home/app/11.2.0.4/grid/cfgtoollogs/opatch/opatch2017-04-27_21-05-50PM_1.log

Invoking prereq "checkactivefilesandexecutables"

Prereq "checkActiveFilesAndExecutables" for patch 22502505 passed.

Prereq "checkActiveFilesAndExecutables" for patch 23054319 passed.

Prereq "checkActiveFilesAndExecutables" for patch 17478514 failed.

The details are:

Following executables are active :
/home/app/11.2.0.4/grid/lib/libclntsh.so.11.1

Prereq "checkActiveFilesAndExecutables" for patch 18031668 passed.

Prereq "checkActiveFilesAndExecutables" for patch 18522509 passed.

Prereq "checkActiveFilesAndExecutables" for patch 19121551 passed.

Prereq "checkActiveFilesAndExecutables" for patch 19769489 passed.

Prereq "checkActiveFilesAndExecutables" for patch 20299013 passed.

Prereq "checkActiveFilesAndExecutables" for patch 20760982 passed.

Prereq "checkActiveFilesAndExecutables" for patch 21352635 passed.

Prereq "checkActiveFilesAndExecutables" for patch 21948347 passed.

Prereq "checkActiveFilesAndExecutables" for patch 22502456 passed.

Prereq "checkActiveFilesAndExecutables" for patch 23054359 passed.

Prereq "checkActiveFilesAndExecutables" for patch 24006111 failed.

The details are:

Following executables are active :
/home/app/11.2.0.4/grid/lib/libsqlplus.so

Prereq "checkActiveFilesAndExecutables" for patch 24732075 passed.

OPatch succeeded.

也是報相同的錯誤，使用fuser和lsof檢查

root@ygc26 app]# fuser /home/app/11.2.0.4/grid/lib/libclntsh.so.11.1
/home/app/11.2.0.4/grid/lib/libclntsh.so.11.1: 25740m
[root@ygc26 app]#
[root@ygc26 app]# lsof /home/app/11.2.0.4/grid/lib/libclntsh.so.11.1
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/42/gvfs
Output information may be incomplete.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sqlplus 25740 grid mem REG 253,2 53796056 932514326 /home/app/11.2.0.4/grid/lib/libclntsh.so.11.1

原來是我剛纔通過sqlplus進入asm實例，一直沒退出來，殺掉。該進程，繼續patch 一切順利。