ORACLE集羣管理-補丁升級CRS fails to start after applying 11.2.0.4 Oct 2018 GI PSU or 11.2.0.4 Oct 2018 BP (

CRS fails to start after applying 11.2.0.4 Oct 2018 GI PSU or 11.2.0.4 Oct 2018 BP (Doc ID 2477177.1

 

 

問題概述：

1 採用手動的方式對11.2.0.4版本集羣進行補丁升級，

執行#GI_HOME/crs/install/rootcrs.pl  -unlock 

報錯 ： envem報錯，解決方式爲 安裝perl-ENv rpm包即可。

2 本次只升級了GI PSU 未升級ocw補丁，在執行root腳本

#GI_HOME/crs/install/rootcrs.pl  -patch 過程中gpnp進程無法啓動直接掛起。

以下文檔爲gpnp進程掛起bug。

APPLIES TO:

Oracle Database - Enterprise Edition - Version 11.2.0.4 to 11.2.0.4 [Release 11.2]
Information in this document applies to any platform.

SYMPTOMS

After applying 11.2.0.4 Oct 2018 GI PSU or 11.2.0.4 Oct 2018 BP, CRS fails to start with the following error:

Oracle Grid Infrastructure stack start initiated but failed to complete

CHANGES

 Applied 11.2.0.4 Oct 2018 GI PSU or 11.2.0.4 Oct 2018 BP

CAUSE

There are two causes:

Cause 1:

Bug 28973509 - CRS FAILS TO START AFTER 11.2.0.4 OCW OCT 2018 PSU IS APPLIED BUT DB PSU IS NOT APPLIED TO GI HOME

If the 11.2.0.4 OCW Oct 2018 PSU (patch 27735020) is applied but if the 11.2.0.4 DB Oct 2018 PSU (patch 28204707) is not applied, then the gpnpd fails to start.

The gpnpdOUT.log shows repeated lines of
<GI HOME>/bin/gpnpd.bin: symbol lookup error: <$GRID_HOME>/lib/libhasgen11.so: undefined symbol: ztpk_SetKeyInfo

To confirm if this is the cause, issue "opatch lsinventory -og <$GRID_HOME>" to check if patch 27735020 is applied but patch 28204707 is not applied.
 

There are many reasons that the 11.2.0.4 OCW Oct 2018 PSU (patch 27735020) is applied but the 11.2.0.4 DB Oct 2018 PSU (patch 28204707) is not applied to GI HOME.
The following are two of the common reasons:

1. manually applied the 11.2.0.4 OCW Oct 2018 PSU (patch 27735020) but did not apply the 11.2.0.4 DB Oct 2018 PSU (patch 28204707)
2. "opatch auto" ran into an error while applying the 11.2.0.4 DB Oct 2018 PSU (patch 28204707), so the patch 28204707 was not applied

 

Cause 2:

If 11.2.0.4 DB Oct 2018 PSU (patch 28204707) applied but the 11.2.0.4 OCW Oct 2018 PSU (patch 27735020) is not applied to GI HOME

The gpnpd.trc shows repeated restart of gpnpd.  The terminating error is

2018-12-10 23:52:00.077: [ GPNP][938252064]clsgpnp_InitSetFreeCK: [at clsgpnp0.c:4359] Result: (10016) CLSGPNP_unknown. Cannot set primary certkey for id=1
2018-12-10 23:52:00.078: [ GPNP][938252064]clsgpnp_InitIdSetCtxCK: [at clsgpnp0.c:4401] Result: (10016) CLSGPNP_unknown. Cannot set certkey for id=1
2018-12-10 23:52:00.078: [ GPNP][938252064]clsgpnp_Init: [at clsgpnp0.c:925] Result: (10016) CLSGPNP_unknown. Error getting certkeys.
2018-12-10 23:52:00.078: [ GPNP][938252064]clsgpnp_Init init failed. Error: CLSGPNP_ERR (1) .
2018-12-10 23:52:00.078: [ default][938252064]Fatal error: cannot initialize GPnP, CLSGPNP_ERR (Generic GPnP error).
2018-12-10 23:52:00.282: [ default][938252064]clsgpnpd_main STOP init failed - terminating.
 

 To confirm if this is the cause, issue "opatch lsinventory -og <$GRID_HOME>" to check if patch 28204707 is applied but patch 27735020 is not applied.

 

SOLUTION

Case 1:

Manually apply the 11.2.0.4 DB Oct 2018 PSU (patch 28204707) to GI  HOME.

Refer to the Section 5 "Manual Steps for Apply/Rollback Patch" in the "Oracle Grid Infrastructure 11.2.0.4.x Patch Set Update SUPPLEMENTAL README Document 1641136.1"

 

Case 2:

Manually apply the 11.2.0.4 OCW Oct 2018 PSU (patch 27735020) to GI HOME.

Refer to the Section 5 "Manual Steps for Apply/Rollback Patch" in the "Oracle Grid Infrastructure 11.2.0.4.x Patch Set Update SUPPLEMENTAL README Document 1641136.1"

REFERENCES

NOTE:2490056.1 - After applying or rolling back 11.2.0.4 Oct 2018 PSU or later, either CRS or database fails to start