08 Oct Silent Out Of Place Patching for ORACLE 19c GI
I am going through a process of patching an Oracle Grid Infrastructure from 19.6 to 19.8 (Patch 31305339) and I wanted to try out Out Of Place Patching (OOP) for it, but since I don’t have access to their GUI environment, I proceeded to use the silent option. Below is how it went
I started with verifying that the GI services are up and running using Fred Dennis’s script and the current patches that are applied to the GI. This is just a sanity check before I start to work on this patch.
[grid@node1 ~]$ ./rac_status.sh -a
Cluster rene-ace-c
Type | Name | node1 | node2 |
---------------------------------------------------------------------
asm | asm | Online | Online |
asmnetwork | asmnet1 | Online | Online |
chad | chad | Online | Online |
cvu | cvu | Online | - |
dg | DATA | Online | Online |
dg | RECO | Online | Online |
network | net1 | Online | Online |
ons | ons | Online | Online |
qosmserver | qosmserver | Online | - |
vip | node1 | Online | - |
vip | node2 | - | Online |
vip | scan1 | - | Online |
vip | scan2 | Online | - |
vip | scan3 | Online | - |
---------------------------------------------------------------------
x : Resource is disabled
: Has been restarted less than 24 hours ago
Listener | Port | node1 | node2 | Type |
------------------------------------------------------------------------------------
ASMNET1LSNR_ASM| TCP:1525 | Online | Online | Listener |
LISTENER | TCP:1521 | Online | Online | Listener |
LISTENER_SCAN1 | TCP:1521 | - | Online | SCAN |
LISTENER_SCAN2 | TCP:1521 | Online | - | SCAN |
LISTENER_SCAN3 | TCP:1521 | Online | - | SCAN |
------------------------------------------------------------------------------------
DB | Version | node1 | node2 | DB Type |
---------------------------------------------------------------------------------------
renedev | 19.6.0.0 (1) | - | Open | SINGLE (P) |
reneqa | 19.6.0.0 (2) | Open | Open | RAC (P) |
[grid@node1 ~]$ echo $ORACLE_HOME
/u01/app/19.3.0.0/grid
[grid@node1 ~]$ cd $ORACLE_HOME/OPatch
[grid@node1 OPatch]$ ./opatch lspatches
30655595;TOMCAT RELEASE UPDATE 19.0.0.0.0 (30655595)
30557433;Database Release Update : 19.6.0.0.200114 (30557433)
30489632;ACFS RELEASE UPDATE 19.6.0.0.0 (30489632)
30489227;OCW RELEASE UPDATE 19.6.0.0.0 (30489227)
OPatch succeeded.
Now that I have done that, I checked if the patches for 19.8 are applicable to the 19.6 GI and there were no conflicts or issues that should arise from the patch.
## Note.- I trimmed the log for readability [root@node1 ~]$ . oraenv ORACLE_SID = [root] ? +ASM1 The Oracle base has been set to /u01/app/oracle [root@node1 ~]$ export PATH=$ORACLE_HOME/OPatch:$PATH [root@node1 ~]$ opatchauto apply /u01/patches/30501910 -analyze ... Host:node1 CRS Home:/u01/app/19.6.0.0/grid Version:19.0.0.0.0 ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /u01/patches/31305339/31281355 Log: /u01/app/19.3.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-07_11-41-25AM_1.log Patch: /u01/patches/31305339/31304218 Log: /u01/app/19.3.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-07_11-41-25AM_1.log Patch: /u01/patches/31305339/31305087 Log: /u01/app/19.3.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-07_11-41-25AM_1.log Patch: /u01/patches/31305339/31335188 Log: /u01/app/19.3.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-07_11-41-25AM_1.log OPatchauto session completed at Wed Oct 7 11:42:46 2020 Time taken to complete the session 2 minutes, 41 seconds
Once I saw that the 19.8 patch was not going to cause an Issue , I decided to proceed with this methodology of patching.
The first thing that you have to do, is to do unzip the 19.3 GI software to the ORACLE_HOME that will be the 19.8 patched GI. The software can be downloaded from either OTN or edelivery, in my case I used the edelivery
I also updated the OPatch version to the one that the 19.8 (31305339) patch needs, which is OPatch vesion 12.2.0.1.21 for DB 19.x releases (Apr 2020).
[grid@node1 grid]$ unzip -q /u01/patches/v982068-01.zip -d /u01/app/19.8.0.0/grid [grid@node1 grid]$ mv /u01/app/19.8.0.0/grid/OPatch /u01/app/19.8.0.0/grid/OPatch.old [grid@node1 grid]$ unzip -q /u01/patches/p6880880_190000_Linux-x86-64.zip -d /u01/app/19.8.0.0/grid
What I normally do in these cases is to unset my Oracle variables, just so that whatever I do, I don’t hurt whatever is currently running. After that I set the ORACLE_HOME to the 19.8 environment.
# a. Unset all Oracle related shell variables [grid@node1 ~]$ unset ORACLE_BASE [grid@node1 ~]$ unset ORACLE_HOME [grid@node1 ~]$ unset ORACLE_SID [grid@node1 ~]$ unset ORA_CRS_HOME [grid@node1 ~]$ unset ORA_NLS10 [grid@node1 ~]$ unset TNS_ADMIN #b. The following must return nothing [grid@node1 ~]$ env | egrep "ORA|TNS" | wc -l 0 #c. Set the 19.8 ORACLE_HOME [grid@node1 ~]$ export ORACLE_HOME=/u01/app/19.8.0.0/grid
Now I was ready to do run the gridSetup.sh in silent mode applying the 19.8 patch passing the response file parameters via command line.
When I did this, I faced the the patch error ZOP-51. I tried looking for a special character in what I was passing, but saw nothing and even though that I verified that the patch binaries where owned by grid:oinstall and that everything on that location look good to me, I kept running into this error.
[grid@node1 ~]$ cd $ORACLE_HOME [grid@node1 ~]$ pwd /u01/app/19.8.0.0/grid [grid@node1 grid]$ ./gridSetup.sh -ignorePrereq -waitforcompletion -silent \ > -ApplyRU /u01/patches/31305339 \ > -responseFile /u01/app/19.8.0.0/grid/install/response/gridsetup.rsp \ > INVENTORY_LOCATION=/u01/app/oraInventory \ > SELECTED_LANGUAGES=en,en_GB \ > oracle.install.option=CRS_SWONLY \ > ORACLE_BASE=/u01/app/grid \ > oracle.install.asm.OSDBA=asmdba \ > oracle.install.asm.OSOPER=asmoper \ > oracle.install.asm.OSASM=asmadmin \ > oracle.install.crs.config.ClusterConfiguration=STANDALONE \ > oracle.install.crs.config.configureAsExtendedCluster=false \ > oracle.install.crs.config.clusterName=rene-ace-c \ > oracle.install.crs.config.gpnp.configureGNS=false \ > oracle.install.crs.config.autoConfigureClusterNodeVIP=false \ > oracle.install.crs.config.clusterNodes=node1.localdomain,node2.localdomain Launching Oracle Grid Infrastructure Setup Wizard... [WARNING] [INS-13014] Target environment does not meet some optional requirements. CAUSE: Some of the optional prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/GridSetupActions2020-10-07_11-51-50AM/gridSetupActions2020-10-07_11-51-50AM.log ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/GridSetupActions2020-10-07_11-51-50AM/gridSetupActions2020-10-07_11-51-50AM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually. The response file for this session can be found at: /u01/app/19.8.0.0/grid/install/response/grid_2020-10-07_11-51-50AM.rsp You can find the log of this install session at: /u01/app/oraInventory/logs/GridSetupActions2020-10-07_11-51-50AM/gridSetupActions2020-10-07_11-51-50AM.log [FATAL] [INS-32500] OPatch tool failed. ACTION: Review /u01/app/oraInventory/logs/GridSetupActions2020-10-07_11-51-50AM/gridSetupActions2020-10-07_11-51-50AM.log for more details. *ADDITIONAL INFORMATION:* - Oracle Interim Patch Installer version 12.2.0.1.21 - Copyright (c) 2020, Oracle Corporation. All rights reserved. - - - ZOP-51: The patch location is not valid for apply, because it doesn't have correct metadata, or it points to a patch directory. - Please check the arguments and try again. - - OPatch failed with error code 135 - Argument(s) Error... Patch location is not valid for apply -
So what I did was build a response file instead with the exact values that I was passing in the command line . Once I did this, I was able to install the CRS without an issue.
[grid@node1 ~]$ cd $ORACLE_HOME [grid@node1 ~]$ pwd /u01/app/19.8.0.0/grid [grid@node1 grid]$ ./gridSetup.sh -ignorePrereq -waitforcompletion -silent -responseFile /home/grid/grid_19.rsp -applyRU /u01/patches/31305339 Preparing the home to patch... Applying the patch /u01/patches/31305339... Successfully applied the patch. The log can be found at: /u01/app/oraInventory/logs/GridSetupActions2020-10-07_12-32-13PM/installerPatchActions_2020-10-07_12-32-13PM.log Launching Oracle Grid Infrastructure Setup Wizard... [WARNING] [INS-13014] Target environment does not meet some optional requirements. CAUSE: Some of the optional prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/GridSetupActions2020-10-07_12-32-13PM/gridSetupActions2020-10-07_12-32-13PM.log ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/GridSetupActions2020-10-07_12-32-13PM/gridSetupActions2020-10-07_12-32-13PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually. The response file for this session can be found at: /u01/app/19.8.0.0/grid/install/response/grid_2020-10-07_12-32-13PM.rsp You can find the log of this install session at: /u01/app/oraInventory/logs/GridSetupActions2020-10-07_12-32-13PM/gridSetupActions2020-10-07_12-32-13PM.log As a root user, execute the following script(s): 1. /u01/app/19.8.0.0/grid/root.sh Execute /u01/app/19.8.0.0/grid/root.sh on the following nodes: [node1, node2] Successfully Setup Software with warning(s). ... [root@node1 patches]# /u01/app/19.8.0.0/grid/root.sh Check /u01/app/19.8.0.0/grid/install/root_node1_2020-10-07_13-01-25-549937823.log for the output of root script [root@node1 patches]# ssh node2 Last login: Wed Oct 7 12:57:07 2020 [root@node2 ~]# /u01/app/19.8.0.0/grid/root.sh Check /u01/app/19.8.0.0/grid/install/root_node2_2020-10-07_13-01-32-411313252.log for the output of root script
When I was ready, I now proceeded to switch the GI to the installed 19.8. Keep in mind that running the root.sh will take a while as this is where basically the GI moves from the 19.6 to 19.8 . Also make sure to never run the root.sh concurrently, it has to run on node1 first and then on node2.
[grid@node1 ~]$ cd /u01/app/19.8.0.0/grid [grid@node1 grid]$ ./gridSetup.sh -switchGridHome -silent Launching Oracle Grid Infrastructure Setup Wizard... You can find the log of this install session at: /u01/app/oraInventory/logs/cloneActions2020-10-07_01-19-16PM.log As a root user, execute the following script(s): 1. /u01/app/19.8.0.0/grid/root.sh Execute /u01/app/19.8.0.0/grid/root.sh on the following nodes: [node1, node2] Run the scripts on the local node first. After successful completion, run the scripts in sequence on all other nodes. Successfully Setup Software. ... [root@node1 patches]# /u01/app/19.8.0.0/grid/root.sh Check /u01/app/19.8.0.0/grid/install/root_node1_2020-10-07_13-32-59-377526581.log for the output of root script [root@node1 patches]# ssh node2 Last login: Wed Oct 7 13:33:03 2020 [root@node2 ~]# /u01/app/19.8.0.0/grid/root.sh Check /u01/app/19.8.0.0/grid/install/root_node2_2020-10-07_13-38-09-323573349.log for the output of root script
After you run the root.sh in each node, I proceed to update the /etc/oratab, verify that the patches are correct and that the services that were up prior to the patching exercise are up after it.
[grid@node1 ~]$ cat /etc/oratab |grep "+ASM"
+ASM1:/u01/app/19.8.0.0/grid:N
[grid@node1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
The Oracle base has been set to /u01/app/grid
[grid@node1 OPatch]$ $ORACLE_HOME/OPatch/opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[grid@node1 ~]$ ./rac_status.sh -a
Cluster rene-ace-c
Type | Name | node1 | node2 |
---------------------------------------------------------------------
asm | asm | Online | Online |
asmnetwork | asmnet1 | Online | Online |
chad | chad | Online | Online |
cvu | cvu | Online | - |
dg | DATA | Online | Online |
dg | RECO | Online | Online |
network | net1 | Online | Online |
ons | ons | Online | Online |
qosmserver | qosmserver | Online | - |
vip | node1 | Online | - |
vip | node2 | - | Online |
vip | scan1 | - | Online |
vip | scan2 | Online | - |
vip | scan3 | Online | - |
---------------------------------------------------------------------
x : Resource is disabled
: Has been restarted less than 24 hours ago
Listener | Port | node1 | node2 | Type |
------------------------------------------------------------------------------------
ASMNET1LSNR_ASM| TCP:1525 | Online | Online | Listener |
LISTENER | TCP:1521 | Online | Online | Listener |
LISTENER_SCAN1 | TCP:1521 | - | Online | SCAN |
LISTENER_SCAN2 | TCP:1521 | Online | - | SCAN |
LISTENER_SCAN3 | TCP:1521 | Online | - | SCAN |
------------------------------------------------------------------------------------
DB | Version | node1 | node2 | DB Type |
---------------------------------------------------------------------------------------
renedev | 19.6.0.0 (1) | - | Open | SINGLE (P) |
reneqa | 19.6.0.0 (2) | Open | Open | RAC (P) |
Last but not least, when you are ready, don’t forget to remove the 19.6 GI Home from the inventory and delete it. I recommend taking at least a few days to delete it as if you need to rollback this is where you will rollback to.
[grid@node1 ~]$ /u01/app/19.8.0.0/grid/oui/bin/runInstaller -detachHome -silent -local ORACLE_HOME="/u01/app/19.6.0.0/grid" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4076 MB Passed The inventory pointer is located at /etc/oraInst.loc You can find the log of this install session at: /u01/app/oraInventory/logs/DetachHome2020-10-07_02-26-40PM.log 'DetachHome' was successful. # As root [root@node1 ~]$ cd /u01/app/ [root@node1 ~]$ rm -rf 19.6.0.0 [grid@node2 ~]$ /u01/app/19.8.0.0/grid/oui/bin/runInstaller -detachHome -silent -local ORACLE_HOME="/u01/app/19.6.0.0/grid" Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4073 MB Passed The inventory pointer is located at /etc/oraInst.loc You can find the log of this install session at: /u01/app/oraInventory/logs/DetachHome2020-10-07_02-26-54PM.log 'DetachHome' was successful. # As root [root@node2 ~]$ cd /u01/app/ [root@node2 ~]$ rm -rf 19.6.0.0
I hope that this blog helps you if you are going to do a silent OOP Patching for your GI . I am also including the grid rsp file I used so that you can use it as reference.
Pingback:René Antúnez - Oracle ACE | Silent Rollback From an Out Of Place Patching For Oracle 19c GI
Posted at 10:27h, 27 October[…] last post was on how to do an Out Of Place Patching for Oracle GI. The next thing that I wanted to try was the rollback procedure for this methodology, but as I […]
Pingback:René Antúnez - Oracle ACE | How I Finished a GI OOP Patching From 19.6 to 19.8 After Facing cluutil: No Such File or Directory and clsrsc-740 Errors
Posted at 03:05h, 11 March[…] I have already blogged in the past on how to do a GI OOP, I won’t go into several details of this process, I will just mention what I was […]