| 1 | As of 11.2 Grid Infrastructure, the private network configuration is not only stored in OCR but also in the gpnp profile. If the private network is not available or its definition is incorrect, the CRSD process will not start and any subsequent changes to the OCR will be impossible. Therefore care needs to be taken when making modifications to the configuration of the private network. It is important to perform the changes in the correct order. Please also note that manual modification of gpnp profile is not supported. |
| Please take a backup of profile.xml on all cluster nodes before proceeding, as grid user: | |
| cd $GRID_HOME/gpnp/<hostname>/profiles/peer/ | |
| cd /app/oracle/product/12.x.x/grid/gpnp/mysrvrahr/profiles/peer | |
| cd /app/oracle/product/12.x.x/grid/gpnp/mysrvrbhr/profiles/peer | |
| cp -p profile.xml profile.xml.bk | |
| 2 | Ensure Oracle Clusterware is running on ALL cluster nodes in the cluster and save current status of resource. |
| /app/oracle/product/12.x.x/grid/bin/crsctl check cluster -all | |
| /app/oracle/product/12.x.x/grid/bin/crsctl status resource -t>/tmp/beforeNewIps.lst | |
| 3 | As grid user ( always curious who might that b, to me it was the Oracle user btw.): |
| Get the existing information. Here you will see only One interconnect in place For example: | |
| /app/oracle/product/12.x.x/grid/bin/oifcfg getif | |
| bond1 172.18.112.208 global cluster_interconnect | |
| bond0 195.233.190.64 global public | |
| ## | |
| Check the interfaces / subnet address can be identified by command for eth specifically: | |
| /app/oracle/product/12.x.x/grid/bin/oifcfg iflist|grep -i eth|sort | |
| eth0 172.18.32.0 | |
| eth2 192.168.10.0 | |
| eth6 192.168.11.0 | |
| or check interfaces / subnets in general on OS with oifcfg: | |
| /app/oracle/product/12.x.x/grid/bin/oifcfg iflist|sort | |
| 4 | Add the new cluster_interconnect information: |
| /app/oracle/product/12.x.x/grid/bin/oifcfg setif -global eth2/192.168.10.0:cluster_interconnect,asm | |
| /app/oracle/product/12.x.x/grid/bin/oifcfg setif -global eth6/192.168.11.0:cluster_interconnect,asm | |
| 5 | Verify the change: |
| /app/oracle/product/12.x.x/grid/bin/oifcfg getif | |
With this information checked and in place it is time for setting up new listeners for asm since the original ASM listener during the installation used eth0 and that eth0 will be dropped – removed from cluster configuration in steps below: Existing listener ASMNET1LSNR will become new one ASMNET122LSNR. srvctl add listener -asmlistener -l ASMNET1221LSNR -subnet 192.168.10.0 (as mentioned this is the eth2 interface that we are going to use). srvctl add listener -asmlistener -l ASMNET1222LSNR -subnet 192.168.11.0 (as mentioned this is the eth6 interface that we are going to use). As always seeing is believing : use crsctl status resource -t to see details. Note: The new ASM listener is created as a resource and it is in a status offline offline on all nodes in the cluster at this point and time. In the next step we will remove the old ASM listener, and use a -f option to prevent errors – messages with regard to dependencies. srvctl update listener -listener ASMNET1LSNR_ASM -asm -remove -force I have checked again with crsctl status resource -t to make sure the old resource is gone now. Removing the old ASM listener In the Mos note there is a little inconsistency because it claims that as a next step the old ASM listener should be stopped. I was able to grep for the listener ( ps -ef|grep -i inherit) and i saw it on OS level on the machine(S). But I am not able to stop that listener since the cluster resource is already gone and lsnrctl did not work. Solution: What I noticed that when I skipped this step and stopped and started the cluster which is mandatory in this scenario, the listener was gone on all nodes. Should have given this command, but that is NOT working: lsnrctl stop ASMNET1LSNR_ASM Check configuration before restarting GI: First command: srvctl config listener -asmlistener Name: ASMNET122LSNR_ASM Type: ASM Listener Owner: oracle Subnet: 192.168.10.0 Home: <CRS home> End points: TCP:1527 Listener is enabled. Listener is individually enabled on nodes: Listener is individually disabled on nodes: Second Command: srvctl config asm ASM home: <CRS home> Password file: +VOTE/orapwASM Backup of Password file: ASM listener: LISTENER ASM instance count: ALL Cluster ASM listener: ASMNET122LSNR_ASM | |
| 6 | In GridInfrastructure: Shutdown Oracle Clusterware on all nodes and disable the Oracle Clusterware as root user ( in my example i was allowed to sudo ): |
| sudo su – | |
| ./app/oracle/product/12.x.x/grid/bin/crsctl stop crs | |
| ./app/oracle/product/12.x.x/grid/bin/crsctl disable crs | |
| 7 | Make the network configuration change at OS level as required, ensure the new interface is available on all nodes after the change. ( check to ping the interfaces on all nodes ). |
| for x in 10 11;do for xx in 75 76 77 78;do ping -c2 192.168.${x}.${xx}|egrep ‘icmp_seq|transmitted’;done;echo;done | |
| for x in a b c d; do for xx in 1 2;do ping -c2 mysrvr${x}hr-hb$xx|egrep ‘icmp_seq|transmitted’;done;echo;done | |
| 8 | Restart Oracle Clusterware on all nodes as root user: |
| sudo su – | |
| ./app/oracle/product/12.x.x/grid/bin/crsctl start crs | |
| 9 | Check |
| /app/oracle/product/12.x.x/grid/bin/crsctl check cluster -all | |
| /app/oracle/product/12.x.x/grid/bin/crsctl status resource -t>/tmp/afterNewIps.lst | |
| sdiff /tmp/afterNewIps.lst /tmp/beforeNewIps.lst | |
| Enable Oracle Clusterware on all nodes as root user: | |
| ./app/oracle/product/12.x.x/grid/bin/crsctl enable crs | |
| 10 | Remove the old interface if required: |
| /app/oracle/product/12.x.x/grid/bin/oifcfg delif -global bond1/172.18.112.208:cluster_interconnect | |
| 11 | Verify the remove: |
| /app/oracle/product/12.x.x/grid/bin/oifcfg getif |
12.2 Grid Infra
Missing or Corrupted Spfile in ASM instance in Rac
Introduction:
Old saying is , always expect the unexpected, well this time this was another proof of that. During patching of an 8 node cluster on the first node we came across 2 issues , both requiring a work around. Issue one was that after applying January 2019 Cluster did not start. This workaround is not part of this note btw. The second issue was that once we had the first Workaround in place the asm instance on the first node would not start. This note is explaining the steps followed to create a new spfile for the ASM instance(s) in a rac cluster.
General information:
This was the scenario where we found ourselves in: Patching in a rolling way had started on the first node with January 2019 on 12.2 Oracle (GI and RDBMS) the first node was patched but crs would not start ( and hmm never liked opatchauto a lot 2 b honest). Together with Oracle support a work around was provided but after that the ASM instance still would not start. During compare of environments it showed one very significant memory setting on this cluster. Maybe this would have been MEMORY_* and we are using hugepages on that cluster) that prevented asm from starting once the patching on the first node completed .
As a work around we created a pfile and once the cluster on node one was up we started the asm instance with the pfile ( which was altered by me).
However we could no longer update the spfile for the other asm instances since oracle told us that in a rolling upgrade mode you cannot make changes to the spfile . That meant that on all 8 Nodes we performed the patching , and once crs was up we had to start the asm with a copy of the init.ora we used on 1st node too. In our case 8 nodes with a local copy of the init.ora which did not make us happy at all. That however brought us to below scenario where you need to bring the asm back to using an spfile .
Important note: Since oracle 11.2 the GPNP profile is the key for such change !!!!
From old days , came up with below scenario to create a pfile, alter that file to meet your needs and bring that as a spfile for the asm instance :
Scenario
- Could be used with a missing spfile – or corrupted .
- With an existing spfile with wrong settings , but where you cannot alter since you started patching already ( spfile updates are prohibited in rolling upgrade scenario ) , so maybe best practice is to analyse spfile before.
- Important message: Scenario has the requirement that the FULL cluster stack is down and you will work with ONE node only!!
#### With spget you can check current location of spfile in asmcmd.
ASMCMD [+] > spget
+VOTE/mysrvr18cl/ASMPARAMETERFILE/registry.253.978015605
#### created and altered the pfile on the first node. ( and copied it to all other nodes during the workaround).
oracle@mysrvr1dr:/app/grid/product/12201/grid/dbs []# cd /app/oracle/admin/+ASM1/pfile
oracle@mysrvr1dr:/app/oracle/admin/+ASM1/pfile []# ls -ltr
total 4
-rw-r–r–. 1 oracle dba 2433 Feb 8 09:23 initASM.ora
##### starting cluster 1st attempt (recalled that the cluster needed 2 b in some part of restricted mode for that, so all of cluster was stopped , then below command was issued). But Oracle showed mercy , telling to use the correct syntax:
mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl start crs restrict
Parse error:
‘restrict’ is an invalid argument
Usage:
crsctl start crs [-excl [-nocrs | -cssonly]] | [-wait | -waithas | -nowait] | [-noautostart]
Start OHAS on this server
where
-excl Start Oracle Clusterware in exclusive mode
-nocrs Start Oracle Clusterware in exclusive mode without starting CRS
-nowait Do not wait for OHAS to start
-wait Wait until startup is complete and display all progress and status messages
-waithas Wait until startup is complete and display OHASD progress and status messages
-cssonly Start only CSS
-noautostart Start only OHAS
## Then started cluster in exclusive mode , that failed too btw since it tried to start the asm instance, which was still holding the original spfile with the incorrect information.
mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl start crs -excl
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr1dr’
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr1dr’ succeeded
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr1dr’
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr1dr’
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr1dr’ succeeded
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr1dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr1dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr1dr’ succeeded
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr1dr’
- CRS-2674: Start of ‘ora.asm’ on ‘mysrvr1dr’ failed
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr1dr’
- ORA-15077: could not locate ASM instance serving a required diskgroup
- CRS-2674: Start of ‘ora.storage’ on ‘mysrvr1dr’ failed
- CRS-2679: Attempting to clean ‘ora.storage’ on ‘mysrvr1dr’
- CRS-2681: Clean of ‘ora.storage’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.ctssd’ on ‘mysrvr1dr’ succeeded
- CRS-4000: Command Start failed, or completed with errors.
## Next attempt , whole cluster was down , on first node performed
- mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl start crs -excl -nocrs
- ## in this nocrs mode we were able to start the +ASM1 instance manually now.
- SQL> STARTUP PFILE=’/app/oracle/admin/+ASM1/pfile/initASM.ora’;
- ## Once ASM had started create the new SPFILE
- create spfile = ‘+VOTE’ from PFILE=’/app/oracle/admin/+ASM1/pfile/initASM.ora’;
- ## in Alert noticed this which is good SINCE this shows gpnp profile had been updated accordingly
- 019-02-08T16:31:32.839547+01:00
- NOTE: updated gpnp profile ASM SPFILE to
- NOTE: header on disk 0 advanced to format #2 using fcn 0.0
- NOTE: header on disk 2 advanced to format #2 using fcn 0.0
- NOTE: updated gpnp profile ASM diskstring: /dev/mapper/ASM_*
- NOTE: updated gpnp profile ASM diskstring: /dev/mapper/ASM_*
- 2019-02-08T16:31:34.381619+01:00
- NOTE: updated gpnp profile ASM SPFILE to +VOTE/mysrvr18cl/ASMPARAMETERFILE/registry.253.999707493
##### Checked our activities in ASMCMD:
- oracle@mysrvr1dr:/app/oracle/admin/+ASM1/pfile [+ASM1]# asmcmd
- [Option -p will be used ]
- ASMCMD [+] > spget
- +VOTE/mysrvr18cl/ASMPARAMETERFILE/registry.253.999707493
### Checked our activities in gpnptool
oracle@mysrvr1dr:/app/oracle/admin/+ASM1/pfile [+ASM1]# gpnptool get
Warning: some command line parameters were defaulted. Resulting command line:
/app/grid/product/12201/grid/bin/gpnptool.bin get -o-
<?xml version=”1.0″ encoding=”UTF-8″?><gpnp:GPnP-Profile Version=”1.0″ xmlns=”http://www.grid-pnp.org/2005/11/gpnp-profile” xmlns:gpnp=”http://www.grid-pnp.org/2005/11/gpnp-profile” xmlns:orcl=”http://www.oracle.com/gpnp/2005/11/gpnp-profile” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd” ProfileSequence=”7″ ClusterUId=”afc024ecfd5ffff8ffbeda0a212bebe1″ ClusterName=”mysrvr18cl” PALocation=””><gpnp:Network-Profile><gpnp:HostNetwork id=”gen” HostName=”*”><gpnp:Network id=”net1″ IP=”198.19.11.0″ Adapter=”bond0″ Use=”public”/><gpnp:Network id=”net2″ IP=”192.168.10.0″ Adapter=”eth3″ Use=”asm,cluster_interconnect”/><gpnp:Network id=”net3″ IP=”192.168.11.0″ Adapter=”eth5″ Use=”cluster_interconnect”/></gpnp:HostNetwork></gpnp:Network-Profile><orcl:CSS-Profile id=”css” DiscoveryString=”+asm” LeaseDuration=”400″/><orcl:ASM-Profile id=”asm” DiscoveryString=”/dev/mapper/ASM_*” SPFile=”+VOTE/mysrvr18cl/ASMPARAMETERFILE/registry.253.999707493″ Mode=”remote” Extended=”false”/><ds:Signature xmlns:ds=”http://www.w3.org/2000/09/xmldsig#“><ds:SignedInfo><ds:CanonicalizationMethod Algorithm=”http://www.w3.org/2001/10/xml-exc-c14n#“/><ds:SignatureMethod Algorithm=”http://www.w3.org/2000/09/xmldsig#rsa-sha1“/><ds:Reference URI=””><ds:Transforms><ds:Transform Algorithm=”http://www.w3.org/2000/09/xmldsig#enveloped-signature“/><ds:Transform Algorithm=”http://www.w3.org/2001/10/xml-exc-c14n#“> <InclusiveNamespaces xmlns=”http://www.w3.org/2001/10/xml-exc-c14n#” PrefixList=”gpnp orcl xsi”/></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm=”http://www.w3.org/2000/09/xmldsig#sha1“/><ds:DigestValue>QH9UPO559zhufkrc7tFxQts6oF0=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>aL2hOnxyLt5YwMcPjGg8LUDx2KD97Y75eLv+
yqvcfQ5O705K8ceQPCnwnsTs4Wn5E1jNeYCEzXnrVp5zM3hMbz9LdEEP2GKk9XJInQprWc39z7JKxm4uEw
NX3Ocs54FqxP1JdBX7PRiMh/
ePd8CoJIVtIaVMD29giX078uGwXcQ=</ds:SignatureValue></ds:Signature></gpnp:GPnP-Profile>
### since we have started cluster with -excl –nocrs time to stop the cluster and start it normally
mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl stop crs
- CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.crsd’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.crsd’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.crf’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.crf’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.gpnpd’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.storage’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.storage’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.mdnsd’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.ctssd’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.cssd’ on ‘mysrvr1dr’
### starting cluster normally on first node in normal mode
mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl start crs
### Had small issue , so decided to stop the cluster on node 1 with force option
mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl stop crs -f
- CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.mdnsd’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.gpnpd’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.evmd’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.asm’ on ‘mysrvr1dr’
- CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.ctssd’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.evmd’ on ‘mysrvr1dr’ succeeded
- CRS-2677: Stop of ‘ora.asm’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.cssd’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.cssd’ on ‘mysrvr1dr’ succeeded
- CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘mysrvr1dr’
- CRS-2677: Stop of ‘ora.gipcd’ on ‘mysrvr1dr’ succeeded
- CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘mysrvr1dr’ has completed
- CRS-4133: Oracle High Availability Services has been stopped.
#### Time to start the cluster in normal mode for all nodes
mysrvr1dr:root:/root $ cd /app/grid/product/12201/grid/bin
mysrvr1dr:root:/app/grid/product/12201/grid/bin $ ./crsctl start cluster -all
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr6dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr6dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr4dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr2dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr3dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr8dr’
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr2dr’
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr4dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr7dr’
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr3dr’
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr8dr’
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr7dr’
- CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘mysrvr5dr’
- CRS-2672: Attempting to start ‘ora.evmd’ on ‘mysrvr5dr’
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr4dr’ succeeded
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr8dr’ succeeded
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr4dr’
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr4dr’
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr8dr’
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr2dr’ succeeded
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr8dr’
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr3dr’ succeeded
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr6dr’ succeeded
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr7dr’ succeeded
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr2dr’
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr6dr’
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr2dr’
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr3dr’
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr6dr’
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr7dr’
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr3dr’
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr4dr’ succeeded
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr7dr’
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr8dr’ succeeded
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr2dr’ succeeded
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr6dr’ succeeded
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr3dr’ succeeded
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr7dr’ succeeded
- CRS-2676: Start of ‘ora.cssdmonitor’ on ‘mysrvr5dr’ succeeded
- CRS-2672: Attempting to start ‘ora.cssd’ on ‘mysrvr5dr’
- CRS-2672: Attempting to start ‘ora.diskmon’ on ‘mysrvr5dr’
- CRS-2676: Start of ‘ora.diskmon’ on ‘mysrvr5dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr6dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr2dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr4dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr8dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr3dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr7dr’ succeeded
- CRS-2676: Start of ‘ora.evmd’ on ‘mysrvr5dr’ succeeded
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr8dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr8dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr8dr’
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr2dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr2dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr2dr’
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr5dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr5dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr5dr’
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr8dr’ succeeded
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr2dr’ succeeded
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr7dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr7dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr7dr’
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr5dr’ succeeded
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr4dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr4dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr4dr’
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr3dr’ succeeded
- CRS-2676: Start of ‘ora.cssd’ on ‘mysrvr6dr’ succeeded
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr3dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr3dr’
- CRS-2672: Attempting to start ‘ora.ctssd’ on ‘mysrvr6dr’
- CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘mysrvr6dr’
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr7dr’ succeeded
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr8dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr8dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr8dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr8dr’
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr4dr’ succeeded
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr3dr’ succeeded
- CRS-2676: Start of ‘ora.ctssd’ on ‘mysrvr6dr’ succeeded
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr2dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr2dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr2dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr2dr’
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr8dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr8dr’
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr8dr’ succeeded
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr2dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr2dr’
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr5dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr5dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr5dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr5dr’
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr2dr’ succeeded
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr5dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr5dr’
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr7dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr7dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr7dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr7dr’
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr5dr’ succeeded
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr7dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr7dr’
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr6dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr6dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr6dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr6dr’
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr4dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr4dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr4dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr4dr’
- CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘mysrvr3dr’ succeeded
- CRS-2672: Attempting to start ‘ora.asm’ on ‘mysrvr3dr’
- CRS-2676: Start of ‘ora.asm’ on ‘mysrvr3dr’ succeeded
- CRS-2672: Attempting to start ‘ora.storage’ on ‘mysrvr3dr’
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr7dr’ succeeded
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr6dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr6dr’
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr4dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr4dr’
- CRS-2676: Start of ‘ora.storage’ on ‘mysrvr3dr’ succeeded
- CRS-2672: Attempting to start ‘ora.crsd’ on ‘mysrvr3dr’
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr6dr’ succeeded
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr4dr’ succeeded
- CRS-2676: Start of ‘ora.crsd’ on ‘mysrvr3dr’ succeeded
- CRS-4690: Oracle Clusterware is already running on ‘mysrvr1dr’ à fine since we kept cluster running on node 1
CRS-4000: Command Start failed, or completed with errors.
### checks performed :
On each node: ps -ef|grep d.bin
On each node: crsctl stat res -t -init
On a node: crsctl check cluster -all
Happy reading,
And till next time
Mathijs
Create password file in 12.2 Oracle on Asm
Introduction.
In current project we are relying on replicated databases for environments like a Pet. On that cloned environment ( 12.2 Rac database with ASM) I received request to add a specific adba group to the user(s) who have sysdba privs in the databases . Below you will find the notes i have created to get there. In the source environment a password file has been setup in asm. During the cloning the database MYDBP is renamed to MYDBA with the nid tool.
Creating the Password file as a first step in ASM
orapwd file=+MYDBP_DATA01 password=hugo10! entries=10
ERROR at line 1:
ORA-01994: GRANT failed: password file missing or disabled
ORA-17503: ksfdopn:2 Failed to open file
+MYDBP_DATA01/MYDBA/PASSWORD/pwdMYDBa.312.981043193
ORA-15173: entry ‘MYDBA’ does not exist in directory ‘/’
ORA-06512: at line 4
ORA-06512: at “SYS.X$DBMS_DISKGROUP”, line 665
ORA-06512: at line 2
orapwd file=’+MYDBP_DATA01/MYDBA/orapwdb’ dbuniquename=’MYDBA’
ASMCMD> pwcreate –dbuniquename <dbuniquename> <file_path> <sys-password>
ASMCMD [+MYDBP_DATA01/MYDBP/PASSWORD] > ls
pwdMYDBp.256.975838613
## below error
ERROR at line 1:
ORA-01994: GRANT failed: password file missing or disabled
ORA-17503: ksfdopn:2 Failed to open file
+MYDBP_DATA01/MYDBA/PASSWORD/pwdMYDBa.312.981043193
ORA-15173: entry ‘MYDBA’ does not exist in directory ‘/’
ORA-06512: at line 4
ORA-06512: at “SYS.X$DBMS_DISKGROUP”, line 665
ORA-06512: at line 2
## as a Workaround:
oracle@Mysrvrr:/app/oracle/product/12201/db/dbs []# asmcmd
[Option -p will be used ]
## In asmcmd created the subdirectory with the name of the cloned database
ASMCMD [+] > cd +MYDBP_DATA01
ASMCMD [+MYDBP_DATA01] > ls
MYDBP/
ASMCMD [+MYDBP_DATA01] > mkdir MYDBA
ASMCMD [+MYDBP_DATA01] > cd MYDBA
ASMCMD [+MYDBP_DATA01/MYDBA] > ls
ASMCMD [+MYDBP_DATA01/MYDBA] > mkdir PASSWORD
ASMCMD [+MYDBP_DATA01/MYDBA] >
### And tried again
orapwd file='+MYDBP_DATA01/MYDBA/orapwdb' dbuniquename='MYDBA' entries=10 oracle@Mysrvrr:/app/oracle/product/12201/db/dbs []# orapwd file='+MYDBP_DATA01/MYDBA/orapwdb' dbuniquename='MYDBA' Enter password for SYS: OPW-00010: Could not create the password file. This resource has a Password File.
## Grrr checked with srvctl config and noticed that this one knows about pwfile too :
Database unique name: MYDBA
Database name:
Oracle home: /app/oracle/product/12201/db
Oracle user: oracle
Spfile:
Password file: +MYDBP_DATA01/MYDBA/PASSWORD/pwdMYDBa.312.981043193
Domain: PROD.NL
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: MYDBP_DATA01,MYDBP_FRA01
Mount point paths:
Services: MYDBA_TAF.prod.nl
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: dba
Database instances: MYDBA1,MYDBA2
Configured nodes: Mysrvrr,nlp1e6dr
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed
## Removed the information about the password file in the cluster.
oracle@Mysrvrr: [MYDBA1]# srvctl modify database -db MYDBA -pwfile ”
##Checked again
oracle@Mysrvrr:/app/oracle/diag/rdbms/local10p/LOCAL10P/trace [MYDBA1]# srvctl config database -d MYDBA
Database unique name: MYDBA
Database name:
Oracle home: /app/oracle/product/12201/db
Oracle user: oracle
Spfile:
Password file:
Domain: PROD.NL
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: MYDBP_DATA01,MYDBP_FRA01
Mount point paths:
Services: MYDBA_TAF.prod.nl
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: dba
Database instances: MYDBA1,MYDBA2
Configured nodes: Mysrvrr,nlp1e6dr
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed
## As they say in german best way to get things done is doning things right.
oracle@Mysrvrr:/app/oracle/diag/rdbms/local10p/LOCAL10P/trace [MYDBA1]# orapwd file=’+MYDBP_DATA01/MYDBA/orapwdb’ dbuniquename=’MYDBA’
Enter password for SYS:
## in sqlplus
SQL> grant sysdba to system ;
Grant succeeded.
SQL> select * from V$PWFILE_USERS;
USERNAME SYSDB SYSOP SYSAS SYSBA SYSDG SYSKM ACCOUNT_STATUS PASSWORD_PROFILE LAST_LOGIN LOCK_DATE EXPIRY_DATE EXTERNAL_NAME AUTHENTI COM CON_ID -------------------------------------------------------------------------------------------------------------------------------- ----- ----- ----- ----- ----- ----- ------------------------------ ------------------------------------------------------------------------------- SYS TRUE TRUE FALSE FALSE FALSE FALSE OPEN PASSWORD NO 0 SYSTEM TRUE FALSE FALSE FALSE FALSE FALSE OPEN DEFAULT PASSWORD NO 0 AIM_DBA TRUE FALSE FALSE FALSE FALSE FALSE OPEN FUNCTIONAL_USER PASSWORD NO 0
Another small puzzle solved , but as always happy me again when being able to help Colleagues / Customers.
Happy reading,
Mathijs
Installing 12.3 Goldengate in different Group
Summary
Do yo know the feeling that sometimes it is easier to send a message to yourself ( no message to rudy ) then taking notes? Well this is another fine example of that . For project had to install Oracle Goldengate in A Cluster and below steps are the ones that were followed to do that. Extra flavor to this assignment due to the fact that the team that will use Goldengate in Bau ( business as usuall) is not the same team as the team i am part of so in this case it was more or less “thou shall not install as oracle:dba”.
Details:
- As a starting point I have created a Diskgroup in ASM , and have setup an ACFS filesystem on it . By doing so This mount-point ( and also the software) becomes shared / usable on all cluster-nodes.
-
Location of the software: oracle@myStartingHost:/opt/oracle/Odrive/depot/software/oracle/GoldenGate/Linux_64/123012_fbo_ggs_Linux_x64_shiphome.zip
On the target cluster: checked current groups. Requested from Linux team that they would add the group adba to the oracle account i am using. (in this scenario it is required to have adba added to the groups for the Oracle account)
oracle@mysrvr:/app/oracle/GG12/# groups oracle
- Set newgroups to install Goldangate tool in an Other teams owned group. in Linux perform this command:
newgrp adba
- unzip the file
- In oracle@mysrvr:/app/oracle/GG12/fbo_ggs_Linux_x64_shiphome/Disk1, this now shows:
oracle@mysrvr:/app/oracle/GG12/fbo_ggs_Linux_x64_shiphome/Disk1 [MIGABP]# ls -lisa total 24 3276804 4 drwxr-xr-x. 5 oracle adba 4096 Dec 9 15:02 . 3276803 4 drwxr-xr-x. 3 oracle adba 4096 Dec 9 15:02 .. 3276808 4 drwxr-xr-x. 4 oracle adba 4096 Dec 9 15:02 install 3276805 4 drwxrwxr-x. 2 oracle adba 4096 Dec 9 15:02 response 3276807 4 -rwxr-xr-x. 1 oracle adba 918 Dec 9 15:02 runInstaller 3276830 4 drwxr-xr-x. 11 oracle adba 4096 Dec 9 15:02 stage
- ran ./ runInstaller
## Issues i came across since the port that Goldengate likes to claim was already in use:
netstat -tulpn | grep 7809 as root to kill process at reinstall
Setting up Goldengate in this way will make proper entry in the OraInventory.
..Next challenge but that is for different post , to add Goldengate to the Oracle Grid Infrastructure as a resource so the cluster-ware will monitor and respond to this resource should the node restart or if a relocation of the resource would help to lift some burden on that server .. Stay tuned.
Happy reading ,
Mathijs
Some things happened (while installing 12.2 GI in a Rac cluster).
Summary:
One of the fine people in the Oracle community once shared a thought where he told about repairing a bike once and repeating the plan of approach N times. As a lesson learned from that scenario it is best to see each bike as a new challenge which deserves a new and fresh approach. In this blog I will describe a number of things i came across when setting up Grid Infra structure 12.2 with January 2018 PSU on a multi-node cluster.
Details – Things to look after:
- Locating the log files of the installation can make a world of difference. Make sure you understand and find the location of it and have it tailed during all of install. In my case in this directory will find subdirs and log file for example: /app/oraInventory/logs/GridSetupActions2018-04-26_09-39-53AM.
- In the past you always had one destination to unzip your Software.zip and during installation the runInstaller would ask for an installation location during setup. With 12.2 ( and in Oracle 18 Grid infra) that is no longer the case. Create the subdirectory where the software is supposed to be installed and unzip your files there as a first step.
- runInstaller is no more … In order to start the installation process you will have to find this command:./gridSetup.sh
- When Installing as in my case on Red Hat Linux 7.4 with a Patched Kernel you might come across ACFS-9154: Loading ‘oracleoks.ko’ driver. > modprobe: ERROR: could not insert ‘oracleoks’: Unknown symbol in module, or unknown parameter (see dmesg) > ACFS-9109: oracleoks.ko driver failed to load. > ACFS-9178: Return code = USM_FAIL > ACFS-9177: Return from ‘ld usm drvs’ > ACFS-9428: Failed to load ADVM/ACFS drivers. A system reboot is recommended. You can Solve that by running the gridSetup.sh with parameters which will install the Patch(es) first and then run the commands:./gridSetup.sh -applyPSU /app/grid/product/12201/grid/27100009. Translated this means that the psu patch needs to be applied first and then the gridSetup can start its setup.
- Images during setup have changed. In my case I have selected this one. Which also brought me FLEX-ASM as per default in 12.2.
When installing GI as a standalone cluster in the follow-up screens you are asked to add the nodes of your cluster either as a Hub or as a Leave. Thus differentiating by default which nodes should have a dedicated ASM instance ( Hub) and which nodes will communicate remotely with one of the Hub-Asm instances. After install i learned that in 12.2 as a default 3 ASM instances will be created no matter how many nodes there are in your cluster.
- Scan listener: Make sure as a preparation that the colleagues from Linux team have added the 3 Ips for your cluster in the Dns and try a nslookup first before installing. During installation when you have to add the clustername ( here presented as mycluster) , the installing tool will also show the scan-name (and most likely you will have to alter it anyhow to meet with the information in dns needed for the setup).( in dns mycluster-scan.prod.nl) needs to be present as 3 IP addresses
- In the clusternode screen you will add all the nodes in your cluster. In this case I intended to set up each node as a hub ( thus expecting that there would be 8 asm instances in place too ( which was not the case but that is elaborated in other topic.
- On this screen you add the nodes using the add button.
- On this screen you can set up SSH connectivity between all the nodes. On the web it was not clear to me in various blogs and in the documentation neither, what is the preferred way to do this. I had the tool setup ssh connectivity between all nodes and i was happy with result.
- Once completed press next and the tool will show something like “validating node readiness”.
- In the specify Network Interface usage screen:
- Best practice / Lessons learned: Make sure you have consulted with the Linux team about the interfaces. In my specific case > ETH0 is admin lan . You should put it to do not use. Eth2 and Eth7 are the private interconnects. Make sure that only one of the is have the option Private, ASM. (In a flex asm cluster ASM needs a way to communicate via its dedicated listener. Since as per default you will have only ASM listener, make sure only one of the private interconnects is using this combo of private and ASM).
Note: This installation was implemented on New Hardware , coming from Dell. During the install we found out that in the original setup the DELL systems use a Range of Ips ( 169.*) which is also used by the HA-IPS of Oracle. And even when the colleagues of Linux might grumble, it is mandatory that the range of 169* is not in use!! In the first setup the Dell systems had the 169* enabled for their idRac interface. This IPs have been disabled.
- For the setup of two Diskgroups ( one for the OCR and voting disks and one for the GIMR ( grid infra structure Management Repository) make sure that the Linux Admins have delivered ASM disks). In my case i got 2 times 3 Disks , so i could setup Normal redundancy Diskgroups for both Diskgroups.
- On the Summary screen , pay extra attention to make sure that all the cluster nodes that you intend to have included in your soon to be cluster are showing ! ( Hub nodes: this should show all the nodes. If this is not the case you can select Edit ( which will rerun all steps as of Cluster node information).
Well that is all for now .. To be continued in a galaxy near you …
As always happy reading and till we meet again.
Mathijs
Installing 12.2 GI on Red hat Linux 74
Introduction.
From one of the Oracle new features books ( I believe it was 11G ) it was stated that the only constant is change. And with recent experience when installing 12.2 Grid Infrastructure as a fine example I could not agree more. But as always if everything with regard to Oracle would be easy…
For a project new installations need to be done. In this specific case that means that I will work with Red Hat Linux 74 together with Oracle 12.2 both Grid Infrastructure and 12.2 Oracle Rdbms including latest PSU (at the moment January 2018). As mentioned in my other post setup for Grid Infrastructure has changes (no more runInstaller there , and unzipping the Oracle provided Zip in the destination which will hold your software ( so no more staging area).
For one of the platforms it was mandatory to setup 12.2 GI ( and database ) on a single server which is also referred to by Oracle as Oracle restart.
Details.
Now this is where the challenging – even scary part comes in. Once the installation was on its way , at app 80% of the progress the famous second window pops up where you are asked to start a second session , become root and run the script root.sh. Well this is what happened on the way to that theater:
CRS-4664: Node mysrvr successfully pinned.
2018/01/30 09:19:28 CLSRSC-330: Adding Clusterware entries to file ‘oracle-ohasd.service’
2018/01/30 09:21:03 CLSRSC-400: A system reboot is required to continue installing.
The command ‘/app/grid/product/12.2.0.1/grid/perl/bin/perl -I/app/grid/product/12.2.0.1/grid/perl/lib -I/app/grid/product/12.2.0.1/grid/crs/install /app/grid/product/12.2.0.1/grid/crs/install/roothas.pl ‘ execution failed
I have looked at the web and it was almost a relieve that more bloggers already came across this phenomena. As always when reading the scenarios that others have followed to solve the issue , it will always be the question does this apply to your specific situation. It was interesting to identify that this issue is caused by acfs drivers when the kernel used in Red Hat 74 ( and also in 73 ) is higher then an expected (default) one thus causing the root.sh to fail. And as a spoiler alert the suggested reboot , did not help in my specific case. Consulted with The Oracle and this came back:
==============================
> ACFS-9154: Loading ‘oracleoks.ko’ driver.
> modprobe: ERROR: could not insert ‘oracleoks’: Unknown symbol in module, or unknown parameter (see dmesg)
> ACFS-9109: oracleoks.ko driver failed to load.
> ACFS-9178: Return code = USM_FAIL
> ACFS-9177: Return from ‘ld usm drvs’
> ACFS-9428: Failed to load ADVM/ACFS drivers. A system reboot is recommended.
==============================
Work Around – Solution:
Either apply a oneoff patch:
25078431 – is for 7.3.
26247490 – is for 7.4.
or
Even better but also a new approach: Apply January 2018 PSU before installation.
Both Scenarios could be interesting but given the fact that i need the psu 2018 as a baseline anyhow this is how the scenario worked:
1 Installation did not complete, so I did not perform a normal de-install.
oracle@mysrvr:/app/grid/product/12.2.0.1 []# cd grid
oracle@mysrvr:/app/grid/product/12.2.0.1/grid []# rm -rf *
2 Unzipped to /app/grid/product/12.2.0.1/grid.
unzip /app/grid/product/12.2.0.1/linuxx64_12201_grid_home.zip -d /app/grid/product/12.2.0.1/grid
3 Latest version of Opatch needed , make sure you download and have this in place.
unzip /app/grid/product/12.2.0.1/p6880880_122010_Linux-x86-64.zip -d /app/grid/product/12.2.0.1/grid
4 this is January 2018 PSU for GI p27100009_122010_Linux-x86-64.zip.
Unzip /app/grid/product/12.2.0.1/p27100009_122010_Linux-x86-64.zip -d /app/grid/product/12.2.0.1/patch
5 Check for correct Opatch.
oracle@mysrvr:/app/grid/product/12.2.0.1/grid []# cd OPatch
oracle@mysrvr:/app/grid/product/12.2.0.1/grid/OPatch []# opatch version
OPatch Version: 12.2.0.1.11
OPatch succeeded.
6 Time to run this command , it will first patch the software tree then start the setup:
cd ..
./gridSetup.sh -applyPSU /app/grid/product/12.2.0.1/patch/27100009
After some minutes this showed : Successfully applied the patch.
The log can be found at: /app/oraInventory/logs/GridSetupActions2018-01-30_02-34-09PM/installerPatchActions_2018-01-30_02-34-09PM.log
right before the gridSetup started.
Then the grid installer started ….
7 And in a popup screen was asked to have root the run :
time to run root.sh
mysrvr:root:/app/grid/product/12.2.0.1/grid $ ./root.sh
This time a much more positive output came in:
Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /app/grid/product/12.2.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /app/grid/product/12.2.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /app/grid/product/12.2.0.1/grid/crs/install/crsconfig_params The log of current session can be found at: /app/oracle/crsdata/mysrvr/crsconfig/roothas_2018-01-30_03-03-44PM.log LOCAL ADD MODE Creating OCR keys for user 'oracle', privgrp 'dba'.. Operation successful. LOCAL ONLY MODE Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-4664: Node mysrvr successfully pinned. 2018/01/30 15:04:19 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'mysrvr' CRS-2673: Attempting to stop 'ora.evmd' on 'mysrvr' CRS-2677: Stop of 'ora.evmd' on 'mysrvr' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'mysrvr' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. mysrvr 2018/01/30 15:06:13 /app/grid/product/12.2.0.1/grid/cdata/mysrvr/backup_20180130_150613.olr 2960767134 2018/01/30 15:06:14 CLSRSC-327: Successfully configured Oracle Restart for a standalone server
And once again a happy me.
Dedication: would like to dedicate this Post to the colleagues from Oracle Acs support for their great and swift help. Always a pleasure to work together.
As always, happy reading and till we meet again,
Mathijs
Install & Patch 12.2 GI in Oracle Restart
Introduction:
Where would we be without challenges. I have become team member of a project team for a new billing environment and this team is aiming to use ( and go live) with Oracle 12.2 Grind Infra structure and Database version). The information of this article will become a baseline for the installation of several Oracle environments on Linux. Oracle is referring to this as Oracle Restart. Next in line after that (and I love it) will be Real application clusters to be set up.
General Preparations 12.2 Grid Kata:
## Identifying ORACLE_BASE and layout of Grid Infrastructure.
echo $ORACLE_BASE /app/oracle echo $ORACLE_HOME /app/grid/product/12.2.0.1/grid ## Identifying ORACLE_BASE and Db software echo $ORACLE_BASE /app/oracle echo $ORACLE_HOME /app/oracle/product/12.2.0.1/db
## So for the 12.2 layout which is in scope for the actions on a Restart or Rac environment:
+ASM1 /app/grid/product/12.2.0.1/grid CRS /app/grid/product/12.2.0.1/grid -MGMTDB /app/grid/product/12.2.0.1/grid MYDB /app/oracle/product/12.2.0.1/db
## Checking Red Hat release:
oracle@mysrvr1hr:/dev/mapper []# cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.9 (Santiago)
## Oracle restart installation for 12.2 instructions to be found:
Interesting point is that in 12.2 the famous, well known runInstaller is replaced by ./gridSetup.sh (when opening runInstaller even ran into errors (oui-10133 wen running runInstaller in 12.2). Second point of interest will be that you have to pre-create the directory where the software will be running.
## Preparations for Installation:
- On the server where you will install the Grid infrastructure create the directory where you want to install the software (the location you will later on call Your ORACLE_HOME). On the source server and in my specific case that meant that: I had to do mkdir -p /app/grid/product/12.2.0.1/grid on the source server.
- From Solltau: oracle@myhost:/opt/oracle/Odrive/depot/software/oracle/12c/GI []# scp linuxx64_12201_grid_home.zip oracle@mysrvr1hr:/app/grid/product/12.2.0.1/grid
- UNSET your environment variables if any on the installation box:
unset ORACLE_BASE unset ORACLE_HOME unset GI_HOME unset ORA_CRS_HOME unset TNS_ADMIN unset ORACLE_SID unset ORA_NLS10
echo $ORACLE_BASE etc.
- ## Check zip file in the destination that will also become your ORACLE_HOME for this install.
oracle@mysrvr1hr:/app/grid/product/12.2.0.1/grid []# ls -ltr total 2924504 -rw-r--r--. 1 oracle dba 2994687209 Jan 3 16:28 linuxx64_12201_grid_home.zip
Make sure you use the future ORACLE_HOME destination to unzip the file. This is mandatory because compared to previous Installations where you can alter the Software installed directory, during the 12.2 installation there will NOT be an option to choose the destination where to setup this installation. ## make sure you are in the directory /app/grid/product/12.2.0.1/grid and extract the zip!!
- ##Once the Zip file is extracted fire below script:
./gridSetup.sh
In one of the following detailed screens make sure you change “change directory path” to discover the disks that you will be using for this installation. In my case this meant that the Linux Admin colleague has set up – has labelled dedicated Luns (disks) as a preparation to my actions.
root # ls -lH /dev/mapper/ASM_*
brw-rw----. 1 oracle dba 253, 6 Dec 22 16:01 /dev/mapper/ASM_ACFS_035_001 brw-rw----. 1 oracle dba 253, 33 Dec 22 16:01 /dev/mapper/ASM_OCRM_008_001 brw-rw----. 1 oracle dba 253, 34 Dec 22 16:01 /dev/mapper/ASM_OCRM_008_002 brw-rw----. 1 oracle dba 253, 25 Dec 22 16:01 /dev/mapper/ASM_VOTE_008_001 brw-rw----. 1 oracle dba 253, 26 Dec 22 16:01 /dev/mapper/ASM_VOTE_008_002 brw-rw----. 1 oracle dba 253, 30 Dec 22 16:01 /dev/mapper/ASM_VOTE_008_003
## Once you have made all the selections needed below screen will appear:
Once you selected install you will be updated by next progress screen:
## In a separate screen as the Root user ./root.sh has to run which will show:
mysrvr1hr:root:/app/grid/product/12.2.0.1/grid # ./root.sh
Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /app/grid/product/12.2.0.1/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /app/grid/product/12.2.0.1/grid/crs/install/crsconfig_params The log of current session can be found at: /app/oracle/crsdata/mysrvr1hr/crsconfig/roothas_2018-01-03_05-02-27PM.log ## logging details of root.sh : LOCAL ADD MODE Creating OCR keys for user 'oracle', privgrp 'dba'.. Operation successful. LOCAL ONLY MODE Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-4664: Node mysrvr1hr successfully pinned. 2018/01/03 17:02:50 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf' CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'mysrvr1hr' CRS-2673: Attempting to stop 'ora.evmd' on 'mysrvr1hr' CRS-2677: Stop of 'ora.evmd' on 'mysrvr1hr' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'mysrvr1hr' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. mysrvr1hr 2018/01/03 17:03:44 /app/grid/product/12.2.0.1/grid/cdata/mysrvr1hr/backup_20180103_170344.olr 0 2018/01/03 17:03:49 CLSRSC-327: Successfully configured Oracle Restart for a standalone server mysrvr1hr:root:/app/grid/product/12.2.0.1/grid #
Resolving possible issues: 12.2 GI standalone : [INS-20802] Automatic Storage Management Configuration Assistant failed (Doc ID 2277224.1)
## The Installation will create the ASM instance with a default spfile. Due to company standards and due to best practice (knowing that size does matter and default settings will not do well in a heavily used environment) you should connect to the ASM instance and alter below values:
## Specific setup for asm Instance
ALTER SYSTEM SET memory_max_target=4096M SCOPE=SPFILE; ALTER SYSTEM SET memory_target=1536M SCOPE=SPFILE; ALTER SYSTEM SET large_pool_size=100M SCOPE=SPFILE; ALTER SYSTEM SET shared_pool_size=512M SCOPE=BOTH; ALTER SYSTEM SET shared_pool_reserved_size=100M SCOPE=SPFILE; ## Nothing to do with performance but mandatory due to Standards. ALTER SYSTEM SET audit_file_dest='/app/oracle/+ASM/admin/adump' SCOPE=SPFILE; ALTER SYSTEM SET background_dump_dest='/app/oracle/diag/asm/+asm/+ASM/trace' SCOPE=BOTH;
## Company Standards as with regard to listener:
- Log destination: /app/oracle/diag/tnslsnr/mysrvr1hr/listener
- One listener per vip
## So I have added a listener with the netca tool running from the GridInfrastructurehome.
## /app/oracle/diag/tnslsnr/<servername>/<listenername>/trace
oracle@mysrvr1hr:/app/grid/product/12.2.0.1/grid/network/admin [+ASM]# lsnrctl status LISTENER_MYSRVR1HR
## Deinstallation when needed. As always you might need a way out (back again) .
Note: For upgrades from previous releases, if you want to uninstall the previous release Grid home, then perform the following steps:
- Log in as the root user.
- Manually change the permissions of the previous release Grid home (see below).
- Run the /app/grid/product/12.2.0.1/grid/deinstall/deinstall command (as oracle User).
For example, on Grid Infrastructure for a standalone server: # chown -R oracle:dba /app/grid/product/12.2.0.1 # chmod -R 775 /app/grid/product/12.2.0.1 In this example: /u01/app/oracle/product/11.2.0/grid is the previous release Oracle Grid Infrastructure for a standalone server home oracle is the Oracle Grid Infrastructure installation owner user dba is the name of the Oracle Inventory group (OINSTALL group) For example, on Oracle Database: # chown -R oracle:dba /app/oracle/product/12.2.0.1 # chmod -R 775 /app/oracle/product/12.2.0.1
If all is well Time to start Patching the Environment !
## patching : GI : p26737266_122010_Linux-x86-64.zip
## oracle@soltau2:/opt/oracle/Odrive/depot/software/oracle/patches/Linuxx86 []# scp p26737266_122010_Linux-x86-64.zip oracle@mysrvr1hr:/app/grid/product/12.2.0.1/stage
## check current situation with opatch before patching.
opatch lsinventory -detail -oh /app/grid/product/12.2.0.1/grid
This shows:
Oracle Interim Patch Installer version 12.2.0.1.6 Copyright (c) 2018, Oracle Corporation. All rights reserved. Oracle Home : /app/grid/product/12.2.0.1/grid Central Inventory : /app/oraInventory from : /app/grid/product/12.2.0.1/grid/oraInst.loc OPatch version : 12.2.0.1.6 OUI version : 12.2.0.1.4 Log file location : /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatch/opatch2018-01-05_14-39-27PM_1.log Lsinventory Output file location : /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2018-01-05_14-39-27PM.txt -------------------------------------------------------------------------------- Local Machine Information:: Hostname: mysrvr1hr.mydomain ARU platform id: 226 ARU platform description:: Linux x86-64 Installed Top-level Products (1): Oracle Grid Infrastructure 12c 12.2.0.1.0 There are 1 products installed in this Oracle Home. Installed Products (99): etc. . . There are 99 products installed in this Oracle Home. There are no Interim patches installed in this Oracle Home. -------------------------------------------------------------------------------- OPatch succeeded.
## Use opatch to check for conflicts:
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /app/grid/product/12.2.0.1/stage/26737266/26710464 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /app/grid/product/12.2.0.1/stage/26737266/26925644 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /app/grid/product/12.2.0.1/stage/26737266/26737232 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /app/grid/product/12.2.0.1/stage/26737266/26839277 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /app/grid/product/12.2.0.1/stage/26737266/26928563
## This did not show any conflicts
## Next step will be : use opatch check for space requirements (you would not want to end up installation failing due to lacking storage:
For Grid Infrastructure Home, as home user:
Create file /tmp/patch_list_gihome.txt with the following content: cat /tmp/patch_list_gihome.txt3. /app/grid/product/12.2.0.1/stage/26737266/269285634. /app/grid/product/12.2.0.1/stage/26737266/268392775. /app/grid/product/12.2.0.1/stage/26737266/267372326. /app/grid/product/12.2.0.1/stage/26737266/269256447. /app/grid/product/12.2.0.1/stage 26737266/26710464
Run the opatch command to check if enough free space is available in the Grid Infrastructure Home:
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt
## this shows
oracle@mysrvr1hr:/app/grid/product/12.2.0.1/stage [+ASM]# $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt Oracle Interim Patch Installer version 12.2.0.1.6 Copyright (c) 2018, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /app/grid/product/12.2.0.1/grid Central Inventory : /app/oraInventory from : /app/grid/product/12.2.0.1/grid/oraInst.loc OPatch version : 12.2.0.1.6 OUI version : 12.2.0.1.4 Log file location : /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatch/opatch2018-01-05_14-55-06PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
## To patch only the GI home:
# opatchauto apply /app/grid/product/12.2.0.1/stage/26737266 -oh /app/grid/product/12.2.0.1/grid
## failed with:
OPATCHAUTO-72046: Invalid wallet parameters. OPATCHAUTO-72046: The wallet path or wallet password provided is not valid. OPATCHAUTO-72046: Please provide valid wallet information. opatchauto bootstrapping failed with error code 46.
## Thank you Mos for elaborating.
| OPATCHAUTO-72046: Invalid wallet parameters (Doc ID 2150070.1) |
opatchauto command is not being run as root user. Opatchauto for Grid PSUs should always be run as root user.
## So as the root user :
/app/grid/product/12.2.0.1/grid/OPatch/opatchauto apply /app/grid/product/12.2.0.1/stage/26737266 -oh /app/grid/product/12.2.0.1/grid
## and it failed again !!!
mysrvr1hr:root:/root # /app/grid/product/12.2.0.1/grid/OPatch/opatchauto apply /app/grid/product/12.2.0.1/stage/26737266 -oh /app/grid/product/12.2.0.1/grid System initialization log file is /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchautodb/systemconfig2018-01-05_03-09-09PM.log. Session log file is /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/opatchauto2018-01-05_03-09-12PM.log The id for this session is 5LQ1 [init:init] Executing OPatchAutoBinaryAction action on home /app/grid/product/12.2.0.1/grid Executing OPatch prereq operations to verify patch applicability on SIHA Home........ [init:init] OPatchAutoBinaryAction action completed on home /app/grid/product/12.2.0.1/grid with failure Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures: Patch Target : mysrvr1hr->/app/grid/product/12.2.0.1/grid Type[siha] Details: [ ---------------------------Patching Failed--------------------------------- Command execution failed during patching in home: /app/grid/product/12.2.0.1/grid, host: mysrvr1hr. Command failed: /app/grid/product/12.2.0.1/grid/OPatch/opatchauto apply /app/grid/product/12.2.0.1/stage/26737266 -oh /app/grid/product/12.2.0.1/grid -target_type has -binary -invPtrLoc /app/grid/product/12.2.0.1/grid/oraInst.loc -persistresult /app/grid/product/12.2.0.1/grid/OPatch/auto/dbsessioninfo/sessionresult_analyze_mysrvr1hr_siha.ser -analyze -online Command failure output: ==Following patches FAILED in analysis for apply: Patch: /app/grid/product/12.2.0.1/stage/26737266/26925644 Log: /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-05_15-09-16PM_1.log Reason: Failed during Analysis: CheckNApplyReport Failed, [ Prerequisite Status: FAILED, Prerequisite output: The details are: Prerequisite check "CheckMinimumOPatchVersion" failed.] Failed during Analysis: CheckMinimumOPatchVersion Failed, [ Prerequisite Status: FAILED, Prerequisite output: The details are: The OPatch being used has version 12.2.0.1.6 while the following patch(es) require higher versions: Patch 26710464 requires OPatch version 12.2.0.1.7. Please download latest OPatch from My Orac ... etc. . . OPatchAuto failed. opatchauto failed with error code 42 mysrvr1hr:root:/root #
## So I downloaded latest opatch version and parked it in a temporary directory on that targeted server:
unzip p6880880_122011_Linux-x86-64.zip -d /app/grid/product/12.2.0.1/grid
## now Opatch shows:
oracle@mysrvr1hr:/app/grid/product/12.2.0.1/opatch [+ASM]# opatch version OPatch Version: 12.2.0.1.11 OPatch succeeded.
## Sometimes you just have to be patient to hear the lambs being silent:
## Next run as the root user :
/app/grid/product/12.2.0.1/grid/OPatch/opatchauto apply /app/grid/product/12.2.0.1/stage/26737266 -oh /app/grid/product/12.2.0.1/grid
## logfiles:
oracle@mysrvr1hr:/app/oracle/crsdata/mysrvr1hr/crsconfig -rw-rw----. 1 oracle dba 17364 Jan 5 15:35 hapatch_2018-01-05_03-34-42PM.log -rw-rw----. 1 oracle dba 23725 Jan 5 15:42 hapatch_2018-01-05_03-42-41PM.log ##showed mysrvr1hr:root:/root # /app/grid/product/12.2.0.1/grid/OPatch/opatchauto apply /app/grid/product/12.2.0.1/stage/26737266 -oh /app/grid/product/12.2.0.1/grid OPatchauto session is initiated at Fri Jan 5 15:33:54 2018 System initialization log file is /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchautodb/systemconfig2018-01-05_03-33-58PM.log. Session log file is /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/opatchauto2018-01-05_03-34-02PM.log The id for this session is XLE2 Executing OPatch prereq operations to verify patch applicability on home /app/grid/product/12.2.0.1/grid Patch applicability verified successfully on home /app/grid/product/12.2.0.1/grid Bringing down CRS service on home /app/grid/product/12.2.0.1/grid Prepatch operation log file location: /app/oracle/crsdata/mysrvr1hr/crsconfig/hapatch_2018-01-05_03-34-42PM.log CRS service brought down successfully on home /app/grid/product/12.2.0.1/grid Start applying binary patch on home /app/grid/product/12.2.0.1/grid Binary patch applied successfully on home /app/grid/product/12.2.0.1/grid Starting CRS service on home /app/grid/product/12.2.0.1/grid Postpatch operation log file location: /app/oracle/crsdata/mysrvr1hr/crsconfig/hapatch_2018-01-05_03-42-41PM.log CRS service started successfully on home /app/grid/product/12.2.0.1/grid OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:mysrvr1hr SIHA Home:/app/grid/product/12.2.0.1/grid Summary: ==Following patches were SUCCESSFULLY applied: Patch: /app/grid/product/12.2.0.1/stage/26737266/26710464 Log: /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-05_15-35-04PM_1.log Patch: /app/grid/product/12.2.0.1/stage/26737266/26737232 Log: /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-05_15-35-04PM_1.log Patch: /app/grid/product/12.2.0.1/stage/26737266/26839277 Log: /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-05_15-35-04PM_1.log Patch: /app/grid/product/12.2.0.1/stage/26737266/26925644 Log: /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-05_15-35-04PM_1.log Patch: /app/grid/product/12.2.0.1/stage/26737266/26928563 Log: /app/grid/product/12.2.0.1/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-05_15-35-04PM_1.log OPatchauto session completed at Fri Jan 5 15:43:05 2018 Time taken to complete the session 9 minutes, 11 seconds
Happy Dba , Installed 12.2 GI and Patched it with RU October 2017.
Thanks for reading and till we meet again,
Mathijs.
Mathijs Bruggink 