Steps to Recreate my Central Inventory in a Real Applications Cluster environment on Linux

Introduction:

In the last quarter of 2012 i have set up  a 4 Node Real Application Cluster on Linux which was great to do as always. In day-to-day business I had lost track of this environment  which in itself is not that bad of course. However this week I needed the environment as a preparation for patching so it would be mandatory to check with opatch last patch installed in the grid infra structure and in the rdbms homes that have been set up on those boxes. Well it was to be expected that other dbas would work on those boxes, maybe / likely might have installed new patches so it was really like getting back together with old friends to see what was under the hood.

Well it was a bit of disappointment that the inventory destination  (/opt/oracle/orInventory) was simply empty . Opatch did not like it either cause it  exited with a returncode 73. But then again on the bright side this meant i could take note of the necessary steps  performed  and turn it into a blog.

Continue reading

Rman Catalog Crosscheck and Delete obsolete an Approach

Introduction

When You are or will be working with Rman  as your backup solution you will have to ask yourself several questions . One of the elementary ones will be that you have to consider whether or not you will be utilizing an Rman Catalog or that  you will have faith in the existing  multiplexed controlfiles.  On the web I see debates going on about the use of a catalog and indeed given the idea that you will not need a catalog , since you are not be using stored scripts, you might get timeouts simply cause working with catalog shows long time no communication ( and you have those hmm lets say aggressive  firewalls in place that will snap at that so-it-seems idle sessions) etc. Well this post is about what you should do if you decide to have a catalog  for rman after all or if a catalog is in place due to company hmm  history – habits – or standards.

What  you should consider to have in place ( of course always  depends on your settings and configuration) Should be  a maintenance job for your Rman Catalog for the specific Database ( target)  you have added there.  With some explain below I would like to show you what was done for that maintenance part  by showing scripts we used for that.

Details:

#!/bin/ksh
#set -vx
PROG=$(basename $0)

if [ -z “$1” ]; then
echo “ERROR – wrong start commando”
echo “EXAMPLE – $PROG <SID>”
exit 1
fi

## Script will take the Instance that has been registered as an input parameter
export ORACLE_SID=$1

## Finding the ORATAB since it will help to set Oracle Home
if [ “${OS}” = “Linux” ];then
export ORATAB=/var/opt/oracle/oratab
else
export ORATAB=/etc/oratab
fi
ORATABLINE=`grep $ORACLE_SID $ORATAB`
export ORACLE_HOME=`echo $ORATABLINE | cut -f2 -d:`

## exporting several Parameters among which logging path and logfilename with a timestamp in it
export LOGPATH=/opt/oracle/scripts/log
export CURRENT_TIMESTAMP=`date +%Y%m%d_%H%M%S`   # Format: YYYYMMDD_HHMISS   e.g.: 20110907_150455
export LOGFILE=${ORACLE_SID}_crosscheck_${CURRENT_TIMESTAMP}.log
export NLS_DATE_FORMAT=”DD-MON-YYYY HH24:MI:SS”
export ORA_NLS32=${ORACLE_HOME}/ocommon/nls/admin/data
export ORA_NLS33=${ORACLE_HOME}/ocommon/nls/admin/data
export NLS_LANG=American_America.WE8ISO8859P15

## Connect to rman and your catalog  and start logging (msglog)  to the specified logdir and logfile.
## then allocate a channel. Note if you make backups to TAPE as we do you HAVE to allocate a channel for maintenance type SBT_TAPE and you HAVE to mention in our case specific Networker Server and Networker Client.
## then you perform the crosscheck backup which will check in the Networker database if the Backups is still available. If it is not the backup will be tagged expired in the Rman catalog.
## Once the  crosscheck is finished we wanted to see a result so did a list backup.
## If you are sure that what you see is correct  delete noprompt expired will erase the no longer existing backups in rman.. MAYBE best to run script first without this. Always best to be safe then sorry.
## After delete another Listing is performed as a record of the post  delete information in the catalog.

${ORACLE_HOME}/bin/rman TARGET  / RCVCAT rman_${ORACLE_SID}/${ORACLE_SID}@RMAN MSGLOG ${LOGPATH}\/${LOGFILE} << EOF
allocate channel for maintenance device type ‘SBT_TAPE’;
send ‘NSR_ENV=(NSR_SERVER=adebckus-nl,NSR_CLIENT=adesnouh)’;
crosscheck backup;
list backup of database summary;
delete noprompt expired backup;
list backup of database summary;
release channel;
exit
EOF

Happy Reading and best  of Luck,

Mathijs

When Starting a Rac database brings you: ORA-29760: instance_number parameter not specified.

Introduction

Recently was supporting a colleague who was unable to start a Real Application Database after he had  created the environment by manual scripts. As i already mentioned in another post nothing wrong with exploring both methods ( either using DBCA (with care but hey any tool will come with a manual and  some shortcomings) or manual. Well if you decide to work manually  of course it will take maybe even  more discipline in setting up init.ora files , registering the database and its instances into the Grid Infra structure layer etc.  But generally speaking there is a challenge in either approach you choose. Well on to our case.  The Database would not start via the clusterware (with srvctl ) but manually  it was possible to start either of the two instances in this Rac.

Investigation:

Well first glimpse to see what might be going on:

SQL> select * from gv$instance;

INST_ID INSTANCE_NUMBER INSTANCE_NAME HOST_NAME VERSION STARTUP_TIME STATUS PAR THREAD# ARCHIVE LOG_SWITCH_WAIT
———- ————— —————- —————————————————————- —————– ——————- ———— — ———- ——- —————
LOGINS SHU DATABASE_STATUS INSTANCE_ROLE ACTIVE_ST BLO
———- — —————– —————— ——— —
1 1 MYDB1 server1hr 11.2.0.3.0 11.04.2013 16:14:31 OPEN YES 1 STARTED
ALLOWED NO ACTIVE PRIMARY_INSTANCE NORMAL NO

2 2 MYDB2 server2hr 11.2.0.3.0 11.04.2013 16:25:54 OPEN YES 2 STARTED
ALLOWED NO ACTIVE PRIMARY_INSTANCE NORMAL NO

That query  showed both instances running. 

Then the colleague tried:

srvctl start database -d  … 

PRCR-1079 : Failed to start resource ora.mydb.db

CRS-5017: The resource action “ora.mydb.db start” encountered the following error:
ORA-29760: instance_number parameter not specified
. For details refer to “(:CLSN00107:)” in “/opt/crs/product/11203_ee_64/crs/log/server1hr/agent/crsd/oraagent_oracle/oraagent_oracle.log”.

CRS-2674: Start of ‘ora.mydb.db’ on ‘server1hr’ failed
CRS-2632: There are no more servers to try to place resource ‘ora.mydb.db’ on that would satisfy its placement policy
CRS-5017: The resource action “ora.mydb.db start” encountered the following error:
ORA-29760: instance_number parameter not specified
. For details refer to “(:CLSN00107:)” in “/opt/crs/product/11203_ee_64/crs/log/server2hr/agent/crsd/oraagent_oracle/oraagent_oracle.log”.

CRS-2674: Start of ‘ora.mydb.db’ on ‘server2hr’ failed

Well then it was time to Google  for ORA-29760: instance_number parameter not specified

A Fellow blogger  mentioned the following Mos Note to solve this issue. And it once again proved that if you really settle for doing manual setup of you Rac environment you have to be well prepared and consistent in the naming of your parameters in you configuration file ( init.ora as a base for an spfile) and the way you enter services in the cluster layer. It once again proved that  names are registered in the way you enter them.  In plain English it is Case sensitive so  the old saying : what you type is what you get is very applicable.

MOS note ‘ORA-29760: instance_number parameter not specified’ When Starting the Database with Srvctl [ID 749515.1]

Happy reading and as always best of luck.

Mathijs.