This week i have patched a preproduction environment with latest PSU patch at this moment ( October 2012). Next week will do the same on the production machines In it self this action is not a big issue when applying it to the Grid infra structure and RDBMS, but ACFS is in use on those boxes. Since I hope this will make an interesting note, I have gathered my steps and want to share them with you.
This weeks challenge has become to set up ACFS on two new to be implemented Real Application Clusters. As with any tooling first question will be , what is your goal if you want to start using it. My purpose is to start using ACFS as a shared mountpoint between the servers, nodes in the cluster in order to have each instance do its logging (ADR), audit and listener log on that mount.
This blog will tell you about my setting it up, the issues i came across and the way it is implemented in a working way.
To share the plot of the story: After implementing I had the issue that right after cluster reboot all services where launched automatiocally BUT the instances would NOT start automatically. In the end i ended up with ACFS being defined as a shared home for an Oracle RDBMS installation. And it worked/works after cluster reboot cause all resources are happy and online now.
For a billing environment I have been asked to change the current Real application solution to a HA solution with an Active – Passive Database in the Cluster ware of Oracle (Grid Infrastructure) (11.2.0.2.0). Even though I have had / seen examples of this working in 11gr1 the fact we use 11Gr2 allowed us to skip steps needed in the 11gr1 environment (such as a dedicated virtual Ip address, able to fail over and a listener that has to come along with that as well). This solution is based on the fact that in this project we have set up scan listener(s) and also that we connect via a Service (taf). Also it should be said that the parameters for the cluster ware have changed again between Oracle 11gr1 and 11gr2. So it was a nice challenge to see, adapt and solve the issues
It is my estimate that a maintenance window of app 1 Hour per Database will be needed, doing the analyses, the changes, the removal and adding it back to the cluster and do relocate tests.
Ps i have marked specific cluster commands in yellow.
Happy Reading.
Always a challenge patching a Real application cluster. My assignment, goal in this is to bring Oracle Clusterware ( 11.1.0.7.0) (without any PSU patches installed) to 11.2.0.3.0. Grid Infrastructure. This will be performed in two steps , in two maintenance windows in order to minimize the change of breaching SLA of maintenance windows.
Happy reading ,
Mathijs
This week I have had the challenge to finally install the Grid infrastructure Software 11.2.0.3.0 on a 4 Node production Real application cluster on Linux. I had followed all the guidelines and came well prepared because more than a month ago I had done the same on the preproduction cluster.
After building the complete Infra structure for Oracle (Grid Infra with Asm) and Rdbms in 11.2.0.2.0. version this Database Administrator is now facing a new challenge made by his Project leader. Please investigate the consequences of a Operating Upgrade from Red Hat Linux 5 ( 5.6) to (5.7) and change your Oracle environment to adapt to the changed Operating system. As they say often say in the Netherlands the Customer is King so this Admin was on his way. And of course set up a plan in which this will be done one step at the time , starting with a Test box. Because even with Kings the Dba should consider a Quote of one of the Oracle gurus (Arup Nanda : What’s more important guts or caution?” I would say pragmatism, bordering on caution; not guts. And i do believe the Guru is right in this matter.
Happy reading,
Mathijs
For a customer I had the privilege to investigate and implement a Logical standby Database. Primary side was a 6 Node RAC and the Standby side was scheduled to become a two Node RAC. The standby side was not meant as a Fail over solution but had to serve the purpose of an almost on-line Reporting environment to the customers ( in such a way also separating them from the production database). The requirements became quite a challenge in more than one aspect giving the facts that the project wanted to go live with it in the very next 2 weeks, that this was unknown territory for the complete team hmm and given the fact that the decision to change from a physical standby database to a logical one was one of the very last minute. I forgot who said it, but with respect I would like to quote the unknown creator of this following thought: .. but if things would always be easy you and I would be out of a job..
This week I came across the fact that the readme files for Applying Psu patches is often a puzzle to people. A PATCH action failed due to the fact that the –OH was NOT used, and in a Rac environment that is always something extra scary with dependencies. So I thought let us have some clearer picture with regard to this matter. Today i had to patch Oracle Restart environment with Grid Infrastructure 112030 and ditto Rdbms.
In order to make a good reminder for myself and for the community, well here are the steps that I performed. I have added some comments where needed.
Introduction
I have been asked to explore a scenario in which a database that is currently on a file systems is recovered with RMAN into ASM. In this web log the steps i have performed are shared.
A single instance/Db 11gr2 under control of 11gr2 grid infrastructure with ASM as storage solution. In ASM the disks are defined as external redundancy due to business rules ( no shooting the messenger please). I was Informed that a database had crashed . Grid Infra structure ( nor manual actions ) were able to start the database again. So I was asked please go take look and fix asap . The good thing was that I was aware of maintenance in the ASM Disk group on that Tuesday September 4th. So when alarms started to go off I had first clues (even Sherlock Holmes needs a first clue right ?). As always the alert log was first place to check for me. This document follows a number of steps during the analyses and the end status which was a proper database open (without reset logs). Bottom-line is give not up too fast. BUT also in this specific scenario it had been better if we had kicked the disk and started from scratch instead of trying to repair. But that being said I know that sounds wise when looking back ….
Mathijs Bruggink 