RE: CPU April 2005 patching madness

  • From: "Srinivasan, Vasan (RBS Insurance)" <Vasan.Srinivasan@xxxxxxxxxxxxx>
  • To: "'jkstill@xxxxxxxxx'" <jkstill@xxxxxxxxx>, 'Oracle-L Freelists' <oracle-l@xxxxxxxxxxxxx>
  • Date: Tue, 24 May 2005 11:05:54 +0100

Hi! Jared,

        How about editing the spfile in Notepad/Wordpad, removing the first
and last squiggly lines and changing the parameters such as remote_pass...
and writing a new pfile. Then start the database using the new pfile, remove
the spfile and write a new spfile. I had to do this when I was experimenting
with NLS and could not restart the database.


Vasan (x5707) 
Mailpoint 28
Vasan Srinivasan                                   * 020 8313 5707
Infrastructure Service Manager              * 020 8313 5646
Oracle Technologies
Churchill Insurance, IT Department
Purple Floor, Phase 1, Churchill Court
1 Westmoreland Road,
Bromley, Kent, BR1 1DP.
* Vasan.Srinivasan@xxxxxxxxxxxxx
Mobile * 07710 154 987
Views Presented here are not necessarily the views
                          of my Employer

-----Original Message-----
From: Jared Still [mailto:jkstill@xxxxxxxxx] 
Sent: 23 May 2005 19:30
To: Oracle-L Freelists
Subject: CPU April 2005 patching madness

Boy, do I love patching.
Especially Oracle.

Especially Oracle on Windows.

You could refer to it as the Full Employment for DBA's Under Pressure act.

Having made the determination to patch the
<>databases on a
new server that is supposed to go live at the end of this week ( OK, it
was determined for me. I didn't really want to) I began the patching 
task Friday Evening, having first practiced it on a test server.

There are three Oracle Homes on this Win23k Server, and three
Oracle databases.

Let's call them dev, qa and prd for simplicity.

Applying the patch on dev and qa went fairly smoothly.

Opatch apply worked on both, run catcpu.sql - no problems.
The only errors were ORA-942's for already existing objects.

Both databases and databases services were stopped and
restarted without incident. The importance of this point will
become apparent later.

At this time I started the patch process for the PRD database
and ran into difficulty. 

The patch would not succeed due to some files being in use.
I could not find any Oracle processes running, so I put them
all on manual start and rebooted the server.

Normally, no big deal.

But, the server did not come back up properly. 

I could ping it.

I could connect to it with the Windows Management Console (t).

But there were several services down that could not be started.

I could not connect to the box with RDP, LanDesk or PC Anywhere.


So, I called my friendly neighborhood SA and he kindly rebooted
the box for me.

This is where the fun starts.

Recall that the QA and DEV databases were successfully stopped
and started prior to the reboot.

Now they will not come up.

Assume the DEV database.

Errors are:

ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file 

What is interesting about this is that the init.ora file location
listed with the error message is incorrect.

I checked the registry. All correct.
I recreated the instance via ORADIM - no joy.

These databases are all using spfile's. 
So I tried starting up the database using a pfile.

SQL> startup pfile="d:\oracleora92\admin\dev\pfile\initdev.ora"

Now there is an error opening the password file:

ORA-1990 - error opening password file.

The password file is D:\oracle\ora92\dev\database\pwddev.ora

The error message for the ORA-1990 would be something like this:
" Could not open password file D:\oracle\ora92\prd\database\pwddev.ora"

The file name is correct, but again, the location is incorrect.

Only by setting remote_login_passwordfile=none was I able to start the 

Both the QA and DEV database must be now started with a script that 
names the pfile, and there can be no password file.

Trying to create a new spfile is no good:

SQL> create spfile='spfile_from_pfile.ora' from pfile;
create spfile='spfile_from_pfile.ora' from pfile
ERROR at line 1:
ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file


Again, the location is wrong.

All registry entries are correct. 

All environment variables are correct.
PATH contains Oracle paths from only one ORACLE_HOME, DEV.

I have opened a TAR on this, but nothing useful has yet come of it.

This server now has one database that operates correctly, the unpatched one.

At this point, I don't know if it is Oracle, Windows, Dell HW or the phase 
of the moon.

If you have any ideas on this, don't hesitate to share them.

If I'm lucky, I'm just overlooking something glaringly obvious and easy to 
fix. :)

Thanks for reading this far.

Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist



 Churchill Insurance Company Limited. Registered in England No 2258947.
Registered office: Churchill Court, Westmoreland Road, Bromley, Kent BR1
1DP.  Authorised and regulated by the Financial Services Authority. Member
of The Royal Bank of Scotland Group. 
 This e-mail message is confidential and for use by the addressee only. If
the message is received by anyone other than the addressee, please return
the message to the sender by replying to it and then delete the message from
your computer. Internet e-mails are not necessarily secure. Churchill
Insurance Company Limited does not accept responsibility for changes made to
this message after it was sent. Whilst all reasonable care has been taken to
avoid the transmission of viruses, it is the responsibility of the recipient
to ensure that the onward transmission, opening or use of this message and
any attachments will not adversely affect its systems or data. No
responsibility is accepted by Churchill Insurance Company Limited in this
regard and the recipient should carry out such virus and other checks as it
considers appropriate.

Other related posts: