Re: corrupt datafile

Re: corrupt datafileTim,

Well said on all points.

1. They (or at least we) would have logged a TAR if they had renewed it. :-(
2. The whole hot backup from Wed isn't available, just the copy of the 
now-corrupted file.  It seems that a current backup overwrites the previous 
backup, and the only reason the Wed copy of the corrupted file exists is that 
it couldn't be copied (due to the corruption) during the backup. :-(
3. The reason that there aren't copies going back is that they couldn't spend 
money on disk space.  As they tell me this I glance at the 120GB Iomega drive I 
use to backup my notebook.  People gotta realize that they don't need Sun 
arrays to keep some backup files around.
4. You're also probably right about the export.  I haven't played with that 
yet, though I have told them it's likely to be the only hope.
5. You're right about them blaming themselves.  We got called in to see if we 
could salvage anything.  It's great to pick up new clients, and it's really 
great if we can be heroes, but then it's depressing if you can't bail them out. 
 Even if it's their fault, you feel bad for them.

Hopefully this will help someone else convince their management/clients of the 
importance of thorough backup strategies.  They not necessarily expensive, and 
when they're not used the expense can be astronomical.

Cheers,

Terry
  Terry,

  Hopefully they have logged a TAR with Oracle Support and are working it that 
way.  It is possible that "dbverify" is returning the response that every block 
is corrupt simply because it cannot read the file-header information, not 
because the entire file is shredded.  This file might have a few bytes out of 
place in the header, in which case it is possible that Oracle Support's Data 
UnLoader (DUL) utility might be able to dump the data from the file.  
Otherwise, if every single block is corrupt, then even DUL probably would not 
be able to retrieve the data.

  A "hot backup" from Wednesday is available but the archivelogs going back to 
that backup aren't?  Other datafile backups from the "hot backup" on Wednesday 
are not available?  It seems that many, many things have gone wrong here.  It 
would be worthwhile to dwell on why the other datafiles and the archivelogs are 
not available, so that this never happens again.  This is a time for brutal 
honesty.

  From what you describe, there are no options.  This database is kaput and 
complete recovery is impossible.  Only two "incomplete recovery" options 
remain:  use DUL to pull data from the dead datafiles or use the 6-month-old 
export.  And given the trail of mistakes, that export was probably not taken 
with CONSISTENT=Y or the database was not fully quiesced, meaning that it is 
probably almost useless too.  So, DUL is probably the only possibility, and 
Oracle Support charges for that.

  These folks should not blame anyone (or anything) but themselves.  They blew 
straight through the many fail-safes that Oracle provides, so that any one of a 
thousand minor mishaps would have landed them in this exact same situation.  It 
was just waiting to happen.

  My sincere condolences...

  -Tim


  on 6/12/04 2:21 PM, Terry Sutton at terrysutton@xxxxxxx wrote:


    A new client has a serious corruption error.  During a hot backup (not 
RMAN) Thursday morning it was discovered that a datafile was corrupted on the 
disk.  When the database is started up, the following error occurs:
     
    "ORA-01122: database file 5 failed verification check
    ORA-01110: data file 5:
    '/ora2/app/oracle/admin/dbn/data/dbn_data_01.dbf'
    ORA-01251: Unknown File Header Version read for file number 5"
    When dbverify is run against this file, every block is indicated as 
corrupt, which isn't surprising.
     
    We have a copy of the datafile from Wednesday's hot backup, but they don't 
have the archivelogs necessary to sync this datafile with the rest of the 
database.  And we don't have the other datafiles from Wednesday's backup, so we 
can't just go back to the state the database was in then.
     
    Any ideas on what they can do?  I can't think of anything (other than going 
back to an export they have from 6 months ago, which seems to be the only 
historical backup they have).
     
    --Terry

Other related posts: