RMAN catalog issue (long)
- From: Jack van Zanen <jack@xxxxxxxxxxxx>
- To: "oracle-l@xxxxxxxxxxxxx" <oracle-l@xxxxxxxxxxxxx>
- Date: Tue, 17 Feb 2009 11:00:26 +1100
Good Morning (depends on which part of the world you are from)
We have a RMAN backup script that is running fine pretty much all the time
but every now and then we have the following issue (very intermittent)
Below is the logfile and the problem lies in the schema's of the constraints
that get violated.
We log on to the catalog using user OMS_P_RMAN. but somehow the first
constraint that gets violated belongs to OMS_T_RMAN. Now I have done my
checks and I can not find any cross schema constraints, as this was my first
thought. But this being an intermittent problem I can not get my head around
this one.
I have opened up a SR with oracle but reached the point very quickly where
they need a trace but it does not reproduce at will :-)
All versions involved (catalog, catalaog db and target db are 10.2.0.2), OS
is AIX
Recovery Manager: Release 10.2.0.2.0 - Production on Thu Feb 12 23:00:01
2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
RMAN>
connected to recovery catalog database
RMAN>
connected to target database: PRODDB (DBID=1195121302)
RMAN>
RMAN>
echo set on
RMAN>
RMAN> # offline/cold backup only
2> ### startup mount;
3>
4> run {
5> allocate channel c1 device type sbt parms
'ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/eapdclparp19/tdpo.opt)';
6> # allocate channel c2 device type sbt parms
'ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/eapdclparp19/tdpo.opt)';
7> # allocate channel c3 device type sbt parms
'ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/eapdclparp19/tdpo.opt)';
8> crosscheck archivelog all;
9> backup full
10> filesperset 5
11> format 'daily_PRODDB_BACKUP_DBF_online_20090212_230000_%s_%p_%c'
12> tag daily_PRODDB_BACKUP
13> database
14> plus archivelog
15> filesperset 5
16> format 'daily_PRODDB_BACKUP_AL_online_20090212_230000_%s_%p_%c'
17> tag daily_PRODDB_BACKUP;
18>
19> backup current controlfile
20> filesperset 5
21> format 'daily_PRODDB_BACKUP_CF_online_20090212_230000_%s_%p_%c'
22> tag daily_PRODDB_BACKUP;
23>
24> # online backup only
25> sql 'alter system archive log current';
26> crosscheck archivelog all;
27> backup
28> filesperset 5
29> format 'daily_PRODDB_BACKUP_AL_online_20090212_230000_%s_%p_%c'
30> tag daily_PRODDB_BACKUP
31> archivelog all not backed up 2 times;
32>
33> # delete noprompt archivelog until time 'sysdate -1' backed up 2 times
to device type sbt;
34>
35> backup spfile
36> filesperset 5
37> format 'daily_PRODDB_BACKUP_SP_online_20090212_230000_%s_%p_%c'
38> tag daily_PRODDB_BACKUP;
39>
40> sql 'alter database backup controlfile to trace';
41>
42> release channel c1;
43> # release channel c2;
44> # release channel c3;
45> }
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of allocate command at 02/12/2009 23:00:22
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of partial resync command on default channel at
02/12/2009 23:00:22
ORA-02291: integrity constraint (OMS_T_RMAN.ROUT_F2) violated - parent key
not found
RMAN>
RMAN> # offline/cold backup only
2> ### alter database open;
3>
4> allocate channel for maintenance device type sbt parms
'ENV=(TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/eapdclparp19/tdpo.opt)';
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of allocate command at 02/12/2009 23:00:26
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of partial resync command on default channel at
02/12/2009 23:00:26
ORA-01400: cannot insert NULL into ("OMS_P_RMAN"."ROUT"."ROUT_SKEY")
RMAN> delete noprompt obsolete;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of delete command at 02/12/2009 23:00:29
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of partial resync command on default channel at
02/12/2009 23:00:29
ORA-01400: cannot insert NULL into ("OMS_P_RMAN"."ROUT"."ROUT_SKEY")
RMAN> delete noprompt expired backup;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of delete command at 02/12/2009 23:00:36
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of partial resync command on default channel at
02/12/2009 23:00:36
ORA-01400: cannot insert NULL into ("OMS_P_RMAN"."ROUT"."ROUT_SKEY")
RMAN> delete noprompt backup completed before 'sysdate - 35' tag
daily_PRODDB_BACKUP ;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of delete command at 02/12/2009 23:00:43
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of partial resync command on default channel at
02/12/2009 23:00:43
ORA-01400: cannot insert NULL into ("OMS_P_RMAN"."ROUT"."ROUT_SKEY")
RMAN> release channel;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of release command at 02/12/2009 23:00:50
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of partial resync command on default channel at
02/12/2009 23:00:50
ORA-01400: cannot insert NULL into ("OMS_P_RMAN"."ROUT"."ROUT_SKEY")
RMAN> exit
Recovery Manager complete.
--
Jack van Zanen
-------------------------
This e-mail and any attachments may contain confidential material for the
sole use of the intended recipient. If you are not the intended recipient,
please be aware that any disclosure, copying, distribution or use of this
e-mail or any attachment is prohibited. If you have received this e-mail in
error, please contact the sender and delete all copies.
Thank you for your cooperation
Other related posts:
