Storage: Dell Clariion CX700 SAN
Dunno yet, but I think so.
/u01/app/oracle//admin/pr09/bdump/pr09_ckpt_29193.trc: ORA-01171: datafile 12 going offline due to error advancing checkpoint ORA-01110: data file 12: '/u01/oradata/pr09/agile_indx101pr09.ora' ^^^^^^^^^^^^^ ... Is this ext3 ?
Yes, this is ext3.
ORA-27072: skgfdisp: I/O error Linux-x86_64 Error: 5: Input/output error
... are there corresponding /var/log/messages entries for these EIO errors tht would impugn the scsi layer or other goodies downwind? Trying to establish if this is an EIO pushed up from a software layer (perhaps powerpath) ? Being intermittant there whould be /var/log/messages goodies to clue us in
No IO error messages that I can find yet, but did find something else:
Jun 7 23:15:07 ordb02 kernel: ctxhx: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffa528 error 4 Jun 7 23:17:53 ordb02 kernel: ctxhx: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffa528 error 4 Jun 7 23:31:39 ordb02 kernel: ctxhx: segfault at 00000000f790c02c rip 00000000f7ea39c3 rsp 00000000ffffa370 error 6 Jun 8 09:55:05 ordb02 kernel: ctxhx: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffb1b8 error 4
There are 2 other databases running the same app, same versions of Oracle, so it is a little difficult to be sure which is causing the problem.
Since most of these are the past 2 days, it could be that this is just a symptom. The IO errors may have crashed ctxhx (I'm assuming it has something to do with building Text indexes)
-- Jared Still Certifiable Oracle DBA and Part Time Perl Evangelist