Re: Strange problem

  • From: De DBA <dedba@xxxxxxxxxx>
  • To: gogala.mladen@xxxxxxxxx, oracle-l <oracle-l@xxxxxxxxxxxxx>
  • Date: Mon, 1 May 2017 08:49:48 +1000

Hi Mladen,

I've seen such errors, but on normal filesystems/ASM. They were at that time 
accompanied by IO errors on the underlying disk device due to failing disks. I 
expect you have already ruled out hardware failure, so I would at this point 
investigate the ZFS volume. Has anyone changed parameters on the underlying 
volume, resulting in the file becoming read-only, such as setting the readonly 
flag on the volume?

Also I suspect that merging a snapshot would change the file addressing space, 
and confuse the database if done with the file open. Not sure whether this is 
actually the case, or ZFS will handle this gracefully.



On 01/05/17 06:22, Mladen Gogala wrote:


I am having a strange problem with an Oracle instance on SUN Solaris 
11.3 (SPARC):

ORA-01110: data file 7: '/oradata/DEV/datafiles/data01b.dbf' 2017-04-28 23:05:11.723000 -04:00 Flush retried for xcb 0x5c0e98a90, pmd 0x5c3faa538 Errors in file /app/oradev/diag/rdbms/oradev/DEV/trace/DEV_pmon_465049.trc: ORA-00372: file 7 cannot be modified at this time ORA-01110: data file 7: '/oradata/DEV/datafiles/data01b.dbf' 2017-04-28 23:05:18.756000 -04:00 Flush retried for xcb 0x5c0e98a90, pmd 0x5c3faa538 Errors in file /app/oradev/diag/rdbms/oradev/DEV/trace/DEV_dbw0_465083.trc: ORA-00372: file 7 cannot be modified at this time ORA-01110: data file 7: '/oradata/DEV/datafiles/data01b.dbf' USER (ospid: 465083): terminating the instance due to error 372 System state dump requested by (instance=1, osid=4295432379 (DBW0)), summary=[abnormal instance termination]. System State dumped to trace file /app/oradev/diag/rdbms/oradev/DEV/trace/DEV_diag_465073_20170428230518.trc 2017-04-28 23:05:23.820000 -04:00 Instance terminated by USER, pid = 465083 2017-04-29 10:58:13.293000 -04:00 Starting ORACLE instance (normal) (OS id: 44563) CLI notifier numLatches:29 maxDescs:1355 All SGA segments were allocated at startup

When I restart the instance all files are available. The instance is patched 
with the January 2017 PSU, soon to be patched with April 2017 PSU. Here are the 
relevant versions:

SunOS sun02 5.11 11.3 sun4v sparc sun4v -bash-4.4$ sqlplus / as sysdba 
SQL*Plus: Release Production on Sun Apr 30 16:15:43 2017 Copyright 
(c) 1982, 2014, Oracle.  All rights reserved. Connected to: Oracle Database 12c 
Enterprise Edition Release - 64bit Production With the Partitioning, 
OLAP, Advanced Analytics and Real Application Testing options

The underlying file system is, of course, ZFS. When the instance is restarted, 
all the files are available and there is nothing in the V$RECOVER_FILE. I 
opened SR with Oracle, but as this is a development system, they are taking 
their time. Has anyone seen this before and, if yes, what would be the 
solution? The system crashes during nightly batches, which load a lot of data 
and are write intensive.

Mladen Gogala
Oracle DBA
Tel: (347) 321-1217

Other related posts: