Hi, I ran into a similar problem. I'm using udev though, so the error messages don't match 1:1. The ORA-600 is there with the same argument, though. I think the source of the problem could very well be the same and might be worth checking out. I get the error when creating a database on a newly set up RAC for a current project after I applied the PSU for GI and DB home. Unsuccessfuly try: [oracle@rac02b ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.2.0 Production on Sat Aug 20 20:34:06 2011 Copyright (c) 1982, 2010, Oracle. All rights reserved. Connected to an idle instance. SQL> startup nomount ORACLE instance started. Total System Global Area 626327552 bytes Fixed Size 2229080 bytes Variable Size 184552616 bytes Database Buffers 436207616 bytes Redo Buffers 3338240 bytes SQL> create database DS0P 2 USER SYS IDENTIFIED BY manager1 USER SYSTEM IDENTIFIED BY manager1 EXTENT MANAGEMENT LOCAL DEFAULT TEMPORARY TABLESPACE TEMP UNDO TABLESPACE UNDOTBS1 DEFAULT TABLESPACE USERS CHARACTER SET WE8ISO8859P15 NATIONAL CHARACTER SET AL16UTF16 ARCHIVELOG FORCE LOGGING; 3 4 5 6 7 8 9 10 11 create database DS0P * ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 28206 Session ID: 1 Serial number: 3 Corresponding alert log entry: Sat Aug 20 20:35:46 2011 create database DS0P USER SYS IDENTIFIED BY *USER SYSTEM IDENTIFIED BY *EXTENT MANAGEMENT LOCAL DEFAULT TEMPORARY TABLESPACE TEMP UNDO TABLESPACE UNDOTBS1 DEFAULT TABLESPACE USERS CHARACTER SET WE8ISO8859P15 NATIONAL CHARACTER SET AL16UTF16 ARCHIVELOG FORCE LOGGING Sat Aug 20 20:35:46 2011 ORA-15025: could not open disk "/dev/asm-disk4" ORA-27041: unable to open file Linux-x86_64 Error: 13: Permission denied Additional information: 9 ORA-15025: could not open disk "/dev/asm-disk6" ORA-27041: unable to open file Linux-x86_64 Error: 13: Permission denied Additional information: 9 ORA-15025: could not open disk "/dev/asm-disk5" ORA-27041: unable to open file Linux-x86_64 Error: 13: Permission denied Additional information: 9 ORA-15025: could not open disk "/dev/asm-disk7" ORA-27041: unable to open file Linux-x86_64 Error: 13: Permission denied Additional information: 9 SUCCESS: diskgroup DATA was mounted Errors in file /u01/app/oracle/diag/rdbms/ds0p/DS0P/trace/DS0P_ora_28310.trc (incident=1817): ORA-00600: internal error code, arguments: [kfioTranslateIO03], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/ds0p/DS0P/incident/incdir_1817/DS0P_ora_28310_i1817.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ERROR: unrecoverable error ORA-600 raised in ASM I/O path; terminating process 28310 Sat Aug 20 20:35:48 2011 Dumping diagnostic data in directory=[cdmp_20110820203548], requested by (instance=1, osid=28310), summary=[incident=1817]. Sat Aug 20 20:36:24 2011 Sweep [inc][1817]: completed Sweep [inc2][1817]: completed Sat Aug 20 20:36:30 2011 SUCCESS: diskgroup DATA was dismounted ASM disk permissions: [oracle@rac02b ~]$ ls -l /dev/asm-disk* brw-rw---- 1 grid asmadmin 8, 17 Aug 20 20:37 /dev/asm-disk1 brw-rw---- 1 grid asmadmin 8, 161 Aug 20 16:55 /dev/asm-disk10 brw-rw---- 1 grid asmadmin 8, 177 Aug 20 20:29 /dev/asm-disk11 brw-rw---- 1 grid asmadmin 8, 33 Aug 20 20:37 /dev/asm-disk2 brw-rw---- 1 grid asmadmin 8, 49 Aug 20 20:37 /dev/asm-disk3 brw-rw---- 1 grid asmadmin 8, 65 Aug 20 20:36 /dev/asm-disk4 brw-rw---- 1 grid asmadmin 8, 81 Aug 20 20:36 /dev/asm-disk5 brw-rw---- 1 grid asmadmin 8, 97 Aug 20 20:36 /dev/asm-disk6 brw-rw---- 1 grid asmadmin 8, 113 Aug 20 20:36 /dev/asm-disk7 brw-rw---- 1 grid asmadmin 8, 129 Aug 20 16:55 /dev/asm-disk8 brw-rw---- 1 grid asmadmin 8, 145 Aug 20 20:29 /dev/asm-disk9 Weird, everything looks ok so far. The error message made me try to add oracle to the asmadmin group though. This is not necessary to my knowledge and as far as I remember I never did that for a RAC before. But it helped. [root@rac02b ~]# usermod -a -G asmadmin oracle [oracle@rac02b ~]$ id uid=1101(oracle) gid=1000(oinstall) groups=1000(oinstall),1020(asmadmin),1021(asmdba),1031(dba),1032(oper) [oracle@rac02b ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.2.0 Production on Sat Aug 20 20:49:14 2011 Copyright (c) 1982, 2010, Oracle. All rights reserved. Connected to an idle instance. SQL> startup nomount ORACLE instance started. Total System Global Area 626327552 bytes Fixed Size 2229080 bytes Variable Size 184552616 bytes Database Buffers 436207616 bytes Redo Buffers 3338240 bytes SQL> CREATE DATABASE DS0P 2 USER SYS IDENTIFIED BY manager1 USER SYSTEM IDENTIFIED BY manager1 EXTENT MANAGEMENT LOCAL DEFAULT TEMPORARY TABLESPACE TEMP UNDO TABLESPACE UNDOTBS1 DEFAULT TABLESPACE USERS CHARACTER SET WE8ISO8859P15 NATIONAL CHARACTER SET AL16UTF16 ARCHIVELOG FORCE LOGGING; 3 4 5 6 7 8 9 10 11 Database created. I did another experiment before that one which I unfortunately did not document. There I had another workaround. It seemed to work okay once I had created a database via dbca. I don't know what it does to fix the issue until know. Will let you know if I get to investigate it further. For the moment: Adding oracle to asmadmin seemed to fix it at least for me. Best regards, Robert http://robertvsoracle.blogspot.com On Sat, Aug 20, 2011 at 12:14 AM, Li Li <litanli@xxxxxxxxx> wrote: > Hi, List, > > Has anyone on the list applied the July CPU? We just installed it on > one of our non-production RAC nodes and now we can't even do a > 'startup nomount' successfully. Logs show the following error: > > ORACLE_BASE from environment = /opt/app/oracle > Errors in file > /opt/app/oracle/diag/rdbms/hrdb/HRDB2/trace/HRDB2_rbal_25712.trc: > ORA-15183: ASMLIB initialization error [driver/agent not installed] > WARNING: FAILED to load library: /opt/oracle/extapi/64/asm/orcl/1/libasm.so > Errors in file > /opt/app/oracle/diag/rdbms/hrdb/HRDB2/trace/HRDB2_rbal_25712.trc: > ORA-15183: ASMLIB initialization error [driver/agent not installed] > SUCCESS: diskgroup HR_DATA was mounted > NOTE: dependency between database HRDB and diskgroup resource > ora.HR_DATA.dg is established > Errors in file /opt/app/oracle/diag/rdbms/hrdb/HRDB2/trace/HRDB2_ora_25586.trc > (incident=7209): > ORA-00600: internal error code, arguments: [kfioTranslateIO03], [], > [], [], [], [], [], [], [], [], [], [] > Incident details in: > /opt/app/oracle/diag/rdbms/hrdb/HRDB2/incident/incdir_7209/HRDB2_ora_25586_i7209.trc > Use ADRCI or Support Workbench to package the incident. > See Note 411.1 at My Oracle Support for error and packaging details. > ERROR: unrecoverable error ORA-600 raised in ASM I/O path; terminating > process 25586 > Dumping diagnostic data in directory=[cdmp_20110819164633], requested > by (instance=2, osid=25586), summary=[incident=7209]. > > I tried to google and can't find much information regarding this and > CPU. Any comments/suggestions? > > thanks, > -Li > -- > //www.freelists.org/webpage/oracle-l > > > -- //www.freelists.org/webpage/oracle-l