ORA-15003 on one node of RAC with ASM

  • From: saurabh manroy <saurabhmanroy@xxxxxxxxx>
  • To: oracle-l <oracle-l@xxxxxxxxxxxxx>
  • Date: Wed, 23 Oct 2013 17:54:39 +0200

Listers,
Here is the problem that we have in one of our production DB and an Oracle
SR is also open for this:
DB/CRS/ASM : 11.1.0.7
Early morning today, RDBMS and ASM instances on one node of a two node
cluster went down. Later, while trying to restart ASM instance on another
node, it reported following errors while trying to mount diskgroups on
failed node in the alert log.

ERROR: diskgroup xxxx was not mounted
ERROR: PST found another heartbeat (grp 2)

On the console
ORA-15032: not all alterations performed
ORA-15003: diskgroup "xxxx" already mounted in another lock name space

Looking at crs_stat -t output shows that each node can't see resources from
another node. For example:

On Node1:
ora.node1.vip  application    ONLINE    ONLINE    node1
ora....SM2.asm application    ONLINE    OFFLINE

Node2:
ora.node1.vip   application    ONLINE    OFFLINE
ora....SM2.asm  application    ONLINE    ONLINE    node2

Oracle support has suggested to perform crs restart on node2 and if it
doesn't help a reboot of complete cluster which we'll perform tonight.
Has anybody seen this before ? What might be the root cause of ORA-15003
error on RAC instance?

Regards,
Saurabh Manroy


--
//www.freelists.org/webpage/oracle-l


Other related posts: