Trouble starting crs - no listener

  • From: orcl <orcl@xxxxxxx>
  • To: oracle-l <oracle-l@xxxxxxxxxxxxx>
  • Date: Sun, 03 Jan 2010 09:23:05 -0500

Hi All  - Happy New Year!

I have a 3 node rac - optiplex1, optiplex2, poweredge

oracle 10.2.0.1
rhel 4.5

Ive preformed a clean install of the clusterware on all nodes without any problems at all, the crs, is running fine I can start the asm and the database - untill a reboot.

When I reboot - the crs logs are saying no listener. From my research there is a directory called /var/tmp/.oracle - which has "socket" files - which somehow act as a listener. The "listener" file that the log is looking for is in the directory - but the log is still showing no listener and refuses to start.

So my question is how to correct the problem of no listener for crsd.

Here is what Ive tried:

clean install ie
from doc http://tinyurl.com/yls9xxv

(MOS looks to be down again)

basically running this
rm -f /etc/oracle/*
rm -f /etc/init.d/init.cssd
rm -f /etc/init.d/init.crs
rm -f /etc/init.d/init.crsd
rm -f /etc/init.d/init.evmd
rm -f /etc/rc2.d/K96init.crs
rm -f /etc/rc2.d/S96init.crs
rm -f /etc/rc3.d/K96init.crs
rm -f /etc/rc3.d/S96init.crs
rm -f /etc/rc5.d/K96init.crs
rm -f /etc/rc5.d/S96init.crs
rm -Rf /etc/oracle/scls_scr
rm -f /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab
rm -f /var/tmp/.oracle/*
rm -f /tmp/.oracle/*
rm -Rf /u01/app/crs/*
rm -f /u02/oradata/racdb/*File*
rm -f /tmp/.oracle/*  ; rm -f var/tmp/.oracle/*
rm -rf /u01/app/crs ; mkdir /u01/app/crs ; chown oracle:oinstall /u01/app/crs*

*Then do the install - no errors
run some cluster verify commands - all look good
verify files exist in /var/tmp/.oracle/ - they do
reboot
crsd does not start and the log shows no listener


Here is some info

[root@*optiplex2* ~]# /u01/app/crs/bin/crs_stat -t

the top 2 OFFLINE are optiplex1

Name           Type           Target    State     Host
------------------------------------------------------------
ora....ex1.gsd application    ONLINE    OFFLINE
ora....ex1.ons application    ONLINE    OFFLINE
ora....ex1.vip application    ONLINE    ONLINE    optiplex2
ora....ex2.gsd application    ONLINE    ONLINE    optiplex2
ora....ex2.ons application    ONLINE    ONLINE    optiplex2
ora....ex2.vip application    ONLINE    ONLINE    optiplex2
ora....dge.gsd application    ONLINE    ONLINE    poweredge
ora....dge.ons application    ONLINE    ONLINE    poweredge
ora....dge.vip application    ONLINE    ONLINE    poweredge


[root@*optiplex1*]/root>ls -ltr /var/tmp/.oracle
total 0
srwxrwxrwx  1 oracle oinstall 0 Jan  2 23:38 soptiplex1DBG_CSSD
srwxrwxrwx  1 oracle oinstall 0 Jan  2 23:38 sOracle_CSS_LclLstnr_crs_1
srwxrwxrwx  1 oracle oinstall 0 Jan  2 23:38 sOCSSD_LL_optiplex1_crs
srwxrwxrwx  1 root   root     0 Jan  3 08:50 sprocr_local_conn_0_PROC
srwxrwxrwx  1 root   root     0 Jan  3 08:50 soptiplex1DBG_CRSD

here is the pertinent section of the log
[root@*optiplex1*]/root> view /u01/app/crs/log/optiplex1/crsd/crsd.log

Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2004, Oracle. All rights reserved
2010-01-03 08:50:47.242: [ default][3086853824]0CRS Daemon Starting
2010-01-03 08:50:47.244: [ CRSMAIN][3086853824]0Checking the OCR device
2010-01-03 08:50:47.258: [ CRSMAIN][3086853824]0Connecting to the CSS Daemon
2010-01-03 08:50:47.737: [ CRSD][3086853824]0Daemon Version: 10.2.0.1.0 Active Version: 10.2.0.1.0 2010-01-03 08:50:47.738: [ CRSD][3086853824]0Active Version and Software Version are same
2010-01-03 08:50:47.738: [ CRSMAIN][3086853824]0Initializing OCR
2010-01-03 08:50:47.757: [ OCRRAW][3086853824]proprioo: for disk 0 (/u02/oradata/racdb/OCRFile), id match (1), my id set (1353305425,355911080) total id sets (1), 1st set (1353305425,355911080), 2nd set (0,0) my votes (1), total votes (2) 2010-01-03 08:50:47.758: [ OCRRAW][3086853824]proprioo: for disk 1 (/u02/oradata/racdb/OCRFile_mirror), id match (1), my id set (1353305425,355911080) total id sets (1), 1st set (1353305425,355911080), 2nd set (0,0) my votes (1), total votes (2) 2010-01-03 08:50:47.843: [ CRSD][3086853824]0ENV Logging level for Module: allcomp 0
...
2010-01-03 08:50:47.992: [ CRSD][3086853824]0ENV Logging level for Module: OCRMAS 0 2010-01-03 08:50:47.993: [ CRSMAIN][3086853824]0Filename is /u01/app/crs/crs/init/optiplex1.pid [ clsdmt][2842594208]Listening to (ADDRESS=(PROTOCOL=ipc)(KEY=*optiplex1DBG_CRSD*)) 2010-01-03 08:50:48.031: [ CRSMAIN][3086853824]0Using Authorizer location: /u01/app/crs/crs/auth/
2010-01-03 08:50:48.073: [ CRSMAIN][3086853824]0Initializing RTI
2010-01-03 08:50:48.075: [CRSTIMER][2819619744]0Timer Thread Starting.
2010-01-03 08:50:48.080: [ CRSRES][3086853824]0Parameter SECURITY = 1, running in USER Mode
2010-01-03 08:50:48.080: [ CRSMAIN][3086853824]0Initializing EVMMgr
*2010-01-03 08:50:48.084: [ COMMCRS][2809129888]clsc_connect: (0x8959980) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=SYSTEM.evm.acceptor.auth))*

2010-01-03 08:50:48.338: [ COMMCRS][2809129888]clsc_connect: (0x8959a78) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=SYSTEM.evm.acceptor.auth))

Other related posts: