Long day. RHEL 6.3 Oracle Clusterware 11.2.0.3 (with patches) Alrighty then: Trying to run $CRS_HOME/crs/config/sonfig.sh . This is a 'bare-metal restore with all new disk, so no ocr, voting disks, anything. But the clusterware binaries were restored from tape. I run through the OUI and create a disk group and set up all the parameters then run root.sh. This is the result: Relinking oracle with rac_on option Using configuration parameter file: /oracle/grid/11203/crs/install/crsconfig_params User ignored Prerequisites during installation OLR initialization - successful Adding Clusterware entries to upstart CRS-2672: Attempting to start 'ora.mdnsd' on 'rchr1p01' CRS-2676: Start of 'ora.mdnsd' on 'rchr1p01' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rchr1p01' CRS-2676: Start of 'ora.gpnpd' on 'rchr1p01' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rchr1p01' CRS-2672: Attempting to start 'ora.gipcd' on 'rchr1p01' CRS-2676: Start of 'ora.cssdmonitor' on 'rchr1p01' succeeded CRS-2676: Start of 'ora.gipcd' on 'rchr1p01' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rchr1p01' CRS-2672: Attempting to start 'ora.diskmon' on 'rchr1p01' CRS-2676: Start of 'ora.diskmon' on 'rchr1p01' succeeded CRS-2676: Start of 'ora.cssd' on 'rchr1p01' succeeded ASM created and started successfully. Disk Group GRID created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'sfcb'.. Operation successful. Start of resource "ora.crsd" failed CRS-2672: Attempting to start 'ora.crsd' on 'rchr1p01' CRS-5017: The resource action "ora.crsd start" encountered the following error: Start action for daemon aborted. For details refer to "(:CLSN00107:)" in "/oracle/grid/11203/log/rchr1p01/agent/ohasd/orarootagent_root/orarootagent_root.log". CRS-2674: Start of 'ora.crsd' on 'rchr1p01' failed CRS-2679: Attempting to clean 'ora.crsd' on 'rchr1p01' CRS-2681: Clean of 'ora.crsd' on 'rchr1p01' succeeded CRS-4000: Command Start failed, or completed with errors. Grid Infrastructure exclusive mode start of Cluster Ready Services failed at /oracle/grid/11203/crs/install/crsconfig_lib.pm line 6823. /oracle/grid/11203/perl/bin/perl -I/oracle/grid/11203/perl/lib -I/oracle/grid/11203/crs/install /oracle/grid/11203/crs/install/rootcrs.pl execution failed The orarootagent_root.log shows: [ clsdmc][1386608384]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rchr1p01DBG_CRSD)) with status 9 2014-08-06 22:13:39.844: [ora.crsd][1386608384] {0:0:2} [check] Error = error 9 encountered when connecting to CRSD 2014-08-06 22:13:39.846: [ora.crsd][1386608384] {0:0:2} [check] DaemonAgent::check returned 1 2014-08-06 22:13:39.846: [ AGFW][1388709632] {0:0:2} ora.crsd 1 1 state changed from: UNKNOWN to: OFFLINE 2014-08-06 22:13:39.846: [ AGFW][1388709632] {0:0:2} Agent sending last reply for: RESOURCE_PROBE[ora.crsd 1 1] ID 4097:96 2014-08-06 22:13:39.847: [ COMMCRS][1375573760]clsc_connect: (0x7f3a3c03ad50) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rchr1p01DBG_MOND)) I have Googled and searched through MOS. I've tried cleanup and restart and can't get by this. I'm ready to just re-install the software, but that's not they way I'm supposed to play the game. Anybody seen this?