Fun With Scan Listener
- From: David Barbour <david.barbour1@xxxxxxxxx>
- To: oracle-l mailing list <oracle-l@xxxxxxxxxxxxx>
- Date: Tue, 3 Jun 2014 16:27:24 -0500
Oracle 11.2.0.3 RHEL 6.3 5-Node RAC
This has me somewhat (okay - totally) baffled. I have a scan listener
that is showing as follows when I run crsctl status resource -t:
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE UNKNOWN rchr1p01
If I check the status via srvctl I get the following:
$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is not running
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node rchr1p02
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node rchr1p03
So I try to start it:
$ srvctl start scan_listener -i 1
PRCR-1079 : Failed to start resource ora.LISTENER_SCAN1.lsnr
CRS-5013: Agent "/oracle/grid/11203/bin/oraagent.bin" failed to start
process "/oracle/grid/11203/bin/lsnrctl" for action "clean": details at
"(:CLSN00008:)" in
"/oracle/grid/11203/log/rchr1p01/agent/crsd/oraagent_oracle/oraagent_oracle.log"
CRS-5013: Agent "/oracle/grid/11203/bin/oraagent.bin" failed to start
process "/oracle/grid/11203/bin/lsnrctl" for action "check": details at
"(:CLSN00008:)" in
"/oracle/grid/11203/log/rchr1p01/agent/crsd/oraagent_oracle/oraagent_oracle.log"
CRS-2680: Clean of 'ora.LISTENER_SCAN1.lsnr' on 'rchr1p01' failed
So I try to stop it:
srvctl stop scan_listener -i 1 -f
PRCR-1065 : Failed to stop resource ora.LISTENER_SCAN1.lsnr
CRS-5013: Agent "/oracle/grid/11203/bin/oraagent.bin" failed to start
process "/oracle/grid/11203/bin/lsnrctl" for action "clean": details at
"(:CLSN00008:)" in
"/oracle/grid/11203/log/rchr1p01/agent/crsd/oraagent_oracle/oraagent_oracle.log"
CRS-5013: Agent "/oracle/grid/11203/bin/oraagent.bin" failed to start
process "/oracle/grid/11203/bin/lsnrctl" for action "check": details at
"(:CLSN00008:)" in
"/oracle/grid/11203/log/rchr1p01/agent/crsd/oraagent_oracle/oraagent_oracle.log"
CRS-2680: Clean of 'ora.LISTENER_SCAN1.lsnr' on 'rchr1p01' failed
So I give up and check the log.
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][2617243392]
{1:53466:11150} [clean] clsn_agent::clean: Exception
SclsProcessSpawnException
2014-06-03 16:19:48.666: [ AGFW][3623876352] {1:53466:11150} Agent
sending reply for: RESOURCE_CLEAN[ora.LISTENER_SCAN1.lsnr 1 1] ID 4100:58347
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][2617243392]
{1:53466:11150} [clean] (:CLSN00106:) clsn_agent::clean }
2014-06-03 16:19:48.666: [ AGFW][2617243392] {1:53466:11150} Command:
clean for resource: ora.LISTENER_SCAN1.lsnr 1 1 completed with status: FAIL
2014-06-03 16:19:48.666: [ AGFW][3623876352] {1:53466:11150} Agent
sending reply for: RESOURCE_CLEAN[ora.LISTENER_SCAN1.lsnr 1 1] ID 4100:58347
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] LsnrAgent::check {
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] lsnrctl status LISTENER_SCAN1
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] getOracleHomeAttrib: oracle_home =
/oracle/grid/11203
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] getOracleHomeAttrib: oracle_home =
/oracle/grid/11203
2014-06-03 16:19:48.666: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Utils::getCrsHome crsHome /oracle/grid/11203
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Utils::execCmd 1
USR_ORA_ENV:ORACLE_BASE=/opt/oracle oracleHome:/oracle/grid/11203
CrsHome:/oracle/grid/11203
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Utils::getCrsHome crsHome /oracle/grid/11203
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Adding Environment Variables
ORACLE_HOME=/oracle/grid/11203
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Adding Environment Variables
TNS_ADMIN=/oracle/grid/11203/network/admin/
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Adding Environment variable from USR_ORA_ENV
ORACLE_BASE=/opt/oracle
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] Utils:execCmd action = 3 flags = 38 ohome = (null)
cmdname = lsnrctl.
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] getOracleHomeAttrib: oracle_home =
/oracle/grid/11203
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] (:CLSN00008:)Utils:execCmd scls_process_spawn()
failed 1
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] (:CLSN00008:) category: -2, operation: fork, loc:
spawnproc28, OS error: 11, other: forked failed [-1]
2014-06-03 16:19:48.667: [ AGENT][3087005440] {1:53466:11150}
UserErrorException: Locale is
2014-06-03 16:19:48.667: [ora.LISTENER_SCAN1.lsnr][3087005440]
{1:53466:11150} [check] clsnUtils::error Exception type=2 string=
CRS-5013: Agent "/oracle/grid/11203/bin/oraagent.bin" failed to start
process "/oracle/grid/11203/bin/lsnrctl" for action "check": details at
"(:CLSN00008:)" in
"/oracle/grid/11203/log/rchr1p01/agent/crsd/oraagent_oracle/oraagent_oracle.log"
I'm willing to go with the part about 'UserErrorException', except I'm not
aware of what I'm doing wrong. Looking through MOS docs but hoping someone
has a suggestion?
Other related posts:
