Re: Shutdown -halt with RAC

  • From: Henry Poras <henry@xxxxxxxxxxxxxxx>
  • To: K Gopalakrishnan <kaygopal@xxxxxxxxx>
  • Date: Tue, 15 Nov 2005 21:52:13 -0500

Here is the crs log. I guess what I was wondering was when Oracle will reboot a node and when it will shut it down. Documentation seems to be weak when it comes to describing these scenarios.

Thanks.

Henry


2005-11-10 15:17:54.024: CRSD-1: Sending Restart Application Request
Oracle Database 10g CRS Release 10.1.0.4.0 Production Copyright 1996, 2004, Oracle. All rights reserved.
2005-11-10 15:17:54 : Oracle CRSD will start boot services
2005-11-10 15:17:54 : Changing directory to /ita/crs/oracle/product/10.1.0/crs/crs/init/
2005-11-10 15:17:54 : Wrote pid 13595 to /ita/crs/oracle/product/10.1.0/crs/crs/init/.lock-serverA
2005-11-10 15:17:54.053: CRS Daemon Starting
2005-11-10 15:17:54.053: Checking the OCR device
2005-11-10 15:17:54.090: Connecting to the CSS Daemon
2005-11-10 15:17:54.154: Initializing OCR
2005-11-10 15:17:54.245: Writing Software Version to OCR
2005-11-10 15:17:54.321: Using Authorizer location: /ita/crs/oracle/product/10.1.0/crs/crs/auth/
2005-11-10 15:17:54.379: Initializing RTI
2005-11-10 15:17:54.383: Parameter SECURITY = 1, running in USER Mode
2005-11-10 15:17:54.383: Initializing EVMMgr
2005-11-10 15:18:58.687: CRSD locked during state recovery, please wait.
2005-11-10 15:18:59.188: CRSD recovered, unlocked.
2005-11-10 15:18:59.190: QS socket on: (ADDRESS=(PROTOCOL=ipc)(KEY=ora_crsqs))
2005-11-10 15:18:59.191: UI socket on: (ADDRESS=(PROTOCOL=ipc)(KEY=serverA_crs_caa))
2005-11-10 15:18:59.192: E2E socket on: (ADDRESS=(PROTOCOL=tcp)(HOST=serverAprv)(PORT=49896))
2005-11-10 15:18:59.192: Starting Threads
2005-11-10 15:18:59.192: CRS Daemon Started.
2005-11-10 15:19:00.470: CRS-1007: Failed after successful dependency consideration

2005-11-10 15:19:01.199: Attempting to start `ora.serverA.vip` on member `serverA`
2005-11-10 15:19:01.783: Start of `ora.serverA.vip` on member `serverA` succeeded.
2005-11-10 15:19:01.893: Attempting to start `ora.serverA.ASM2.asm` on member `serverA`
2005-11-10 15:19:03.183: Start of `ora.serverA.ASM2.asm` on member `serverA` succeeded.
2005-11-10 15:19:04.033: Attempting to start `ora.serverA.LISTENER_serverA.lsnr` on member `serverA`
2005-11-10 15:19:04.619: Start of `ora.serverA.LISTENER_serverA.lsnr` on member `serverA` succeeded.
2005-11-10 15:19:05.045: Attempting to start `ora.serverA.gsd` on member `serverA`
2005-11-10 15:19:05.498: Start of `ora.serverA.gsd` on member `serverA` succeeded.
2005-11-10 15:19:05.905: Attempting to start `ora.serverA.ons` on member `serverA`
2005-11-10 15:19:06.259: Start of `ora.serverA.ons` on member `serverA` succeeded.
2005-11-10 15:19:07.152: Attempting to start `ora.test.test2.inst` on member `serverA`
2005-11-10 15:19:11.412: Start of `ora.test.test2.inst` on member `serverA` succeeded.
2005-11-10 15:17:56.158: CRSD-1: [CMDMAIN:3059703936] Restart waiting for Oracle CRSD to start
2005-11-10 15:18:24.256: CRSD-1: [CMDMAIN:3059703936] Restart waiting for Oracle CRSD to start
2005-11-10 15:18:52.357: CRSD-1: [CMDMAIN:3059703936] Restart waiting for Oracle CRSD to start
2005-11-10 15:19:11.488: CRSD-1: Complete Restart Application Request
2005-11-10 15:29:06.558: [RUNCONTEXT::RUNSCRIPT:2529962928] CheckResource error for ora.serverA.ons error code = 1
`ora.serverA.ons` on `serverA` went OFFLINE unexpectedly
2005-11-10 15:29:06.676: Attempting to stop `ora.serverA.ons` on member `serverA`
2005-11-10 15:29:06.898: Stop of `ora.serverA.ons` on member `serverA` succeeded.
Restarting `ora.serverA.ons` on `serverA`
2005-11-10 15:29:06.962: Attempting to start `ora.serverA.ons` on member `serverA`
2005-11-10 15:29:07.179: Start of `ora.serverA.ons` on member `serverA` succeeded.
Successfully restarted `ora.serverA.ons` on `serverA`
2005-11-10 15:31:05.078: ALERT [CAAOCRLOOKUP:2624658352] OCR api procr_open_key failed for key CRS.CUR.ora!serverA!vip
OCR error code = 3 OCR error msg:
crsd.bin: caaocr.cpp:200: Assertion `0==1' failed.
2005-11-10 15:33:41.329: CRSD-2: Sending Shutdown Application Request
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
2005-11-10 15:33:55.533: CRSD-2: ALERT [MAIN:3059703936] Error sending request to running CRSD
2005-11-10 15:33:55.533: CRSD-2: ALERT [MAIN:3059703936] SHUTDOWNAPPS
Could not send application shutdown request to CRSD
2005-11-10 15:38:06.143: CRSD-1: Sending Restart Application Request


K Gopalakrishnan wrote:

Henry:

What does your CRS log file say? I guess CRS killed instance to avoid
data corruption. THis seems like a simple io-fencing issue.

Regards,
Gopal


On 11/15/05, Henry Poras <henry@xxxxxxxxxxxxxxx> wrote:


I've been playing around with a RAC system (10.1.0.4 on RedHat AS3), and
managed to crash it(no problem, I'm trying to see what can go wrong). It was
probably due to a corrupted Registry file (does ocrconfig -import work?). On
trying to export and then import a Registry file, then restart CRS, one of
our nodes removed itself (no longer visible from the network. Probable
crash). When our SysAdm went down to look, he said Oracle had 'halt'ed the
server. All the doc I have seen talks about 'reboot' under certain
conditions, but I have never seen a halt (or shutdown) mentioned anywhere.
Has anyone else seen (or seen documentation of) this behavior?

Henry




--
Best Regards,
K Gopalakrishnan
Co-Author: Oracle Wait Interface, Oracle Press 2004
http://www.amazon.com/exec/obidos/tg/detail/-/007222729X/




--
//www.freelists.org/webpage/oracle-l


Other related posts:

  1. Home
  2. oracle-l
  3. Archive
  4. 11-2005