RE: ASM Grid Infrastructure voting files not found any more
- From: "Martin Bach" <development@xxxxxxxxxxxxxxxxx>
- To: "D'Hooge Freek" <Freek.DHooge@xxxxxxxxx>
- Date: Tue, 12 Jul 2011 16:12:35 +0200 (CEST)
Martin,
may I recommend the following to mull over-my customer had the same
problem after he changed the asm_diskstring from '/dev/oracleasm/disks/'
to ''. The customer used ASMLib, but hasn't configured it correctly, which
has been visible in v$asm_disk.library before the change.
BTW This is 11.2.0.2 with voting disks and OCR in ASM: diskgroup "+OCR"
(Linux x86-64)
Upon restart CSS discovered all voting files only to discard them :(
The solution was to
- prevent crs from starting automatically (crsctl disable crs)
- reboot
- start clusterware in -excl mode (this started the ASM instance)
- change ASM diskstring to ''
- use crsctl replace <DG_NAME>
Then reboot again, start stack again and voila-it worked. It doesn't mean
that it works in your environment though, please test this first before
applying to production etc and run it past Oracle Support.
Kind regards,
(another) Martin
http://martincarstenbach.wordpress.com
http://www.linkedin.com/in/martincarstenbach
> Martin,
> X-archive-position: 37362
> X-ecartis-version: Ecartis v1.0.0
> Sender: oracle-l-bounce@xxxxxxxxxxxxx
> Errors-to: oracle-l-bounce@xxxxxxxxxxxxx
> X-original-sender: Freek.DHooge@xxxxxxxxx
> Precedence: normal
> Reply-To: Freek.DHooge@xxxxxxxxx
> List-help: <mailto:ecartis@xxxxxxxxxxxxx?Subject=help>
> List-unsubscribe: <oracle-l-request@xxxxxxxxxxxxx?Subject=unsubscribe>
> List-software: Ecartis version 1.0.0
> List-Id: oracle-l <oracle-l.freelists.org>
> X-List-ID: oracle-l <oracle-l.freelists.org>
> List-subscribe: <oracle-l-request@xxxxxxxxxxxxx?Subject=subscribe>
> List-owner: <mailto:steve.adams@xxxxxxxxxxxx>
> List-post: <mailto:oracle-l@xxxxxxxxxxxxx>
> List-archive: <//www.freelists.org/archives/oracle-l>
> X-list: oracle-l
>
> Is this 11.2 with voting files in ASM?
> If so, following can help (but note this is not supported)
>
> The asm diskstring is stored in the profile.xmlf file in the grid
> infrastructure $ORACLE_HOME/ gpnp/profiles/peer/ directory.
>
> This file is a signed xml file, but can be unsigned / signed using the
> gpnptool
> Best is to stop the entire cluster before making modifications to this
> file
>
> Use gpnptool -? to get the syntax on how to unsign /sign the file
>
> To get a formatted outout:
>
> [grid@beantorarac03 peer]$ gpnptool c14n -fmt -fmtonly
> Warning: some command line parameters were defaulted. Resulting command
> line:
> /opt/grid/11.2.0.2/bin/gpnptool.bin c14n -fmt=0,2 -fmtonly
> -p=profile.xml -o-
>
> <?xml version="1.0" encoding="UTF-8"?>
> <gpnp:GPnP-Profile Version="1.0"
> xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile";
> xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile";
> xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile
> gpnp-profile.xsd" ProfileSequence="4"
> ClusterUId="548a704f469c5fceff540a328dc5eaf3" ClusterName="xxxx"
> PALocation="">
> <gpnp:Network-Profile>
> <gpnp:HostNetwork id="gen" HostName="*">
> <gpnp:Network id="net1" IP="xxxx" Adapter="bond0" Use="public"/>
> <gpnp:Network id="net2" IP="xxxx" Adapter="bond1"
> Use="cluster_interconnect"/>
> </gpnp:HostNetwork>
> </gpnp:Network-Profile>
> <orcl:CSS-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/>
> <orcl:ASM-Profile id="asm" DiscoveryString="/dev/oracle/*asm*p1"
> SPFile="+DG_CLUSTER/beantorarac/asmparameterfile/registry.253.730433213"/>
> <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#";>
> <ds:SignedInfo>
> <ds:CanonicalizationMethod
> Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
> <ds:SignatureMethod
> Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
> <ds:Reference URI="">
> <ds:Transforms>
> <ds:Transform
> Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
> <ds:Transform
> Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#";>
> <InclusiveNamespaces
> xmlns="http://www.w3.org/2001/10/xml-exc-c14n#";
> PrefixList="gpnp orcl xsi"/>
> </ds:Transform>
> </ds:Transforms>
> <ds:DigestMethod
> Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
> <ds:DigestValue>5a1vWvd/aWUneNgBBj/k6vYeSus=</ds:DigestValue>
> </ds:Reference>
> </ds:SignedInfo>
>
> <ds:SignatureValue>Q/PJMAMSq6qNxWUiwqNUHCakFJJWlcZW/zA94sDdqcVfc8oUiphDqHwg2TBMaSvnDVfCl+HCq2upiag91XFl55/5O5UGd7ZklkvCPWr3M1nUPf+HdrifgtmuvQ7wUczZ04vUS9U9jVW2YA6ae6Ek/bQARZmozuw/HXg113C4Tq8=</ds:SignatureValue>
> </ds:Signature>
> </gpnp:GPnP-Profile>
>
> Success.
>
>
> The string you are looking for is:
>
> <orcl:ASM-Profile id="asm" DiscoveryString="/dev/oracle/*asm*p1"
>
> Modify the discoverystring to the correct value
>
>
> Again, editing the profile.xml file is not supported by Oracle.
> The supported way is to reconfigure the cluster using the
>
> $GRID_HOME/crs/install/rootcrs.pl -deconfig -force
> $GRID_HOME/crs/install/rootcrs.pl -deconfig -force -lastnode
> And then rerun the root scripts on all nodes.
>
> However, this will erase all the asm diskgroups on which an ocr file is
> located.
> (this is the reason why you should have a separate asm diskgroup just for
> your voting / ocr files)
>
> Regards,
>
> Freek D'Hooge
> Uptime
> Oracle Database Administrator
> email: freek.dhooge@xxxxxxxxx
> tel +32(0)3 451 23 82
> http://www.uptime.be
> disclaimer: www.uptime.be/disclaimer
>
> -----Original Message-----
> From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx]
> On Behalf Of Martin Klier
> Sent: dinsdag 12 juli 2011 13:45
> To: Oracle L
> Subject: ASM Grid Infrastructure voting files not found any more
>
> Dear list,
>
> I messed up my ASM diskstring, after this and a reboot my cssd's do not
> start, because they can't find the voting files any more. Now one node
> is fencing, the other one just attempts to restart the CSSD frequently.
>
> How can I reconfigure the device name mask for the cssd (or the ASM disk
> string - if it helps), to allow it to find its voting files in the right
> way?
>
>
> Thank you very much, any help is greatly appreciated!
>
> Martin Klier
>
>
>
> Here the corresponding part of the ocssd log file. I think the crucial
> part is:
>
> 2011-07-11 09:36:45.976: [ CSSD][3997759248]clssnmvDiskCreate: Found
> a duplicate voting file
> /dev/disk/by-id/scsi-360080e50001c169e000002334dfebb44
> in the discovery queue which appears to be the same physical device as
> the newly discovered disk /dev/disk/by-id/ASMOCW2. Rejecting both these
> files
>
> It does repeat that for all three disks bearing the voting file copies.
> => bad
>
> =======================================================
>
> 2011-07-11 09:36:45.975: [ CSSD][3997759248]clssnmvDiskVerify:
> discovered a potential voting file
> 2011-07-11 09:36:45.975: [ SKGFD][3997759248]Handle 0x88e450 from lib
> :UFS:: for disk :/dev/disk/by-id/ASMOCW2:
>
> 2011-07-11 09:36:45.976: [ CSSD][3997759248]clssnmFindVF: found VF by
> vdin in the discovered queue
> 2011-07-11 09:36:45.976: [ CSSD][3997759248]clssnmFindVF: Duplicate
> voting file found in the queue of previously discovered disks
> queued(/dev/disk/by-id/scsi-360080e5000
> 1c169e000002334dfebb44|[7903ab4c-10b84f92-bfec4e33-5fb310fb]),
> found(/dev/disk/by-id/ASMOCW2|[7903ab4c-10b84f92-bfec4e33-5fb310fb])
> 2011-07-11 09:36:45.976: [ CSSD][3997759248]clssnmvDiskCreate: Found
> a duplicate voting file
> /dev/disk/by-id/scsi-360080e50001c169e000002334dfebb44 in the discovery
> queu
> e which appears to be the same physical device as the newly discovered
> disk /dev/disk/by-id/ASMOCW2. Rejecting both these files
> 2011-07-11 09:36:45.976: [ CSSD][3997759248]clssnmvDiskDestroy:
> removing the voting disk /dev/disk/by-id/ASMOCW2
> 2011-07-11 09:36:45.976: [ SKGFD][3997759248]Lib :UFS:: closing handle
> 0x88e450 for disk :/dev/disk/by-id/ASMOCW2:
>
> 2011-07-11 09:36:45.976: [ CSSD][3997759248]clssnmvDiskVerify:
> discovered a potential voting file
> 2011-07-11 09:36:45.976: [ SKGFD][3997759248]Handle 0x88dab0 from lib
> :UFS:: for disk :/dev/disk/by-id/ASMOCW3:
>
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmFindVF: found VF by
> vdin in the discovered queue
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmFindVF: Duplicate
> voting file found in the queue of previously discovered disks
> queued(/dev/disk/by-id/scsi-360080e5000
> 1c1726000003374e01867c|[70ccc4d5-e1d94f7d-bface149-7b3d8e5f]),
> found(/dev/disk/by-id/ASMOCW3|[70ccc4d5-e1d94f7d-bface149-7b3d8e5f])
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskCreate: Found
> a duplicate voting file
> /dev/disk/by-id/scsi-360080e50001c1726000003374e01867c in the discovery
> queu
> e which appears to be the same physical device as the newly discovered
> disk /dev/disk/by-id/ASMOCW3. Rejecting both these files
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskDestroy:
> removing the voting disk /dev/disk/by-id/ASMOCW3
> 2011-07-11 09:36:45.977: [ SKGFD][3997759248]Lib :UFS:: closing handle
> 0x88dab0 for disk :/dev/disk/by-id/ASMOCW3:
>
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskVerify:
> discovered a potential voting file
> 2011-07-11 09:36:45.977: [ SKGFD][3997759248]Handle 0x88dab0 from lib
> :UFS:: for disk :/dev/disk/by-id/ASMOCW1:
>
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmFindVF: found VF by
> vdin in the discovered queue
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmFindVF: Duplicate
> voting file found in the queue of previously discovered disks
> queued(/dev/disk/by-id/scsi-360080e5000
> 1c1726000002a54dfebb58|[416b1bc2-1f764f0f-bfc1430b-8e447d30]),
> found(/dev/disk/by-id/ASMOCW1|[416b1bc2-1f764f0f-bfc1430b-8e447d30])
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskCreate: Found
> a duplicate voting file
> /dev/disk/by-id/scsi-360080e50001c1726000002a54dfebb58 in the discovery
> queu
> e which appears to be the same physical device as the newly discovered
> disk /dev/disk/by-id/ASMOCW1. Rejecting both these files
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskDestroy:
> removing the voting disk /dev/disk/by-id/ASMOCW1
> 2011-07-11 09:36:45.977: [ SKGFD][3997759248]Lib :UFS:: closing handle
> 0x88dab0 for disk :/dev/disk/by-id/ASMOCW1:
>
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskVerify:
> Successful discovery of 3 disks
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248]clssnmCompleteInitVFDiscovery: Completing initial
> voting file discovery
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248]clssnmCompleteVFDiscovery: Completing voting file
> discovery
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskStateChange:
> state from discovered to deconfigured disk
> /dev/disk/by-id/scsi-360080e50001c1726000002a54dfebb58
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskStateChange:
> state from discovered to deconfigured disk
> /dev/disk/by-id/scsi-360080e50001c1726000003374e01867c
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]clssnmvDiskStateChange:
> state from discovered to deconfigured disk
> /dev/disk/by-id/scsi-360080e50001c169e000002334dfebb44
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248]clssnmvVerifyCommittedConfigVFs: Insufficient voting
> files found, found 0 of 3 configured, needed 2 voting files
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248](:CSSNM00020:)clssnmvVerifyCommittedConfigVFs: voting
> file 0, id 416b1bc2-1f764f0f-bfc1430b-8e447d30 not found
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248](:CSSNM00020:)clssnmvVerifyCommittedConfigVFs: voting
> file 1, id 7903ab4c-10b84f92-bfec4e33-5fb310fb not found
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248](:CSSNM00020:)clssnmvVerifyCommittedConfigVFs: voting
> file 2, id 70ccc4d5-e1d94f7d-bface149-7b3d8e5f not found
> 2011-07-11 09:36:45.977: [ CSSD][3997759248]ASSERT clssnm1.c 3141
> 2011-07-11 09:36:45.977: [
> CSSD][3997759248](:CSSNM00021:)clssnmCompleteVFDiscovery: Found 0 voting
> files, but 2 are required. Terminating due to insufficient configure
> d voting files
>
>
> --
> Usn's IT Blog for Linux, Oracle, Asterisk
> http://www.usn-it.de
>
> --
> //www.freelists.org/webpage/oracle-l
>
>
> --
> //www.freelists.org/webpage/oracle-l
>
>
>
--
//www.freelists.org/webpage/oracle-l
Other related posts:
