Re: ASM disk path in exadata
- From: "Gaurav Mehta" <gaurav_mehta77@xxxxxxxxxxx>
- To: <acolvin@xxxxxxxxxxx>, <frits.hoogland@xxxxxxxxx>
- Date: Wed, 22 Feb 2012 23:10:54 +0530
Thanks.
-----Original Message-----
From: Andy Colvin
Sent: Tuesday, February 21, 2012 7:16 PM
To: frits.hoogland@xxxxxxxxx
Cc: oracle-l
Subject: Re: ASM disk path in exadata
Right…the ASM disks on Exadata do not exist on the database servers as
physical devices. It's not like a typical system that uses block or raw
devices for the ASM disks. They're connected using the iDB protocol over
RDS or UDP.
[enkdb01:root] /root
> fdisk -l
Disk /dev/sda: 291.9 GB, 291999055872 bytes
255 heads, 63 sectors/track, 35500 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 7832 62910508+ 83 Linux
/dev/sda2 7833 9920 16771860 82 Linux swap / Solaris
/dev/sda3 9921 35500 205471350 83 Linux
[enkdb01:root] /root
> kfod disks=all
--------------------------------------------------------------------------------
Disk Size Path User Group
================================================================================
1: 1501184 Mb o/192.168.12.3/DATA_CD_00_enkcel01 <unknown>
<unknown>
2: 1501184 Mb o/192.168.12.3/DATA_CD_01_enkcel01 <unknown>
<unknown>
3: 1501184 Mb o/192.168.12.3/DATA_CD_02_enkcel01 <unknown>
<unknown>
4: 1501184 Mb o/192.168.12.3/DATA_CD_03_enkcel01 <unknown>
<unknown>
5: 1501184 Mb o/192.168.12.3/DATA_CD_04_enkcel01 <unknown>
<unknown>
6: 1501184 Mb o/192.168.12.3/DATA_CD_05_enkcel01 <unknown>
<unknown>
7: 1501184 Mb o/192.168.12.3/DATA_CD_06_enkcel01 <unknown>
<unknown>
8: 1501184 Mb o/192.168.12.3/DATA_CD_07_enkcel01 <unknown>
<unknown>
9: 1501184 Mb o/192.168.12.3/DATA_CD_08_enkcel01 <unknown>
<unknown>
10: 1501184 Mb o/192.168.12.3/DATA_CD_09_enkcel01 <unknown>
<unknown>
11: 1501184 Mb o/192.168.12.3/DATA_CD_10_enkcel01 <unknown>
<unknown>
12: 1501184 Mb o/192.168.12.3/DATA_CD_11_enkcel01 <unknown>
<unknown>
13: 29808 Mb o/192.168.12.3/DBFS_DG_CD_02_enkcel01 <unknown>
<unknown>
14: 29808 Mb o/192.168.12.3/DBFS_DG_CD_03_enkcel01 <unknown>
<unknown>
15: 29808 Mb o/192.168.12.3/DBFS_DG_CD_04_enkcel01 <unknown>
<unknown>
16: 29808 Mb o/192.168.12.3/DBFS_DG_CD_05_enkcel01 <unknown>
<unknown>
17: 29808 Mb o/192.168.12.3/DBFS_DG_CD_06_enkcel01 <unknown>
<unknown>
18: 29808 Mb o/192.168.12.3/DBFS_DG_CD_07_enkcel01 <unknown>
<unknown>
19: 29808 Mb o/192.168.12.3/DBFS_DG_CD_08_enkcel01 <unknown>
<unknown>
20: 29808 Mb o/192.168.12.3/DBFS_DG_CD_09_enkcel01 <unknown>
<unknown>
21: 29808 Mb o/192.168.12.3/DBFS_DG_CD_10_enkcel01 <unknown>
<unknown>
22: 29808 Mb o/192.168.12.3/DBFS_DG_CD_11_enkcel01 <unknown>
<unknown>
23: 375344 Mb o/192.168.12.3/RECO_CD_00_enkcel01 <unknown>
<unknown>
24: 375344 Mb o/192.168.12.3/RECO_CD_01_enkcel01 <unknown>
<unknown>
25: 375344 Mb o/192.168.12.3/RECO_CD_02_enkcel01 <unknown>
<unknown>
26: 375344 Mb o/192.168.12.3/RECO_CD_03_enkcel01 <unknown>
<unknown>
27: 375344 Mb o/192.168.12.3/RECO_CD_04_enkcel01 <unknown>
<unknown>
28: 375344 Mb o/192.168.12.3/RECO_CD_05_enkcel01 <unknown>
<unknown>
29: 375344 Mb o/192.168.12.3/RECO_CD_06_enkcel01 <unknown>
<unknown>
30: 375344 Mb o/192.168.12.3/RECO_CD_07_enkcel01 <unknown>
<unknown>
31: 375344 Mb o/192.168.12.3/RECO_CD_08_enkcel01 <unknown>
<unknown>
32: 375344 Mb o/192.168.12.3/RECO_CD_09_enkcel01 <unknown>
<unknown>
33: 375344 Mb o/192.168.12.3/RECO_CD_10_enkcel01 <unknown>
<unknown>
34: 375344 Mb o/192.168.12.3/RECO_CD_11_enkcel01 <unknown>
<unknown>
35: 1501184 Mb o/192.168.12.4/DATA_CD_00_enkcel02 <unknown>
<unknown>
36: 1501184 Mb o/192.168.12.4/DATA_CD_01_enkcel02 <unknown>
<unknown>
37: 1501184 Mb o/192.168.12.4/DATA_CD_02_enkcel02 <unknown>
<unknown>
38: 1501184 Mb o/192.168.12.4/DATA_CD_03_enkcel02 <unknown>
<unknown>
39: 1501184 Mb o/192.168.12.4/DATA_CD_04_enkcel02 <unknown>
<unknown>
40: 1501184 Mb o/192.168.12.4/DATA_CD_05_enkcel02 <unknown>
<unknown>
41: 1501184 Mb o/192.168.12.4/DATA_CD_06_enkcel02 <unknown>
<unknown>
42: 1501184 Mb o/192.168.12.4/DATA_CD_07_enkcel02 <unknown>
<unknown>
43: 1501184 Mb o/192.168.12.4/DATA_CD_08_enkcel02 <unknown>
<unknown>
44: 1501184 Mb o/192.168.12.4/DATA_CD_09_enkcel02 <unknown>
<unknown>
45: 1501184 Mb o/192.168.12.4/DATA_CD_10_enkcel02 <unknown>
<unknown>
46: 1501184 Mb o/192.168.12.4/DATA_CD_11_enkcel02 <unknown>
<unknown>
47: 29808 Mb o/192.168.12.4/DBFS_DG_CD_02_enkcel02 <unknown>
<unknown>
48: 29808 Mb o/192.168.12.4/DBFS_DG_CD_03_enkcel02 <unknown>
<unknown>
49: 29808 Mb o/192.168.12.4/DBFS_DG_CD_04_enkcel02 <unknown>
<unknown>
50: 29808 Mb o/192.168.12.4/DBFS_DG_CD_05_enkcel02 <unknown>
<unknown>
51: 29808 Mb o/192.168.12.4/DBFS_DG_CD_06_enkcel02 <unknown>
<unknown>
52: 29808 Mb o/192.168.12.4/DBFS_DG_CD_07_enkcel02 <unknown>
<unknown>
53: 29808 Mb o/192.168.12.4/DBFS_DG_CD_08_enkcel02 <unknown>
<unknown>
54: 29808 Mb o/192.168.12.4/DBFS_DG_CD_09_enkcel02 <unknown>
<unknown>
55: 29808 Mb o/192.168.12.4/DBFS_DG_CD_10_enkcel02 <unknown>
<unknown>
56: 29808 Mb o/192.168.12.4/DBFS_DG_CD_11_enkcel02 <unknown>
<unknown>
57: 375344 Mb o/192.168.12.4/RECO_CD_00_enkcel02 <unknown>
<unknown>
58: 375344 Mb o/192.168.12.4/RECO_CD_01_enkcel02 <unknown>
<unknown>
59: 375344 Mb o/192.168.12.4/RECO_CD_02_enkcel02 <unknown>
<unknown>
60: 375344 Mb o/192.168.12.4/RECO_CD_03_enkcel02 <unknown>
<unknown>
61: 375344 Mb o/192.168.12.4/RECO_CD_04_enkcel02 <unknown>
<unknown>
62: 375344 Mb o/192.168.12.4/RECO_CD_05_enkcel02 <unknown>
<unknown>
63: 375344 Mb o/192.168.12.4/RECO_CD_06_enkcel02 <unknown>
<unknown>
64: 375344 Mb o/192.168.12.4/RECO_CD_07_enkcel02 <unknown>
<unknown>
65: 375344 Mb o/192.168.12.4/RECO_CD_08_enkcel02 <unknown>
<unknown>
66: 375344 Mb o/192.168.12.4/RECO_CD_09_enkcel02 <unknown>
<unknown>
67: 375344 Mb o/192.168.12.4/RECO_CD_10_enkcel02 <unknown>
<unknown>
68: 375344 Mb o/192.168.12.4/RECO_CD_11_enkcel02 <unknown>
<unknown>
69: 1501184 Mb o/192.168.12.5/DATA_CD_00_enkcel03 <unknown>
<unknown>
70: 1501184 Mb o/192.168.12.5/DATA_CD_01_enkcel03 <unknown>
<unknown>
71: 1501184 Mb o/192.168.12.5/DATA_CD_02_enkcel03 <unknown>
<unknown>
72: 1501184 Mb o/192.168.12.5/DATA_CD_03_enkcel03 <unknown>
<unknown>
73: 1501184 Mb o/192.168.12.5/DATA_CD_04_enkcel03 <unknown>
<unknown>
74: 1501184 Mb o/192.168.12.5/DATA_CD_05_enkcel03 <unknown>
<unknown>
75: 1501184 Mb o/192.168.12.5/DATA_CD_06_enkcel03 <unknown>
<unknown>
76: 1501184 Mb o/192.168.12.5/DATA_CD_07_enkcel03 <unknown>
<unknown>
77: 1501184 Mb o/192.168.12.5/DATA_CD_08_enkcel03 <unknown>
<unknown>
78: 1501184 Mb o/192.168.12.5/DATA_CD_09_enkcel03 <unknown>
<unknown>
79: 1501184 Mb o/192.168.12.5/DATA_CD_10_enkcel03 <unknown>
<unknown>
80: 1501184 Mb o/192.168.12.5/DATA_CD_11_enkcel03 <unknown>
<unknown>
81: 29808 Mb o/192.168.12.5/DBFS_DG_CD_02_enkcel03 <unknown>
<unknown>
82: 29808 Mb o/192.168.12.5/DBFS_DG_CD_03_enkcel03 <unknown>
<unknown>
83: 29808 Mb o/192.168.12.5/DBFS_DG_CD_04_enkcel03 <unknown>
<unknown>
84: 29808 Mb o/192.168.12.5/DBFS_DG_CD_05_enkcel03 <unknown>
<unknown>
85: 29808 Mb o/192.168.12.5/DBFS_DG_CD_06_enkcel03 <unknown>
<unknown>
86: 29808 Mb o/192.168.12.5/DBFS_DG_CD_07_enkcel03 <unknown>
<unknown>
87: 29808 Mb o/192.168.12.5/DBFS_DG_CD_08_enkcel03 <unknown>
<unknown>
88: 29808 Mb o/192.168.12.5/DBFS_DG_CD_09_enkcel03 <unknown>
<unknown>
89: 29808 Mb o/192.168.12.5/DBFS_DG_CD_10_enkcel03 <unknown>
<unknown>
90: 29808 Mb o/192.168.12.5/DBFS_DG_CD_11_enkcel03 <unknown>
<unknown>
91: 375344 Mb o/192.168.12.5/RECO_CD_00_enkcel03 <unknown>
<unknown>
92: 375344 Mb o/192.168.12.5/RECO_CD_01_enkcel03 <unknown>
<unknown>
93: 375344 Mb o/192.168.12.5/RECO_CD_02_enkcel03 <unknown>
<unknown>
94: 375344 Mb o/192.168.12.5/RECO_CD_03_enkcel03 <unknown>
<unknown>
95: 375344 Mb o/192.168.12.5/RECO_CD_04_enkcel03 <unknown>
<unknown>
96: 375344 Mb o/192.168.12.5/RECO_CD_05_enkcel03 <unknown>
<unknown>
97: 375344 Mb o/192.168.12.5/RECO_CD_06_enkcel03 <unknown>
<unknown>
98: 375344 Mb o/192.168.12.5/RECO_CD_07_enkcel03 <unknown>
<unknown>
99: 375344 Mb o/192.168.12.5/RECO_CD_08_enkcel03 <unknown>
<unknown>
100: 375344 Mb o/192.168.12.5/RECO_CD_09_enkcel03 <unknown>
<unknown>
101: 375344 Mb o/192.168.12.5/RECO_CD_10_enkcel03 <unknown>
<unknown>
102: 375344 Mb o/192.168.12.5/RECO_CD_11_enkcel03 <unknown>
<unknown>
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
================================================================================
+ASM1 /u01/app/11.2.0.3/grid
+ASM2 /u01/app/11.2.0.3/grid
Andy Colvin
Principal Consultant
Enkitec
andy.colvin@xxxxxxxxxxx
http://blog.oracle-ninja.com
On Feb 21, 2012, at 6:07 AM, Frits Hoogland wrote:
> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;
> charset=us-ascii
>
> There is no udev/path translation on the database/computing layer in >
> exadata.
>
> During startup, ASM first needs to find the disks. It does that by >
> picking up the cellserver information (addresses) in >
> /etc/oracle/cell/network-config/cellip.ora
> Because the discovery string is set to "o/*/*", it picks up all >
> griddisks available on the cellservers defined in cellip.ora
>
> Because of the entirely different path, ASM knows it needs to >
> communicate over the network to get to the griddisks, instead of using a >
> locally attached disk. This combination is RDS (infiniband) on Exadata, >
> but can be UDP (ethernet).
>
> In fact, the discovery can manually be done by using kfod:
> kfod asm_diskstring="o/*/*", disks=all
> this is what the asmca uses behind the GUI to discover disks, also on >
> normal platforms.
> When specifying the o/*/* diskstring, kfod too reads >
> /etc/oracle/cell/network-config/cellip.ora to know where to look.
>
>
> Frits Hoogland
>
> http://fritshoogland.wordpress.com
> mailto:frits.hoogland@xxxxxxxxx
> cell: +31 6 53569942
>
> On Feb 21, 2012, at 10:32 AM, gaurav mehta wrote:
>
>> I have been trying to figure out how exadata griddisks are presented > as
>> candidate asm disks. We know that the disk discovery path for asm >
>> instances using exadata is "o/<cell_ip_address>*/<griddisk_name>*". I >
>> was wondering what actual path / udev in the operating system on the db >
>> machine does this translate to ?.
>> --
>> //www.freelists.org/webpage/oracle-l
>>
>>
>
>
>
>
> --
> //www.freelists.org/webpage/oracle-l
>
>
--
//www.freelists.org/webpage/oracle-l
--
//www.freelists.org/webpage/oracle-l
Other related posts:
