Re: ASM disk path in exadata

Right?the ASM disks on Exadata do not exist on the database servers as physical 
devices.  It's not like a typical system that uses block or raw devices for the 
ASM disks.  They're connected using the iDB protocol over RDS or UDP.
[enkdb01:root] /root 
> fdisk -l

Disk /dev/sda: 291.9 GB, 291999055872 bytes
255 heads, 63 sectors/track, 35500 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        7832    62910508+  83  Linux
/dev/sda2            7833        9920    16771860   82  Linux swap / Solaris
/dev/sda3            9921       35500   205471350   83  Linux

[enkdb01:root] /root 
> kfod disks=all
--------------------------------------------------------------------------------
 Disk          Size Path                                     User     Group   
================================================================================
   1:    1501184 Mb o/192.168.12.3/DATA_CD_00_enkcel01       <unknown> <unknown>
   2:    1501184 Mb o/192.168.12.3/DATA_CD_01_enkcel01       <unknown> <unknown>
   3:    1501184 Mb o/192.168.12.3/DATA_CD_02_enkcel01       <unknown> <unknown>
   4:    1501184 Mb o/192.168.12.3/DATA_CD_03_enkcel01       <unknown> <unknown>
   5:    1501184 Mb o/192.168.12.3/DATA_CD_04_enkcel01       <unknown> <unknown>
   6:    1501184 Mb o/192.168.12.3/DATA_CD_05_enkcel01       <unknown> <unknown>
   7:    1501184 Mb o/192.168.12.3/DATA_CD_06_enkcel01       <unknown> <unknown>
   8:    1501184 Mb o/192.168.12.3/DATA_CD_07_enkcel01       <unknown> <unknown>
   9:    1501184 Mb o/192.168.12.3/DATA_CD_08_enkcel01       <unknown> <unknown>
  10:    1501184 Mb o/192.168.12.3/DATA_CD_09_enkcel01       <unknown> <unknown>
  11:    1501184 Mb o/192.168.12.3/DATA_CD_10_enkcel01       <unknown> <unknown>
  12:    1501184 Mb o/192.168.12.3/DATA_CD_11_enkcel01       <unknown> <unknown>
  13:      29808 Mb o/192.168.12.3/DBFS_DG_CD_02_enkcel01    <unknown> <unknown>
  14:      29808 Mb o/192.168.12.3/DBFS_DG_CD_03_enkcel01    <unknown> <unknown>
  15:      29808 Mb o/192.168.12.3/DBFS_DG_CD_04_enkcel01    <unknown> <unknown>
  16:      29808 Mb o/192.168.12.3/DBFS_DG_CD_05_enkcel01    <unknown> <unknown>
  17:      29808 Mb o/192.168.12.3/DBFS_DG_CD_06_enkcel01    <unknown> <unknown>
  18:      29808 Mb o/192.168.12.3/DBFS_DG_CD_07_enkcel01    <unknown> <unknown>
  19:      29808 Mb o/192.168.12.3/DBFS_DG_CD_08_enkcel01    <unknown> <unknown>
  20:      29808 Mb o/192.168.12.3/DBFS_DG_CD_09_enkcel01    <unknown> <unknown>
  21:      29808 Mb o/192.168.12.3/DBFS_DG_CD_10_enkcel01    <unknown> <unknown>
  22:      29808 Mb o/192.168.12.3/DBFS_DG_CD_11_enkcel01    <unknown> <unknown>
  23:     375344 Mb o/192.168.12.3/RECO_CD_00_enkcel01       <unknown> <unknown>
  24:     375344 Mb o/192.168.12.3/RECO_CD_01_enkcel01       <unknown> <unknown>
  25:     375344 Mb o/192.168.12.3/RECO_CD_02_enkcel01       <unknown> <unknown>
  26:     375344 Mb o/192.168.12.3/RECO_CD_03_enkcel01       <unknown> <unknown>
  27:     375344 Mb o/192.168.12.3/RECO_CD_04_enkcel01       <unknown> <unknown>
  28:     375344 Mb o/192.168.12.3/RECO_CD_05_enkcel01       <unknown> <unknown>
  29:     375344 Mb o/192.168.12.3/RECO_CD_06_enkcel01       <unknown> <unknown>
  30:     375344 Mb o/192.168.12.3/RECO_CD_07_enkcel01       <unknown> <unknown>
  31:     375344 Mb o/192.168.12.3/RECO_CD_08_enkcel01       <unknown> <unknown>
  32:     375344 Mb o/192.168.12.3/RECO_CD_09_enkcel01       <unknown> <unknown>
  33:     375344 Mb o/192.168.12.3/RECO_CD_10_enkcel01       <unknown> <unknown>
  34:     375344 Mb o/192.168.12.3/RECO_CD_11_enkcel01       <unknown> <unknown>
  35:    1501184 Mb o/192.168.12.4/DATA_CD_00_enkcel02       <unknown> <unknown>
  36:    1501184 Mb o/192.168.12.4/DATA_CD_01_enkcel02       <unknown> <unknown>
  37:    1501184 Mb o/192.168.12.4/DATA_CD_02_enkcel02       <unknown> <unknown>
  38:    1501184 Mb o/192.168.12.4/DATA_CD_03_enkcel02       <unknown> <unknown>
  39:    1501184 Mb o/192.168.12.4/DATA_CD_04_enkcel02       <unknown> <unknown>
  40:    1501184 Mb o/192.168.12.4/DATA_CD_05_enkcel02       <unknown> <unknown>
  41:    1501184 Mb o/192.168.12.4/DATA_CD_06_enkcel02       <unknown> <unknown>
  42:    1501184 Mb o/192.168.12.4/DATA_CD_07_enkcel02       <unknown> <unknown>
  43:    1501184 Mb o/192.168.12.4/DATA_CD_08_enkcel02       <unknown> <unknown>
  44:    1501184 Mb o/192.168.12.4/DATA_CD_09_enkcel02       <unknown> <unknown>
  45:    1501184 Mb o/192.168.12.4/DATA_CD_10_enkcel02       <unknown> <unknown>
  46:    1501184 Mb o/192.168.12.4/DATA_CD_11_enkcel02       <unknown> <unknown>
  47:      29808 Mb o/192.168.12.4/DBFS_DG_CD_02_enkcel02    <unknown> <unknown>
  48:      29808 Mb o/192.168.12.4/DBFS_DG_CD_03_enkcel02    <unknown> <unknown>
  49:      29808 Mb o/192.168.12.4/DBFS_DG_CD_04_enkcel02    <unknown> <unknown>
  50:      29808 Mb o/192.168.12.4/DBFS_DG_CD_05_enkcel02    <unknown> <unknown>
  51:      29808 Mb o/192.168.12.4/DBFS_DG_CD_06_enkcel02    <unknown> <unknown>
  52:      29808 Mb o/192.168.12.4/DBFS_DG_CD_07_enkcel02    <unknown> <unknown>
  53:      29808 Mb o/192.168.12.4/DBFS_DG_CD_08_enkcel02    <unknown> <unknown>
  54:      29808 Mb o/192.168.12.4/DBFS_DG_CD_09_enkcel02    <unknown> <unknown>
  55:      29808 Mb o/192.168.12.4/DBFS_DG_CD_10_enkcel02    <unknown> <unknown>
  56:      29808 Mb o/192.168.12.4/DBFS_DG_CD_11_enkcel02    <unknown> <unknown>
  57:     375344 Mb o/192.168.12.4/RECO_CD_00_enkcel02       <unknown> <unknown>
  58:     375344 Mb o/192.168.12.4/RECO_CD_01_enkcel02       <unknown> <unknown>
  59:     375344 Mb o/192.168.12.4/RECO_CD_02_enkcel02       <unknown> <unknown>
  60:     375344 Mb o/192.168.12.4/RECO_CD_03_enkcel02       <unknown> <unknown>
  61:     375344 Mb o/192.168.12.4/RECO_CD_04_enkcel02       <unknown> <unknown>
  62:     375344 Mb o/192.168.12.4/RECO_CD_05_enkcel02       <unknown> <unknown>
  63:     375344 Mb o/192.168.12.4/RECO_CD_06_enkcel02       <unknown> <unknown>
  64:     375344 Mb o/192.168.12.4/RECO_CD_07_enkcel02       <unknown> <unknown>
  65:     375344 Mb o/192.168.12.4/RECO_CD_08_enkcel02       <unknown> <unknown>
  66:     375344 Mb o/192.168.12.4/RECO_CD_09_enkcel02       <unknown> <unknown>
  67:     375344 Mb o/192.168.12.4/RECO_CD_10_enkcel02       <unknown> <unknown>
  68:     375344 Mb o/192.168.12.4/RECO_CD_11_enkcel02       <unknown> <unknown>
  69:    1501184 Mb o/192.168.12.5/DATA_CD_00_enkcel03       <unknown> <unknown>
  70:    1501184 Mb o/192.168.12.5/DATA_CD_01_enkcel03       <unknown> <unknown>
  71:    1501184 Mb o/192.168.12.5/DATA_CD_02_enkcel03       <unknown> <unknown>
  72:    1501184 Mb o/192.168.12.5/DATA_CD_03_enkcel03       <unknown> <unknown>
  73:    1501184 Mb o/192.168.12.5/DATA_CD_04_enkcel03       <unknown> <unknown>
  74:    1501184 Mb o/192.168.12.5/DATA_CD_05_enkcel03       <unknown> <unknown>
  75:    1501184 Mb o/192.168.12.5/DATA_CD_06_enkcel03       <unknown> <unknown>
  76:    1501184 Mb o/192.168.12.5/DATA_CD_07_enkcel03       <unknown> <unknown>
  77:    1501184 Mb o/192.168.12.5/DATA_CD_08_enkcel03       <unknown> <unknown>
  78:    1501184 Mb o/192.168.12.5/DATA_CD_09_enkcel03       <unknown> <unknown>
  79:    1501184 Mb o/192.168.12.5/DATA_CD_10_enkcel03       <unknown> <unknown>
  80:    1501184 Mb o/192.168.12.5/DATA_CD_11_enkcel03       <unknown> <unknown>
  81:      29808 Mb o/192.168.12.5/DBFS_DG_CD_02_enkcel03    <unknown> <unknown>
  82:      29808 Mb o/192.168.12.5/DBFS_DG_CD_03_enkcel03    <unknown> <unknown>
  83:      29808 Mb o/192.168.12.5/DBFS_DG_CD_04_enkcel03    <unknown> <unknown>
  84:      29808 Mb o/192.168.12.5/DBFS_DG_CD_05_enkcel03    <unknown> <unknown>
  85:      29808 Mb o/192.168.12.5/DBFS_DG_CD_06_enkcel03    <unknown> <unknown>
  86:      29808 Mb o/192.168.12.5/DBFS_DG_CD_07_enkcel03    <unknown> <unknown>
  87:      29808 Mb o/192.168.12.5/DBFS_DG_CD_08_enkcel03    <unknown> <unknown>
  88:      29808 Mb o/192.168.12.5/DBFS_DG_CD_09_enkcel03    <unknown> <unknown>
  89:      29808 Mb o/192.168.12.5/DBFS_DG_CD_10_enkcel03    <unknown> <unknown>
  90:      29808 Mb o/192.168.12.5/DBFS_DG_CD_11_enkcel03    <unknown> <unknown>
  91:     375344 Mb o/192.168.12.5/RECO_CD_00_enkcel03       <unknown> <unknown>
  92:     375344 Mb o/192.168.12.5/RECO_CD_01_enkcel03       <unknown> <unknown>
  93:     375344 Mb o/192.168.12.5/RECO_CD_02_enkcel03       <unknown> <unknown>
  94:     375344 Mb o/192.168.12.5/RECO_CD_03_enkcel03       <unknown> <unknown>
  95:     375344 Mb o/192.168.12.5/RECO_CD_04_enkcel03       <unknown> <unknown>
  96:     375344 Mb o/192.168.12.5/RECO_CD_05_enkcel03       <unknown> <unknown>
  97:     375344 Mb o/192.168.12.5/RECO_CD_06_enkcel03       <unknown> <unknown>
  98:     375344 Mb o/192.168.12.5/RECO_CD_07_enkcel03       <unknown> <unknown>
  99:     375344 Mb o/192.168.12.5/RECO_CD_08_enkcel03       <unknown> <unknown>
 100:     375344 Mb o/192.168.12.5/RECO_CD_09_enkcel03       <unknown> <unknown>
 101:     375344 Mb o/192.168.12.5/RECO_CD_10_enkcel03       <unknown> <unknown>
 102:     375344 Mb o/192.168.12.5/RECO_CD_11_enkcel03       <unknown> <unknown>
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME                                                          
================================================================================
     +ASM1 /u01/app/11.2.0.3/grid                                               
     +ASM2 /u01/app/11.2.0.3/grid                         

Andy Colvin

Principal Consultant
Enkitec
andy.colvin@xxxxxxxxxxx
http://blog.oracle-ninja.com

On Feb 21, 2012, at 6:07 AM, Frits Hoogland wrote:

> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;
>       charset=us-ascii
> 
> There is no udev/path translation on the database/computing layer in > 
> exadata.
> 
> During startup, ASM first needs to find the disks. It does that by > picking 
> up the cellserver information (addresses) in > 
> /etc/oracle/cell/network-config/cellip.ora
> Because the discovery string is set to "o/*/*", it picks up all > griddisks 
> available on the cellservers defined in cellip.ora
> 
> Because of the entirely different path, ASM knows it needs to > communicate 
> over the network to get to the griddisks, instead of using a > locally 
> attached disk. This combination is RDS (infiniband) on Exadata, > but can be 
> UDP (ethernet).
> 
> In fact, the discovery can manually be done by using kfod:
> kfod asm_diskstring="o/*/*", disks=all
> this is what the asmca uses behind the GUI to discover disks, also on > 
> normal platforms.
> When specifying the o/*/* diskstring, kfod too reads > 
> /etc/oracle/cell/network-config/cellip.ora to know where to look.
> 
> 
> Frits Hoogland
> 
> http://fritshoogland.wordpress.com
> mailto:frits.hoogland@xxxxxxxxx
> cell: +31 6 53569942
> 
> On Feb 21, 2012, at 10:32 AM, gaurav mehta wrote:
> 
>> I have been trying to figure out how exadata griddisks are presented > as 
>> candidate asm disks.  We know that the disk discovery path for asm > 
>> instances using exadata is "o/<cell_ip_address>*/<griddisk_name>*".  I > was 
>> wondering what actual path / udev in the operating system on the db > 
>> machine does this translate to ?.   
>> --
>> http://www.freelists.org/webpage/oracle-l
>>  
>>  
> 
> 
> 
> 
> --
> http://www.freelists.org/webpage/oracle-l
> 
> 


--
http://www.freelists.org/webpage/oracle-l


Other related posts: