Hi Oracle Fans,
I need some help and advice on tracking down source of excessive IO.
Our 10.2.0.[2,3] RAC enviroments are configured with shared
ORACLE_HOME, residing on Netapp storage. The same volume also contains
the voting disk and OCR for the RAC.
This volume, that we assumed will have almost no load, actually has
about 6000 IOPS (IO operations per second), sometimes up to 13000. For
comparison, our data volume has about 1500 IOPS.
Interesting findings:
1) From Netapp reports and from some network sniffing, it appears that
over 90% of the IO is "access" and "get attribute" operations -
neither reads, nor writes.
2) This does not happen to the voting disks on RAC systems that have
no shared home. (although there are other configuration differences as
well).
3) This also does not happen in stand alone systems that have
ORACLE_HOME on the Netapp.
I can't use IOSTAT to debug because it does not report NFS. Network
monitoring aggregates over all NFS traffic, so I can't seperate normal
data traffic from strange ORACLE_HOME traffic (it is all on same
storage interface). Linux does not report network activity per process
(not without special patches). Wait event tables will not show traffic
to Oracle_home or to voting disk.
Any suggestions where to look?
Anyone can guess what we could have configured wrong to cause such an
issue? ( I know that we are all against random guessing, but it feels
like all investigation channels are insufficient).
Thanks,
Chen Shapira
--
//www.freelists.org/webpage/oracle-l
