caution on Linux asmlib in DW setups

  • From: Marc Slemko <identd@xxxxxxxxx>
  • To: oracle-l@xxxxxxxxxxxxx
  • Date: Mon, 7 Feb 2005 02:28:04 -0800

Just a note so anyone else thinking about this can hopefully avoid the
same problems I ran into.  I haven't found any other mentions of this.

I've been working on a new 10g install on Linux.  We have been using
ASM, with Oracle's  asmlib kernel module to simply disk configuration.

We have a system setup with 5 10k RPM SCSI disks and were only getting
about 60 megabytes/sec out during full table scans.  Each disk by
itself can do that, so something didn't add up.  Since this system is
a data warehouse system, this sort of throughput is somewhat
problematic.

When I dug into it, I found that the asmlib kernel module limited each
IO to 64k!  From the code:

/*
 * Max I/O is 64K.  Why 64?  Because the MegaRAID card supports only
 * 26 scatter/gather segments.  26 * 4k page == 104k.  The power of 2
 * below that is 64K.  The intention is for per-device discovery of this
 * value in the future.  FIXME.
 */
#define ASM_MAX_IOSIZE    (1024 * 64)

Needless to say, we aren't using a megaraid, and 64k IOs are pretty limiting.

When I changed this to 256k, throughput during full table scans went
up from 60 megabytes/sec to 180 megabytes/sec.

I have opened a TAR on the issue, but since we are strapped for time
have just decided to skip asmlib since I'm not too keen on the idea of
all my IO going through a not-yet-ready-for-primetime kernel module. 
You don't need to use asmlib to use ASM, it just makes administering
it a bit easier.

Anyway, just FYI in case you are thinking of using asmlib on Linux in
an environment where full table scans are frequent.  I woudln't expect
it would make too much difference in the canonical OLTP environment...
but since full table scans are increasingly becoming relevant in a lot
of environments, you never know.
--
//www.freelists.org/webpage/oracle-l

Other related posts: