Re: caution on Linux asmlib in DW setups

  • From: Marc Slemko <identd@xxxxxxxxx>
  • To: oracle-l@xxxxxxxxxxxxx
  • Date: Mon, 7 Feb 2005 14:20:31 -0800

On Mon, 7 Feb 2005 02:28:04 -0800, Marc Slemko <identd@xxxxxxxxx> wrote:
> Just a note so anyone else thinking about this can hopefully avoid the
> same problems I ran into.  I haven't found any other mentions of this.
> 
> I've been working on a new 10g install on Linux.  We have been using
> ASM, with Oracle's  asmlib kernel module to simply disk configuration.
> 
> We have a system setup with 5 10k RPM SCSI disks and were only getting
> about 60 megabytes/sec out during full table scans.  Each disk by
> itself can do that, so something didn't add up.  Since this system is
> a data warehouse system, this sort of throughput is somewhat
> problematic.
> 
> When I dug into it, I found that the asmlib kernel module limited each
> IO to 64k!  From the code:
> 
> /*
>  * Max I/O is 64K.  Why 64?  Because the MegaRAID card supports only
>  * 26 scatter/gather segments.  26 * 4k page == 104k.  The power of 2
>  * below that is 64K.  The intention is for per-device discovery of this
>  * value in the future.  FIXME.
>  */
> #define ASM_MAX_IOSIZE    (1024 * 64)
> 
> Needless to say, we aren't using a megaraid, and 64k IOs are pretty limiting.
> 
> When I changed this to 256k, throughput during full table scans went
> up from 60 megabytes/sec to 180 megabytes/sec.
> 
> I have opened a TAR on the issue, but since we are strapped for time
> have just decided to skip asmlib since I'm not too keen on the idea of
> all my IO going through a not-yet-ready-for-primetime kernel module.
> You don't need to use asmlib to use ASM, it just makes administering
> it a bit easier.
> 
> Anyway, just FYI in case you are thinking of using asmlib on Linux in
> an environment where full table scans are frequent.  I woudln't expect
> it would make too much difference in the canonical OLTP environment...
> but since full table scans are increasingly becoming relevant in a lot
> of environments, you never know.

Just to keep anyone who is interested updated... I got the following
response to my TAR:

  This was reported in internal bug 4223517 about 2 weeks ago. I will
  update the bug to let
  development know that you are experiencing the same issue.

  Hdr: 4113517 10.1 8 1.0.3 PRODID-1309 PORTID-46
  Abstract: ASMLIB FOR LINUX REPORTS A MAXIO SIZE OF 64K

For now we've just switched to not use it, it just isn't worth it and
doesn't seem fully baked.  Since asmlib isn't just used for discovery
of devices, but every single IO operation goes through another layer,
it isn't something to toy with.

Also note the patch notice Oracle has stuck up at
http://www.oracle.com/technology/tech/linux/asmlib/index.html :

"It is recommended that customers using the ASMLIB for Linux download
and apply the following patch from metalink to avoid some possible
data corruption issues which might arise in extremely heavy workload
environments. The patch number is 3800160 and it fixes a bug found on
the Oracle database kernel side of the interface. The patch is
currently available for x86 Linux only. It will be available on IA64
Linux and AMD64 Linux shortly and it will be included in the 10.1.0.4
patchset. "

The bug referenced by the patch doesn't seem to be public though so
not sure of any details beyond that.
--
//www.freelists.org/webpage/oracle-l

Other related posts: