Re: OT - Blog entry on hugepages

  • From: kyle Hailey <kylelf@xxxxxxxxx>
  • To: rjoralist2@xxxxxxxxxxxxxxxxxxxxx
  • Date: Thu, 5 Apr 2012 11:03:36 -0700

Thanks for the added info.
AFAIK  filesystemio_options=setall
but this was over NFS and if the NFS mount doesn't have cio on it then I'm
not sure that cio will be used.
( Contrary to the IBM literature, we found that the cio option on NFS
mounts was actually slower.  )

Unfortunately I don't have output from the OS stat tools during the run and
memory of such things is less than dependable, but basically the story goes
like this:

Running an I/O benchmark tool on an NFS mount and  the read I/Os were 7ms
or so.
Then 2 databases with 8GB of SGA were started.
The I/O latency went up to 30ms or so.
According to customer there was no paging not even any scanning and there
was plenty of idle CPU. though CPU usage was heavily weighted toward sys.
Shutdown the databases. Set up largepages, start up the databases with
locked SGAs.
Now the I/O latency is back to 7ms,
Now overall CPU usage is lower and sys CPU is down.

What I don't  understand, and maybe the data recounted to me was incorrect,
but how there could be idle CPU and the NFS reads be so drastically
These NFS reads were separate from the database. We were running a stand
alone I/O benchmark and just stopping and starting two Oracle databases.

- Kyle Hailey

On Wed, Apr 4, 2012 at 2:10 PM, Rich Jesse <rjoralist2@xxxxxxxxxxxxxxxxxxxxx
> wrote:

> Hey Kyle,
> >   Does anyone have metrics to identify memory access problems on AIX
> other
> > than paging and scanning stats that might be solved with largepages?
> >   Recently an AIX customer  was seeing slow I/Os from an NFS filer but
> fast
> > local I/O response times.  Of course the customer blamed the NFS filer,
> but
> > the filer reported constant speeds during good periods as well as bad. I
> Was CPU consistent between the tests?
> Does the init.ora filesystemio_options='SETALL'?  I'm not sure how that
> affects the NFS mount, but for JFS2 mounts, that will cause Oracle to
> automagically use CIO (Ora10g and up) and also AIO, if enabled, IIRC.
> Oracle does this regardless of the mount options specified in
> /etc/filesystems, again for JFS2.
> I'm theorizing that if CIO's not used, there could be Oracle data files in
> the filecache of the local storage, which could result in higher CPU while
> still appearing as PIO to Oracle.  This would also depend on the VMO
> filecache settings of minperm/maxperm/maxclient.  I watch the filecache
> constantly in AIX's excellent nmon in the memory window, as well as a
> vmstat
> monitoring script containing:
> vmstat -v|grep -E 'numperm|file pages'
> Hope this drivel helps!  I don't claim to be an expert in this, but have
> been delving into this stuff now to help me determine how big I can make my
> buffer cache w/o causing AIX (v5.3) paging.
> GL!
> Rich
> --


Other related posts: