RE: OT - Blog entry on hugepages

  • From: "CRISLER, JON A" <JC1706@xxxxxxx>
  • To: "Walker, Jed S" <Jed_Walker@xxxxxxxxxxxxxxxxx>, "oracle-l@xxxxxxxxxxxxx" <oracle-l@xxxxxxxxxxxxx>
  • Date: Thu, 5 Apr 2012 22:23:23 +0000

A round-about way would be to set up the /etc/security/limits.conf file so that 
only the "oracle" user has rights to hugepages:


-- this sets up permission to use about 80gb of memory, but does not actually 
allocate anything
oracle hard memlock 83886080
oracle soft memlock 83886080

log out / back in, shutdown and restart oracle, then check /proc/meminfo.  

Example-

HugePages_Total: 25000
HugePages_Free:   6092
HugePages_Rsvd:   5669

What I don't get is why some memory is Rsvd but not used sometimes: probably 
some pool has not grown much and used the memory.

Are you using Oracle 11g AMM ? If so you need to unset memory_max_size and 
memory_target, otherwise they try to use /dev/shm and will not use hugepages.  
On Linux, AMM and hugepages are mutually exclusive, but you can use both on AIX 
and Solaris I believe.  In other words, on Oracle 11g, in order to use 
hugepages you have to revert to 10g style memory management.  In my experience 
for larger systems (> 10gb SGA) this is a worthwhile tradeoff for better 
performance.

-----Original Message-----
From: Walker, Jed S [mailto:Jed_Walker@xxxxxxxxxxxxxxxxx] 
Sent: Thursday, April 05, 2012 5:44 PM
To: CRISLER, JON A; oracle-l@xxxxxxxxxxxxx
Subject: RE: OT - Blog entry on hugepages

Yes, I can see that hugepages is active via /proc/meminfo, I'm asking how to 
actually tell if it really is Oracle using it. That is why I'm asking about 
ipcs.

Mark sent this link which I'll read: 
http://yong321.freeshell.org/oranotes/HugePages.txt


-----Original Message-----
From: CRISLER, JON A [mailto:JC1706@xxxxxxx] 
Sent: Thursday, April 05, 2012 3:28 PM
To: Walker, Jed S; oracle-l@xxxxxxxxxxxxx
Subject: RE: OT - Blog entry on hugepages

I am only familier with this on Linux- this does not seem to prove or disprove 
hugepages, only shared memory segments right?
To look at hugepages on Linux, do a cat /proc/meminfo- other OS's require 
different methods, and the default hugepage size in Linux is 2mb.

-----Original Message-----
From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On 
Behalf Of Walker, Jed S
Sent: Thursday, April 05, 2012 4:28 PM
To: oracle-l@xxxxxxxxxxxxx
Subject: RE: OT - Blog entry on hugepages

The Oracle documents say you can tell hugepages are being used  by the 
following:
      To make sure that the configuration is valid, the HugePages_Free value 
should be smaller than HugePages_Total and there should be some HugePages_Rsvd. 
The sum of Hugepages_Free and HugePages_Rsvd may be smaller than your total 
combined SGA as instances allocate pages dynamically and proactively as needed.
This does seem to indicate they are being used, but I was wondering if there's 
any other way to truly see it. I'm wondering if anyone can confirm my thinking 
on this. I used

>ipcs
 ------ Shared Memory Segments --------
 key        shmid      owner     perms      bytes       nattch     status
 0x00000000 29097988   oracle    640        4096       0
 0x00000000 29130757   oracle    640        4096       0
 0x0e2f3c64 29163526   oracle    640        4096       0
 0x00000000 29655047   oracle    640        268435456  59
 0x00000000 29687816   oracle    640        51271172096 59
 0x2894b058 29720585   oracle    640        2097152    59

I could be wrong, but 51271172096/1024/1024/1000H (since pages are in kbytes, 
this would seem to be my 48G SGA). I'm not sure what the one above it is and it 
is big enough to make me wonder.

Does that look like it makes sense?

(Still wish someone had some test results to show it really helps)

Thanks,

Jed

-----Original Message-----
From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On 
Behalf Of Rich Jesse
Sent: Wednesday, April 04, 2012 3:10 PM
To: oracle-l@xxxxxxxxxxxxx
Subject: Re: OT - Blog entry on hugepages

Hey Kyle,

>   Does anyone have metrics to identify memory access problems on AIX 
> other than paging and scanning stats that might be solved with largepages?
>   Recently an AIX customer  was seeing slow I/Os from an NFS filer but 
> fast local I/O response times.  Of course the customer blamed the NFS 
> filer, but the filer reported constant speeds during good periods as 
> well as bad. I

Was CPU consistent between the tests?

Does the init.ora filesystemio_options='SETALL'?  I'm not sure how that affects 
the NFS mount, but for JFS2 mounts, that will cause Oracle to automagically use 
CIO (Ora10g and up) and also AIO, if enabled, IIRC.
Oracle does this regardless of the mount options specified in /etc/filesystems, 
again for JFS2.

I'm theorizing that if CIO's not used, there could be Oracle data files in the 
filecache of the local storage, which could result in higher CPU while still 
appearing as PIO to Oracle.  This would also depend on the VMO filecache 
settings of minperm/maxperm/maxclient.  I watch the filecache constantly in 
AIX's excellent nmon in the memory window, as well as a vmstat monitoring 
script containing:

vmstat -v|grep -E 'numperm|file pages'

Hope this drivel helps!  I don't claim to be an expert in this, but have been 
delving into this stuff now to help me determine how big I can make my buffer 
cache w/o causing AIX (v5.3) paging.

GL!

Rich

--
//www.freelists.org/webpage/oracle-l




--
//www.freelists.org/webpage/oracle-l


--
//www.freelists.org/webpage/oracle-l

Other related posts:

  1. Home
  2. oracle-l
  3. Archive
  4. 04-2012