RE: OT - Blog entry on hugepages

  • From: "Walker, Jed S" <Jed_Walker@xxxxxxxxxxxxxxxxx>
  • To: Andy Gmail <andrew.kerber@xxxxxxxxx>, "jc1706@xxxxxxx" <jc1706@xxxxxxx>
  • Date: Tue, 10 Apr 2012 14:02:37 +0000

Thanks Andy,

I did find info in the alert log.

Good->
 Starting ORACLE instance (normal)
 ****************** Large Pages Information *****************
 Parameter use_large_pages = ONLY  

 Total Shared Global Region in Large Pages = 48 GB (100%)

 Large Pages used by this instance: 24577 (48 GB)
 Large Pages unused system wide = 3 (6144 KB) (alloc incr 128 MB)
 Large Pages configured system wide = 24580 (48 GB)
 Large Page size = 2048 KB
 ***********************************************************

Not good->
 Starting ORACLE instance (normal)
 ****************** Large Pages Information *****************
 
 Total Shared Global Region in Large Pages = 0 KB (0%)

 Large Pages used by this instance: 0 (0 KB)
 Large Pages unused system wide = 0 (0 KB) (alloc incr 128 MB)
 Large Pages configured system wide = 0 (0 KB)
 Large Page size = 2048 KB

 RECOMMENDATION:
   Total Shared Global Region size is 48 GB. For optimal performance,
   prior to the next instance restart increase the number
   of unused Large Pages by atleast 24577 2048 KB Large Pages (48 GB)
   system wide to get 100% of the Shared
   Global Region allocated with Large pages
 ****************** Large Pages Information *****************

-----Original Message-----
From: Andy Gmail [mailto:andrew.kerber@xxxxxxxxx] 
Sent: Thursday, April 05, 2012 5:11 PM
To: JC1706@xxxxxxx
Cc: Walker, Jed S; oracle-l@xxxxxxxxxxxxx
Subject: Re: OT - Blog entry on hugepages

Oracle 11.2 shows huge page usage in the alert log on startup, at least on 
Linux.  I think that would be conclusive in that case.

Sent from my iPhone

On Apr 5, 2012, at 5:23 PM, "CRISLER, JON A" <JC1706@xxxxxxx> wrote:

> A round-about way would be to set up the /etc/security/limits.conf file so 
> that only the "oracle" user has rights to hugepages:
> 
> 
> -- this sets up permission to use about 80gb of memory, but does not 
> actually allocate anything oracle hard memlock 83886080 oracle soft 
> memlock 83886080
> 
> log out / back in, shutdown and restart oracle, then check /proc/meminfo.  
> 
> Example-
> 
> HugePages_Total: 25000
> HugePages_Free:   6092
> HugePages_Rsvd:   5669
> 
> What I don't get is why some memory is Rsvd but not used sometimes: probably 
> some pool has not grown much and used the memory.
> 
> Are you using Oracle 11g AMM ? If so you need to unset memory_max_size and 
> memory_target, otherwise they try to use /dev/shm and will not use hugepages. 
>  On Linux, AMM and hugepages are mutually exclusive, but you can use both on 
> AIX and Solaris I believe.  In other words, on Oracle 11g, in order to use 
> hugepages you have to revert to 10g style memory management.  In my 
> experience for larger systems (> 10gb SGA) this is a worthwhile tradeoff for 
> better performance.
> 
> -----Original Message-----
> From: Walker, Jed S [mailto:Jed_Walker@xxxxxxxxxxxxxxxxx]
> Sent: Thursday, April 05, 2012 5:44 PM
> To: CRISLER, JON A; oracle-l@xxxxxxxxxxxxx
> Subject: RE: OT - Blog entry on hugepages
> 
> Yes, I can see that hugepages is active via /proc/meminfo, I'm asking how to 
> actually tell if it really is Oracle using it. That is why I'm asking about 
> ipcs.
> 
> Mark sent this link which I'll read: 
> http://yong321.freeshell.org/oranotes/HugePages.txt
> 
> 
> -----Original Message-----
> From: CRISLER, JON A [mailto:JC1706@xxxxxxx]
> Sent: Thursday, April 05, 2012 3:28 PM
> To: Walker, Jed S; oracle-l@xxxxxxxxxxxxx
> Subject: RE: OT - Blog entry on hugepages
> 
> I am only familier with this on Linux- this does not seem to prove or 
> disprove hugepages, only shared memory segments right?
> To look at hugepages on Linux, do a cat /proc/meminfo- other OS's require 
> different methods, and the default hugepage size in Linux is 2mb.
> 
> -----Original Message-----
> From: oracle-l-bounce@xxxxxxxxxxxxx 
> [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On Behalf Of Walker, Jed S
> Sent: Thursday, April 05, 2012 4:28 PM
> To: oracle-l@xxxxxxxxxxxxx
> Subject: RE: OT - Blog entry on hugepages
> 
> The Oracle documents say you can tell hugepages are being used  by the 
> following:
>      To make sure that the configuration is valid, the HugePages_Free value 
> should be smaller than HugePages_Total and there should be some 
> HugePages_Rsvd. The sum of Hugepages_Free and HugePages_Rsvd may be smaller 
> than your total combined SGA as instances allocate pages dynamically and 
> proactively as needed.
> This does seem to indicate they are being used, but I was wondering if 
> there's any other way to truly see it. I'm wondering if anyone can 
> confirm my thinking on this. I used
> 
>> ipcs
> ------ Shared Memory Segments --------
> key        shmid      owner     perms      bytes       nattch     status
> 0x00000000 29097988   oracle    640        4096       0
> 0x00000000 29130757   oracle    640        4096       0
> 0x0e2f3c64 29163526   oracle    640        4096       0
> 0x00000000 29655047   oracle    640        268435456  59
> 0x00000000 29687816   oracle    640        51271172096 59
> 0x2894b058 29720585   oracle    640        2097152    59
> 
> I could be wrong, but 51271172096/1024/1024/1000H (since pages are in kbytes, 
> this would seem to be my 48G SGA). I'm not sure what the one above it is and 
> it is big enough to make me wonder.
> 
> Does that look like it makes sense?
> 
> (Still wish someone had some test results to show it really helps)
> 
> Thanks,
> 
> Jed
> 
> -----Original Message-----
> From: oracle-l-bounce@xxxxxxxxxxxxx 
> [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On Behalf Of Rich Jesse
> Sent: Wednesday, April 04, 2012 3:10 PM
> To: oracle-l@xxxxxxxxxxxxx
> Subject: Re: OT - Blog entry on hugepages
> 
> Hey Kyle,
> 
>>  Does anyone have metrics to identify memory access problems on AIX 
>> other than paging and scanning stats that might be solved with largepages?
>>  Recently an AIX customer  was seeing slow I/Os from an NFS filer but 
>> fast local I/O response times.  Of course the customer blamed the NFS 
>> filer, but the filer reported constant speeds during good periods as 
>> well as bad. I
> 
> Was CPU consistent between the tests?
> 
> Does the init.ora filesystemio_options='SETALL'?  I'm not sure how that 
> affects the NFS mount, but for JFS2 mounts, that will cause Oracle to 
> automagically use CIO (Ora10g and up) and also AIO, if enabled, IIRC.
> Oracle does this regardless of the mount options specified in 
> /etc/filesystems, again for JFS2.
> 
> I'm theorizing that if CIO's not used, there could be Oracle data files in 
> the filecache of the local storage, which could result in higher CPU while 
> still appearing as PIO to Oracle.  This would also depend on the VMO 
> filecache settings of minperm/maxperm/maxclient.  I watch the filecache 
> constantly in AIX's excellent nmon in the memory window, as well as a vmstat 
> monitoring script containing:
> 
> vmstat -v|grep -E 'numperm|file pages'
> 
> Hope this drivel helps!  I don't claim to be an expert in this, but have been 
> delving into this stuff now to help me determine how big I can make my buffer 
> cache w/o causing AIX (v5.3) paging.
> 
> GL!
> 
> Rich
> 
> --
> //www.freelists.org/webpage/oracle-l
> 
> 
> 
> 
> --
> //www.freelists.org/webpage/oracle-l
> 
> 
> --
> //www.freelists.org/webpage/oracle-l
> 
> 
--
//www.freelists.org/webpage/oracle-l

Other related posts:

  1. Home
  2. oracle-l
  3. Archive
  4. 04-2012