Neil pointed out, used = ( total - free ) + reserved. I also thought it
pertinent to reverse that and say that actual free = free - reserved. A common
mistake is expecting free to be usable pages. Pages are reserved up front but
not removed from the free list until they are actually used. A recently
started instance will reserve its pages but they will not be removed from free
until they are accessed for the first time. The reserved pages are not
available for requests.
The page tables are something worthy of understanding although running with
huge pages you are 512 times less likely to feel the pain. Each page table
entry is 8 bytes and a page table entry is created when a user process accesses
a page of memory in the shared memory segment. Each process has it’s own page
table map and it grows as the user reads pages. All processes might not be
created equal but applications that use connection pools tend to grow over time
and the consumption can surprise the unaware. Here are some numbers
Assuming 80G SGA and 2,000 connections that eventually over time access every
page ( not likely but worst case ). Linux default page size is 4K and Page
Table Entry ( PTE ) is 8 bytes.
Without huge pages
80G/4k = 20971520 pages per process
20971520 * 8 bytes = 160MB per process
160MB * 2000 processes = 312GB of memory for page tables.
If you’re not familiar with page tables you’re probably not thinking about the
need for an extra 312GB of RAM to support your database users.
Now, this same workload using 2MB huge pages
80G/2MB = 40960 pages per process
40960 * 8 bytes = 320k per process
320k * 2000 processes = 625MB
625MB seems a little more manageable.
BTW, you can see total page table allocation in /proc/meminfo and per process
in the /proc/{PID}/stat file. I think it’s the stat file, that or status.
Can’t recall exactly and too lazy to connect to a system right now to confirm.
6) Some initial testing I was doing with OEL 6 showed horrible performance
until I realized the hard way I was being affected by transparent huge pages.
Once disabled everything returned to normal. I don’t have good numbers
available but trust me it was a very large impact on performance.
7) There are claims of performance gains but I’ve never tested, or researched,
them. I would suspect it would be a small percent improvement at best. I
believe the biggest benefit you get from using huge pages is in the reduction
of page tables and being able to use that memory for the SGA. If you take the
example above you could divert the 312GB - 625MB of memory to your SGA and
hopefully get a sizable lift in performance from improved caching.
Kenny
On Sep 5, 2015, at 12:50 PM, Neil Chandler <neil_chandler@xxxxxxxxxxx> wrote:
The huge pages used is (total - free) + reserved. Don't forget ASM and
-MGMTDB will also use huge pages if allocated. Also, you must use ASMM, not
AMM. (Does anyone successfully use AMM on Linux? Stupid idea. Use ASMM.)
The main reason for using huge pages is to minimise the mapping tables.
Normal pages are small, so it will take 1gb of memory to map each 8gb of
normal pages. With huge pages, it costs a few tens of mb for every 8gb. 40 I
think (it's Saturday night). So you gain memory and have less management
overhead.
Only the SGA comes out of huge pages, as Andrew says. You should never
allocate more than about 65% of server memory for huge pages (OLTP) and
somewhat less for data warehouse, to leave room for pga ( which should be
bigger in a DWh.)
Regs
Neil Chandler
sent from my phone
On 5 Sep 2015, at 19:12, Andrew Kerber <andrew.kerber@xxxxxxxxx> wrote:--
First of all, PGA will not use huge pages at all. Only the sga. Next
transparent huge pages are the same as anonymous huge pages. And they look
to be disabled on your system. Look at your alert log for how oracle is
using huge pages. Information is printed there on boot. I would have
expected the huge pages reserved to be the number in use, but those don't
quite add up. So I'm not sure about that.
Sent from my iPad
On Sep 5, 2015, at 12:47 PM, Dba DBA <oracledbaquestions@xxxxxxxxx> wrote:--
Redhat 6
Oracle RAC 11.2.0.4
Grid: 11.2.0.4
Huge page size: 2048
I set this up on a few clusters recently. I followed the oracle notes and a
few pages on the web. I am trying to understand huge pages a little lower
level. I am wondering if anyone can help me out.
1. If I size my PGA_AGGREGATE_TARGET at 5 gb and my Huge Pages are sized at
2 GB, PGA will still need to use 3 huge pages and will effectively reserve
6 GB in memory right? Is it the same with SGA_MAX_SIZE
2. When I use 'free -m' to see my free memory, I see that all of my Huge
Page memory is already allocated and does not show up here. When users
connect to a DB, do the user sessions use Huge Page or memory from free
memory? Anything else associated with Oracle that will not use huge pages?
3. See below from one of my boxes. I have read some unix sites and I still
can't figure out what HugePages_RSVD and HugePages_Surp means to an oracle
DB? I think all I care about Total and Free.
grep -i huge /proc/meminfo
AnonHugePages: 0 kB
HugePages_Total: 26752
HugePages_Free: 20097
HugePages_Rsvd: 11781
HugePages_Surp: 0
Hugepagesize: 2048 kB
4. second huge page output is from the same 2 node cluster as the one
above. The exact same DB and settings are running on each node. Why would
there be so much less HugePages Free across servers?
I believe SGA is static. You set SGA_MAX_SIZE and at startup and this grabs
a static amount of memory. Between 4 DBs I have 10 GB for
PGA_AGGREGATE_TARGET. This isn't a static allocation right? The DBs are not
busy and nothing is going on since we are off hours. So it doesn't look
like PGA would be allocated. I bounced the DBs last night as part of a
planned outage and the HugePage free was differnet as soon as I started
them. I dont know what it was before. I did not have time to stop and start
instances one by one on each node to see the variance in huge pages free by
DB.
AnonHugePages: 0 kB
HugePages_Total: 26752
HugePages_Free: 12266
HugePages_Rsvd: 3950
HugePages_Surp: 0
Hugepagesize: 2048 kB
5. How do I tell which processes use huge pages vs. which use regular
memory? I don't see a way to distinguish in top or any of the utilities?
6. I found out from a coworker last night that oracle sent out an alert
recently about 'transparent huge pages' and they have to be disabled due to
a bug. I have not read the article yet. Does anyone know of any other known
bugs with huge pages?
7. anyone bench mark this huge pages to see an actual performance
difference? I see oracle recommend it. I see all over the place people talk
about implementing it, but what I don't see is a benchmark showing that it
actually improves performances.
//www.freelists.org/webpage/oracle-l
//www.freelists.org/webpage/oracle-l
