Re: Questions on Memory Address Space under Linux x64
- From: Mike Pall <mike-1207@xxxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Wed, 4 Jul 2012 15:07:29 +0200
Armin Pies wrote:
> In several posts I have read of a Luajit memory limit of 1 GB in
> the lowest 2GB of the process address space.
The actual limit is 2GB. The 1GB is due to a hardcoded Linux
kernel limit for the MAP_32BIT feature. Adventurous minds may
consider patching the kernel.
> We have a 64bit server application which highly uses memory mapping of many
> big files (often greater than 4GB).
>
> On the other hand we use Luajit to call Lua scripts from that application.
>
> To avoid "out of memory" by Luajit we use light userdata.
Why lightuserdata? A userdata object may carry a pointer to the
mapping. That's a very small object, even if the mapping is big.
> 1. But how can we ensure (or is - better - can it be done by
> Luajit itself?) that the needed address space is not already
> occupied by another component/object in our application?
The default mmap() code in the Linux/x64 kernel never allocates
from the low 4GB, so you should be safe.
> 2. Is it possible that luajit does not acquire the 1 GB address space at once?
IMHO neither necessary, nor very embedding-friendly.
> 3. Do all push functions use that 1 GB address space for storing
> the Luajit data or do any of them use a kind of indirection,
> e.g. storing strings by a pointer in the 1GB, which reference
> the "real" string data in an extra memory region controlled by
> Luajit?
Yes. No.
In any case, you should take a look at the FFI: it has 64 bit
pointers and allows you to use the full x64 address space. And you
can call mmap() from your Lua code (no C code needed).
--Mike
Other related posts:
