Questions on Memory Address Space under Linux x64

  • From: Armin Pies <armin.pies@xxxxxxxxxxx>
  • To: "luajit@xxxxxxxxxxxxx" <luajit@xxxxxxxxxxxxx>
  • Date: Wed, 4 Jul 2012 14:14:57 +0200

In several posts I have read of a Luajit memory limit of 1 GB in the lowest 2GB 
of the process address space.

We have a 64bit server application which highly uses memory mapping of many big 
files (often greater than 4GB).

On the other hand we use Luajit to call Lua scripts from that application.

To avoid "out of memory" by Luajit we use light userdata.

1. But how can we ensure (or is - better - can it be done by Luajit itself?) 
that the needed address space is not already occupied by another 
component/object  in our application?

In my tests I used similar code as in lj_alloc.c to just mmap a huge amount of 
data (with MAP_32BIT) before calling our Lua code from our application.
Sometimes we could run our Lua scripts, sometimes  we got "out of memory" 
errors.

These errors appear at different stages. So the second question comes up:
2. Is it possible that luajit does not acquire the 1 GB address space at once?

And at last:
3. Do all push functions use that 1 GB address space for storing the Luajit 
data or do any of them use a kind of indirection, e.g. storing strings by a 
pointer in the 1GB, which reference the "real" string data in an extra memory 
region controlled by Luajit?

Sorry, I know that these are three questions at once but they all belong 
together.

Thanks in advance,

Armin

Other related posts: