Workarounds for LuaJIT's 4GB limit
- From: Nicholas Hutchinson <nick.hutchinson@xxxxxxxxxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Wed, 15 Oct 2014 23:28:31 +0100
Hi all,
A (64-bit Linux) product I'm working on is running into issues relating to:
- LuaJIT's requirement that the interpreter state reside in the first 4GB of
your process' address space
- its default allocator (see lj_alloc.c) getting memory from the system using
mmap() with the MAP_32BIT flag, which further restricts LuaJIT to using using
the first 2GB of your process' address space
I gather there are cases where we're trying to initialise a Lua interpreter
state quite late in the piece, and that this fails as we've exhausted that
lower 2GB portion of our address space. I'd love to hear other people's
experiences and how they're working around this limitation. The suggestions
I've read seem to boil down to:
- switching to a different memory allocator such as tcmalloc or jemalloc --
tcmalloc can be configured to use mmap() rather than sbrk() to request memory
from the system, and jemalloc seems to always use mmap(). As I understand it,
mmap() generally (always?) returns memory from >= 4GB, so the the lower portion
would then be free for LuaJIT.
- customising LuaJIT's own memory allocator -- at process startup, allocate
some suitably large chunk of memory from the < 4GB region and dole it out to
LuaJIT in chunks.
Am I missing something simpler?
Finally, is there any chance restriction is going to be relaxed in a future
version of LuaJIT?
Thanks,
Nick
Other related posts:
