Re: Workarounds for LuaJIT's 4GB limit
- From: Alex <initrd.gz@xxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Wed, 15 Oct 2014 19:30:12 -0400
The 4GB limit only applies to Lua objects and cdata allocated with ffi.new
or from ffi.typeof constructors. If you use malloc directly (via declaring
it with ffi.cdef then using it with ffi.C.malloc), you can use memory
outside of the limit.
On Wed, Oct 15, 2014 at 6:28 PM, Nicholas Hutchinson <
nick.hutchinson@xxxxxxxxxxxxxxxx> wrote:
> Hi all,
>
> A (64-bit Linux) product I'm working on is running into issues relating to:
> - LuaJIT's requirement that the interpreter state reside in the first 4GB
> of your process' address space
> - its default allocator (see lj_alloc.c) getting memory from the system
> using mmap() with the MAP_32BIT flag, which further restricts LuaJIT to
> using using the first 2GB of your process' address space
>
> I gather there are cases where we're trying to initialise a Lua
> interpreter state quite late in the piece, and that this fails as we've
> exhausted that lower 2GB portion of our address space. I'd love to hear
> other people's experiences and how they're working around this limitation.
> The suggestions I've read seem to boil down to:
>
> - switching to a different memory allocator such as tcmalloc or jemalloc
> -- tcmalloc can be configured to use mmap() rather than sbrk() to request
> memory from the system, and jemalloc seems to always use mmap(). As I
> understand it, mmap() generally (always?) returns memory from >= 4GB, so
> the the lower portion would then be free for LuaJIT.
> - customising LuaJIT's own memory allocator -- at process startup,
> allocate some suitably large chunk of memory from the < 4GB region and dole
> it out to LuaJIT in chunks.
>
> Am I missing something simpler?
>
> Finally, is there any chance restriction is going to be relaxed in a
> future version of LuaJIT?
>
> Thanks,
> Nick
>
--
Sincerely,
Alex Parrill
Other related posts:
