Re: Any plans to increase the GC upper limit of LuaJIT on Linux x86_64?
- From: Mike Pall <mike-1208@xxxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Fri, 10 Aug 2012 11:56:55 +0200
agentzh wrote:
> Hello!
> I'm wondering if you have any plans to reduce the hard limit on LuaJIT
> 2.0 or even 2.1.
The Linux MAP_32BIT 1 GB limit is a kernel limit (which could be
patched). The actual limit is 2 GB.
For LuaJIT 2.1 I'm planning to extend this to a 4 GB limit on all
architectures, with some preference to allocate in the lowest 2 GB
for x64. The Linux MAP_32BIT feature will no longer be needed then.
However, you should really investigate what's causing so much
memory to be retained in Lua objects. I'd guess something's
inefficient there, since most of the overhead should be in buffers
and such, which are allocated outside of Lua.
Also, the user might consider to switch from e.g. huge Lua tables
to FFI arrays and structures. If you allocate these with malloc()
and not ffi.new(), they get access to the full x64 address range.
> Also, there's one interesting report on FreeBSD x86_64 [2] that shared
> memory zones created by Nginx's fastcgi modules (and possibly also
> others) can make LuaJIT 2.0 panic at the "not enough memory" exception
> even when the GC is allocating hundreds of kilobytes of memory. Does
> any work-around exist here?
Maybe this is fragmentation or maybe the sbrk space is somehow in
the low 4G. In the latter case you could either tell the FreeBSD
memory allocator not to use sbrk space or just put up a guard page
right after the sbrk top, which should coerce it into using mmap.
--Mike
Other related posts:
