Re: Any plans to increase the GC upper limit of LuaJIT on Linux x86_64?
- From: agentzh <agentzh@xxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Fri, 10 Aug 2012 10:43:29 -0700
Hello!
On Fri, Aug 10, 2012 at 2:56 AM, Mike Pall <mike-1208@xxxxxxxxxx> wrote:
> agentzh wrote:
>
> For LuaJIT 2.1 I'm planning to extend this to a 4 GB limit on all
> architectures, with some preference to allocate in the lowest 2 GB
> for x64. The Linux MAP_32BIT feature will no longer be needed then.
>
That's great news! Thanks!
> However, you should really investigate what's causing so much
> memory to be retained in Lua objects. I'd guess something's
> inefficient there, since most of the overhead should be in buffers
> and such, which are allocated outside of Lua.
>
> Also, the user might consider to switch from e.g. huge Lua tables
> to FFI arrays and structures. If you allocate these with malloc()
> and not ffi.new(), they get access to the full x64 address range.
>
Yes, these are all very good optimizations :)
>> Also, there's one interesting report on FreeBSD x86_64 [2] that shared
>> memory zones created by Nginx's fastcgi modules (and possibly also
>> others) can make LuaJIT 2.0 panic at the "not enough memory" exception
>> even when the GC is allocating hundreds of kilobytes of memory. Does
>> any work-around exist here?
>
> Maybe this is fragmentation or maybe the sbrk space is somehow in
> the low 4G. In the latter case you could either tell the FreeBSD
> memory allocator not to use sbrk space or just put up a guard page
> right after the sbrk top, which should coerce it into using mmap.
>
Maybe. I'll have a closer look at this issue :)
Thanks!
-agentzh
Other related posts:
