Re: Any plans to increase the GC upper limit of LuaJIT on Linux x86_64?

  • From: agentzh <agentzh@xxxxxxxxx>
  • To: luajit@xxxxxxxxxxxxx
  • Date: Fri, 10 Aug 2012 10:43:29 -0700


On Fri, Aug 10, 2012 at 2:56 AM, Mike Pall <mike-1208@xxxxxxxxxx> wrote:
> agentzh wrote:
> For LuaJIT 2.1 I'm planning to extend this to a 4 GB limit on all
> architectures, with some preference to allocate in the lowest 2 GB
> for x64. The Linux MAP_32BIT feature will no longer be needed then.

That's great news! Thanks!

> However, you should really investigate what's causing so much
> memory to be retained in Lua objects. I'd guess something's
> inefficient there, since most of the overhead should be in buffers
> and such, which are allocated outside of Lua.
> Also, the user might consider to switch from e.g. huge Lua tables
> to FFI arrays and structures. If you allocate these with malloc()
> and not, they get access to the full x64 address range.

Yes, these are all very good optimizations :)

>> Also, there's one interesting report on FreeBSD x86_64 [2] that shared
>> memory zones created by Nginx's fastcgi modules (and possibly also
>> others) can make LuaJIT 2.0 panic at the "not enough memory" exception
>> even when the GC is allocating hundreds of kilobytes of memory. Does
>> any work-around exist here?
> Maybe this is fragmentation or maybe the sbrk space is somehow in
> the low 4G. In the latter case you could either tell the FreeBSD
> memory allocator not to use sbrk space or just put up a guard page
> right after the sbrk top, which should coerce it into using mmap.

Maybe. I'll have a closer look at this issue :)


Other related posts: