Re: Extending LuaJIT's memory limit to 4Gbytes.

  • From: Dmitri Shubin <sbn@xxxxxxxxxxx>
  • To: luajit@xxxxxxxxxxxxx
  • Date: Wed, 22 Aug 2012 13:20:43 +0400

On 21.08.2012 13:46, Robert G. Jakabosky wrote:
On Tuesday 21, Dmitri Shubin wrote:
Just FYI.

To support LuaJIT on Solaris 64-bit I had to do the similar thing -- use
custom page allocator for low 2GiB of address space.

Is there some restriction keeping you from using up to 4Gbytes of address
space?

From what I read I have an impression that only (low?) 2GiB could be used by LuaJIT on 64-bit system: see e.g. [1]

Can you build PIC executables [1] on Solaris?

Looks like the answer is no.
I.e. probably (my) code in executable could be made position-independent (by using -fPIC), but - AFAIU special CRT files should be linked in (crt1.o vs Scrt1.o on my Linux box);
 - kernel/runtime-linker should load it above the low 4GiB.


"reserve" this space using MAP_NORESERVE and used vmem
resource allocator from umem library [1][2] to allocate pages from it.
Are you "reserving" all of the low 2Gb with mmap?  How do you return pages
back to the OS?  Does vmem provide a way to allocate at a fixed address?  If
not how did you handle the mmap call at src/lj_mcode.c:101?
Yes I "reserve" almost all of the low 2GiB on start (.init) using mmap(MAP_FIXED | MAP_NORESERVE) to avoid using this virtual space by someone else. (Probably this is not actually required since Solaris scans address space from high addresses down) vmem is used to allocate address space extents (i.e. as resource allocator) that are then mmap()-ed using MAP_FIXED. When LuaJIT releases the memory chunk (via CALL_MUNMAP()) I munmap() it and return free extent to vmem allocator.

mmap() call in lj_mcode.c is rewritten to use our custom mmap()-er.

BTW as Mike pointed out here [2] approach with MAP_FIXED could be problematic but we have full control on allocation in our process.

[1] //www.freelists.org/post/luajit/Questions-on-Memory-Address-Space-under-Linux-x64,1
[2] //www.freelists.org/post/luajit/Solarisx64-support-for-LuaJIT,1

Other related posts: