Re: Projects I may be willing to sponsor
- From: Pierre-Yves Gérardy <pygy79@xxxxxxxxx>
- To: luajit@xxxxxxxxxxxxx
- Date: Thu, 11 Dec 2014 01:15:12 +0100
On Thu, Dec 11, 2014 at 12:27 AM, Mike Pall <mike-1412@xxxxxxxxxx> wrote:
> Joe Ellsworth wrote:
>> What I really want is a patch that will allow me to use 40 gig of
>> FFI allocations on a server that contains 80 Gig of Ram.
>
> local x = ffi.cast("double *", ffi.C.malloc(40*2LL^30))
>
> But, yes ... you need to manually manage the memory. There's
> simply no way to get reliable, performant and automatic memory
> management under these constraints.
>
> [Ask the Java people about the horrendous things they need to do
> to work around the 'best of their class' garbage collectors for
> huge memory loads.]
If I understand correctly, Joe is using large objects. He may malloc
the large array, and reference them in managed ffi structs that use
the __gc metamethod to clean things up.
Here's what I use for buffers.
local BUFF_INIT_SIZE = 16
local charsize = ffi.sizeof"char"
ffi.cdef"void* malloc (size_t size);"
ffi.cdef"void free (void* ptr);"
local Buffer = ffi.metatype(
-- size, index, array
"struct{uint32_t s; uint32_t i; unsigned char* a;}",
{__gc = function(self) ffi.C.free(self.a) end}
)
local function make_buffer()
local b = Buffer(
BUFF_INIT_SIZE,
0,
ffi.C.malloc(BUFF_INIT_SIZE * charsize)
)
return b
end
local function reserve (buf, size) -- ensure the buffer can
accomodate `size` bytes.
if size <= buf.s then return end
repeat buf.s = buf.s * 2 until size <= buf.s end
local a = C.malloc(buf.s * charsize)
if a == nil then error("out of memory") end
ffi.copy(a, buf.a, buf.i)
ffi.C.free(buf.a)
buf.a = a
end
—Pierre-Yves
> --Mike
>
Other related posts:
