Re: Key lookup optimizations...

  • From: Lance Thornblad <lance.thornblad@xxxxxxxxx>
  • To: luajit@xxxxxxxxxxxxx
  • Date: Tue, 1 Sep 2015 16:48:51 -0600

On Tue, Sep 1, 2015 at 4:19 PM, Coda Highland <chighland@xxxxxxxxx> wrote:

On Tue, Sep 1, 2015 at 2:51 PM, Lance Thornblad
<lance.thornblad@xxxxxxxxx> wrote:
On Tue, Sep 1, 2015 at 3:18 PM, Coda Highland <chighland@xxxxxxxxx>
wrote:

On Tue, Sep 1, 2015 at 2:10 PM, Lance Thornblad
<lance.thornblad@xxxxxxxxx> wrote:
On Tue, Sep 1, 2015 at 3:05 PM, Coda Highland <chighland@xxxxxxxxx>
wrote:

On Tue, Sep 1, 2015 at 1:59 PM, Lance Thornblad
<lance.thornblad@xxxxxxxxx> wrote:
In my opinion, you should go ahead and write that logic in Lua,
inject
the Lua code into the environment from the host environment (so
you
can treat it as part of the binding API and not part of the
script)
and then lock down the metatable so that user code can't mess with
it.
Likewise, you can wrap FFI calls in Lua functions, keeping the FFI
namespace as an upvalue and removing the ffi object from the
global
environment so that user code can't use that, either.

There are, of course, limits to sandboxing, but it's not generally
very hard to block the obvious routes.

/s/ Adam


Hmm, there is likely some merit to that approach. I'll have to
consider
that against my project's other goals. One advantage that I have
currently
is that the code needed to interface my objects to some other
language
is
pretty small. The more I tie in directly to Lua, the harder that
becomes,
but perhaps I just have to accept that things will not be optimized
if I
don't...

As Alex suggested, can you implement the lookup as a C function
instead of a Lua C API function? Or perhaps as a set of them, if they
need to be able to return different types of values, with another
function to determine which lookup function you'll need?

/s/ Adam


What's the difference, when you say C function vs. Lua C API function?

When I say C function, I mean something invoked using the FFI instead
of using the Lua C API. It looks something like this:

/* C code, compiled into binary */
int getValue(char* param) {
return strlen(param); // just a silly example
}

-- Lua code --
local ffi = require("ffi")
ffi.cdef[[int getValue(char* param);]]

local result = ffi.C.getValue("abc")

It's important to remember that functions invoked by the FFI *must
not* under any circumstances touch the Lua state. (LuaJIT will try to
catch this and throw an error if you do it.) If you're just making
bindings, that's generally not a problem because important state won't
be kept in the Lua state.

More information:

http://luajit.org/ext_ffi.html
http://luajit.org/ext_ffi_tutorial.html

/s/ Adam


I did some experiments with FFI some time ago, but it was about passing
cdata back and forth (I wanted proper ulongs). I ran into some issues,
likely because I'm using a BlitzMax LuaJIT module (e.g. the CData
constant
was missing) and I didn't want to put that much time into it. Another
thing, my end user shouldn't have to know anything about FFI...

They don't. You'd wrap the FFI calls in small (potentially even
autogenerated, depending on your needs) Lua wrappers -- just enough to
be able to wrap them up into a table or clean up the API or something.

The part about not touching the state has me concerned. How would that
even
work? My current __index C function takes the key and looks for the
object
(a method, usually) that it represents. When it finds it, it pushes it
onto
the stack or calls it, if it's a function object, and then that result is
pushed. If it doesn't find it, then it pushes a closure that represents
an
error or warning, depending on the severity.

Isn't all of that "touching the stack?" It's sounding as though I'd
need to
wrap the FFI call in another Lua function, no?

At that point I would have all of the relevant function objects /
closures stored in a table somewhere (feel free to use the Lua C API
to set this up -- you only have to do it once, on startup), and your
FFI function would just return a key (char* or int, probably, but
potentially a pointer type or something) that the Lua-side code would
use to look it up in the table.

/s/ Adam


Hmm, I seem to be doing something similar, but without FFI. I'm sure that
it would benefit the project to look into it further.

Thanks for the advice, at any rate. :)

Lance

Other related posts: