Implicit casting issues when binding to C++

  • From: "Janis Britals" <jbritals@xxxxxxxxx>
  • To: <luajit@xxxxxxxxxxxxx>
  • Date: Fri, 6 Jul 2012 23:21:09 +0200

I've been working for some time on building Lua bindings for Qt using FFI
and am amazed at the ease and efficiency of this approach.

Of course, I understand Mike's reservations about extending FFI to cover C++
and completely agree with the reasoning. However in my project I needed both
performance (i.e. LuaJIT) and a rich, user-friendly GUI/Net/Sys library
(i.e. Qt), so I decided to look closer at the issue of binding to C++ via
FFI.

A lot of the intricacies of C++ has to do with type juggling (pointer
conversions, implicit arguments, constructors/destructors, etc.) and almost
all C++ stuff can be reduced to plain C calls just using pure Lua
declarations. I've done this by generating a Lua binding module for every Qt
class that declares all possible methods with their argument types and using
it as metatable to the corresponding CTYPE. (I'm looking up exports by their
mangled names using the 'asm' feature of cdef declarations, btw. Works like
a charm:).

It all worked out amazingly well, to the point where I can take a piece of
C++ code using Qt and convert it to Lua by basically replacing scope
operators ('::' in C++ to '.' in Lua) and member operators ('.' and '->' in
C== to ':' in Lua) plus a couple of very minor tweaks here and there.

I tried really hard to cover all 'C++ features' in pure Lua and almost
succeeded, except for only ONE thing: the implicit type conversions of C++.

In C++ it is very handy to call a function requiring one type of argument
(e.g. QLabel(const QString & text)) and supplying to it a completely
different type (e.g. QLabel("Text")), or just supplying a derived class
pointer to an argument expecting a base class. In order to wrap this
behavior in Lua I would need to take every call to the external C++
function, go over every argument and check whether its type can be converted
to the type required by function. In principle it could be done this way,
but it would be a terrible waste.

After breaking my head around this problem for a while I couldn't find any
other solution than to hack into the FFI internals and to place this
argument type check where it is actually happening, i.e. inside the
'lj_cconv_ct_ct' function. I needed to look at 3 scenarios: conversion from
one class type to another (via implicit constructors), conversion from class
type to pointer to a different class type (either via static cast to a base
type or via implicit constructor to a const reference type) or conversion
from a pointer to one class type to pointer to a different class type (again
either via static downcasting or implicit constructors between "const &"
types).

The problem of finding the right type of conversion was solved quite easily:
I just placed the necessary information in the CDATA metatables. When
encountering a different type of struct than expected, I need to look first
in the metatable of the supplied struct (the source type) and check whether
it contains the key equal to the CTtypeID of dest type. If not found then I
check the metatable of the required argument struct (the dest type) and
check whether it contains the key equal to the *negative* CTypeID of source
type. If either one of these values exists, then I can proceed to
conversion: if the value is a number (an integer) then it is the pointer
offset for the static cast between related types: the only thing I need to
do is to adjust the pointer value and I'm done. In any other case the value
is the conversion function (or callable table) taking one argument and
returning one result - the converted value. This is the tricky part: I need
to push this function and the original CDATA value on the Lua stack, call
the function, and then run the 'lj_cconv_ct_tv' function on the converted
value.

I compiled this hack and it seems to be working pretty well, except that
sometimes I get strange assertion errors from trace recorder. Now it could
be unrelated, but I'm pretty sure that it's caused by my hack, as I have
only a vague idea about the whole JIT compiler thing and I think the extra
manipulations on the Lua stack inside the conversion function probably
throws the recorder off track.

In any case I did this hack only in order to achieve the proof of concept:
that it is possible to bind to a sophisticated C++ library (Qt) via FFI
keeping 100% of the functionality and ease of use. The proof succeded with
flying colors, but to make it work I need to turn my hack into a workable
solution. Here I think only Mike can help.

Mike, can you, please, look at my hack (I've attached it to this message. It
is enabled by #defining LJ_LPP) and give your opinion on the feasibility of
such an extension to LuaJIT FFI? I understand your reticence regarding
covering C++, but maybe this one feature (implicit ctype conversions) is not
too much of a work, and I already have proved that all the rest can be done
in 100% Lua. This would open FFI up for complete C++ library bindings and
I'm sure that many of us FFI users would greatly welcome such extention.

--Janis

Other related posts: