[gameprogrammer] Re: Scripting engines: Upvalues and function refs
- From: David Olofson <david@xxxxxxxxxxx>
- To: gameprogrammer@xxxxxxxxxxxxx
- Date: Wed, 14 Apr 2004 13:17:57 +0200
On Tuesday 13 April 2004 17.27, Bob Pendleton wrote:
[...refcounting and/or GC...]
> Depends on the language and how the code is structured. Adding any
> kind of memory manager after the fact is very hard. To add a GC you
> have to have the code structured in such a way that the GC can find
> every pointer so that it can mark it and follow it.
[...]
Well, the VM is mostly designed with this in mind... References have
explicit type info, like all other values, whether they're constants,
variables or temporary values in "registers". There's room for GC
info (mark bits or whatever), and it's trivial to add code that gets
executed whenever you create a reference to something and stuff like
that.
Besides, it's all on the VM level; only the actual script code has to
be involved. Compiling scripts isn't a real time operation, so it
doesn't have to be done in the same context as the VM runs in.
> Reference counting is really hard to retrofit to C code but can be
> done to C++ without losing to much sleep and blood... You have to
> create a smart pointer type template and replace all pointer
> variables with that new type. Once that is done reference counting
> "just works".
I have refcounted strings already, so that logic is in place. The
actual string storage isn't handled by an RT memory manager yet, but
that's another issue. I could use the same logic for other types with
a minimal amount of hacking.
> In both cases you run into problems with local variables, temporary
> variables, and pointer arithmetic.
Yeah, that's the messy part right now. Does a temporary VM register
that holds a reference own an object or not? If it doesn't, the VM
doesn't have to explicitly free a register before overwriting it -
but then who keeps track of object lifetime...? Who keeps track of
which local variables need proper deallocation? (The other ones can
just be dropped without being touched. Any dynamically typed variable
may contain a reference - so I can't just throw those away without
looking at them.)
Anyway, this stuff is mostly under control, although I've probably not
found the optimal solutions. It feels kind of wasteful to explicitly
check values before deleting them all the time, just in case they're
not integers, reals or other self-contained values. I guess that
can't be avoided with dynamically typed values...
[...]
> There are a lot of other languages out there. Have you taken a good
> hard look at them?
I've looked around quite a bit, but so far, the only mature RT safe
alternative I've found is SuperCollider; an RT synth/language which
was formerly a proprietary application for Mac OS only. It's now
Free/Open Source - but unfortunately, it's GPL, which is a
showstopper for pretty much everything I need it for. (Must be LGPL,
X11 or similar.)
I found one or two other RT scripting languages, but they were in an
experimental state and/or not actively maintained. I figured I'd be
better off finishing and maintaining my own C code than someone
else's C++ code...
I also looked into making a "normal" scripting engine RT safe one way
or another. Small, Pike, Lua and some others seemed like interesting
candidates, though some aren't as portable as I'd like. (Win32,
Linux/x86, Linux/PPC and Mac OS X is a minimum requirement.) Whereas
I could find no signs of the Pike community being interested in RT
programming, it seems like a major part of the Lua community is doing
soft RT stuff and wouldn't mind "harder RT". (Tons of Free and
commercial games are using it.)
However, it turned out that it'd be quite some work to get Lua to the
level of "RT safeness" I need. (Goes for most languages not designed
for RT, and the higher level ones tend to have other issues than just
the GC.) The Lua team is working on an incremental GC, but they're
aiming at video (30-150 fps, soft RT), while I need to deal with low
latency audio (1000+ fps, hard RT). Lua 6.0 (or whatever) *might* do
the job, but I can't wait for it, and I can't risk that it arrives
and still doesn't cut it.
Of course, I'm still interested in other alternatives, to use as is,
or to study, but I don't have the time to investigate anything
seriously right now. EEL is almost ready to do the job now, and it'll
have to do for now, even if there is a better solution out there,
somewhere.
[...]
> > > Then there is the point that RT systems tend to lag behind
> > > desktop and server systems in overall memory and performance.
> >
> > Yes... Although that doesn't exactly apply to games - at least no
> > the=20 state-of-the-art 3D games. Then again, games are rarely
> > considered=20 hard RT systems these days, unfortunately. :-/ Not
> > much point if they=20 have to run on desktop OSes without proper
> > RT scheduling, so I can't=20 say I blame game programmers in
> > general for having limited knowledge=20 of, or interest in, hard
> > RT systems.
>
> I don't think games have ever been thought of as hard real time
> systems. Seriously, even when they were, the programmers didn't
> think about them that way.
Well, at least on the Amiga, game programmers were often extremely
picky about dropped frames in scrollers and stuff like that. Dropped
frames are *very* obvious in a fast scrolling game that normally runs
locked at the 50 or 60 Hz refresh rate - especially since the game
logic was generally not designed to account for it. That is
essentially a hard RT requirement; never drop a frame == never miss a
vblank deadline.
Now, PC games were always a completely different matter, even in the
DOS + VGA days, when you *could* lock to the refresh rate and tune
scrolling and animations to it. Dunno why... Maybe the "arcade smooth
scrolling" people simply stayed as far away from PCs as possible?
[...]
> > > > Speaking of which;
> > > >
> > > > A) No type specifier =3D=3D> default type. (Say, integer.)
> > > >
> > > > B) No type specifier =3D=3D> object becomes dynamically
> > > > typed.
> > > >
> > > > C) Type specifiers are not optional. Say "dynamic" if
> > > > that's what you want.
> > > >
> > > > What's your vote?
> > >
> > > I prefer to either have strong typing or dynamic typing. Never
> > > both.
> >
> > Why not both? (Though I'd agree that it's usually better to pass
> > an OO=20 interfaces around, and derive various classes from that
> > interfaces as=20 needed. That avoids run time binding of member
> > names as well, since=20 you gan use statically typed object
> > references...)
>
> I don't like having both because it is confusing and error prone.
> You get into situations where the programmer declares things
> incorrectly and yet they still work. Or, they work for a long time
> and break when a small change is made to the code.
Yeah... I'm thinking of the dynamic type as a lightweight alternative
to wrapping simple things up in classes with inheritance and stuff,
or structs with a type field + a union and stuff like that. It's not
really meant to be used all over the place; just where it makes
things a lot cleaner and simpler.
That is, to me, having static typing and a "dynamic" type is like a
statically typed language with a "minor" extension, rather than a
dynamically typed language with support for statically typed
variables. Either way, it's the programmer that decides. I'd
recommend against using the "dynamic" type unless it *really* fits
in, but others may have other ideas... And then we get this wonderful
C++ situation where you have to decide on a common coding style, so
programmers on a project can understand each other's code at all. ;-)
> > =46rom the language POV, I tend to think of static typing mainly
> > as a=20 way of telling the compiler exactly what you mean, so you
> > simply=20 can't compile code that won't work, or won't do
> > anything sensible=20 anyway.
>
> I see it as a way to let the compiler find errors at compile time
> instead of run time.
Yeah, that's the net result I want.
> A way to reduce the size of the language by
> eliminating the need to have a type testing mechanism in the
> language. And, as a way to generate better code because you don't
> have to include code for run time type testing.
Good reasons for not having the "dynamic" type at all...
> In a completely dynamic system you can derive type information my
> looking at the types of constants and other hints and a good job of
> assigning types to variables without having to declare them at all.
OTOH, then you still need a VM with instructions for dealing with both
static and dynamic type data, or there isn't much point in deriving
that information, is there?
//David Olofson - Programmer, Composer, Open Source Advocate
.- Audiality -----------------------------------------------.
| Free/Open Source audio engine for games and multimedia. |
| MIDI, modular synthesis, real time effects, scripting,... |
`-----------------------------------> http://audiality.org -'
--- http://olofson.net --- http://www.reologica.se ---
Other related posts:
