[haiku-development] weaks symbols and friends
- From: Ingo Weinhold <ingo_weinhold@xxxxxx>
- To: haiku-development@xxxxxxxxxxxxx
- Date: Mon, 15 Nov 2010 19:00:56 +0100
Howdy,
sorry for the longish mail. The short version is that recently I have
worked on introducing support for weak symbols and enabling symbol
preemption in a branch [1, 2] and, since that has certain consequences (not
all of them desirable), we have to decide whether/when the branch shall be
merged.
The long version:
In order to satisfy the requirement of the Itanium C++ ABI (which despite
its name is what gcc 4 implements also for other architectures) that two
type info pointers must be equal, if they refer to the same class [3], I
implemented weak symbol support in Haiku and disabled the hardcoded
symbolic linking in the gcc 4 tool chain. Furthermore I enabled an
optimization (respectively disabled the work-around) in gcc 4 for comparing
type infos for equality (now done by comparing pointers as opposed to
comparing class names).
A bit of technical background:
When compiling a source file the compiler generates a type info, i.e. an
instance of the class type_info, for every type that is involved in an
operation that requires some kind of runtime type information (RTTI). This
includes the C++ typeid operator (obviously, as it returns the type_info),
dynamic_cast, exception handling (catching that type), and definitions of
constructors and destructors of classes that have or have inherited at
least one virtual method.
Here's where the weak symbols come into play. The type infos are marked
weak, which basically means that if multiple definitions of a type info are
encountered only one (and always the same one!) shall be used. That's why
the linker ignores all but one definition (instead of failing, what it
usually does when encountering multiple definitions for the same symbol).
Still, there can be a type info for the same class in several shared
objects (application, libraries, add-ons), e.g. a BWindow type info will be
in libbe.so and it could also be in an application using it. That's why the
runtime loader must essentially do the same as the linker and make sure
that all references point to the same type info, while the additional ones
are completely ignored.
Regarding the symbolic linking (the -Bsymbolic linker command line option).
That's something we've inherited from BeOS' gcc 2 configuration and which
has also made it into our gcc 4. It's not exactly a feature but rather
something that disables a feature, namely symbol preemption. Normally a
symbol exported by a shared object is preemptable, that is it can be
overridded with another definition. E.g. if one thinks the memory allocator
in libroot.so sucks for a particular application, one can override
malloc(), free(), and friends in the application and not only the
application itself but also all other libraries used by the application,
including libroot.so (!), will use those overrides. Symbolic linking
prevents that by requiring the linker and the runtime loader to look up a
symbol reference from a symbolically linked shared object first in that
shared object before using the standard lookup order. This does not only
kill symbol preemption for that shared object, but it also screws up the
handling of weak symbols by the runtime loader, since the symbol lookup
order for each loaded shared object may be different, potentially causing
the same symbol to be resolved to different weak definitions.
The reasons why Be hardcoded symbolic linking are probably that BeOS'
runtime loader did neither support weak symbols nor symbol preemption
anyway, and using it allows the linker to optimize away a level of
indirection required for these features. The performance difference
executing the code should be virtually non-existent (e.g. on x86 that
indirection should only be a single additional machine instruction for a
call to an exported function). The runtime loader performance, however,
differs significantly. With symbolic linking all internal calls to
functions defined in a library are resolved by the linker. The most the
runtime loader has to do is apply a relocation offset for the calls (to
compensate for the variable memory address a library can be loaded to).
Without symbolic linking the runtime loader has to search for the symbol in
all shared objects loaded earlier. For weak symbols it has to search
through all loaded shared objects, since a non-weak definition can override
a weak one.
With my test installation (on a Core i7) I couldn't really feel much of a
difference, but a quick measurement verified that there is one. A "time
/system/apps/StyledEdit" with an already running instance of the
application reported 85 ms instead of 69 ms. The same for WebPositive
yielded 277 ms vs. 100 ms. There's a bit potential for optimization (e.g.
implementing lazy binding and/or a symbol resolution cache), but I can't
estimate how much that will improve things.
Another implication of disabling symbolic linking is that one has to be
considerably more careful what symbols libraries and applications export. I
already fixed a few instances where symbols were undesirably preempted by
ones in other libraries or the application. This can generally be avoided
by using proper namespaces for private stuff, and/or, even better, restrict
the (ELF symbol) visibility. Marking something hidden (gcc (4) provides
attributes and pragmas for that purpose) has not only the advantage that it
cannot clash with definitions in other shared objects, it also allows (the
compiler and) the linker the same optimizations as with symbolic linking
(or even better ones). Consequently we should build everything with hidden
visibility and only explicitly export what we want to export. This is also
generally recommended good practice [4].
So far I have only worked with gcc 4 in my branch. I'd expect the same
issues with gcc 2. Regarding binary compatibility, the weak symbol support
won't introduce any issues AFAICT. Disabling symbolic linking could have
undesirable effects, though. Since the runtime loader already uses a
BeOS-compatible symbol lookup algorithm when a BeOS executable is loaded,
continuing to do that should retain the current behavior in this case.
Neither weak symbols nor symbol preemption would work (which also means
that enabling the type info comparison optimization in gcc 2 would not be
possible). Using old BeOS libraries in new programs can lead to those
libraries preempting symbols of libraries loaded later (e.g. libroot or
libbe), which they wouldn't do ATM.
Unfortunately gcc 2 itself doesn't support the symbol visibility features
(only the binutils do), which prevents using those as a measure to avoid
clashes due to symbol preemption, if we disabled symbolic linking for gcc 2.
I think I've discussed all consequences that come with enabling weak
symbols and symbol preemption, so let me come to the big questions
whether/when to merge that goodness into the trunk. Since the Itanium ABI
requires it and by not complying we risk broken programs, I believe the
question for our gcc 4 world is really only: When? All gcc 4 packages have
been rebuilt, so in principle it could be done any time. Since it's not
unlikely that regressions will be introduced (hopefully mostly of the easy
to notice crashing kind), we could postpone it until after the next release
(or even after R1), though.
Regarding gcc 2 I'm rather undecided. AFAIK there's no direct (or indirect)
issue that would be fixed by enabling weak symbols and symbol preemption.
At least none that didn't already exist under BeOS. New binary
compatibility issues might be introduced (not sure how likely) and the
missing visibility features in gcc aren't exactly helping either. On the
other hand enabling weak symbols and symbol preemption for gcc 4 but not
for gcc 2 is bound to cause annoyances as well.
Opinions?
CU, Ingo
[1]
http://dev.haiku-os.org/browser/haiku/branches/developer/bonefish/weak-symbols
[2] http://dev.haiku-os.org/browser/buildtools/branches/gcc4-weak-symbols
[3] http://refspecs.freestandards.org/cxxabi-1.83.html#rtti (cf. 2.9.1)
[4] http://gcc.gnu.org/wiki/Visibility
Other related posts:
