[haiku-development] Re: A modest (FatELF) proposal
- From: Ingo Weinhold <ingo_weinhold@xxxxxx>
- To: haiku-development@xxxxxxxxxxxxx
- Date: Tue, 20 Nov 2012 00:17:30 +0100
On 11/18/2012 07:02 PM, Landon Fuller wrote:
[...]
In terms of solutions, the issue of ABI compatibility isn't just one of
selecting the right libraries at runtime, either, and can't be solved
just by dynamically providing the correct library ABI on demand.
Applications that host plugins must have an ABI that matches that of the
plugins they load. This introduces some interesting issues during
transitionary periods, as not all plugins will be updated immediately,
and as such, it proves impossible to allow for a gradual transition by
third party plugins to a new architecture -- it must be all or nothing.
I fail to see what that has to do with fat binaries. You can't load an
x86 add-on from an x86-64 application or vice versa. So regardless of
whether fat binaries are or aren't used, for an add-on to be loadable it
must match the application's architecture.
Fat binaries ease this transitions considerably; they were leveraged by
NeXT when they were moving beyond m68k, by Apple when they were
transitioning from m68k to Intel, and further leveraged heavily during
the Mac OS X transitions from PPC to PPC64, PPC to x86, and mixed
x86/x86-64 installations, in addition to the iOS transitions from
ARMv6->ARMv7->ARMv7s.
An important difference is that we don't control the hardware. We're not
migrating to x86-64 and phasing out gcc2 x86 over a few years. We're
actually going to keep the latter around as long as reasonable and we're
spreading to other architectures (e.g. hopefully ARM sometimes soon).
Fat binaries make it easy for the OS vendor, user, and software
developer to support new architectures through gradual migration, by
pushing the cost of doing so onto the development toolchain. Rather than
compile and distribute something N times for N different architectures,
your build process can run the compiler with the appropriate -arch
flags, and compile a binary that runs on all the available
architectures.
1. What you compile you should test. 2. Not everything can be
cross-compiled, so this approach isn't universal anyway. 3. The only
difference is indeed the distribution, not the building. Whether you
cross-compile the code for N architectures into one package or
cross-compile the code N times into N packages doesn't make much of a
difference for the developer.
When hosting plugins, the hosting application can
automatically 'downgrade' to the lowest common denominator ABI by
relaunching itself, thus allowing for gradual transitions between old
ABIs and new ABIs -- this was done by Apple in System Preferences when
the entire OS switched over to x86-64, allowing System Preferences to
switch back to i386 in the case where the user still had legacy plugins
installed.
As already mentioned above, this has nothing to do with fat binaries.
The same can be done with separate slim binaries.
From the user's perspective, there's nothing to think about. There's a
single binary to download, and a single installation CD/DVD to run, and
the right ABI is used depending on their host environment.
For commercial products that are distributed via CD/DVD that might
indeed be an advantage. For downloads too, though that automatically
comes with the disadvantage that the download size increases. In either
case I don't consider it that much of a burden for the user to select
the installation package/download for their architecture (in the former
case it could even be done automatically by a script). Overall I'd say
the advantage is very small.
However, the vast majority of software will be installed via the package
management once it is ready. And in that context fat binaries are just
superfluous.
From the
software developer's perspective, things are equally easy -- by adding
new -arch flags to their compiler invocations, they can automatically
target multiple architectures/ABIs straight from the same build, and
provide a single binary to run.
I addressed the building part above. The only difference is that you
build one package for N architectures instead of N packages.
> Transitions from one architecture to
another can occur gradually, without every single third-party developer
and OS vendor supplied application making the transition at once.
That again has nothing to to with fat binaries. We already address that
issue with our hybrid builds. Though ATM the tool chain isn't configured
correctly.
[...]
I think it's the case that fat binaries can augment package management,
as well as work entirely without it: fat binaries make it easier for a
developer to produce multi-architecture packages, which can either be
thinned server-side into single-architecture packages, or thinned
client-side, depending on requirements.
Well, multi-architecture packages could be done without fat binaries.
It's really just a question of the package format supporting it. So far
we haven't planned anything like this. The current plan is to simply
have a separate package for the secondary architecture.
However, not shipping actual fat
packages makes transitions such as that of x86-32 to x86-64 much more
difficult (generally requiring reinstall, or lockstep update of all
components), and I think that shipping fat binaries makes sense.
Admittedly not having to put the binaries for the secondary architecture
into separate directories makes things look cleaner. But that aside, I
don't see why fat binaries would ease the transition. I mean this really
is the only difference: fat binary == everything is rolled into one
file, slim binaries == there's a file for each architecture.
Functionally there's little difference between the two approaches.
AFAICT if fat binaries are allowed to be slimmed down, this is going to
be a major headache for package management. Maybe this could be solved
elegantly somehow. I haven't thought about it.
To sum it up, the arguments that stick IMO:
pros:
* A developer only has to distribute a single package instead of one per
architecture.
* A user doesn't have to pick the package for their architecture.
cons:
* Size.
* Probably package management unfriendly.
CU, Ingo
PS: What I'd really like to see is support for LLVM bytecode binaries.
Why fat when it can be portable?
Other related posts:
