[haiku-development] Re: A modest (FatELF) proposal
- From: Dustin Howett <alaricx@xxxxxxxxx>
- To: "haiku-development@xxxxxxxxxxxxx" <haiku-development@xxxxxxxxxxxxx>
- Date: Sun, 18 Nov 2012 14:47:29 -0800
On Nov 18, 2012, at 10:02 AM, Landon Fuller <landonf@xxxxxxxxxxxxxx> wrote:
> Hello All --
>
> This would be my first post to the list, and while I'd intended to not send
> anything until I sat with patches in-hand, I was convinced on IRC that I
> ought to float my idea of introducing FatELF before spending too much time on
> it, on the chance that it proves to be non-viable for inclusion. My hope is
> that any controversy can wait until I have a least a proof-of-concept
> working, and that this posting will instead elicit some suggestions or advice
> from those more knowledgable than I.
>
> With the combination of x86-gcc2, x86-gcc4, x86-64, and potentially ARM,
> there are a plethora of architectures that Haiku currently does -- or will --
> support. In addition, certain architectures -- such as ARM -- are evolving
> relatively rapidly compared to x86, with new functionality introduced across
> rapidly deployed revisions of the processors. On iPhones, we've seen the
> rapid introduction of various iterations of VFP and NEON, THUMB-2, etc. On
> Linux, they've struggled with weighing the costs of breaking ABI with the
> advantages of adopting new functionality available in new processors.
>
> These ABI shifts have the potential to introduce significant maintenance
> burden on just about everyone -- the user has to choose between different
> binary versions, the OS vendor has to decide at which point to break ABI (if
> at all), and the developer has to rebuild their application individually for
> each ABI, and provide separate downloads or packages. We see from the Linux,
> BSD, and Windows worlds that even on x86, this results in completely distinct
> OS distributions and packages. Transitioning from x86-32 to x86-64 requires a
> complete reinstall, as would (for example) transitioning from armv7 to
> apple's armv7s -- a small iterative improvement that adds VFPv4, and one that
> hardly justifies a full OS reinstall.
>
> In terms of solutions, the issue of ABI compatibility isn't just one of
> selecting the right libraries at runtime, either, and can't be solved just by
> dynamically providing the correct library ABI on demand. Applications that
> host plugins must have an ABI that matches that of the plugins they load.
> This introduces some interesting issues during transitionary periods, as not
> all plugins will be updated immediately, and as such, it proves impossible to
> allow for a gradual transition by third party plugins to a new architecture
> -- it must be all or nothing.
>
> Fat binaries ease this transitions considerably; they were leveraged by NeXT
> when they were moving beyond m68k, by Apple when they were transitioning from
> m68k to Intel, and further leveraged heavily during the Mac OS X transitions
> from PPC to PPC64, PPC to x86, and mixed x86/x86-64 installations, in
> addition to the iOS transitions from ARMv6->ARMv7->ARMv7s.
>
> Fat binaries make it easy for the OS vendor, user, and software developer to
> support new architectures through gradual migration, by pushing the cost of
> doing so onto the development toolchain. Rather than compile and distribute
> something N times for N different architectures, your build process can run
> the compiler with the appropriate -arch flags, and compile a binary that runs
> on all the available architectures. When hosting plugins, the hosting
> application can automatically 'downgrade' to the lowest common denominator
> ABI by relaunching itself, thus allowing for gradual transitions between old
> ABIs and new ABIs -- this was done by Apple in System Preferences when the
> entire OS switched over to x86-64, allowing System Preferences to switch back
> to i386 in the case where the user still had legacy plugins installed.
>
> From the user's perspective, there's nothing to think about. There's a single
> binary to download, and a single installation CD/DVD to run, and the right
> ABI is used depending on their host environment. From the software
> developer's perspective, things are equally easy -- by adding new -arch flags
> to their compiler invocations, they can automatically target multiple
> architectures/ABIs straight from the same build, and provide a single binary
> to run. Transitions from one architecture to another can occur gradually,
> without every single third-party developer and OS vendor supplied application
> making the transition at once.
>
> The downsides are minimal; most of an application's size is consumed by
> resources, rather than code, and it is entirely possible to strip out unused
> architectures for the particularly space-concious user. In reality, this
> hasn't been an issue, even on iOS where application downloads are performed
> over cellular, capped at 20MB, and have to fit on devices with as 'little' as
> 8GB of space.
>
> Apple's solution is based on Mach-O, and involves changes to their runtime
> linker, kernel loading, a compiler driver that sits in front of the actual
> cross-compilers, binutils changes to support linking, gdb and lldb support
> for parsing 'universal' Mach-O binaries. For ELF systems, Ryan Gordon has
> gone to the considerable lengths of designing a formal specification, a set
> of portable tools to work with fat ELF files, and patches for glibc, the
> Linux kernel, file(1), binutils, gdb, etc: http://icculus.org/fatelf/ His
> work appears to be particularly solid, and as a proof of concept, I'd like to
> bring it over to Haiku.
>
> All that said, FatELF was met with staunch objection in the Linux world.
> Despite the advantages of fat binaries demonstrated across 20 years of
> architectural transitions and ABI shifts, there was considerable pushback
> from Linux developers that believed that packaging negates the need for fat
> binaries, or that fat binaries are simply not useful for users or developers
> at all. This, despite evidence of painful ABI updates, the inability to
> easily and iteratively transition from x86-32 to x86-64 operating systems
> without a complete reinstall, and considerable evidence that fat binaries
> greatly simplify architectural transitions for users and developers alike,
> including commercial developers targeting a platform.
>
> I think it's the case that fat binaries can augment package management, as
> well as work entirely without it: fat binaries make it easier for a developer
> to produce multi-architecture packages, which can either be thinned
> server-side into single-architecture packages, or thinned client-side,
> depending on requirements. However, not shipping actual fat packages makes
> transitions such as that of x86-32 to x86-64 much more difficult (generally
> requiring reinstall, or lockstep update of all components), and I think that
> shipping fat binaries makes sense.
>
> Additionally, I've been told that there's no need to handle multiple
> architectures this at the fat binary layer, and packaging could do this at a
> higher level. That may be true, but the advantage of doing this with binaries
> is in simplicity for both user, developer, and implementation. The developer
> doesn't need to build multiple binaries and merge them into a package, they
> can simply apply -arch flags to their build configuration, linking against
> the fat libraries available on their system. The user doesn't need to
> 'install' a package, they can run a binary in-place, wherever it is, and the
> right architecture will be selected and run. This is something my company
> leverages in our build systems, where we can check in external tools as
> binaries (such as protobuf's protoc), and then simply run them in place,
> whether or the user is on a PPC, i386, or x86-64 system. The runtime
> implementation itself is extremely simple, requiring only minimal parsing of
> the file header before selecting the section of the file to map -- most of
> the cost is placed on the toolchain pieces responsible for linking.
>
> TL;DR In conclusion, I wanted to float the idea of FatELF, since it has a
> broad impact on tooling and distribution issues. That said, I don't think the
> question of inclusion has to be decided up-front without even a
> proof-of-concept implementation -- I'm working on bringing over the FatELF
> tools now, and would like to get to the point that a proof-of-concept
> gcc2/gcc4 fat hybrid is running. However, I also don't want to waste my time
> on it if, like the Linux community, the response is a resounding and absolute
> "no, never ever!" from all parties involved. I haven't been here long, but so
> far I haven't met any Ulrich Dreppers in Haiku:
>
> http://www.redhat.com/archives/fedora-devel-list/2009-October/msg01118.html
>
> Thanks for making it through all of the above (or at least reading the tl;dr)
> :)
>
> -landonf
I had seen FatELF some time ago, and always thought it absurd that it was
rejected so harshly. I am likewise not a Haiku developer, but feel that it
would be an awesome addition to a great platform.
-- Dustin L. Howett.
Other related posts:
