[haiku-development] Bootstrapping Haiku
- From: Jonathan Schleifer <js-haiku-development@xxxxxxxxxxx>
- To: haiku-development@xxxxxxxxxxxxx
- Date: Tue, 11 Feb 2014 12:49:35 +0100
Hi everbody,
it seems like really nobody except olta did this before, so here's what
obstacles I had to overcome. There's even hardcoded /home/zooey at some places.
I haven't yet got it to fully bootstrap, but I got quite far. I already found
quite a few things that need fixing, but first I wanted to get it working at
all.
I first tried it on OS X, but after I found so many problems, I decided to get
it working on Linux first.
The first problem that affects Linux and OS X is that the generated
haikuports.conf contains relative paths - relative to the Haiku checkout. This
is a problem, as directly before calling Haiku, the current working directory
is changed. It took me quite a while to figure this out, because all it said
was "[Error 2] No such file or directory". It would not tell me which file, it
would not tell you which module even caused that. Basically, to find out, I had
to use dtruss -f on jam and look through huge list of syscalls to find out.
I manually changed the haikuports.conf, but this is not enough: During the
build, another one is created, and later on, a file called ports_list is passed
by jam to haikuporter - again with a relative path that is wrong after the cd.
I guess this would be easy to fix, I'm sure there's a function to get the
absolute path in jam. I have not looked into that yet and my jam knowledge is
quite limited. So if someone wants to help out here, that'd be great!
Another problem that exists on OS X and *BSD is that at some places sed is
called, expecting it is GNU sed. However, if you have GNU sed installed as gsed
and sed is the BSD sed, it will of course not work. Fixing this should be easy
(check if there's gsed, if so, use that. Maybe even check sed --version | grep
GNU returns something). This is something easy to fix that I plan to do once I
got it working on Linux. A similar problem is libtoolize vs glibtoolize.
A real problem on systems with a case insensitive file system is PATCH: For
some reason, it calls PATCH, but as the file system is case insensitive, it
calls patch. I tried to reproduce this on the shell, but couldn't:
bash-4.2$ PATCH() {
> echo patching
> }
bash-4.2$ PATCH
patching
So I have no idea what's going on here. This is again an issue I'll look into
once I got it building on Linux. I tried just pressing ^D (because patch was
waiting for input) with which I seem to have gotten quite far. Yet another
problem is that bash is expected in /bin/bash (not a problem on OS X, but on
BSD*) and that it is expected to be at least version 4 (a problem on OS X).
Some substitutions are used that seem to be only available in bash 4, while OS
X ships bash 3 as /bin/bash and bash 4 can be installed in e.g.
/usr/local/bin/bash. The question here is if it would make sense to just
replace those substitutions. Again, this is something for once I got it working
on Linux.
Then another OS X only problem: Building libicu_bootstrap results in the linker
complaining about symbols being duplicate. If you look at the code, it becomes
obvious why: They check if you are on OS X, and if you are, they define inline
to nothing and include <OS/Atomic.h> - which of course defines them all as
inline. Just removing that code and using the GCC built-ins solved that.
And now, finally, the problem where I'm stuck on Linux: After all bootstrap
packages have been built, it tries to build meta-ports/meta_portsfile. For
that, it passes a packages_list file, which lists all packages including
version that should be built. The problem however is that it can't find the
specified version, even though it exists. If you change the packages_list to
use the newest version, it works. However, there's one port for which this
can't be done: Python. Because that would mean using Python 3.2 instead of 2.7.
I have no idea why it is refusing to find anything but the newest version. I
tried changing the versions in the packages_list and deleting Python 3.2 - but
that wouldn't help, it would still not find Python 2.7 or 2.6. So if anybody
got an idea here, that'd be really appreciated!
My long term goal is to make bootstrapping as easy as possible. I consider the
state in which bootstrapping is right now alarming, as PM is still quite new,
yet so much is already broken. Fast forward a few years and it might be almost
impossible to bootstrap Haiku. That's why I want to make bootstrapping as easy
and well-tested as possible and ideally have a build bot at some point which
does nightly bootstraps, so we do notice if things break. It really worries me
that it seems that nobody besides olta ever bootstrapped Haiku. Ideally,
bootstrapping would at one point allow you to build all of Haiku without the
need to boot up a VM and continue building packages there.
Another, in my opinion really important, reason to do this is that we currently
all depend on unsigned binaries that are downloaded over an insecure
connection. If bootstrapping at one point breaks because we don't do it
regularly, it would mean we all depend on binaries we can't really trust. So I
think it's really important that we always have bootstrapping working. And also
to port it to other CPUs, of course!
--
Jonathan
Other related posts:
