[openbeos] New Build System
- From: Ingo Weinhold <bonefish@xxxxxxxxxxxxxxx>
- To: openbeos@xxxxxxxxxxxxx
- Date: Sat, 29 Oct 2005 18:34:14 +0200
Howdy,
I've been working several weeks on improving the build system, and have now
merged those changes into the main trunk, i.e. they become effective to all
developers as soon as they update their working copy.
Although I intended -- and still intend -- to write a bit of documentation
for the build system, ATM there is still as little as there was before,
that is only the inline documentation of the jam rules. So, let me list
what changes for the build system user:
1. Jam 2.5rc3 Required
Theoretically the old build system worked sufficiently fine with jam 2.4.
In practice, due to bugs in jam, one had to use the jam version (2.5rc3)
from our repository, which had those bugs fixed. Now the build system
officially requires jam 2.5rc3, since it uses new features introduced with
that version. Again, in theory one could use the pristine jam as available
from Perforce, but in practice one wants to use the one from our
repository, due to fixed bugs. Since I recently fixed another bug, it is
recommended to update the jam executable; otherwise errors may occur when
building libz or libpng.
A compiled version of our jam can be downloaded here:
http://tfs.cs.tu-berlin.de/~bonefish/download/jam/jam-2.5rc3-haiku-2005-10-29.zip
2. Strict Separation of Host and Target
With the old build system we could specify different target platforms
(Haiku, BeOS R5, BONE, Dano), but we had no clear distinction between the
tool chain, headers, and libraries to be used for building things. Now the
build system knows two different sets of variables describing the build
environment for the host platform and for the one specified as target.
Tools like rc or xres, that are needed by the build system itself are built
for the host platform, while the "meaty" stuff is built for the target
platform.
A significant restriction I introduced, is that it is only possible to
build for a non-Haiku target platform (i.e. BeOS R5, BONE, Dano) on a host
platform that is at least compatible to it (e.g. one can build for R5 under
Dano, but not the other way around). The reason for this is that it
considerably simplifies the build system, since in case of such a target
platform it will use the tools, headers, and libraries of the host
environment.
Besides this there a no consequences for the build system user. One can
still specify the target platform by passing it to "configure", or by
defining the environment variable TARGET_PLATFORM.
3. Linux Host Platform Support
As a result of 2. it was possible to add Linux as a host platform for
building Haiku. To build Haiku under Linux you have to use cross
compilation tools. The easiest way is to check out both, the Haiku trunk
and the buildtools trunk. Assuming the current working directory is the
root directory of the trunk and the buildtools are checked out alongside
the Haiku trunk, one would invoke configure like that:
./configure --build-cross-tools ../buildtools
This builds the cross compilation tools from the sources and prepares the
Haiku build system to use them.
4. Reorganized Output Directory Structure
The former bunch of output directories ("objects", "distro", "tests",...)
has been reorganized. Now there is only a single output directory which
contains similar subdirectories. "objects" and "tests" are still there,
"distro" is gone, though, since it is no longer needed. By default the
single output directory is named "generated" and placed in the root
directory of the checked out sources -- this happens if the "configure"
script is invoked just there. But it is also possible to choose another
output directory by invoking "configure" from that directory. A Jamfile is
created in the directory, so that one can directly build things there.
When updating an existing working copy to the new build system, it is
necessary to invoke "configure" again, and recommended to delete the old
output directories.
5. makehdimage Gone
As so often, interim solutions have an incredibly long lifetime. That of
the makehdimage script is finally over, though, since the jam build system
has taken over the task. "jam haiku-image" builds an image, whose name,
directory, and size are specified by the jam or environment variables
HAIKU_IMAGE_NAME, HAIKU_IMAGE_DIR, and HAIKU_IMAGE_SIZE. Defaults are
"haiku.image", "generated", and "80" (MB), i.e. an 80 MB image will be
created at "generated/haiku.image". "jam install-haiku" installs Haiku into
a directory (e.g. a mounted volume), which is specified via the
HAIKU_INSTALL_DIR variable (default is "/haiku").
What belongs to a Haiku installation is specified in build/jam/HaikuImage
using the Add{Files,Directory,Symlink}ToHaikuImage rules.
The makehaikufloppy script Axel introduced recently is still there, but
since I haven't had a closer look yet, I would suspect it to be broken now
(mainly due to the changed directory structure).
6. Customization/Configuration
A new feature I've called configuration variables has been introduced.
Those are jam variables that are set to a well-defined configuration in
each Jamfile. Meaning two things:
* One can change them at one's leisure in a Jamfile without affecting their
values in other Jamfiles.
* One can define their values for any Jamfile in a central place. The file
intended for that purpose is build/jam/UserBuildConfig, which is included
by the build system if existent, but ignored by svn. So a developer can
customize the build according to own needs (e.g. turning on debugging,
adding defines, even adding new targets, etc.) using that file, without
changing any Jamfile. For samples confer build/jam/UserBuildConfig.sample.
What variables are configuration variables is defined by the variable
AUTO_SET_UP_CONFIG_VARIABLES, including e.g. CCFLAGS, C++FLAGS, DEBUG,
DEFINES, HDRS, OPTIM,... For a complete list have a look at the bottom of
the files build/jam/ConfigRules and build/jam/BuildSetup.
7. System Headers
Something we inherited from the standard jam rules was the missing
discrimination between system and local header search paths (for <...> vs.
"..." includes). This is now remedied. The formerly existing rules for
headers -- SubDirHdrs, ObjectHdrs, SourceHdrs -- do now add to the local
include paths, while the new rules SubDirSysHdrs, ObjectSysHdrs,
SourceSysHdrs extend the system header search paths. The new variables
coming with them are SYSHDRS and SUBDIRSYSHDRS. The UsePublicHeaders rule
always adds system header search paths, while UsePrivateHeaders and
UseHeaders have a new boolean parameter indicating whether the added paths
are system search paths (defaults are "true" and "false" respectively).
Note, that this change breaks the build of components that do not correctly
use "..." and <...> or use system headers other than the public or private
headers. I have fixed everything that was built by makehdimage, but beyond
that very little.
8. PLATFORM/SUPPORTED_PLATFORMS
The platform a target is actually being built for is determined by the
variable PLATFORM. It is allowed to set it to either of two values -- to
the value of TARGET_PLATFORM (which is the default) or to "host". One
should actually not need to meddle with this variable explicitly. If one
wants to build something for the host platform (e.g. a tool needed for
building other things), the BuildPlatform{Objects,Main,SharedLibrary,...}
rule can be used, which set the PLATFORM to "host" for that target.
Of more interest is the SUPPORTED_PLATFORMS variable. It lists all
platforms a target can be built for. If for a target the PLATFORM is not
included in the SUPPORTED_PLATFORMS, this target simply won't exist. E.g.
if TARGET_PLATFORM is set to "r5", a "jam bfs" will only yield a "don't
know how to make bfs" error message, since the BFS module can only be built
for Haiku.
Both PLATFORM and SUPPORTED_PLATFORMS are configuration variables, so one
can directly set their values in a Jamfile without affecting other
Jamfiles. The rules SetPlatformForTarget, SetSubDirPlatform,
SetSupportedPlatformsForTarget, and SetSubDirSupportedPlatforms are
preferrable though. The rule SetSubDirSupportedPlatformsBeOSCompatible,
which doesn't take any arguments, sets the supported platforms of the
subdirectory to all platforms that are BeOS compatible, i.e. "r5", "bone",
"dano", and "haiku".
9. Per Target Variable Settings
A lot of variables that control the compilation/linking of targets are now
settable per target. Most notably DEBUG which formerly always affected all
targets. Now it can be set per directory (using the configuration variable
feature) or directly per target. The compilation flags as well as the
output directory for the generated file will be set accordingly.
Just to name a few other variables that can be set per target: WARNINGS,
{HOST,TARGET}_HDRS, {HOST,TARGET}_DEFINES,
{HOST,TARGET}_WARNING_{CC,C++}_FLAGS,...
10. Renamed/Refactored Rules and Variables
A lot of renaming and refactoring has been going on. Here're the most
relevant things I remember off the top of my head:
* All OBOS_* variables have been renamed to HAIKU_*. There are now three
sets of variables: HAIKU_*, HOST_*, and TARGET_*. The ones that don't have
either prefix are most likely variables inherited from jam.
* {R5,BONE,DANO,HAIKU}_COMPATIBLE still exist, but are deprecated. The
TARGET_PLATFORM_{BEOS,BONE,DANO,HAIKU}_COMPATIBLE variables should be used
instead. There are also analog HOST_* variables.
* The App rule is now named Application.
* LinkSharedOSLibs is more suitably named LinkAgainst.
* The R5* rules are gone (e.g. R5KernelAddon). The rules without the prefix
have taken over the chores.
* The {Object,Source}[Sys]Hdrs and Object{CCFlags,C++Flags,Defines} all
support more than one supplied file and can now be invoked *before* the
rule that builds the respective objects, which is also the recommended way
to do it.
* The Lex++ and Bison rules are gone, instead the Lex and Yacc rules have
been extended.
11. LinkAgainst Library Name Translation
When building for target "haiku", the LinkAgainst rule (formerly
LinkSharedOSLibs) does now translate library names, like "be" to
"libbe.so", "translation" to "libtranslation.so" etc. (if something is
missing, feel free to search for HAIKU_LIBRARY_NAME_MAP in
build/jam/BuildSetup and add it). This simplifies things for targets that
can be built not only for Haiku, but also for BeOS. The Application,
BinCommand, Server, Addon,... rules that take a library list support the
same feature.
Also note, that it is no longer necessary to link against "root" or
"libroot.so" explicitely. The Application, BinCommand, SharedLibrary,...
rules automatically add it, if necessary.
12. Broken Things
Please note, that several things that formerly worked, won't build anymore.
I only took care, that everything makehdimage was building plus a few other
components still build. Anything else might be broken at the moment. I
basically foresee two kinds of problems:
* As described in 7. the discrimination between local and system headers
can cause problems. Third party (e.g. GNU) tools for instance often include
their local "config.h" and other headers as system headers, which doesn't
work, unless the respective directories are explicitely added as system
header search dirs.
* Components that could be built for target "r5" don't build anymore.
Usually this can be remedied by added an invocation of
SetSubDirSupportedPlatformsBeOSCompatible at the beginning of the
respective Jamfile. I did that for all Jamfiles that use the Package rule
(I figured that packages were built mainly for R5/BONE/Dano). I didn't test
all of them, though.
Furthermore it is known that the unit tests are completely broken. Unless
someone needs them urgently, I would take care of that over the course of
the next weeks.
If you encounter anything that doesn't work and feel confident to fix it,
please do so. Otherwise drop me a mail.
CU, Ingo
Other related posts:
