[openbeos] Re: New Build System

One thing to say: Congratulations.
Here works like a charm on a fresh checkout.

On 2005-10-29 at 18:34:14 [+0200], Ingo Weinhold <bonefish@xxxxxxxxxxxxxxx> 
wrote:
> Howdy,
> 
> I've been working several weeks on improving the build system, and have now
> merged those changes into the main trunk, i.e. they become effective to all
> developers as soon as they update their working copy.
> 
> Although I intended -- and still intend -- to write a bit of documentation
> for the build system, ATM there is still as little as there was before,
> that is only the inline documentation of the jam rules. So, let me list
> what changes for the build system user:
> 
> 1. Jam 2.5rc3 Required
> 
> Theoretically the old build system worked sufficiently fine with jam 2.4.
> In practice, due to bugs in jam, one had to use the jam version (2.5rc3)
> from our repository, which had those bugs fixed. Now the build system
> officially requires jam 2.5rc3, since it uses new features introduced with
> that version. Again, in theory one could use the pristine jam as available
> from Perforce, but in practice one wants to use the one from our
> repository, due to fixed bugs. Since I recently fixed another bug, it is
> recommended to update the jam executable; otherwise errors may occur when
> building libz or libpng.
> 
> A compiled version of our jam can be downloaded here:
> 
> http://tfs.cs.tu-berlin.de/~bonefish/download/jam/jam-2.5rc3-haiku-2005-10-29.zip
> 
> 
> 2. Strict Separation of Host and Target
> 
> With the old build system we could specify different target platforms
> (Haiku, BeOS R5, BONE, Dano), but we had no clear distinction between the
> tool chain, headers, and libraries to be used for building things. Now the
> build system knows two different sets of variables describing the build
> environment for the host platform and for the one specified as target.
> Tools like rc or xres, that are needed by the build system itself are built
> for the host platform, while the "meaty" stuff is built for the target
> platform.
> 
> A significant restriction I introduced, is that it is only possible to
> build for a non-Haiku target platform (i.e. BeOS R5, BONE, Dano) on a host
> platform that is at least compatible to it (e.g. one can build for R5 under
> Dano, but not the other way around). The reason for this is that it
> considerably simplifies the build system, since in case of such a target
> platform it will use the tools, headers, and libraries of the host
> environment.
> 
> Besides this there a no consequences for the build system user. One can
> still specify the target platform by passing it to "configure", or by
> defining the environment variable TARGET_PLATFORM.
> 
> 
> 3. Linux Host Platform Support
> 
> As a result of 2. it was possible to add Linux as a host platform for
> building Haiku. To build Haiku under Linux you have to use cross
> compilation tools. The easiest way is to check out both, the Haiku trunk
> and the buildtools trunk. Assuming the current working directory is the
> root directory of the trunk and the buildtools are checked out alongside
> the Haiku trunk, one would invoke configure like that:
> 
>   ./configure --build-cross-tools ../buildtools
> 
> This builds the cross compilation tools from the sources and prepares the
> Haiku build system to use them.
> 
> 
> 4. Reorganized Output Directory Structure
> 
> The former bunch of output directories ("objects", "distro", "tests",...)
> has been reorganized. Now there is only a single output directory which
> contains similar subdirectories. "objects" and "tests" are still there,
> "distro" is gone, though, since it is no longer needed. By default the
> single output directory is named "generated" and placed in the root
> directory of the checked out sources -- this happens if the "configure"
> script is invoked just there. But it is also possible to choose another
> output directory by invoking "configure" from that directory. A Jamfile is
> created in the directory, so that one can directly build things there.
> 
> When updating an existing working copy to the new build system, it is
> necessary to invoke "configure" again, and recommended to delete the old
> output directories.
> 
> 
> 5. makehdimage Gone
> 
> As so often, interim solutions have an incredibly long lifetime. That of
> the makehdimage script is finally over, though, since the jam build system
> has taken over the task. "jam haiku-image" builds an image, whose name,
> directory, and size are specified by the jam or environment variables
> HAIKU_IMAGE_NAME, HAIKU_IMAGE_DIR, and HAIKU_IMAGE_SIZE. Defaults are
> "haiku.image", "generated", and "80" (MB), i.e. an 80 MB image will be
> created at "generated/haiku.image". "jam install-haiku" installs Haiku into
> a directory (e.g. a mounted volume), which is specified via the
> HAIKU_INSTALL_DIR variable (default is "/haiku").
> 
> What belongs to a Haiku installation is specified in build/jam/HaikuImage
> using the Add{Files,Directory,Symlink}ToHaikuImage rules.
> 
> The makehaikufloppy script Axel introduced recently is still there, but
> since I haven't had a closer look yet, I would suspect it to be broken now
> (mainly due to the changed directory structure).
> 
> 
> 6. Customization/Configuration
> 
> A new feature I've called configuration variables has been introduced.
> Those are jam variables that are set to a well-defined configuration in
> each Jamfile. Meaning two things:
> 
> * One can change them at one's leisure in a Jamfile without affecting their
> values in other Jamfiles.
> 
> * One can define their values for any Jamfile in a central place. The file
> intended for that purpose is build/jam/UserBuildConfig, which is included
> by the build system if existent, but ignored by svn. So a developer can
> customize the build according to own needs (e.g. turning on debugging,
> adding defines, even adding new targets, etc.) using that file, without
> changing any Jamfile. For samples confer build/jam/UserBuildConfig.sample.
> 
> What variables are configuration variables is defined by the variable
> AUTO_SET_UP_CONFIG_VARIABLES, including e.g. CCFLAGS, C++FLAGS, DEBUG,
> DEFINES, HDRS, OPTIM,... For a complete list have a look at the bottom of
> the files build/jam/ConfigRules and build/jam/BuildSetup.
> 
> 
> 7. System Headers
> 
> Something we inherited from the standard jam rules was the missing
> discrimination between system and local header search paths (for <...> vs.
> "..." includes). This is now remedied. The formerly existing rules for
> headers -- SubDirHdrs, ObjectHdrs, SourceHdrs -- do now add to the local
> include paths, while the new rules SubDirSysHdrs, ObjectSysHdrs,
> SourceSysHdrs extend the system header search paths. The new variables
> coming with them are SYSHDRS and SUBDIRSYSHDRS. The UsePublicHeaders rule
> always adds system header search paths, while UsePrivateHeaders and
> UseHeaders have a new boolean parameter indicating whether the added paths
> are system search paths (defaults are "true" and "false" respectively).
> 
> Note, that this change breaks the build of components that do not correctly
> use "..." and <...> or use system headers other than the public or private
> headers. I have fixed everything that was built by makehdimage, but beyond
> that very little.
> 
> 
> 8. PLATFORM/SUPPORTED_PLATFORMS
> 
> The platform a target is actually being built for is determined by the
> variable PLATFORM. It is allowed to set it to either of two values -- to
> the value of TARGET_PLATFORM (which is the default) or to "host". One
> should actually not need to meddle with this variable explicitly. If one
> wants to build something for the host platform (e.g. a tool needed for
> building other things), the BuildPlatform{Objects,Main,SharedLibrary,...}
> rule can be used, which set the PLATFORM to "host" for that target.
> 
> Of more interest is the SUPPORTED_PLATFORMS variable. It lists all
> platforms a target can be built for. If for a target the PLATFORM is not
> included in the SUPPORTED_PLATFORMS, this target simply won't exist. E.g.
> if TARGET_PLATFORM is set to "r5", a "jam bfs" will only yield a "don't
> know how to make bfs" error message, since the BFS module can only be built
> for Haiku.
> 
> Both PLATFORM and SUPPORTED_PLATFORMS are configuration variables, so one
> can directly set their values in a Jamfile without affecting other
> Jamfiles. The rules SetPlatformForTarget, SetSubDirPlatform,
> SetSupportedPlatformsForTarget, and SetSubDirSupportedPlatforms are
> preferrable though. The rule SetSubDirSupportedPlatformsBeOSCompatible,
> which doesn't take any arguments, sets the supported platforms of the
> subdirectory to all platforms that are BeOS compatible, i.e. "r5", "bone",
> "dano", and "haiku".
> 
> 
> 9. Per Target Variable Settings
> 
> A lot of variables that control the compilation/linking of targets are now
> settable per target. Most notably DEBUG which formerly always affected all
> targets. Now it can be set per directory (using the configuration variable
> feature) or directly per target. The compilation flags as well as the
> output directory for the generated file will be set accordingly.
> 
> Just to name a few other variables that can be set per target: WARNINGS,
> {HOST,TARGET}_HDRS, {HOST,TARGET}_DEFINES,
> {HOST,TARGET}_WARNING_{CC,C++}_FLAGS,...
> 
> 
> 10. Renamed/Refactored Rules and Variables
> 
> A lot of renaming and refactoring has been going on. Here're the most
> relevant things I remember off the top of my head:
> 
> * All OBOS_* variables have been renamed to HAIKU_*. There are now three
> sets of variables: HAIKU_*, HOST_*, and TARGET_*. The ones that don't have
> either prefix are most likely variables inherited from jam.
> 
> * {R5,BONE,DANO,HAIKU}_COMPATIBLE still exist, but are deprecated. The
> TARGET_PLATFORM_{BEOS,BONE,DANO,HAIKU}_COMPATIBLE variables should be used
> instead. There are also analog HOST_* variables.
> 
> * The App rule is now named Application.
> 
> * LinkSharedOSLibs is more suitably named LinkAgainst.
> 
> * The R5* rules are gone (e.g. R5KernelAddon). The rules without the prefix
> have taken over the chores.
> 
> * The {Object,Source}[Sys]Hdrs and Object{CCFlags,C++Flags,Defines} all
> support more than one supplied file and can now be invoked *before* the
> rule that builds the respective objects, which is also the recommended way
> to do it.
> 
> * The Lex++ and Bison rules are gone, instead the Lex and Yacc rules have
> been extended.
> 
> 
> 11. LinkAgainst Library Name Translation
> 
> When building for target "haiku", the LinkAgainst rule (formerly
> LinkSharedOSLibs) does now translate library names, like "be" to
> "libbe.so", "translation" to "libtranslation.so" etc. (if something is
> missing, feel free to search for HAIKU_LIBRARY_NAME_MAP in
> build/jam/BuildSetup and add it). This simplifies things for targets that
> can be built not only for Haiku, but also for BeOS. The Application,
> BinCommand, Server, Addon,... rules that take a library list support the
> same feature.
> 
> Also note, that it is no longer necessary to link against "root" or
> "libroot.so" explicitely. The Application, BinCommand, SharedLibrary,...
> rules automatically add it, if necessary.
> 
> 
> 12. Broken Things
> 
> Please note, that several things that formerly worked, won't build anymore.
> I only took care, that everything makehdimage was building plus a few other
> components still build. Anything else might be broken at the moment. I
> basically foresee two kinds of problems:
> 
> * As described in 7. the discrimination between local and system headers
> can cause problems. Third party (e.g. GNU) tools for instance often include
> their local "config.h" and other headers as system headers, which doesn't
> work, unless the respective directories are explicitely added as system
> header search dirs.
> 
> * Components that could be built for target "r5" don't build anymore.
> Usually this can be remedied by added an invocation of
> SetSubDirSupportedPlatformsBeOSCompatible at the beginning of the
> respective Jamfile. I did that for all Jamfiles that use the Package rule
> (I figured that packages were built mainly for R5/BONE/Dano). I didn't test
> all of them, though.
> 
> Furthermore it is known that the unit tests are completely broken. Unless
> someone needs them urgently, I would take care of that over the course of
> the next weeks.
> 
> If you encounter anything that doesn't work and feel confident to fix it,
> please do so. Otherwise drop me a mail.
> 
> CU, Ingo
-- 
"A man does what he does because he sees the world as he sees it" A.K
Radio404 - Ze Radio for Ze music - www.radio404.org
Blog: http://unamsbox.blogspot.com

Other related posts: