[openbeos-build-team] Goals for the new build system

  • From: "Ryan Leavengood" <leavengood@xxxxxxxxxxxx>
  • To: "OpenBeOS Build Team" <openbeos-build-team@xxxxxxxxxxxxx>
  • Date: Thu, 04 Jul 2002 13:25:28 EDT

I know you all wanted the new build system done yesterday 
(metaphorically speaking), but I think something like this requires 
more than just a few days of trial and error hacking.  Plus I hope you 
would all agree that it would be a waste of effort to totally revamp 
the build system in our current CVS tree when Erik is working on a new 
tree and we would just have to spend extra time "porting" the new build 
system over to that tree.

So, given that, I've decided to explain what I feel are some of the 
goals for the new build system.  Here we go:

1. Building needs to be as easy as possible.  All someone should need 
to do is "cd current; ./configure; jam" to get the whole tree built.

We pretty much already have this now, so I don't think this goal is a 
difficult one.  The main thing to do is make sure we get everything in 
the tree to build correctly (which doesn't happen now) and also to get 
rid of some compilation warnings.  Of course a lot of this is coding 
related, and while it might be better if the authors of the modules 
giving warnings fixed them, it might just be the case that my build 
team fixes them for the sake of expediency.

2. The Jam files should be as simple and easy to maintain as possible.  
That way it won't take legions of build team members to maintain the 
tree, but just a few newsletter articles from myself explaining Jam and 
our build system to the masses so that they themselves can learn how to 
maintain the tree.

In my opinion we definitely don't have this now.  No disrespect to the 
current Jamfile/Jamrules authors, but even I can barely figure them out 
;)  I can't know for sure if this goal is possible given our complex 
build requirements, but I think it at least deserves to be tried.  
Also, mostly what I mean by "maintaining the tree" is that new people 
should be able to easily add their modules with proper Jamfiles, so 
that the build team doesn't have to go back in later to fix them. 

3. We should reduce the hard-coding of source files in Jamfiles as much 
as possible.  This is especially true in the kernel Jamfiles where we 
have (for example) all the libc .c source file listed out in each libc 
sub-directory, and then in the higher level kernel Jamfile, we have the 
libc .o files listed out, with GRIST and all, so they can be build into 
a static lib.

That is not how we should do this.  I will address this more in my next 
point, but basically if one is to hardcode (for example) bcopy.? 
anywhere, it should only be once, and it should be bcopy.c in the libc/
string sub-directory.  But in fact, I'm not entirely convinced we even 
need the hard-coding at all.  Since we are using Jam 2.4 now, which has 
the GLOB rule, we could just use KernelObjects [ GLOB . *.c ] ; in each 
kernel sub-directory.  Or, if we also need the $(SOURCE_GRIST) appended 
to each source (which must be the case given the current Jamfiles), we 
could write a rule to do this using GLOB and a for loop, so that again 
we have avoided the hard-coding.  I say if we can use the power of Jam 
to make our build files simpler, we should.

The only problem with GLOB is that if we have .c files in the current 
directory that we don't want built (for some reason) they would 
"accidentally" be added to the list.  Though I would really only 
imagine this happening during development on someone's local tree, it 
might happen.  In that case it would be trivial to add a second 
optional parameter to the rule I described above that would be a list 
of files to ignore.  Within the for loop there would be an if statement 
seeing if the current file was in the ignore list, and if it was, it 
wouldn't be added to the final list.  Doesn't that sound more elegant 
than hard-coding all the file names?

4. Each sub-directory of the tree should be self-contained, and higher 
level directories should only make use of the "biggest" targets in each 
of their sub-directories.  This means I should be able to go into any 
directory of the tree, type jam, and get the targets in that directory 
and lower built.

It seems we pretty much already have this, except for my second point.  
As I mentioned in point 3, the kernel Jamfile actually builds the 
static libraries for libc and libm, as well as the kernel itself.  In 
my opinion, the only thing the kernel Jamfile should be doing is 
putting together all the various high-level targets from each sub-
directory into the kernel.  For example, libc.a should be built by the 
Jamfile in the libc directory, and then just linked in with the kernel 
by the kernel Jamfile.  Basically I'm just advocating low-coupling and 
high-cohesion in our build system.

5. We will have new pseudo-targets for most major functions of the 
build.  To create a distribution, once will be able to do jam dist.  To 
install things on your local machine, one will be able to do jam 
install.  To build the unit tests, one will be able to do jam tests.  
To run those tests, one will be able to do jam runtests.

Of course right now we have none of this, and I'm sure you all agree 
they need to be added.

6. One final point, though it isn't really a requirement of the build 
system: could you guys explain the reasoning behind the separate 
objects directory?  In my opinion (which may not be the best since I've 
done very little C/C++ development), outputting the object files to a 
separate directory doesn't give that much benefit, but does have 
several drawbacks.  It makes the Jamfiles harder to write and it 
"hides" the object files such that if someone is interested in looking 
at them, they have to cd into another entire hierarchy to find them.  
Also, why use x86.R5 directory in the objects directory?  I would 
imagine that until we have a system for cross-compiling (which I doubt 
we will), it is pretty redundant to list the platform something was 
built on, since one tends to already know that already (i.e. you can be 
pretty sure those objects files are for x86 if you are building on 
x86.)

The reason I say this is that it just seems more "clean" and organized 
to me to keep the objects and other targets for each directory and 
Jamfile in the same directory as the sources and Jamfile.  If you feel 
that having all those .o files with the .c files makes things "messy", 
how about an objects directory in each source directory?  Hmmm, though 
I'm not sure about that as far as Jam is concerned.

I guess I'm sort of on the fence on this issue, but I would like to 
read opinions as to why the whole separate objects tree should be used 
(and not just "it makes the source tree cleaner".)  Because keep in 
mind that eventually we will be adding more pseudo-targets like I said 
above, and I have a feeling having this separate object tree will make 
those a lot more complicated.  Also keep in mind that "jam clean" works 
very well and is completely automatic, so it is not like putting the 
objects with the sources makes that any more complicated.

So let me know what you think about this as well as any other views or 
input on the new build system.

Ryan Leavengood
OpenBeOS Build Team Lead

Other related posts: