[hipl-dev] Re: binary packaging

Hi Diego,
(sorry for late reply due to the Nordsec 2010 conference which I was attending) 

On 10/27/2010 01:51 PM, Diego Biurrun wrote:

What's with the top-posting all of a sudden?

On Tue, Oct 26, 2010 at 10:30:23PM +0300, Miika Komu wrote:


On 26/10/10 20:57, Diego Biurrun wrote:

So I've been looking at our binary packaging infrastructure the past few
days.  The RPM and fake Debian packages generated from the spec files
are split into no less than 8 packages for hipd, hipfw, dnsproxy, doc,
tools plus virtual packages for a complete and a minimal installation
of HIPL and a library package that currently only contains useless
static libraries since we compile statically by default.

Is it worth going to all that trouble of splitting up HIPL into many
small parts?  HIPL is still experimental software in heavy flux and
the target systems are not so small (nowadays) that a few MBs of
software exceed capacity.


there's not much new features coming to HIPL, so I don't think it's in a
heavy flux anymore. As you have experienced, it has been more about
polishing HIPL for the past year.

What do you mean by experimental? Most of the features in HIPL are
driven by standardization. While the RFCs are experimental, this will
change quite soon.


Depends what you call heavy flux then.


The "experimental" implementation actually originates from 2001. We have
quite a lot of experience in running the software since then. We have
been running it successfully in small deployments for many years. All
the polishing and removal of features have increased better quality for
code and code stability.


I think you misjudge the state of HIPL.  On ohloh.net you wrote "I think
the code base is getting quite mature thanks to the contributions from
RWTH Aachen." - well, HIPL is surely improving and probably even much
quicker than ever before, but it's not in a state where one could let it
loose upon the world with a clear conscience.


I was just trying out ohloh and I believe in positive encouragement :)


This thing is supposed to be infrastructure software that touches every
single packet passing through a machine.  For that it needs to become
ultrafast and very reliable.  At the moment it is neither.
Not really all packets, just the HIP-related ones. Ultrafast and reliable calls for testing and performance optimization, not just polishing the code base. 


We have come a long way, no doubt about that, but there is still a
long and rocky road ahead.


I suggest we rethink if and how we split the HIPL binary packages.
Is one monolithic package enough?  Are hipd and hipfw independent
enough to be separated?


I think we should keep the current split with the packages because this
is the way it works in most other standard packages and I assume we want
HIPL to go into a mainstream linux distribution at some point.


The thought of HIPL in widespread use scares me ;)

Trolling aside, split packages make sense for large software packages
or for minimal systems.  Standard RPM/DEB packages of HIPL fall in
neither category IMO.
HIPL is not large, but we have needed just partial functionality of HIPL earlier. Splitting the packages was the most sensible way to me to avoid activating some parts of the software. The idea behind the minimal package is to provide minimal HIP functionality for the following cases: 

1. Servers that don't need DNS proxy and hipfw
2. P2P-SIP over HIP prototype has it's own way of resolving HIs, no need for DNS proxy or the LSI support provided by hipfw 


Also, P2P-SIP over HIP prototype used just a minimal installation of
HIPL (libs + hipd), so there has been a prior need to select only
certain parts of the software.


But the "minimal" package is nonsense.  It just depends on hipd and the
tools, but the tools are hardly necessary for any installation, much
less a minimal one.
"hipconf" is the one really needed from tools. Virtual packages are quite common. 


The lib package contains all the Python libraries for the hipdnsproxy.
The dnsproxy package contains them .. again.

The lib package contains static libraries (static build is the default)
that are (maybe) useful for development but by no means a dependency
for hipd and hipfw.  Plus some debug binaries that are also in the
debuginfo package.

So it's quite a mess overall and I wonder why it was never noticed.
I agree that the current way of packaging is not perfect but "fixing" it my monolithic packaging is not perfect either. 


I think there's not much benefit of merging the packages. For example, I
see more benefit in migrating from the brittle debbuild script to a
better way of handling the packaging (perhaps on a new branch). After
all, it does not even work on Debian at moment...


We should get rid of debbuild, yes.  We already have Debian packaging in
place, it just needs to be extended.  But since we're overhauling the
binary packaging infrastructure anyway, it makes sense to rethink now
which parts of it are good and which need to be changed.

Once we have the RPM infrastructure properly cleaned up, I will tackle
the Debian packaging issue.  Right now real Debian packaging stuff
exists in the debian/ subdirectory, but it builds a single monolithic
package.
Go ahead with the monolithic package if you really believe it's the best way to go. However, you should then make sure to modify packaging correctly so that there is "Replaces: hipl-all, hipl-daemon, .." dependency included in the rpm and deb packaging. This way, we can ensure a flawless upgrade path from current split packages to the monolithic version. If there's naming changes in packaging, please remind me to update the installation instructions at hipl.hiit.fi.

Other related posts: