[pisa] Re: hipl braindeadness^w^w suggestions for tiny hipl


On Nov 3, 2009, at 10:08 PM, Miika Komu wrote:


Tobias Heer wrote:

Hi.
I think it is important to fix as many of these issues in HIP before we split it into HIPL and TinyHIP. Otherwise the HIPL code will cease to be a valid development platform at some point. I would like to see the future of TinyHIP and HIPL like this: HIPL stays a development and experimentation platform but tries to improve as much as possible. TinyHIP advances to the point where it can be used in production systems. => None of both should die or be let behind because both are important. Hence, I think it is an error to hope that TinyHIP will magically solve all the problems. It won't.
I didn't really claim that TinyHIP is a silver bullet, I just have doubts how well the changes can be merged between HIPL and TinyHIP. Also, focusing usually gives better results.
Tim is currently getting rid of at least most of the warnings in tiny branch and we are striving for a similar warning level as already reached by pisa. I guess a good way of maintaining HIPL and tinyHIP in parallel is to differentiate very well between functional code changes and bugfixes. We should then try to apply bugfixes whenever possible in both implementations and check if the same bug still exists in case the patch is not valid any further. I think the major issue - at least in the beginning - is the required communication to ensure that bugfixes are always noticed. 


More comments inline.
Am 03.11.2009 um 16:34 schrieb Miika Komu:

Diego Biurrun wrote:

Hi,
Rene and Tobi, there's also some requests for comments for you in this email.
René asked me to write down some of the complaints I had about hipl. Since Miika is directly listening here, I figured I could send them to 
this list where they are archived and can be noticed and discussed.
It's quite a rant, so hang tight and don't take it too seriously, it's 
much more fun to write rants than dry issue listings :)
[All the numbers I took from the midauth2 branch.]
So here goes:
** still using tla (OK, Miika is working on this)
tla is a PITA and a resource hog. Knowing tla is a dead skill and the tool is more of a hindrance than a help IMO. Compared to what could 
be achieved with the alternatives, this is a net drain on developer
productivity.
I am not confident on the tla branch imports done by git. I'd suggest that we'd switch to git for the tinyhip stuff because it's going to diverge from HIPL quite radically. This could be done after merging userspace (that already contains midauth stuff thanks to Rene) to the tinyhip branch. What do folks think about that?
I already created a git project to "repo.or.cz" called hipl.git. Would that make a nice new home for tinyhip? 
We could also set up a repository here. Any reason not to do so?
we could also host it as well, but my suggestion was to make it look even more like an open source software project and to appeal developers outside of the research circles. I am not also sure if we all are going to do this stuff for 20 years to come, so at some point somebody, if not us, has to move the software elsewhere.
Besides this, the suggested repository avoids a number of annoying maintenance things like: 

* maintaining a unix machine
* securing and upgrading it
* setting and maintaining a web interface
* adding unix accounts and making sure that only git is allowed
* making back ups
The site supports these already in an automatized fashion. It is used by a number of other projects which increases at least my confidence for the site. 

(Note: hosting web pages should still be done somewhere else)
In my opinion, we should first get the core and our required extensions working before making tiny publicly available. I don't see the point in getting patches for additional functionality and bugfixes from an eventually existing community, while we're still redesigning the whole thing and shifting around/deleting/modifying a substantial part of the code. Once the basic implementation is done, I agree, that that having an open source project can be beneficial. 


** most files are executable - WTF ??
Source files should not be set executable.  Neither should anything
except, surprise, executables. This is incredibly lame, it looks as 
if it had been developed on a FAT32 drive from which proper Unix
semantics could not be derived. Sheesh, this is not some lame Windows 
project..


This is the umask syndrome...

This is simple to fix and should be fixed in HIPL ASAP.


We need to stick to the same umask setting then.


** massive -I flag abuse
The includes throughout hipl are handled in a terrible way. Instead of using proper relative paths in the headers themselves, the compiler 
flags are bloated with literally dozens of -I flags.  This is ugly
enough by itself, but as a bonus, if you want to use a hipl header
outside of hipl, you have to recreate this mess...


I hope this can be solved with tinyhip's superior building system :)
Rewriting the autobuild system may be boring and cumbersome but this should definitely be done for HIPL. The current rewrite of the PISA build system (which was a HIPL copy & paste) could be a template for this. The resulting Makefile.am is surprisingly simple and short. I guess this can be done with little effort as soon as we have sorted out the remaining problems in PiSA. 

This would be nice.


** massive CFLAG abuse
The compiler calls are bloated with faaaaaaaaaaaaaaaaaaaaaaaaar too
many -D flags. Such #defines should be in config.h. Only definitions 
that are specific to certain files should be passed as -D flags.


It doesn't really bother my sleep at nights, but I tend to agree.

I agree, too. This should be fixed.



** horrible misuse of recursive automake
My views on auto* are basically unprintable, but hipl takes the misuse 
of the autotools to another extreme.  They are used in a recursive
fashion and there is recursion aplenty. All Makefile.ams combined are more than 3000 lines long, hipl is about a million lines of code. My raw Makefile for MPlayer is 1100 lines for about half a million lines of code, but I list one source/object file per line, so you can count 
it as about 600 lines for the purposes of this comparison.


Issue to be solved in TinyHIP?
It will be solved in TH but it also needs to be solved in HIPL (see above). 


** gazillions of warnings
The number of warnings just drows everything.  No wonder they don't
get fixed, I mean, who could tell that file X now prints 101 instead of 100 warnings? Many of them are harmless (but incredibly lame) like 
"unused this" and "redundant that", but I bet half of them are real
bugs that nobody ever bothered to fix.
Issue to be solved in TinyHIP by enforcing -Wall or something even more scarier?
I guess this would also be essential for HIPL. This is a measure to improve and maintain quality. I guess this could save countless hours of debugging and maintaining buggy code of other people (e.g. students that left the project). It takes some time to get there but Diego and Thomas have proven that it can be done... and it feels good :-) 

See above.




** rampant #ifdeffery
Don't get me started, let's just say they are too many :)


Agree.

This will be tackled in TinyHIP
So much for this rant of mine. You asked for it, you got it. The worst part is that I really did not dig deeply, on the contrary. I shudder to think of the horrors that a detailed investigation will uncover. Also, much of what I mentioned could be fixed very quickly, in minutes, hours, 
days, but apparently nobody could ever be bothered so far...
I could write an essay as a response to this comment and list all the excuses but I'd rather not. Suffice to say that we should have a more strict compilation environment and some sort of automated test system. 
Yep. Don't wait for TinyHIP.


Ok. Then we just need some volunteers for all these issues :)
I should probably point out that some of these changes are less painful to handle with TinyHIP because a) it has less dependencies (libinet6 is the worst) and b) TinyHIP is not yet in wide use, so we can still blow it up :)
My suggestion is to not only fix these things in tiny hipl, but fix them 
in all of hipl.  At least don't spread the ugliness any further...
For me, TinyHIP will be the successor of HIPL and I'd like to concentrate at least my efforts on that rather spreading them on multiple places.
I have no idea if we can still merge changes from HIPL to TinyHIP. If we can, it would make sense to solve quite many of the issues in HIPL now before making the split. 

I have some doubts about merging between TinyHIP and HIPL.
I think the merging consist of cherry picking and a lot of manual conflict resolution. This is why I also suggested to switch to git in TinyHIP as soon as possible.
I'm not sure about which version control system to you, but I want tiny to move to something else than tla as soon as possible. For me the 2 competitors are good old SVN and git as a more modern candidate.
Since this is supposed to be a list of things to improve for tiny hipl, add anything that comes to your mind. If our hearts stay pure, we may 
see a decent hipl in our lifetimes...
Although I agree on your worries on code purity issues for the purposes of portability, the system test environment has more priority to me at the moment. It could be also because of the fact that Gurtov keeps asking me that every second day :) 
Will that system be used in TinyHIP as well?


This is the idea.


Ciao,
Tobi
PS: Call me sentimental but I would hate to see HIPL die.


Call me religious but I see TinyHIP as a reincarnation of HIPL :)
I have also suggested calling it HIPLv2 to claim its predecessors fame and fortune, but this is your decision. The "L" would stand for "Lite" and hopefully not "Lame" :)
Well, you're right that HIPL is known throughout the HIP community. Still, it's rather known as a research product and that's not exactly what we are aiming for. Our mission is to provide a small and stable implementation, yet pluggable. Especially concerning the co-existance of HIPL and tinyHIP, I guess HIPLv2 will rather harm that. Finally, rewording the L in HIPL won't really stick to people's mind, I think, and we really wanna be portable with tiny as well. 

These are my thoughts on the matter :-)
René


---
Dipl.-Inform. René Hummen, Ph.D. Student
Distributed Systems Group
RWTH Aachen University, Germany
tel: +49 241 80 20772
web: http://ds.rwth-aachen.de/members/hummen

Other related posts: