[haiku-development] Re: Rant about quality and testing

  • From: Stephan Assmus <superstippi@xxxxxx>
  • To: haiku-development@xxxxxxxxxxxxx
  • Date: Sun, 27 Feb 2011 15:49:59 +0100

Hi,

Am 27.02.2011 15:12, schrieb Oliver Tappe:

from my POV, this rant is quite welcome ;-)

I basically spent all day yesterday trying to replace the hardcoded paths
used by existing code with corresponding find_directory() invocations.
Irritatingly, I found several hardcoded paths in pretty new code, too.
I feel your pain, though, in this case the code even "worked" -- before you needed it to work in a new situation. ;-) 


On 2011-02-27 at 14:23:04 [+0100], Stephan Assmus<superstippi@xxxxxx>  wrote:



[ ... ]

Today it went something like this: I wanted to look at a problem in
AboutSystem with the building of the translated Haiku license string.
Basically, the translated name for the license is inserted broken. Now,
this means someone didn't test their changes. Either it was a
contributor of a patch. Then the lack of testing began there, and
continued at the committer of the patch who didn't test it properly
either and just trusted the contributor. When I wanted to log the bug
from WebPositive in Haiku, I could not log into Trac. "Protocol not
supported". HTTPS is not supported in WebPositive anymore, I suspect
because of the very recent update of libcurl. Again, something was
replaced that worked perfectly fine before, without the necessary
testing. WebPositive is a tool that I suspect many are using when they
use Haiku. So extra care has to be taken when changing it or components
that it depends on. Especially so when you make a commit for no obvious
benefit. Then I wanted to look at the AboutSystem problem myself, but
now Pe does not open, because there is a problem with a missing libpcre.
Again, some package was changed without proper testing prior to the
commit. That just did for today and now I am writing this mail.


In order to lessen the problems related to packaging, I think we should
maintain at least two different package repositories: haiku and
haiku-testing. Freshly built packages would be added to the haiku-testing
and only after they've passed a (yet to be defined) testing process, will
they be added to haiku-core.
While it doesn't make sense to (more or less) blindly add new packages to
our repository, it's still nice to be following the head of development in
the testing repository in order to see problems of current packages as soon
as possible.
I don't find it so important to be able to test new packages as soon as possible. If something works well, why update it? Who cares if we are using libcurl 0.22.6 or 0.22.7? If someone has the time and feels the motivation to update it, I have nothing against that, but then the new package should be tested properly, by the same person who does the update. Anything else just has way too much overhead. I find it unrealistic that people would give testing packages a spin, if those were available separately. If there are obscure problems with a new package that are not easily encountered, I don't blame anyone for updating that package, but if the problems are pretty quickly found, I do think the commit was premature. It all depends on who spends time on what and it's just not good style to expect or require other people to run into obvious problems and spend the time on fixing them. We have just too little resources overall for this to work well, since the overhead is just too much. Those things will obviously happen from time to time, but if this gets too frequent, the frustration will become too much and outweight the happiness about the overall progress and frequent commits.
Basically, don't trust the tools so much. Don't trust haiku-porter to reproduce packages that work as good as the last versoin, and obviously don't trust the compiler and think it works because it compiles... :-D 

Best regards,
-Stephan

Other related posts:

  1. Home
  2. haiku-development
  3. Archive
  4. 02-2011