[openbeos] Re: Quality Assurance
- From: Niels Reedijk <niels.reedijk@xxxxxxxxx>
- To: openbeos@xxxxxxxxxxxxx
- Date: Fri, 15 Apr 2005 10:18:35 +0200
On 4/14/05, Axel Dörfler <axeld@xxxxxxxxxxxxxxxx> wrote:
> Niels Reedijk <niels.reedijk@xxxxxxxxx> wrote:
> > I've always been intrigued by quality assurance and the lack of it in
> > Open Source Software. At the current rate of developments we are
> > propelling towards the first generation of alpha and beta builds, and
> > man, will these get attention. The downside of this, is that the
> > project isn't ready to handle that yet. I've made several
> > observations
> > on the issue, and I've posted these on my web-log
> > (http://nielx.blogspot.com)
>
> While I agree with you that QA is a critical point, I don't see how we
> could realize a system you are proposing - if we had QA people that
> qualified, they would probably be coding themselves.
I don't know if this is entirely true. There's a difference between
testers and QA people. Let's face it: testers have the most boring job
in the world. Repeatedly breaking things can be very boring, and as
Joel (on software) once said, they will probably come and go. That's
why I'm making a difference between testing and a Quality Assurance
person. The latter distills bug reports, provides user-developer
communication and make sure builds will roll out as good as possible.
While this job isn't as steady as doing development, it does provide a
lower entry point (you get to know the code and the team), and it
might even promote new developers entering the project.
At the other hand, don't be surprised that I'll rather do the QA than
being a developer. I lack the experience and talent for coding. I'm
much more a manager anyway - and a QA job is managerial in many ways -
which is much closer to my nature. So I guess there will always be
people who feel their talent is much more toward the QA side of things
(just as there are doc writers, translators, web site maintainers). I
think it's a common pitfall to think that developing is the ultimate
position in Open Source Software. I do recognise the fact that at the
moment that is the way, but that's just because there hasn't been a
proper way of doing QA yet (that wins the respect from developers).
Finally I'd like to point out that translators are a very strong force
within the KDE project (so if a group is recognised, they do feel good
about their job).
> We should definitely have a QA team, though, and it should start
> setting up those tinderboxes, as well as making sure our unit tests
> work as expected. If anyone is interested in setting something like
> this up for us, be welcomed.
I've been looking at Mozilla's Tinderbox a few times. Man, that's too
ugly for me to handle. I'll volunteer for writing a cleaner
replacement. I'll host it on my server for the time being, but I need
someone with a machine that can do the building.
> And while alpha/beta releases might not always be a good replacement
> for inhouse testing, it's pretty important to us, as we probably can't
> buy all the hardware that is out there (and even test them in all
> possible configurations). That's a special attribute of OS and driver
> development, though.
Well, naturally that's true. I don't imagine we can catch every bug in
smaller testing releases, but it would filter out the common pitfalls.
What if Haiku R1-alpha1 has a bug that crashes Tracker as soon as you
move more than two files to another disk. This sort of things are bugs
that will be very likely to be noticed, since this is a young OSS
project, people will actually file bug reports, and boom, you've got
30 duplicates. These issues can be controlled a little more by doing
in-house testing beforehand.
> What I could imagine, though, is something you also mentioned in your
> text more or less: that a QA team will try to reproduce the bug, or
> resolve any issues in a bug report. Then they could either acknowledge
> the bug, or remove it. The developers would only get to the
> acknowledged and complete bug reports, so that they can start fix it
> right away. That would lower the requirements for the QA team quite a
> lot, and would still help the development process as well.
Okay, so we do have a common vision here. To be honest, it might be
wise to start assembling one or two people who try to set up the
procedures. But the trick is to find a balance between 'making rules'
and how it works in practise. I'm also not sure about how and when to
start. I guess it might be a good idea for a few people to come
together and work things out (for example, how can the UI of bugzilla
be improved, what will be the exact procedures, etc.) These are all
issues that are already playing (read: how often bug reporting tools
are discussed, etc.), but it's not something I can decide (I don't
have the power, nor the exact knowledge).
I'm hoping for a constructive discussion.
Niels
Other related posts:
