FreeLists / openbeos / [openbeos] Re: testing enhancement

["Ingo Weinhold" <bonefish@xxxxxxxxxxxxxxx>]

> right now, after the build finishes, since the OS isn't ready to be 
> used by itself, most testing is done using the unittester..
> when it comes to performance comparing we have the time it took.
> but this doesn't really show us if it's faster/slower than the 
> equivalent R5 implementation, or from recent implementations of the 
code.
> thus, i suggest the "score" would be kept somehow.
> a possible implementation would be a directory called speed
> it would have R5.txt obos[date].txt and obos[now-date].txt
> 
> then, when the unit tester measures the performance it can also 
> show if we're faster/slower.
> since it's command line, we can also dedicate a machine to doing a 
> test every X minutes and notify if there has been a slowdown.
> so the one in charge would be notified and think of an optimization.

While that is probably not too bad an idea in principle, I feel that it 
is still a bit early for general performance testing. Especially in our 
libopenbeos we have a couple of work-arounds that make objective 
measurement virtually impossible. I think, most of the stuff can't be 
thouroughly performance-tested before it runs on top of our own kernel. 
And for some classes, like BString and BList, tests have already been 
done.

CU, Ingo