[liblouis-liblouisxml] Re: [liblouis] r642 committed - Added version2 of runHarness, based on nose tests....

  • From: Michael Whapples <mwhapples@xxxxxxx>
  • To: liblouis-liblouisxml@xxxxxxxxxxxxx
  • Date: Tue, 29 May 2012 12:54:59 +0100

Hello,
I have been watching this thread, however as i was away it wasn't so convenient 
to reply. Having seen what has been written, some of the features of using nose 
for tests looks good, however the dependency part is something I ideally would 
have liked to avoid.

As for extending the JSON test files for other cases, I have some ideas. It 
mainly revolves around an idea that a "type" field could be given to specify 
what type of test is to be performed (if type is not given then the standard 
translate test is assumed), and then one gives relevant fields for that test 
type, some fields may be required others might be optional. This should mean 
that all test types would be fairly compact.

I still see advantages to the doctest system as it provides examples and can 
run any type of code, however it seems harder to support python2 and python3 in 
a single file for them.

Michael Whapples
On 23 May 2012, at 20:03, Mesar Hameed wrote:

> On Wed 23/05/12,15:06, Christian Egli wrote:
>> Mesar Hameed <mesar.hameed@xxxxxxxxx> writes:
>>> Nose is more flexible than unittests, and the major advantage over
>>> unittests is that it allows for test generators
>> 
>> In what way would test generators helps?
> 
> unittests mean that each test has to be written in python, much like doctests 
> but more closely integrated with the actual project code, i.e. 
> each test is a function.
> 
> nose allows for test generators, which mean that these tests can be created 
> programmatically instead of human written, i.e. data driven, 
> which is suitable for our situation.
> Each test can then be submitted to the infrastructure and delt with as 
> normal, with minimal overhead.
> 
>>> Both with v1 and v2, json is being used to provide the data, but with
>>> v2 we dont have to do any counting, and it takes care of reporting.
>> 
>> That sounds very good. I generally like the direction in which the
>> harness is going (use json as input format, base on existing test
>> framework, etc).
> 
> Yes, I had your consern about frameworks in mind when investigating 
> alternatives to runHarness v1.
> Moving to json will also remove much of the p2/p3 issues regarding unicode, 
> since the json parser will give us the write type depending on version.
> 
>> We probably need to consider the general use case. Most
>> of the time you just want to compare the translation of liblouis with a
>> known good translation, i.e. string comparison.
> 
> Agreed, this was and continues to be easy, its driven by the data defined for 
> each test in the json file.
> 
>> There are some other more obscure scenarios where you want to test cursor 
>> positions, inpos
>> arrays or back translations. Maybe the test framework should make the
>> main use case easy and concise and make the others possible, i.e. make
>> it easy to add just translation tests and have a (less concise)
>> possibility to test the other stuff.
> 
> Yes, I will work on this, v2 makes this very straightforward, with minimal 
> amount of new code.
> 
> Thanks.
> Mesar
> For a description of the software, to download it and links to
> project pages go to http://www.abilitiessoft.com

For a description of the software, to download it and links to
project pages go to http://www.abilitiessoft.com

Other related posts:

  1. Home
  2. liblouis-liblouisxml
  3. Archive
  4. 05-2012