Hi John, Please find attached patch that includes the updated harness. points of discussion: 1. should the expected braille be unicode braille chars, or is the ascii-braille ok? 2. if ascii then is "liblouis ascii-braille like encoding" the correct term? from the updated documentation: "The dots should be encoded in the liblouis ascii-braille like encoding." 3. The third test case in en-gb-g2_harness.py has been corrected, will submit more test cases as time permits. Thank you for your review. Mesar On Sun 26/02/12,06:04, John J. Boyer wrote: > Hi Mesar, > > Thanks for updating the documentation. That is a big problem right now. > > Your improvements to the harness are great. > > I think a subdirectory of tests would be best for the harness. > > John > > On Sun, Feb 26, 2012 at 10:44:19AM +0000, Mesar Hameed wrote: > > Hi John, > > On Sun 26/02/12,04:17, John J. Boyer wrote: > > > Hi Mesar, > > > > > > I've looked at your test harness. It looks good to me. > > > > Thank you for the review. > > > > I have improved the harness to include my area of intrest, which is cursor > > positions and translation modes, which should help > > us debug braille cursor problems which is what we often see in nvda and > > orca. > > I am just updating the documentation to include the test harness info, as > > soon as that is done I will send the changes > > to the list. > > > > > We already have a > > > tests directory. The tests there check tables for validity. Your code > > > checks for accuracy of transltion. Could the test harness be placed in > > > the tests directory? > > > > Yes sure, do you want it directly in tests, or in tests/harness > > I ask this because we will potentially have many harness files, one for > > each table. > > > > Thanks. > > Mesar > > For a description of the software, to download it and links to > > project pages go to http://www.abilitiessoft.com > > -- > John J. Boyer; President, Chief Software Developer > Abilitiessoft, Inc. > http://www.abilitiessoft.com > Madison, Wisconsin USA > Developing software for people with disabilities > > For a description of the software, to download it and links to > project pages go to http://www.abilitiessoft.com
Index: tests/harness/runHarness.py =================================================================== --- tests/harness/runHarness.py (revision 0) +++ tests/harness/runHarness.py (revision 0) @@ -0,0 +1,105 @@ +#!/usr/bin/env python +# -*- coding: UTF-8 -*- +# Liblouis test harness +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Library General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Library General Public License for more details. +# +# You should have received a copy of the GNU Library General Public +# License along with this library; if not, write to the +# Free Software Foundation, Inc., Franklin Street, Fifth Floor, +# Boston MA 02110-1301 USA. +# +# Copyright (c) 2012, liblouis team, Mesar Hameed. + +"""Liblouis test harness: +Please see the liblouis documentation for information of how to add a new harness or more tests for your braille table. + +@author: Mesar Hameed <mhameed@xxxxxxxxxxxxx> +""" + +import sys +import louis +from glob import iglob + +def showCurPos(length, pos1, marker1="^", pos2=None, marker2="*"): + """A helper function to make a string to show the position of the given cursor.""" + display = [" "] *length + display[pos1] = marker1 + if pos2: display[pos2] = marker2 + return "".join(display) + +def reportFailure(text, actualBRL, expectedBRL, cursorPos, actualBRLCursorPos, expectedBRLCursorPos): + """Function to layout and print a failure report. + + Works out the where the missmatch is occuring and presents the necessary information, with markers. + """ + + template = "%-25s '%s'" + + report = [template % ("text:", text), + template %("CursorAt: %d" %cursorPos, showCurPos(len(text), cursorPos) )] + if actualBRL != expectedBRL and actualBRLCursorPos != expectedBRLCursorPos: + report.insert(0,"--- Braille and cursor Difference Failure: ---") + report.extend([ + template % ("expected brl:", expectedBRL), + template %("expectedCursorAt: %d" %expectedBRLCursorPos, showCurPos(len(expectedBRL), expectedBRLCursorPos) ), + + template % ("actual brl:", actualBRL), + template %("actualCursorAt: %d" %actualBRLCursorPos, showCurPos(len(actualBRL), actualBRLCursorPos) ), + ]) + elif actualBRL != expectedBRL: + report.insert(0,"--- Braille Difference Failure: ---") + report.extend([ + template % ("expected brl:", expectedBRL), + template % ("actual brl:", actualBRL), + template %("brlCursorAt: %d" %actualBRLCursorPos, showCurPos(len(actualBRL), actualBRLCursorPos) ), + ]) + else: + report.insert(0, "--- Braille Cursor Difference Failure: ---") + report.extend([ + template % ("received brl:", actualBRL), + template % ("BRLCursorAt %d expected %d:" % (actualBRLCursorPos, expectedBRLCursorPos), + showCurPos(len(actualBRL), actualBRLCursorPos, pos2=expectedBRLCursorPos)) + ]) + report.append("--- end ---") + print "\n".join(report) + +# relative path to tables directory +tablePath = '../../tables/' + +# Process all *_harness.py files in the current directory. +for harness in iglob('*_harness.py'): + try: + harnessModule = __import__(harness[:-3]) + except Exception, e: + # Doesn't look like the harness is a valid python file. + print "Warning: could not import %s" %harness + print e + continue + table = tablePath+ harnessModule.table + if not iglob(table): + print "WARNING: unable to locate %s, please check capitalization and extension." %table + continue + print "Processing %s" %harness + failed = 0 + tableList = [table] + for test in harnessModule.tests: + text = test['txt'] + mode = test.get('mode', 0) + cursorPos = test.get('cursorPos', 0) + expectedBRLCursorPos = test.get('BRLCursorPos', 0) + expectedBRL = test['brl'] + + actualBRL, BRL2rawPos, raw2BRLPos, actualBRLCursorPos = louis.translate(tableList, text, mode=mode, cursorPos=cursorPos, typeform=None) + if actualBRL != expectedBRL or actualBRLCursorPos != expectedBRLCursorPos: + failed += 1 + reportFailure(text, actualBRL, expectedBRL, cursorPos, actualBRLCursorPos, expectedBRLCursorPos) + print "%d of %d tests failed." %(failed, len(harnessModule.tests)) Property changes on: tests/harness/runHarness.py ___________________________________________________________________ Added: svn:executable + * Index: tests/harness/en-GB-g2_harness.py =================================================================== --- tests/harness/en-GB-g2_harness.py (revision 0) +++ tests/harness/en-GB-g2_harness.py (revision 0) @@ -0,0 +1,29 @@ +# -*- coding: UTF-8 -*- + +""" +Liblouis test harness for the U.K. English Grade 2 Braille Contraction Table + +Please see the liblouis documentationfor more information. +""" + +import louis + +table = 'en-GB-g2.ctb' + +tests = [ + { # check that "the" is correctly contracted + 'txt': u'the cat sat on the mat', + 'brl': u'! cat sat on ! mat' + }, + { # Checking "to" is contracted correctly and joined to next word. + 'txt': u'to the moon', + 'brl': u'6! moon' + }, + { # Check that "to" at end of line doesnt get contracted, and that "went" is expanded when cursor is positioned within the word. + 'txt': u'you went to', + 'mode': louis.compbrlAtCursor, + 'cursorPos': 4, + 'brl': u'y went to', + 'BRLCursorPos': 2, + } +] Index: doc/liblouis.texi =================================================================== --- doc/liblouis.texi (revision 537) +++ doc/liblouis.texi (working copy) @@ -94,6 +94,7 @@ * Introduction:: * Test Programs:: * How to Write Translation Tables:: +* How to Write a Translation Table Test Harness:: * Notes on Back-Translation:: * Programming with liblouis:: * Opcode Index:: @@ -417,7 +418,7 @@ You will see a few lines telling you how to use the program. -@node How to Write Translation Tables, Notes on Back-Translation, Test Programs, Top +@node How to Write Translation Tables, How to Write a Translation Table Test Harness, Test Programs, Top @chapter How to Write Translation Tables Many translation (contraction) tables have already been made up. They @@ -1734,7 +1735,41 @@ @deprecatedopcode{literal, characters, compbrl} @end table -@node Notes on Back-Translation, Programming with liblouis, How to Write Translation Tables, Top + +@node How to Write a Translation Table Test Harness, Notes on Back-Translation, How to Write Translation Tables, Top +@chapter How to Write a Translation Table Test Harness + +The translation table test harness a set of tests to ensure the correctness of a given table. +Having a harness ensures that the table is correct for all the listed tests, and if any fail, a report will be produced +showing the error and the actual and expected outputs. +To ensure complete coverage, you should have at least one test for each contraction rule. + +Each harness file is a simple utf8 encoded python file, which has two entries. +@code{table} The name of the table which the tests should be run against. +@code{tests} The list of tests that should be run. + +Each test case has the following entries: + +@table @code +@code{txt} +The unicode text to be tested (required). +@code{cursorPos} +The position of the cursor within the given text (optional). +Useful when simulating screenreader interaction, to debug contraction and cursor behaviour. +@code{brl} +The expected braille output (required). +The dots should be encoded in the liblouis ascii-braille like encoding. +@code{BRLCursorPos} +The expected position of the braille cursor in the braille output (optional). +Useful when simulating screenreader interaction, to debug contraction and cursor behaviour. +@code{mode} +The liblouis translation mode that should be used for this test (optional). +If not defined defaults to 0. +@end table + +For a full example please see @code{en-GB-g2_harness.py} in the harness directory. + +@node Notes on Back-Translation, Programming with liblouis, How to Write a Translation Table Test Harness, Top @chapter Notes on Back-Translation Back-translation is carried out by the function