[liblouis-liblouisxml] Re: proposal for a test harness
- From: Mesar Hameed <mesar.hameed@xxxxxxxxx>
- To: Christian Egli <christian.egli@xxxxxx>
- Date: Wed, 29 Feb 2012 12:31:06 +0000
Ok, please find attached, liblouis is now imported directly from source.
Thanks.
Mesar
On Wed 29/02/12,12:39, Christian Egli wrote:
> Mesar Hameed <mesar.hameed@xxxxxxxxx> writes:
>
> > Am I correct in saying that for the time being we have all agreed over
> > the improved test harness?
>
> I'm fine with adding a Python based test harness as long as we can make
> sure it runs out of the src dir, i.e. without having to install the
> liblouis Python bindings first.
>
> --
> Christian Egli
> Swiss Library for the Blind, Visually Impaired and Print Disabled
> Grubenstrasse 12, CH-8045 Zürich, Switzerland
>
> -----
> Jetzt kostenlos eidgenoessische und kantonale Abstimmungsunterlagen aus 17
> Kantonen
> zum Hoeren auf CD abonnieren: medienverlag@xxxxxx
Index: tests/harness/runHarness.py
===================================================================
--- tests/harness/runHarness.py (revision 0)
+++ tests/harness/runHarness.py (revision 0)
@@ -0,0 +1,109 @@
+#!/usr/bin/env python
+# -*- coding: UTF-8 -*-
+# Liblouis test harness
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Library General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Library General Public License for more details.
+#
+# You should have received a copy of the GNU Library General Public
+# License along with this library; if not, write to the
+# Free Software Foundation, Inc., Franklin Street, Fifth Floor,
+# Boston MA 02110-1301 USA.
+#
+# Copyright (c) 2012, liblouis team, Mesar Hameed.
+
+"""Liblouis test harness:
+Please see the liblouis documentation for information of how to add a new
harness or more tests for your braille table.
+
+@author: Mesar Hameed <mhameed@xxxxxxxxxxxxx>
+"""
+
+import sys
+
+# Make sure we are using the liblouis python bindings that come with this
package, and not the once
+# that may have been installed on the system.
+sys.path.insert(1, "../../python/")
+from louis import translate
+from glob import iglob
+
+def showCurPos(length, pos1, marker1="^", pos2=None, marker2="*"):
+ """A helper function to make a string to show the position of the given
cursor."""
+ display = [" "] *length
+ display[pos1] = marker1
+ if pos2: display[pos2] = marker2
+ return "".join(display)
+
+def reportFailure(text, actualBRL, expectedBRL, cursorPos, actualBRLCursorPos,
expectedBRLCursorPos):
+ """Function to layout and print a failure report.
+
+ Works out where the missmatch is occuring and presents the necessary
information, with markers.
+ """
+
+ template = "%-25s '%s'"
+
+ report = [template % ("text:", text),
+ template %("CursorAt: %d" %cursorPos, showCurPos(len(text),
cursorPos) )]
+ if actualBRL != expectedBRL and actualBRLCursorPos != expectedBRLCursorPos:
+ report.insert(0,"--- Braille and cursor Difference Failure: ---")
+ report.extend([
+ template % ("expected brl:", expectedBRL),
+ template %("expectedCursorAt: %d" %expectedBRLCursorPos,
showCurPos(len(expectedBRL), expectedBRLCursorPos) ),
+
+ template % ("actual brl:", actualBRL),
+ template %("actualCursorAt: %d" %actualBRLCursorPos,
showCurPos(len(actualBRL), actualBRLCursorPos) ),
+ ])
+ elif actualBRL != expectedBRL:
+ report.insert(0,"--- Braille Difference Failure: ---")
+ report.extend([
+ template % ("expected brl:", expectedBRL),
+ template % ("actual brl:", actualBRL),
+ template %("brlCursorAt: %d" %actualBRLCursorPos,
showCurPos(len(actualBRL), actualBRLCursorPos) ),
+ ])
+ else:
+ report.insert(0, "--- Braille Cursor Difference Failure: ---")
+ report.extend([
+ template % ("received brl:", actualBRL),
+ template % ("BRLCursorAt %d expected %d:" % (actualBRLCursorPos,
expectedBRLCursorPos),
+ showCurPos(len(actualBRL), actualBRLCursorPos,
pos2=expectedBRLCursorPos))
+ ])
+ report.append("--- end ---")
+ print "\n".join(report)
+
+# relative path to tables directory
+tablePath = '../../tables/'
+
+# Process all *_harness.py files in the current directory.
+for harness in iglob('*_harness.py'):
+ try:
+ harnessModule = __import__(harness[:-3])
+ except Exception, e:
+ # Doesn't look like the harness is a valid python file.
+ print "Warning: could not import %s" %harness
+ print e
+ continue
+ table = tablePath+ harnessModule.table
+ if not iglob(table):
+ print "WARNING: unable to locate %s, please check capitalization and
extension." %table
+ continue
+ print "Processing %s" %harness
+ failed = 0
+ tableList = [table]
+ for test in harnessModule.tests:
+ text = test['txt']
+ mode = test.get('mode', 0)
+ cursorPos = test.get('cursorPos', 0)
+ expectedBRLCursorPos = test.get('BRLCursorPos', 0)
+ expectedBRL = test['brl']
+
+ actualBRL, BRL2rawPos, raw2BRLPos, actualBRLCursorPos =
translate(tableList, text, mode=mode, cursorPos=cursorPos, typeform=None)
+ if actualBRL != expectedBRL or actualBRLCursorPos !=
expectedBRLCursorPos:
+ failed += 1
+ reportFailure(text, actualBRL, expectedBRL, cursorPos,
actualBRLCursorPos, expectedBRLCursorPos)
+ print "%d of %d tests failed." %(failed, len(harnessModule.tests))
Property changes on: tests/harness/runHarness.py
___________________________________________________________________
Added: svn:executable
+ *
Index: tests/harness/en-GB-g2_harness.py
===================================================================
--- tests/harness/en-GB-g2_harness.py (revision 0)
+++ tests/harness/en-GB-g2_harness.py (revision 0)
@@ -0,0 +1,33 @@
+# -*- coding: UTF-8 -*-
+
+"""
+Liblouis test harness for the U.K. English Grade 2 Braille Contraction Table
+
+Please see the liblouis documentationfor more information.
+"""
+
+import sys
+# Make sure we are using the liblouis python bindings that come with this
package, and not the once
+# that may have been installed on the system.
+sys.path.insert(1, "../../python/")
+import louis
+
+table = 'en-GB-g2.ctb'
+
+tests = [
+ { # check that "the" is correctly contracted
+ 'txt': u'the cat sat on the mat',
+ 'brl': u'! cat sat on ! mat'
+ },
+ { # Checking "to" is contracted correctly and joined to next word.
+ 'txt': u'to the moon',
+ 'brl': u'6! moon'
+ },
+ { # Check that "to" at end of line doesnt get contracted, and that "went"
is expanded when cursor is positioned within the word.
+ 'txt': u'you went to',
+ 'mode': louis.compbrlAtCursor,
+ 'cursorPos': 4,
+ 'brl': u'y went to',
+ 'BRLCursorPos': 2,
+ }
+]
Index: doc/liblouis.texi
===================================================================
--- doc/liblouis.texi (revision 537)
+++ doc/liblouis.texi (working copy)
@@ -94,6 +94,7 @@
* Introduction::
* Test Programs::
* How to Write Translation Tables::
+* How to Write a Translation Table Test Harness::
* Notes on Back-Translation::
* Programming with liblouis::
* Opcode Index::
@@ -417,7 +418,7 @@
You will see a few lines telling you how to use the program.
-@node How to Write Translation Tables, Notes on Back-Translation, Test
Programs, Top
+@node How to Write Translation Tables, How to Write a Translation Table Test
Harness, Test Programs, Top
@chapter How to Write Translation Tables
Many translation (contraction) tables have already been made up. They
@@ -1734,7 +1735,41 @@
@deprecatedopcode{literal, characters, compbrl}
@end table
-@node Notes on Back-Translation, Programming with liblouis, How to Write
Translation Tables, Top
+
+@node How to Write a Translation Table Test Harness, Notes on
Back-Translation, How to Write Translation Tables, Top
+@chapter How to Write a Translation Table Test Harness
+
+The translation table test harness a set of tests to ensure the correctness of
a given table.
+Having a harness ensures that the table is correct for all the listed tests,
and if any fail, a report will be produced
+showing the error and the actual and expected outputs.
+To ensure complete coverage, you should have at least one test for each
contraction rule.
+
+Each harness file is a simple utf8 encoded python file, which has two entries.
+@code{table} The name of the table which the tests should be run against.
+@code{tests} The list of tests that should be run.
+
+Each test case has the following entries:
+
+@table @code
+@code{txt}
+The unicode text to be tested (required).
+@code{cursorPos}
+The position of the cursor within the given text (optional).
+Useful when simulating screenreader interaction, to debug contraction and
cursor behaviour.
+@code{brl}
+The expected braille output (required).
+The dots should be encoded in the liblouis ascii-braille like encoding.
+@code{BRLCursorPos}
+The expected position of the braille cursor in the braille output (optional).
+Useful when simulating screenreader interaction, to debug contraction and
cursor behaviour.
+@code{mode}
+The liblouis translation mode that should be used for this test (optional).
+If not defined defaults to 0.
+@end table
+
+For a full example please see @code{en-GB-g2_harness.py} in the harness
directory.
+
+@node Notes on Back-Translation, Programming with liblouis, How to Write a
Translation Table Test Harness, Top
@chapter Notes on Back-Translation
Back-translation is carried out by the function
Other related posts:
