[liblouis-liblouisxml] Re: Performance Question
- From: Ken Perry <kperry@xxxxxxx>
- To: "liblouis-liblouisxml@xxxxxxxxxxxxx" <liblouis-liblouisxml@xxxxxxxxxxxxx>
- Date: Sat, 26 Oct 2013 18:05:13 +0000
One thing I would like to do if we move to a new library and I had told John
this in an off list mail is to create a library that is the controlling library
and have it be drop in replaceable for liblouis. The reason as has already
been stated is liblouis is used on Chrome books, nvda, bp18, braille pblaster
and a whole list of other things. I would like to be able to drop the new
library in and if we had new routines and functionality allow people to use
those but if for example there were only Korean tables written in liblouis it
would default to what we have now. If that is not clear enough imagine a
liblouis that has a couple scripting languages embedded with the same functions
as liblouis for legacy support and all those functions do is call liblouis now
if the braille code being requested was not already re-written. In this way
we could have a stub starting library in a few weeks and slowly as new
functions and braille translators were written it could replace liblouis fully.
In that way when liblouis finally was removed no one would know the new
liblouis would just replace it.
Ken
-----Original Message-----
From: liblouis-liblouisxml-bounce@xxxxxxxxxxxxx
[mailto:liblouis-liblouisxml-bounce@xxxxxxxxxxxxx] On Behalf Of John Gardner
Sent: Saturday, October 26, 2013 12:47 PM
To: liblouis-liblouisxml@xxxxxxxxxxxxx
Subject: [liblouis-liblouisxml] Re: Performance Question
Hello all, thanks for the great feedback. Let me repeat and emphasize some
points from my original proposal. One is that we use a well-known
well-supported scripting language so that we can be reasonably confident that
we won't have to use our time debugging our engine. I absolutely disagree with
John Boyer about writing our own scripting language. That is a step backward.
We must make our fundamental scripts accessible to mere mortals if we really
want liblouis braille translation to be excellent for all languages and to stay
that way. I am a physicist and am convinced that simple is always better.
I am concerned that we are setting up "performance" as too much of a holy
grail. We need performance on a human time scale, and humans are not getting
faster. Computers are. So even if the new system was somewhat slower than the
current liblouis, it wouldn't be slower in a couple of years. Instead of
rejecting suggestions because we fear that performance will be too bad, we
should set up a benchmark and insist that the new system meet that benchmark.
I would suggest a benchmark based on the present speed times some factor. My
question of you is whether a factor of ten slower would adversely affect any
present application? A factor of 2? Mesar reports a speed of roughly 20
microseconds per word now. His test was 5 seconds on 241k words. That's about
a 250 page book. Would you object to taking 50 seconds to translate that book
if you had confidence that the translation was really right? And in five years
the time would probably shrink to 5 seconds again. Jamie, is 200 microseconds
per word gonna slow down NVDA?
Of the comments made so far, I am most concerned by Jamie's comment about
conflicting Python run-time engines. It seems clear to me now that if we do
use Python, it would need to be compiled before final use, although the
run-time environment would be used for de-bugging. I do not know Cython, so
maybe my understanding is just wrong that it will compile into a C module that
can then be run independently of Cython. Are there Cython experts available
for advice.
I do understand that algorithm design is important. In my simple mind, this is
just a different way to saying that we need to build fast compiled subroutines
to do the common tasks of translation. However, if we adopt Susan's long-time
suggestion of translating most words from big look-up tables, the time spent on
other translations would probably not be important. I claim that performance
is less critical for formatting (ie liblouisutdml). Am I wrong?
Finally I do strongly agree that we need to consider other scripting languages.
What are the advantage of Lua? Are there other candidates? My personal
acceptahnce critieria would be that the final format work on all OS, be
well-established and likely to have legs into the future and that it be as
humanly readable as possible.
Thank you all. This is a fruitful discussion, and I hope we keep it that way.
John G
-----Original Message-----
From: liblouis-liblouisxml-bounce@xxxxxxxxxxxxx
[mailto:liblouis-liblouisxml-bounce@xxxxxxxxxxxxx] On Behalf Of Mesar Hameed
Sent: Friday, October 25, 2013 11:27 PM
To: liblouis-liblouisxml@xxxxxxxxxxxxx
Subject: [liblouis-liblouisxml] Re: Performance Question
Hi Ken,
I don't have the performance issues that you describe, but maybe thats because
your test is written in bash.
My python scripts that I refered to earlier, take 5 seconds to process a
wordlist of 241 thousand, which seems more realistic.
The reason for the slowdown in your script is because you are using a separate
process call for each word being translated.
Hope that helps.
Mesar
On Sat 26/10/13,00:13, Ken Perry wrote:
> Here is what I mean by performance issue. The current liblouis takes
> 15
minutes to convert a 99,107 word file. That is too long. If this parser was
generated by the same tools I have used in the past to write a compiler that
spits out some pretty nasty c++ using grammers and lex files that file could be
done in seconds or at most a minute or two not 15 minutes. Add the definite
slow down when you use scripting languages even those which are jit compiled
you will start seeing much worse than 15 minutes.
>
> Ken
>
> -----Original Message-----
> From: liblouis-liblouisxml-bounce@xxxxxxxxxxxxx
> [mailto:liblouis-liblouisxml-bounce@xxxxxxxxxxxxx] On Behalf Of Susan
> Jolly
> Sent: Friday, October 25, 2013 6:04 PM
> To: liblouis-liblouisxml@xxxxxxxxxxxxx
> Subject: [liblouis-liblouisxml] Performance Question
>
> Hi everyone,
>
> I've been lurking on this list for a long time but have now joined
> because
I believe that what I've learned from my practical experience in developing
braille software over the last dozen years could be helpful as far as making
plans for the future.
>
> However I will need some background. My first question is why you
> have a
concern about performance issues. I've always written my braille software
entirely in Java and even on my prevous PC running Windows XP I could translate
entire books of fiction to English contracted braille in only a few seconds.
>
> Susan Jolly
>
> For a description of the software, to download it and links to project
> pages go to http://www.abilitiessoft.com For a description of the
> software, to download it and links to project pages go to
> http://www.abilitiessoft.com
For a description of the software, to download it and links to project pages go
to http://www.abilitiessoft.com
For a description of the software, to download it and links to
project pages go to http://www.abilitiessoft.com
Other related posts:
