[brailleblaster] Re: utdml2brl.temp
- From: Keith Creasy <kcreasy@xxxxxxx>
- To: "brailleblaster@xxxxxxxxxxxxx" <brailleblaster@xxxxxxxxxxxxx>
- Date: Mon, 14 May 2012 09:24:52 -0400
It is in C++ and no it isn't open source so I can't share it. I can share some
of the techniques I think, have to make sure it's OK though.
It isn't complicated however. It boils down to using C++, making sure
everything is encapsulated in the object classes, and being sure that every
thread that uses the API creates it's own instance of the object. You also have
to be sure every instance gets destroyed. A braille translater doesn't need to
have any mutable data that is shared between threads so some of the more
complex issues are not involved.
Here is some almost-pseudocode that illustrates:
The "void*" returned here becomes the handle to the translator object. One
advantage to this is that the code that uses the library can be in any language
since it doesn't have to know anything about the translator object. It just
uses the handle, which is basically an int.
void* createTranslator() {
void* handle = new Translator();
return handle;
}
void* destroyTranslator( void* handle) {
if( handle!=null)
delete handle;
handle=null;
return handle; // Return null for success.
}
const char* translate( void* handle, const char* text) {
translator* t = (translator*)handle;
if( t!=null)
return t->translate( text);
}
Obviously this is very simplified code but you probably get the idea.
Keith
From: brailleblaster-bounce@xxxxxxxxxxxxx
[mailto:brailleblaster-bounce@xxxxxxxxxxxxx] On Behalf Of Alex Jurgensen
Sent: Monday, May 14, 2012 8:58 AM
To: brailleblaster@xxxxxxxxxxxxx
Subject: [brailleblaster] Re: utdml2brl.temp
Hi Keith,
I'd like to have a look at your translator for some inspiration.
Your translator isn't open-source, is it?
What is it written in?
Regards,
Alex,
Alex Jurgensen,
Community Coordinator,
The Camp Bowen Society for the Visually Impaired,
Explore Camp Bowen online at: http://www.campbowen.ca/
Phone: 778-908-0521
E-mail: ASquared21@xxxxxxxxx<mailto:ASquared21@xxxxxxxxx>
On 2012-05-14, at 5:39 AM, Keith Creasy wrote:
Hi, just picked up on this thread.
My advice is to take the time to make the library thread-safe. I can't point to
any examples but do know some techniques that I used on our own braille
translator, which is thread safe and quite robust. Basically it boils down to
avoiding any global variables in the API and creating the API such that it just
creates an instance of the library object; i.e. a translator, and returns it as
a new object. Everything is encapsulated. The API is then used to destroy the
object on completion. Objects that do the work are not shared between processes
or threads.
A unique ID can be used to create directories for temp files if necessary.
It isn't really hard or complicated but converting a library that is not
thread-safe can obviously be error prone and should be introduced with care.
Keith
-----Original Message-----
From:
brailleblaster-bounce@xxxxxxxxxxxxx<mailto:brailleblaster-bounce@xxxxxxxxxxxxx>
[mailto:brailleblaster-bounce@xxxxxxxxxxxxx] On Behalf Of John J. Boyer
Sent: Monday, May 14, 2012 8:15 AM
To: brailleblaster@xxxxxxxxxxxxx<mailto:brailleblaster@xxxxxxxxxxxxx>
Subject: [brailleblaster] Re: utdml2brl.temp
What is involved in making a library thread-safe? Is there a tutorial with
examples?
Thanks,
John
On Mon, May 14, 2012 at 12:12:52PM +0100, Michael Whapples wrote:
Hello,
Yes I agree with what you say about liblouisutdml not being thread
safe and so the only real answer at the moment is to use it from
synchronized blocks.
If you don't intend to make it thread safe, then may be it would be
worth making a note somewhere in the documentation explaining it is
not thread safe and there are currently no plans to make it thread safe.
As for what you suggest about using the file system of each machine in
the cluster, this would not always work:
* What about multi-core processors, there is only one file system in
that case and so the available processing power is under utilised.
* I don't know how the various cluster systems work, but I thought
they are trying to work towards making the fact its running on a
cluster more transparent and so there may be cases or the time might
come where one could not do such a configuration as you describe.
Whether its worth your time, I don't know, but as time goes on
multi-threaded environments will become more and more improtant and so
a thread safe option may be needed, is there one?
Michael Whapples
On 14/05/2012 11:44, John J. Boyer wrote:
I think the synchronized block is the only way to assure correct
performance. There are lots of shared variables. Making liblouisutdml
and liblouis thread-save would be a big unndertaking and there are
better ways to spend my time. Another solution would be to give each
machine in the cluster its own directories. The synchronized block
should not then exact much of a performance penalty.
John
On Mon, May 14, 2012 at 10:44:20AM +0100, Michael Whapples wrote:
While synchronized blocks may prevent the issue showing, it might
not be the real answer as it can lead to poor performance. In this
case this probably is true.
The reason is that synchronized means that other threads have to
wait for the current thread to finish the synchronized block. In
this case as there are multiple machines and really separate
translations should be, well, separate, there should not really be a
need to have to do translations sequentially. Sequential translation
is not the most efficient/intended way to use a cluster environment.
The original suggestion of having separate temp files for each
translation is the correct and fastest way for this to work in a
cluster environment. May be having it based on the file name is one
solution, another would be to allow the calling application to set
the temp file name for the current translation.
Having said the above, even if the file name thing was seen to,
there could very well be other bugs with using it in a
multi-threaded environment (eg. there may be shared variables which
multiple translations both may try to use, etc).
So may be John's suggestion of using synchronized is the only way to
do this in the immediate future, significant work could be needed to
make liblouisutdml thread safe and so may simply not be the right
choice for such a use.
Michael Whapples
On 14/05/2012 03:35, John J. Boyer wrote:
Hi Alex,
If I remember right, you should do something like this
synchronized {
file2brl (arguments);
}
or you could call translateFile within the block.
John
On Sun, May 13, 2012 at 06:46:36PM -0700, Alex Jurgensen wrote:
Hi John,
My translation system consists of aprox 100 Thousand concurrent
translations over a small cluster of machines. Do you have any
advice for how I should implement the LibLouisUTDML interface in
Java?
Regards,
Alex,
Alex Jurgensen,
Community Coordinator,
The Camp Bowen Society for the Visually Impaired,
Explore Camp Bowen online at: http://www.campbowen.ca/
Phone: 778-908-0521
E-mail: ASquared21@xxxxxxxxx<mailto:ASquared21@xxxxxxxxx>
On 2012-05-13, at 6:37 PM, John J. Boyer wrote:
Hi Alex,
If you are using Java it would be best to use a synchronized
block. If a translation is started while another is in progress
they will interfere with each other, even if you use a method
that does not generate file2brl.temp. Each translation takes only
a few seconds, so your users should not notice much delay.
John
On Sat, May 12, 2012 at 06:13:50PM -0700, Alex Jurgensen wrote:
Hi John,
To add to my previous question, I was wondering if such a
feature could work as follows:
/usr/bin/file2brl --tempfile /tmp/myfile.temp
/home/Alex/myfile.xml /home/Alex/myfile.brl
Regards,
Alex,
Alex Jurgensen,
Community Coordinator,
The Camp Bowen Society for the Visually Impaired,
Explore Camp Bowen online at: http://www.campbowen.ca/
Phone: 778-908-0521
E-mail: ASquared21@xxxxxxxxx<mailto:ASquared21@xxxxxxxxx>
On 2012-05-12, at 5:23 PM, John J. Boyer wrote:
Hi Alex,
I assume you are referring to file2brl.temp This file is
overwritten each time the file2brl command-line program or the
file2brl method in the Java bindings is run. It is not intended
for any permanent use, but may be useful for debuging. What are
your reasons for wanting a file name that reflects the source
file?
Thanks,
John
On Sat, May 12, 2012 at 01:06:56PM -0700, Alex Jurgensen wrote:
Hi John,
Is it possible to have LibLouisUTDML create a temporary file
that is not named utdml2brl.temp, but rather, uses the name of
the source document one is translating? For example,
"file1.temp" or "file2.temp"
Regards,
Alex,
Alex Jurgensen,
Software Engineer,
ASInnovations,
"We might be blind, but we have a vision."
Phone: 778-908-0521
E-mail: VoiceOverTrainer@xxxxxx<mailto:VoiceOverTrainer@xxxxxx>
--
John J. Boyer; President, Chief Software Developer
Abilitiessoft, Inc.
http://www.abilitiessoft.com
Madison, Wisconsin USA
Developing software for people with disabilities
--
John J. Boyer; President, Chief Software Developer Abilitiessoft,
Inc.
http://www.abilitiessoft.com
Madison, Wisconsin USA
Developing software for people with disabilities
--
John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc.
http://www.abilitiessoft.com
Madison, Wisconsin USA
Developing software for people with disabilities
Other related posts:
