[brailleblaster] Re: utdml2brl.temp
- From: "John J. Boyer" <john.boyer@xxxxxxxxxxxxxxxxx>
- To: brailleblaster@xxxxxxxxxxxxx
- Date: Mon, 14 May 2012 08:11:37 -0500
Hi Keith,
The need to make liblouis and liblouisutdml thread-safe simply hasn't
arisen until now. I'm not sure it is needed for BrailleBlaster.
Both libraries have a great lmany static variables. I imagine that these
would have to be put into some sort of state description.
libxml2 claims to be thread-safe, but I haven't seen how they do it.
Thanks,
John
On Mon, May 14, 2012 at 08:39:50AM -0400, Keith Creasy wrote:
> Hi, just picked up on this thread.
>
> My advice is to take the time to make the library thread-safe. I can't point
> to any examples but do know some techniques that I used on our own braille
> translator, which is thread safe and quite robust. Basically it boils down to
> avoiding any global variables in the API and creating the API such that it
> just creates an instance of the library object; i.e. a translator, and
> returns it as a new object. Everything is encapsulated. The API is then used
> to destroy the object on completion. Objects that do the work are not shared
> between processes or threads.
>
> A unique ID can be used to create directories for temp files if necessary.
>
> It isn't really hard or complicated but converting a library that is not
> thread-safe can obviously be error prone and should be introduced with care.
>
> Keith
>
>
> -----Original Message-----
> From: brailleblaster-bounce@xxxxxxxxxxxxx
> [mailto:brailleblaster-bounce@xxxxxxxxxxxxx] On Behalf Of John J. Boyer
> Sent: Monday, May 14, 2012 8:15 AM
> To: brailleblaster@xxxxxxxxxxxxx
> Subject: [brailleblaster] Re: utdml2brl.temp
>
> What is involved in making a library thread-safe? Is there a tutorial with
> examples?
>
> Thanks,
> John
>
> On Mon, May 14, 2012 at 12:12:52PM +0100, Michael Whapples wrote:
> > Hello,
> > Yes I agree with what you say about liblouisutdml not being thread
> > safe and so the only real answer at the moment is to use it from
> > synchronized blocks.
> >
> > If you don't intend to make it thread safe, then may be it would be
> > worth making a note somewhere in the documentation explaining it is
> > not thread safe and there are currently no plans to make it thread safe.
> >
> > As for what you suggest about using the file system of each machine in
> > the cluster, this would not always work:
> > * What about multi-core processors, there is only one file system in
> > that case and so the available processing power is under utilised.
> > * I don't know how the various cluster systems work, but I thought
> > they are trying to work towards making the fact its running on a
> > cluster more transparent and so there may be cases or the time might
> > come where one could not do such a configuration as you describe.
> >
> > Whether its worth your time, I don't know, but as time goes on
> > multi-threaded environments will become more and more improtant and so
> > a thread safe option may be needed, is there one?
> >
> > Michael Whapples
> > On 14/05/2012 11:44, John J. Boyer wrote:
> > >I think the synchronized block is the only way to assure correct
> > >performance. There are lots of shared variables. Making liblouisutdml
> > >and liblouis thread-save would be a big unndertaking and there are
> > >better ways to spend my time. Another solution would be to give each
> > >machine in the cluster its own directories. The synchronized block
> > >should not then exact much of a performance penalty.
> > >
> > >John
> > >
> > >On Mon, May 14, 2012 at 10:44:20AM +0100, Michael Whapples wrote:
> > >>While synchronized blocks may prevent the issue showing, it might
> > >>not be the real answer as it can lead to poor performance. In this
> > >>case this probably is true.
> > >>
> > >>The reason is that synchronized means that other threads have to
> > >>wait for the current thread to finish the synchronized block. In
> > >>this case as there are multiple machines and really separate
> > >>translations should be, well, separate, there should not really be a
> > >>need to have to do translations sequentially. Sequential translation
> > >>is not the most efficient/intended way to use a cluster environment.
> > >>
> > >>The original suggestion of having separate temp files for each
> > >>translation is the correct and fastest way for this to work in a
> > >>cluster environment. May be having it based on the file name is one
> > >>solution, another would be to allow the calling application to set
> > >>the temp file name for the current translation.
> > >>
> > >>Having said the above, even if the file name thing was seen to,
> > >>there could very well be other bugs with using it in a
> > >>multi-threaded environment (eg. there may be shared variables which
> > >>multiple translations both may try to use, etc).
> > >>
> > >>So may be John's suggestion of using synchronized is the only way to
> > >>do this in the immediate future, significant work could be needed to
> > >>make liblouisutdml thread safe and so may simply not be the right
> > >>choice for such a use.
> > >>
> > >>Michael Whapples
> > >>On 14/05/2012 03:35, John J. Boyer wrote:
> > >>>Hi Alex,
> > >>>
> > >>>If I remember right, you should do something like this
> > >>>
> > >>>synchronized {
> > >>>file2brl (arguments);
> > >>>}
> > >>>
> > >>>or you could call translateFile within the block.
> > >>>
> > >>>John
> > >>>
> > >>>On Sun, May 13, 2012 at 06:46:36PM -0700, Alex Jurgensen wrote:
> > >>>>Hi John,
> > >>>>
> > >>>>My translation system consists of aprox 100 Thousand concurrent
> > >>>>translations over a small cluster of machines. Do you have any
> > >>>>advice for how I should implement the LibLouisUTDML interface in
> > >>>>Java?
> > >>>>
> > >>>>Regards,
> > >>>>Alex,
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>Alex Jurgensen,
> > >>>>Community Coordinator,
> > >>>>The Camp Bowen Society for the Visually Impaired,
> > >>>>
> > >>>>Explore Camp Bowen online at: http://www.campbowen.ca/
> > >>>>
> > >>>>Phone: 778-908-0521
> > >>>>E-mail: ASquared21@xxxxxxxxx
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>On 2012-05-13, at 6:37 PM, John J. Boyer wrote:
> > >>>>
> > >>>>>Hi Alex,
> > >>>>>
> > >>>>>If you are using Java it would be best to use a synchronized
> > >>>>>block. If a translation is started while another is in progress
> > >>>>>they will interfere with each other, even if you use a method
> > >>>>>that does not generate file2brl.temp. Each translation takes only
> > >>>>>a few seconds, so your users should not notice much delay.
> > >>>>>
> > >>>>>John
> > >>>>>
> > >>>>>On Sat, May 12, 2012 at 06:13:50PM -0700, Alex Jurgensen wrote:
> > >>>>>>Hi John,
> > >>>>>>
> > >>>>>>To add to my previous question, I was wondering if such a
> > >>>>>>feature could work as follows:
> > >>>>>>
> > >>>>>>/usr/bin/file2brl --tempfile /tmp/myfile.temp
> > >>>>>>/home/Alex/myfile.xml /home/Alex/myfile.brl
> > >>>>>>
> > >>>>>>Regards,
> > >>>>>>Alex,
> > >>>>>>
> > >>>>>>
> > >>>>>>Alex Jurgensen,
> > >>>>>>Community Coordinator,
> > >>>>>>The Camp Bowen Society for the Visually Impaired,
> > >>>>>>
> > >>>>>>Explore Camp Bowen online at: http://www.campbowen.ca/
> > >>>>>>
> > >>>>>>Phone: 778-908-0521
> > >>>>>>E-mail: ASquared21@xxxxxxxxx
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>On 2012-05-12, at 5:23 PM, John J. Boyer wrote:
> > >>>>>>
> > >>>>>>>Hi Alex,
> > >>>>>>>
> > >>>>>>>I assume you are referring to file2brl.temp This file is
> > >>>>>>>overwritten each time the file2brl command-line program or the
> > >>>>>>>file2brl method in the Java bindings is run. It is not intended
> > >>>>>>>for any permanent use, but may be useful for debuging. What are
> > >>>>>>>your reasons for wanting a file name that reflects the source
> > >>>>>>>file?
> > >>>>>>>
> > >>>>>>>Thanks,
> > >>>>>>>John
> > >>>>>>>
> > >>>>>>>On Sat, May 12, 2012 at 01:06:56PM -0700, Alex Jurgensen wrote:
> > >>>>>>>>Hi John,
> > >>>>>>>>
> > >>>>>>>>Is it possible to have LibLouisUTDML create a temporary file
> > >>>>>>>>that is not named utdml2brl.temp, but rather, uses the name of
> > >>>>>>>>the source document one is translating? For example,
> > >>>>>>>>"file1.temp" or "file2.temp"
> > >>>>>>>>
> > >>>>>>>>Regards,
> > >>>>>>>>Alex,
> > >>>>>>>>
> > >>>>>>>>Alex Jurgensen,
> > >>>>>>>>Software Engineer,
> > >>>>>>>>ASInnovations,
> > >>>>>>>>
> > >>>>>>>>"We might be blind, but we have a vision."
> > >>>>>>>>
> > >>>>>>>>Phone: 778-908-0521
> > >>>>>>>>E-mail: VoiceOverTrainer@xxxxxx
> > >>>>>>>>
> > >>>>>>>--
> > >>>>>>>John J. Boyer; President, Chief Software Developer
> > >>>>>>>Abilitiessoft, Inc.
> > >>>>>>>http://www.abilitiessoft.com
> > >>>>>>>Madison, Wisconsin USA
> > >>>>>>>Developing software for people with disabilities
> > >>>>>>>
> > >>>>>>>
> > >>>>>--
> > >>>>>John J. Boyer; President, Chief Software Developer Abilitiessoft,
> > >>>>>Inc.
> > >>>>>http://www.abilitiessoft.com
> > >>>>>Madison, Wisconsin USA
> > >>>>>Developing software for people with disabilities
> > >>>>>
> > >>>>>
> >
>
> --
> John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc.
> http://www.abilitiessoft.com
> Madison, Wisconsin USA
> Developing software for people with disabilities
>
>
--
John J. Boyer; President, Chief Software Developer
Abilitiessoft, Inc.
http://www.abilitiessoft.com
Madison, Wisconsin USA
Developing software for people with disabilities
Other related posts:
