[liblouis-liblouisxml] Re: A new liblouis?
- From: "Joseph Lee" <joseph.lee22590@xxxxxxxxx>
- To: <liblouis-liblouisxml@xxxxxxxxxxxxx>
- Date: Sun, 27 Oct 2013 13:33:51 -0700
Hi all,
Many Chinese characters do have Unicode representation. At present, we do
have Chinese tables for both Taiwan and Hong Kong which has braille dot
representations for Unicode Chinese characters (they have dictionary-based
tables). As some people said before, the way these characters are pronounced
makes a huge impact on meaning of characters, and in extension, braille
dots.
The best way to accommodate CJK (Chinese, Japanese, Korean ) character set
for now is to use either the Unicode characters themselves (such as what we
have now for Chinese and Korean tables) or specify their hex values, which
means writing down all possible character combinations (we're talking about
up to several thousand characters). Korean braille is interesting in that
one can write either dictionary-based or rule-based tables (we currently use
dictionary approach, but with an introduction or Unicode range calculation
subroutine, it becomes rule-based (more towards hybrid approach where we
first define character parts and dots for these parts which makes up that
character, then based on the character we're parsing, we can assign
different braille dots based on calculating Unicode values for these
character parts). Another interesting fact, which may be solved using regex
(not implemented yet) is handling of foreign alphabet in Korean braille (we
use special braille symbols to denote beginning and end of English strings).
For a rationale behind Takuya's post, see a request on NVDA core ticket
Trac:
http://community.nvda-project.org/ticket/2736
For more info about how CJK characters work in computers, take a look at
this Wikipedia article:
http://en.wikipedia.org/wiki/CJK
I think we need to work on a separate thread if we want to tackle CJK issues
in Liblouis later...
Cheers,
Joseph
-----Original Message-----
From: liblouis-liblouisxml-bounce@xxxxxxxxxxxxx
[mailto:liblouis-liblouisxml-bounce@xxxxxxxxxxxxx] On Behalf Of John Gardner
Sent: Sunday, October 27, 2013 10:24 AM
To: liblouis-liblouisxml@xxxxxxxxxxxxx
Subject: [liblouis-liblouisxml] Re: A new liblouis?
Hello Takuya, it would be fantastic if you could introduce Japanese braille
translation into liblouis. I know that Japanese and Chinese braille
translation are particularly difficult, because they are based on the spoken
language, not the written one. So they require the same AI that is needed
for a speech engine in that language. I am impressed that you will be using
such a routine. It could result in excellent Japanese braille.
Have you been keeping up with the present conversation about possibly moving
to a scripting language? We will be using an Apache 2 license for the new
development, and this is compatible with the BSD license on the software you
need. I do not understand your reference to the Unicode support, so I would
suggest you look at both Python and Lua to know if either or both will
support your development. At present, those are the leading candidates for
the liblouis gen 2 scripting language.
John Gardner
John Gardner
-----Original Message-----
From: liblouis-liblouisxml-bounce@xxxxxxxxxxxxx
[mailto:liblouis-liblouisxml-bounce@xxxxxxxxxxxxx] On Behalf Of Takuya
Nishimoto
Sent: Saturday, October 26, 2013 6:48 PM
To: liblouis-liblouisxml@xxxxxxxxxxxxx
Subject: [liblouis-liblouisxml] Re: A new liblouis?
Dear list members,
I have been thinking about Japanese language support of the liblouis, the
most popular open-source braille translation library.
According to my experience of developing NVDA Japanese version, which now
supports most of Japanese braille translation rules, our braille translation
library must use "Part-of-Speech and Morphological Analyzer,"
such as MeCab open source project (under BSD, LGPL, GPL):
https://code.google.com/p/mecab/
Written Japanese sentence uses ideographic characters and does not use
spaces between words.
On the other hand, Japanese braille only uses phonetic characters, and
spaces for word breaks.
Another important feature we need is Unicode normalization, such as
unicodedata module of Python language.
I agree that Japanese braille support consumes disk spaces for text
processing dictionary and may not affordable as a standard function,
however, if some DSL commands which can take the path for MeCab library or
dictionary, we will be possible to integrate our Japanese braille engine and
next generation of liblouis.
I am aware that many important open-source products does not support
Japanese braille, because of limitations of liblouis.
If necessary, I will call for more contribution to next generation of
liblouis from Japanese language users.
Best regards,
--
Takuya Nishimoto
nishimotz@xxxxxxxxx
2013/10/27 John J. Boyer <john.boyer@xxxxxxxxxxxxxxxxx>:
> If we want to free ourselves from the LGPL we can't use any of the
> existing code for liblouis or liblouisutdml. The LGPL has meant that
> liblouis is rejected for many applications, such as those in the Apple
> store.
>
> John B
>
> On Sat, Oct 26, 2013 at 08:26:49PM +0200, Mesar Hameed wrote:
>> It sounds like what is being suggested is to rebuild liblouis from
>> scratch.
>> Might it be more beneficial to enforce a feature freeze for an
>> extended period of time, where no new code is allowed to enter the
repository.
>> During this refactoring time, we try to untangle the spaghetti
>> code/rewrite those parts, and make things more modular.
>> This would mean that segments of the existing liblouis could be
>> tackled, cleaned up and when tested to be equivilent to the one in
>> trunk, merged back.
>> At the end of the process we have improved liblouis, and positioned
>> it hopefully in a better place to tackle new challenges.
>>
>> The effort required is somewhat less than starting from scratch, but
>> is also not negligible.
>> Mesar
>> On Sat 26/10/13,07:40, James Teh wrote:
>> > Hi.
>> >
>> > I certainly agree that we need to think about succession planning.
>> >
>> > Even as a huge fan of Python (I use it for everything I can), I'm
>> > concerned about performance for something like this. I realise that
>> > you can make something very fast in Python or Cython "if properly
>> > made". However, the primary reason you cite for moving to such a
>> > language is that more people can write and maintain it. I'd argue
>> > this isn't so much the case if coding for high performance is a
>> > requirement, which requires a fairly advanced programmer anyway, in
>> > which case they can probably learn to code well enough in C or C++.
>> >
>> > There may be all sorts of people wanting to call the library. I
>> > guess there must be a way to build a shared library which exposes
>> > Python code so it can be called from native code (though I'm not
>> > familiar with it). However, that still requires a Python
>> > interpreter to be loaded and initialised, which is a fair amount of
overhead.
>> > It's also not possible in some situations. For example, an
>> > application may already be using a Python interpreter for something
>> > else, in which case the library can't load its own interpreter. You
>> > also can't load two Python interpreters of differing versions. (We
>> > actually hit this problem with NVDA and a speech synth that was
>> > using Python for some, but not all, of its internals.)
>> >
>> > We need to think about embedded and mobile situations. It might be
>> > fine for a notebook, desktop or server computer to run a braille
>> > translator in Python, but that may not be feasible for mobile or
>> > embedded. Even if it were, it may not perform well enough on these
>> > lower powered processors.
>> >
>> > I'm unclear on one point. It could be interpreted from this message
>> > that each braille code will have its own script. If so, this would
>> > be fairly bulky. I think there still needs to be a core library
>> > which then loads data (in Python or otherwise) for each braille
>> > code.
>> >
>> > Finally, I would hope the new library can load the current liblouis
>> > braille tables or that there will be a converter for them. It would
>> > be a shame to see all of the current work on tables need to be done
>> > from scratch, especially as some of the contributors may not be
>> > willing to do this.
>> >
>> > Just my AUD$0.02. :)
>> >
>> > Jamie
>> >
>> > On 26/10/2013 2:25 AM, John Gardner wrote:
>> > >Hello all. John Boyer has been the guiding light of liblouis from
>> > >the beginning, and it is hard to imagine the project without him.
>> > >But all good businesses need to have succession plans, and it is
>> > >hard for many of us to imagine how anybody else could take over
>> > >the liblouis project as it now exists. I believe that we should
>> > >begin a twin project to remake liblouis and liblouisutdml into
>> > >something that uses modern scripting languages that many more
>> > >people can write and maintain. I have previously suggested
>> > >adopting a good well-known well-supported scripting language so
>> > >that the translator for any given language could just be a script.
>> > >This approach has many advantages, including the
fact
>> > >that almost anybody can learn a good scripting language well
>> > >enough to write and maintain translators. The project would be
>> > >big, and it
would
>> > >need to be done largely by volunteers over time. We would
>> > >certainly
be
>> > >using the present system for years. Let me emphasize that I am
>> > >definitely not suggesting that present developers divide their
>> > >time -
we
>> > >cannot afford that. Any parallel development must be done by
>> > >other people and by current developers during personal time.
>> > >
>> > >The problem with scripting languages has always been that they are
>> > >typically much slower than good compiled languages. Consequently,
>> > >a number of people on this list have objected to such a change.
>> > >In a
few
>> > >private discussions with APH developers, we have worked out a plan
that
>> > >has the advantages of a scripting language, a way to introduce it
>> > >without disrupting the way liblouis is currently being used, and
>> > >that does not sacrifice speed.
>> > >
>> > >Ken has offered to write up a library that permits one to
>> > >substitute a script for the current liblouis table+engine for any
>> > >given language,
so
>> > >that as a new script is perfected, it can be slipped in with no
>> > >change in any software that uses it. Ken needs to explain the
>> > >details
though.
>> > >
>> > >The speed penalty can be minimized by letting any "heavy lifting"
>> > >be done by compiled routines, for example a look-up table that
>> > >will translate 95% of the words probably faster than the current
method.
If
>> > >necessary, the script can be compiled. If properly made, the
>> > >compiled version can run as fast as a hand-coded routine.
>> > >
>> > >I have proposed using Python as the scripting language, and
>> > >Michael Whapples has propposed using it in a controlled manner so
>> > >that it can
be
>> > >compiled to C with Cython. I believe that this is a good
>> > >approach,
and
>> > >I like it because I am more or less capable of writing Python, so
>> > >I would volunteer to start the process by writing a new translator
>> > >for Nemeth and UEB.
>> > >
>> > >Okay, it is now time for all you stakeholders to weigh in.
>> > >Succession planning is not really an option - we have to do
>> > >something. The question really is "What?" We all understand about
>> > >the need to keep translation fast and the need not to take up any
>> > >significant amount of present developers' time. What are other
>> > >issues, and what are other answers to "what?"?
>> > >
>> > >John Gardner
>> > >
>> >
>-----------------------------------------------------------------------
>-
>> > >
>> > >John Gardner
>> > >
>> > >
>> > >
>> > > |
>> > >
>> > >
>> > >
>> > >President
>> > >
>> > >
>> > >
>> > >|
>> > >
>> > >
>> > >
>> > >Description: Description: Description: ViewPlus
>> > >
>> > >541.754.4002 x 200
>> > >
>> > >
>> > >
>> > > |
>> > >
>> > >
>> > >
>> > >www.viewplus.com <http://www.viewplus.com/>
>> > >
>> > >
>> > >
>> >
>-----------------------------------------------------------------------
>-
>> > >
>> > >PRIVILEGED AND CONFIDENTIAL: This message and any files
>> > >transmitted
with
>> > >it may be proprietary and are intended solely for the use of the
>> > >individual to whom they are addressed. If you are not the intended
>> > >recipient, any use, copying, disclosure, dissemination or
>> > >distribution is strictly prohibited; please notify the sender and
>> > >delete the
message.
>> > >ViewPlus Technologies, Inc. accepts no liability for damage of any
kind
>> > >resulting from this email.
>> > >
>> >
>> > --
>> > James Teh
>> > Director, NV Access Limited
>> > Ph + 61 7 5667 8372
>> > www.nvaccess.org
>> > Facebook: http://www.facebook.com/NVAccess
>> > Twitter: @nvaccess
>> > For a description of the software, to download it and links to
>> > project pages go to http://www.abilitiessoft.com
>
>
>
> --
> John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc.
> http://www.abilitiessoft.com
> Madison, Wisconsin USA
> Developing software for people with disabilities
>
> For a description of the software, to download it and links to project
> pages go to http://www.abilitiessoft.com
For a description of the software, to download it and links to project pages
go to http://www.abilitiessoft.com
For a description of the software, to download it and links to project pages
go to http://www.abilitiessoft.com
For a description of the software, to download it and links to
project pages go to http://www.abilitiessoft.com
Other related posts:
