[liblouis-liblouisxml] Re: Questions about the VS projects

From: "Norbert Markus" <hamilfonz@xxxxxxxxx>
To: <liblouis-liblouisxml@xxxxxxxxxxxxx>
Date: Sun, 7 May 2017 18:11:48 +0200

Paulw.torchtrust signatureHi Paul,

Unfortunately I am still struggling with persisting problems.
Following your kind advice, I've managed to obtain the vs projects (with
cumulative commits) of the then latest liblouis, liblouisutdml and file2brl.
These were made folders under "visual studio 2015\projects" as you described,
then were built in the required sequence: liblouis > liblouisutdml > file2brl.
The builds were successful.
Then I updated the older binaries with these freshly built ones in my testing
environment and tried to process documents into braille files.
Unfortunately, I had to experience the same symptoms as before when I worked
with pre-built dll's and exe found in your one-year old project.
As long as it is about English texts translated using en-us-g1.ctb or
en-us-g2.ctb, also when combined with us-table.dis and the hyphenation list,
the translation is done normally and the result is appropriate.
So is it wwith german materials and de-de-g2.ctb and hyph_de_DE.dic.
But I encounter strange things when I try to translate Hungarian materials with
the relevant Hungarian tables (and this is our main aim).
The controversy seems to revolve around the parameter "literaryTextTable".
If it is defined in the .cfg file, a comma separated table list is not
tolerated by the program. Thus:
literaryTextTable us-table.dis,hu-hu-g1.ctb,hyph_hu_HU.dic
provokes errors like these in the stderr output captured in a text file:

Cannot resolve table 'us-table.dis'

1 errors found.

us-table.dis,hu-hu-g1.ctb,hyph_hu_HU.dic could not be found

found table .\lbu_files\us-table.dis

found table .\lbu_files\hu-hu-g1.ctb

found table .\lbu_files\hyph_hu_HU.dic

...
Begin read_configuration_file

preferenceshun.cfg:97: Table 'us-table.dis,hu-hu-g1.ctb,hyph_hu_HU.dic' cannot
be found.

preferenceshun.cfg:97: invalid literaryTextTable

Finish read_configuration_file

...
and so on

The text however is processed but the braille output is largely unreadable
because all the accented letters are replaced by two question marks (digit 5 in
USA braille corresponds to braille dots 2 6 that signifies question mark in the
Hungarian braille).
After a lot of experiments, it turned out that if this table list definition is
placed on the command line following the -C for file2brl.exe, and only
literaryTextTable hu-hu-g1.ctb
is specified in the config file, then there is no apparent error message but
the translation is hung or frozen so that the only way out is to press ctrl-c,
and no braille output is produced.
Later I found another variation: when the hyphenation definition file is
removed from the table list, so that only
-CliteraryTextTable=us-table.dis,hu-hu-g1.ctb
is specified then the translation completes and the result is appropriate,
except that there is no hyphenation in the braille file that is a great deficit.

At the same time I must underline that the same Hungarian related liblouis
tables and hyphenation dictionary work fine with the same texts when using the
Linux version of liblouis, liblouisutdml and file2brl.
hu-hu-g1.ctb and the related table files and also hyph_hu_HU.dic are compliant
with the liblouis standards and are in utf8 format.

Not long ago, I discovered that there are more recent commits to liblouis and I
downloaded the latest state of the vs projects, built liblouis.dll and
subsequently built liblouisutdml and file2brl upon it.
Unfortunately, the problems described above still persist.
I even tried to set the Unicode bits, UCS=2 to UCS=4 in configure.mk and
rebuilt everything but it didn't help either.

Testing the freeze with different texts, it appears that the procedure gets
stuck when it encounters a runover line. I am not exactly sure about it this is
just an impression.

I got the hint that maybe the utf8 format of the hyphenation dictionary caused
the problem, so I ran a test with the Czech tables (their hyphenation
dictionary is also in utf8) but it worked fine.

The above mentioned hu-hu-g1.ctb, all its dependent table files and
hyph_hu_HU.dic are part of the official liblouis distribution.
We also tested the grade 2 Hungarian tables and got the same frustrating
results.

Do you happen to have any idea what may cause this problem? Did I do something
wrong when collecting the sources or during the build process?

I tried to doublecheck liblouis' version.
My entrypoint for the download was
https://github.com/liblouis/liblouis/pull/194/commits

The title of the page is:

"Visual Studio projects for 3.0.0 by torchtrust · Pull Request #194 ·
liblouis/liblouis · GitHub"

Perhaps a bit confusing for someone looking for version 3.1.0.　

Among the "Commits on Apr 21, 2017" there is a commit with a comment "Update
for 3.1.0".

I hoped it meant what I thought it meant. 　

But I went to check the version in the source code.

liblouis\liblouis\compileTranslationTable.c:

#include "config.h"

...

char *EXPORT_CALL

lou_version ()

{

static char *version = PACKAGE_VERSION;

return version;

}

　

liblouis\windows\include\config.h:

#define PACKAGE_VERSION "liblouis-3.1.0"

#define PACKAGE_NAME "Liblouis"

#define VERSION "3.1.0"

After having found these above, I felt assured to have the latest vs projects
for liblouis.dll.
I also checked the most recent commits to liblouisutdml and found that I
probably have the latest or nearly the latest version 2.7.
However, it is a bit strange that liblouis' target is debug, that of
liblouisutdml is release and for file2brl is debug again. Hope this cannot
cause the translation problems I described above.

When I try to pinpoint a particular cause, my attention gets drawn to
hyph_hu_HU.dic but it is contradicted by the fact that it works properly under
Linux. And, I don't know how to get more detailed debug info (saying "debug
yes" in the config does not make any difference at least as far as I can tell).

Is there a way to find out what resource, component or build propriety may
cause this problem?

Thank you and anyone else for any help in advance.

Best Regards, Norbert.

From: Paul Wood (Torch)
Sent: Sunday, April 23, 2017 1:02 PM
To: liblouis-liblouisxml@xxxxxxxxxxxxx
Subject: [liblouis-liblouisxml] Re: Questions about the VS projects

Hi Norbert,

I too could not find the 'Download Zip' button when I first looked at this on
Thursday/Friday! So what I did was download the GitHub Extension for visual
Studio, which made life a lot easier especially to rebase and update changes,
but I have also now found the illusive button!

from https://github.com/liblouis/liblouisutdml/pull/23

click on 'Commits'! https://github.com/liblouis/liblouisutdml/pull/23/commits

Then click on last commit, "Browse the repository at this point"
https://github.com/liblouis/liblouisutdml/tree/8b9ffc90865ea77038fa40939ffabcd7eb089e23

Now you have the link to download!
https://github.com/liblouis/liblouisutdml/archive/8b9ffc90865ea77038fa40939ffabcd7eb089e23.zip

As you have probably worked out this is the state of the whole repository with
my latest commits in place as well as all the other commits. I wish I had know
this last week! Anyway as I said the GitHub extension did help me with pushing
the changes. I detailed the steps for my future reference and also so you can
follow similar steps for the Liblouis projext etc.

For the final code you will need all the dll's including libxml2 and
libiconv.dll. Let me know if you can't find those.

Cheers
Paul

On 23/04/2017 11:17, Norbert Markus wrote:

  Hi Paul,
  Sorry, I am encountering problems when attempting to download.
  I clicked the link in Your latest message (have updated #23).
  I find myself on Github's page for the Visual Studio 2015 Solution for
Liblouisutdml, and I am unable to find my way further.
  Last time, about a month ago, when I visited #23, somehow I stumbled upon a
button or menu item "download zip", and by that I pulled a full package
containing all sources, resource files and vs project files.
  Now I am unable to find such a menu offering to download a zip archive.
  When I clicked "Code" then there was that menu and download zip, but it
appeared to pull a purer liblouisutdml package with no vs files, no binaries,
etc. Jus for comparison: when I did it the first time some weeks ago, the
download size was around 25 mb, and now it was just 2.5 mb.
  Tumbling around, I scrolled down the list of your commits and found at the
bottom of the list "Update to 2.7". When I click it, a new page shows with the
diff details of the most recent commit and offers to show file contents, but I
cannot find any control to download a package related to vs. When I click
"code", the download zip option is there again, but the size is around 1.5 mb.

  What am I doing wrong? Could you please provide some guidance for how to get
the latest stuff downloaded.

  Thanks in advance.
  Best Regards, Norbert.


  From: Paul Wood
  Sent: Friday, April 21, 2017 5:25 PM
  To: liblouis-liblouisxml@xxxxxxxxxxxxx
  Subject: [liblouis-liblouisxml] Re: Questions about the VS projects

  Hi Norbert,

  Have updated https://github.com/liblouis/liblouisutdml/pull/23 with relative ;
paths. Dependants are libxml-2.7.2 and LibLouis. The VS projects for LibLouis
are https://github.com/liblouis/liblouis/pull/194 but are not tested with ;
master. I have an issue when I try to build with tools, but you don't need the
tools to run liblouisUTDML. file2brl is a separate solution. If I knew what I
know now then I would combine file2brl with liblouisUTDML. If you want it to
work without changing anything in the paths then setup your folders so
liblouis, liblouisutdml and libxml2-2.7.2 are all the folder names of the
different parts.
  I used a spare PC which already had Visual Studio 2015 Community and then
Cloned the repos of the master then the pull requests of liblouis and
liblouisUTDML. Copied my version of libxml2-2.7.2 from my development PC. so
AFAIK should all work for you or whoever. Please let me know how you get on.

  I will now look to see if everything works with the latest versions.
  Cheers
  Paul

  On 20/04/2017 22:48, Norbert Markus wrote:

    Hi Paul,
    I am just after having downloaded pull request #23 and dealing with the
sources included, that is, I am afraid, liblouisutdml 2.5 that was recent at
that time (June or July 2016).
    My first modest objective is to build the solution as it is just to see
that I can do it over here too. Once I succeed, I would like to make the source
up to date and repeat the build to get binaries corresponding to the latest
version.
    What do you think, is this a logical path?

    I still need to find out which pull request contains the VS solution for
the then matching liblouis version and how to deal with the libxml2 dependency.

    Hope I did not forget to mention any important detail.

    Thanks for your assistance!

    Best Regards, Norbert.

    From: Paul Wood
    Sent: Thursday, April 20, 2017 3:10 PM
    To: liblouis-liblouisxml@xxxxxxxxxxxxx
    Subject: [liblouis-liblouisxml] Re: Questions about the VS projects

    Hi Norbert,

    where are you at now? which source are you using 3.0.0 or 3.1 or main
repository? I want to try and start where you are.

    Thanks
    Paul

    On 19/04/2017 22:08, Norbert Markus wrote:

      Hi Neal,
      There is a separate VS solution specifically for file2brl.exe. I don't
want to state any non-sense before I've gotten that far myself but I can simply
see that pull request #23 contains two solutions: one for liblouisutdml.dll and
another for file2brl.exe.
      Thanks for drawing my attention to the fact that nmake scripts are
encapsulated into the VS project's build mechanism. I wasn't aware of it. I am
just beginning to look around in the stuff. It seems there is still a lot to
understand.
      Thanks a lot for the nmake excerpt. Any such information is welcome.
      Best Regards, Norbert.

      From: Neil Soiffer
      Sent: Wednesday, April 19, 2017 9:28 PM
      To: liblouis-liblouisxml@xxxxxxxxxxxxx
      Subject: [liblouis-liblouisxml] Re: Questions about the VS projects

      FYI: I too am using the VS solution. If you look at the .vcxproj file, it
basically calls nmake (copied relevant commands below for your convenience), so
there shouldn't be any  difference from doing the commands yourself from a
command line. Of course if you are debugging or developing something, that's
where using VS's IDE is really useful.

      In case you weren't aware, the makefiles do not mention file2brl and so
they (and the VS solution) won't build it. It looks like you need to modify
Makefile.nmake to include building and invoking the makefile.am in the tools
dir, but I don't need that file and I'm out of my league in answering questions
about building it.

      ======= from vcxproj file =======

      mkdir libslib
      set LXCCFLAGS= /wd4005
      nmake  /f makefile.nmake
      del /f /q liblouisutdml.lib
      nmake /f makefile.gen liblouisutdml.lib
      copy liblouisutdml.lib libslib

      On Wed, Apr 19, 2017 at 1:40 AM, Norbert Markus <hamilfonz@xxxxxxxxx>
wrote:

        Hi Neil,
        I was relatively lucky with the nmake script for liblouisutdml. It has
gone through with no fatal errors and built the dll but it did not create
file2brl.exe that I still badly need. I modified MakeFile.nmake neither for
liblouis nor for liblouisutdml. I don't know what factorsaved me from fatal
errors.

         Readme.windows says that Libxml2 version 2.7.2 is needed. First I
tried to deviate from it but my lack of discipline was quickly retaliated, so I
urgently fetched the right version but I could only find the source. Thanks for
the hint about how to build its binaries. The paths are correct in configure.mk
I believe.

        However, after experimenting with the command line tools for weeks with
partial success, I decided to turn my hand to the Visual Studio solution.
        Thank You all in advance if any progress is made on my side.

        Norbert.

        From: Neil Soiffer
        Sent: Wednesday, April 19, 2017 12:08 AM
        To: liblouis-liblouisxml@xxxxxxxxxxxxx
        Subject: [liblouis-liblouisxml] Re: Questions about the VS projects

        I'm currently trying to build liblouis and liblousutdml with VS 2015. I
am building from the current github versions (some suggested fixes below).  I
have run into some problems with linking, but I think I can answer a few
questions. I'm jumping into the middle of this discussion, so apologies if
these have already been answered or my answers are off base...

        libxml2 2.7.8 can be found at
https://git.gnome.org/browse/libxml2/tag/?h=v2.7.8

        I believe it gets built due to lines in windows\make_makefile.c

        I don't have and don't seem to need libiconv.dll and libiconv.lib.

        Make sure you edit windows\configure.mk in your liblouisutdml dir to
point to the right locations.

        What I have found so far that needs changing are:
        liblouis ERRORS:
        liblouis\windows\Makefile.nmake
        commonTranslationFunctions.c not part of makefile

        change to:
        OBJ = commonTranslationFunctions.obj compileTranslationTable.obj
lou_translateString.obj \
            lou_backTranslateString.obj logging.obj wrappers.obj pattern.obj

        add as first compile rule (to keep alphabetical):
        commonTranslationFunctions.obj: $(SRCDIR)\commonTranslationFunctions.c \
            $(HEADERS)
            $(CC) $(CFLAGS) $(SRCDIR)\commonTranslationFunctions.c

        liblouis WARNINGS:
        liblouis\compileTranslationTable.c:(4011) : 'ptn_regex': unreferenced
local variable

        Fix by removing ptn_regex

        liblouisUTDML link WARNINGS/ERRORS:
        As I mentioned at the start, I haven't gotten liblouisUTDML to link
yet. Some warnings are:
        4>LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of
other libs; use /NODEFAULTLIB:library

        Lots like:
        4>liblouis.lib(lou_translateString.obj) : warning LNK4217: locally
defined symbol _calloc imported in function _hyphenate

        And finally the fatal errors:
        4>liblouis.lib(compileTranslationTable.obj) : error LNK2019: unresolved
external symbol __imp___stricmp referenced in function _findOpcodeNumber
        4>liblouis.lib(pattern.obj) : error LNK2019: unresolved external symbol
__imp__puts referenced in function _pattern_output_expression
        4>MSVCRT.lib(_chandler4gs_.obj) : error LNK2019: unresolved external
symbol __except_handler4_common referenced in function __except_handler4

        If someone has seen these before and knows a fix, it would save me time
figuring out what's not right with the linking.

        Thanks,

            Neil

        On Tue, Apr 18, 2017 at 8:36 AM, Norbert Markus <hamilfonz@xxxxxxxxx>
wrote:

          Hi Paul,
          Thanks for your immediate response.
          I will appreciate if you can deal with my questions from Thursday on.
          In the meanwhile, I am trying to find the pull request for the
liblouis VS project and to sort out the path controversy.
          Until no better solution is suggested, I am going to use the libxml
binaries included in #23.

          Have a nice holiday!

          Best Regards, Norbert.

          From: Paul Wood
          Sent: Tuesday, April 18, 2017 5:32 PM
          To: liblouis-liblouisxml@xxxxxxxxxxxxx
          Subject: [liblouis-liblouisxml] Re: Questions about the VS projects

          Hi Norbert,
          I am on holiday, but back Thursday. I hope to look at this this week,
but I have plenty of work piled up, but I am keen to find out why your build is
nor working. Liblouis is needed by liblouisutdml and both are needed by
file2brl.
          Bye for now
          Paul

          On 18 Apr 2017 4:16 p.m., "Norbert Markus" <hamilfonz@xxxxxxxxx>
wrote:

            Hi Paul,

            I am trying to get under control your liblouisutdml solution for
Visual Studio downloaded from your pull request #23. My first objective is to
be able to build the projects with ctrl-F5 encountering no errors. This is not
yet achieved, unfortunately.
            A further objective is to update the source to the latest liblouis
and liblouisutdml versions and to successfully build the projects including the
DLLs and EXEs, thus to obtain an up-to-date runnable version (especially of
file2brl.exe) for windows.

            In your previous related message, You wrote that you have created
two projects, one for liblouis and another for liblouisutdml. What is the pull
request number of the former (i.e., for liblouis).

            What is the relationship between these solutions from the respect
of Visual Studio? Is there a common entry point to start the build process of
all components, or should these projects be built in a particular order and to
be referenced to each other?
            For instance, should I first build the project for liblouis, then
another project for liblouisutdml referring to the result(s) of the liblouis
build, and then in a third pass, should the project for file2brl be built based
on the results of the former two?

            I have copied the folder for the liblouisutdml solution downloaded
from git to:
            C:\users\Norbert\documents\Visual Studio 2015\projects
            Here I truncated the original download folder name to liblouisutdml.

            I found all the VS related stuff in subfolders of the windows
folder. I assume, not everything is needed for VS in the complete downloaded
folder, but at this moment I am unable to figure out what to jettison with no
severe consequences.
            I found two .sln files:
            windows\msvc\file2brl\file2brl.sln
            windows\msvc\liblouisutdml\liblouisutdml.sln

            In both cases, I began the procedure by clicking on one of these
that launched VS and loaded the solution. I did not hope it will work for the
first try but I pressed ctrl-F5 and got a question: "This project is out of
date. Would you like to build it anyway?" I am not exactly sure what it means
but I answered yes to proceed.
            Then: build failed.
            In case of liblouisutdml.sln, there were 24 errors and 2 warnings.
I checked a few of the errors complaining about that liblouisutdml.h and
liblouis.h (several of them) cannot be found here:
            c:\users\Norbert\documents\Visual Studio
2015\projects\liblouisutdml\liblouisutdml\liblouisutdml.h

            It hints, that I copied too many depths of the downloaded folder to
the projects directory, but it is not exactly clear what path to provide for a
successful build.
            Can you help me find out where to copy what of the material you've
uploaded last year?
            I assume manually correcting all instances of the references to
such missing files wouldn't be a wise method.

            A similar attempt to build the project for file2brl produces 8
errors of similar kind. In addition to missing header files, there are errors
missing certain type definitions, e.g., among others: loglevels and widechar if
I remember well.

            What are the necessary steps to get these solutions configured
appropriately?

            Some further questions:
            In the windows\msvc\liblouisutdml\lib folder, I found libiconv.dll,
libiconv.lib, libxml2.dll and libxml2.lib needed for xml parsing.
            How did you obtain these files?
            Have you created another vs project to build them? (I haven't found
anything in your liblouisutdml project to hint that libxml2 is built as part of
that solution.)
            Have you created these files by running a command line build
process included in the libxml2 source folders?
            Or have you found and downloaded prebuilt binaries?

            Earlier, I ran a few rounds with this: first I obtained the latest
libxml2 version (2.9.4), but its header files caused lots of error messages
when I tried to build liblouisutdml using a command line nmake tool.
            Later I found the source for version 2.7.2 that appeared to be
adequate but the nmake script that builds liblouisutdml seems not to build the
libxml2 binaries.
            Then I looked around to download prebuilt binaries but I could not
find versions as old as 2.7.2. Even the more recent packages include no .lib
files.

            Please help me if you can, configure and build your original VS
solution on my local machine.

            Thanks in advance, your help would be invaluable.
            Best Regards, Norbert.

    --

    Paul Wood, Chief Technical Officer
    Torch Trust
    Torch House, Torch Way,
    Market Harborough, Leics. LE16 9HL, UK
    Direct Line: +44(0)1858 438269
    Tel: +44(0)1858 438260, Fax: +44(0)1858 438275
    Email: paulw@xxxxxxxxxxxxxx
    Website: www.torchtrust.org

    ____________________________________________________

    Chief Executive: Dr Gordon Temple
    Charity No. 1095904

    Privileged/Confidential Information may be contained in this message.
    If you are not the intended recipient please destroy this message
    and kindly notify the sender by reply email. The computer from which
    this mail originates is equipped with virus screening software.
    However Torch Trust cannot guarantee that the mail and its attachments
    are free from virus infection.

  --

  Paul Wood, Chief Technical Officer
  Torch Trust
  Torch House, Torch Way,
  Market Harborough, Leics. LE16 9HL, UK
  Direct Line: +44(0)1858 438269
  Tel: +44(0)1858 438260, Fax: +44(0)1858 438275
  Email: paulw@xxxxxxxxxxxxxx
  Website: www.torchtrust.org

  ____________________________________________________

  Chief Executive: Dr Gordon Temple
  Charity No. 1095904

  Privileged/Confidential Information may be contained in this message.
  If you are not the intended recipient please destroy this message
  and kindly notify the sender by reply email. The computer from which
  this mail originates is equipped with virus screening software.
  However Torch Trust cannot guarantee that the mail and its attachments
  are free from virus infection.

--

Paul Wood, Chief Technical Officer
Torch Trust
Torch House, Torch Way,
Market Harborough, Leics. LE16 9HL, UK
Direct Line: +44(0)1858 438269
Tel: +44(0)1858 438260, Fax: +44(0)1858 438275
Email: paulw@xxxxxxxxxxxxxx
Website: www.torchtrust.org

____________________________________________________

Chief Executive: Dr Gordon Temple
Charity No. 1095904

Privileged/Confidential Information may be contained in this message.
If you are not the intended recipient please destroy this message
and kindly notify the sender by reply email. The computer from which
this mail originates is equipped with virus screening software.
However Torch Trust cannot guarantee that the mail and its attachments
are free from virus infection.

[liblouis-liblouisxml] Re: Questions about the VS projects

Other related posts: