I guess what I'm asking is, since you have the master strategy and seem pretty firm on it, what if any of this should Chuck, Brandon, and I take responsibility for? At this point I am unsure how to proceed, or should we just wait? -----Original Message----- From: John J. Boyer [mailto:john.boyer@xxxxxxxxxxxxxxxxx] Sent: Thursday, January 10, 2013 11:21 AM To: Keith Creasy Cc: Michael Whapples; John Gardner Subject: Re: Steering: Design outline Hi Keith, I really can't come up with specific suggestions for Chuck and Brandon. You know them better than I do. However, I would like to know what information the developers want to pass in and out of the document package, so I can set up methods in the Document class. These are technical matters and should be on the list. John B On Thu, Jan 10, 2013 at 02:00:14PM +0000, Keith Creasy wrote: > Yes, well I think we already agree on those two points. I also think your > strategy for using semantic actions is fine too. I probably would not have > done it that way but it most likely doesn't matter much. It boils down to the > fact that I'd have used conventional XML features more; things like schemas, > XSLT, and XPath. > > I am trying to get my head around your strategy and I think I'm getting > there. Your recent changes actually helped. > > I am anxious to get things to a point that we can assign different components > to our developers and let them get going with it. Any suggestions for Chuck > and Brandon? > > Styled text works for what we want to do initially. I still haven't found the > reference to support for images but then I haven't looked at all the links > Michael sent. I'm excited that image support might be in there because it > opens up all kinds of possibilities. > > > I still think the document methods need to be event driven, threaded, and > queued. > > -----Original Message----- > From: John J. Boyer [mailto:john.boyer@xxxxxxxxxxxxxxxxx] > Sent: Wednesday, January 09, 2013 6:18 PM > To: Keith Creasy > Cc: Yuemei Sun; John Gardner; Michael Whapples > Subject: Re: Steering: Design outline > > I think we can agree on two things. It would be desirable for BrailleBlaster > to be vocabulary-agnostic. To accomplish this it is necessary to have classer > or a wrapper that encapsulates knowledge of the vocabulary of the particular > document and presents the rest of the application with an abstraction. I am > familiar with the use of semantic-action files and have been considering > extending them from their use in liblouisutdml to generalized rendering and > editing for quite some time. Other approaches to attaining the two goals > listed above are possible, but this is one backed by years of experience and > refinemment. > > On other matters, StyledText and StyledTextContent require considerable > mental work. There are other classes in the SWT custom package which we might > find helpful, including a printing class. StyledText itself can convert to > RTF, so it may take care of one export need. > > If we agree on the two things stated at the beginning of this message I think > we can proceed. I will work on the details of editing and updating and will > also investigate the use of XPath. The Document class will consist of a large > number of convenience methods, some with short bodies. > > John B > > On Wed, Jan 09, 2013 at 08:32:18PM +0000, Keith Creasy wrote: > > Thank you. I understood all this but your explanation is appreciated. > > > > > > > > -----Original Message----- > > From: Yuemei Sun [mailto:yuemei.sun@xxxxxxxxxxxx] > > Sent: Wednesday, January 09, 2013 3:24 PM > > To: John J. Boyer > > Cc: Keith Creasy; John Gardner; Michael Whapples > > Subject: Re: Steering: Design outline > > > > I will see if I can explain a little bit about the use of > > libouisutdml/semantic files. > > > > ViewPlus uses liblouisUTDML to translate OpenOffice.org Writer and Calc > > document into braille documents with matching ink texts interlined . We > > extract the content xml from the Zipped OOo file and send it to > > libouisutdml. In return, we get an XML with UTDML (<brl> blocks) embedded. > > The <brl> blocks has braille formatting information, such as <newline>, > > <newPage> etc, and the mapping info from braille to the original texts. > > > > We process the returned document in a way that the Braille will be > > formatted as specified in the <brl> block, and the texts will be extracted > > according to the mapping info. The braille will be put in one layer > > (transparent) and the ink will be in another layer (also transparent), but > > formatted in a way so that it will appear between braille lines. The UTDML > > block is then removed from the returned document, and the document is > > zipped back into an ODT or ODS document which will be opened again in > > OpenOffice.org. > > > > One of the most important reasons that ViewPlus uses liblouisutdml is the > > correct braille formatting included in the translated braille, for example, > > the braille page numbers included in the translated texts. > > This can only be done during the translation stage. Braille > > formatting/style has their complex rules and liblouistutdml knows how to > > handle them. > > > > In my understanding, semantic-action files, used by liblouisutdml, is a way > > to specify how every element in the document is to be handled. When an OOo > > xml file was first sent to liblouisutdml, a .sem file > > (document-content.sem), named after the root element, was created, with all > > the element listed and set to default action "no", which means no action is > > to be performed. In order to translate, I need to edit the .sem file to > > specify which elements should be performed a translation action. Later on, > > I learned to use the .sem file to specify more customized action, such as > > applying specific translation table for elements having certain attributes. > > > > The semantic-action mechanism used in liblouisutdml has it flexibility of > > introducing new actions and of specifying actions on individual elements. > > Those actions do not have to be a translate action. I can see why John B. > > argues that the same mechanism can be used for editing XML document in > > general. Semantic-action machanism is independent from UTDML. It was > > first used in liblouisxml; we added UTDML spec later in order to have the > > ink-braille mapping info in the translated file. > > > > Yuemei > > > > > > On 1/9/2013 6:16 AM, John J. Boyer wrote: > > > Keith, you are by no means stupid, but this involves thinking > > > outside the box. I think I have stated what semantic actions do quite > > > clearly. > > > It's true that in liblouisutdml they are used to render documents > > > into Braille. In BrailleBlaster I have used what I learned in > > > liblouisutdml. > > > The role of semantic action files is to tell the program how to > > > handle documents and how to edit them. > > > > > > Please remember to use reply-to-all when answering messages to the > > > steering committee. There are people at ViewPlus who do understand > > > semantic-action files. Perhaps they can help. > > > > > > John B > > > > > > On Wed, Jan 09, 2013 at 01:59:27PM +0000, Keith Creasy wrote: > > >> John. > > >> > > >> I'm come to the conclusion that I am completely in the dark about > > >> exactly what semantics and styles do, outside a view, and it is very > > >> confusing that you use semantic action files in LibLouisUTDML and have > > >> something with the same name that does something different in the > > >> document class. If LibLouis UTDML does everything that is needed to > > >> style and format braille then what is the point? > > >> > > >> I do, of course, understand what "semantics" are and that they give some > > >> commonly understood meaning to elements. Your strategy on how they get > > >> used and where you chose to put it completely baffles me. > > >> > > >> Does anyone else but you understand this? It totally makes no sense to > > >> me. Maybe someone who does understand it can call and explain it because > > >> the comments and email just isn't doing the trick. > > >> > > >> Sorry. Maybe I'm just stupid but I'm here anyway and really do need to > > >> understand. > > >> > > >> > > >> Keith Creasy > > >> Software Developer > > >> American Printing House for the Blind KCreasy@xxxxxxx > > >> Phone: 502.895.2405 > > >> Skype: keith537 > > >> > > >> > > >> -----Original Message----- > > >> From: John J. Boyer [mailto:john.boyer@xxxxxxxxxxxxxxxxx] > > >> Sent: Wednesday, January 09, 2013 8:44 AM > > >> To: Keith Creasy > > >> Cc: Michael Whapples; John Gardner; Yuemei Sun > > >> Subject: Re: Steering: Design outline > > >> > > >> Hi Keith, > > >> > > >> We are still talking past each other. The Semantics class has very > > >> little to do with Braille markup. Everything pertaining to Braille and > > >> tactile graphics is contained in UTDML. Semantic-action files and the > > >> Semantics class in general attach meaning to markup in the document. It > > >> is analogous to how compilers attach code to tokens and constructs in a > > >> grammar. Semantics deals with the entire document. > > >> > > >> Styles are used to layout content in a wordprocessor-like view of the > > >> document. They have nothing to do with Braille, because Braille is > > >> completely described by UTDML. > > >> > > >> I see the Document wrapper class as making the semantics machinery a > > >> "black box". Other parts of BrailleBlaster would call methods in this > > >> class or trigger events which would be picked up by listeners. > > >> > > >> Taking a tree view as an example, the view would call a method that > > >> would provide the information for each element. This information would > > >> include depth. > > >> > > >> I think this matter is for the steering committee only and should not be > > >> on the public list. > > >> > > >> John B > > >> > > >> On Wed, Jan 09, 2013 at 01:07:59PM +0000, Keith Creasy wrote: > > >>> OK, that's great about the styled text view. I didn't see anything in > > >>> the docs I read about graphics or hosting controls but if it does that > > >>> then it makes everything easier. > > >>> > > >>> I don't know how it effects your renaming of the document and the > > >>> Semantics class. One thing for sure, and at the risk of beating a dead > > >>> horse, I think the Semantics stuff belongs in a Semantics class that > > >>> the document uses to add braille markup and not in the document class > > >>> itself. In other words, all the semantics functionality needs to be > > >>> encapsulated. > > >>> > > >>> How would you like to harmonize my design outline and your own plans > > >>> for implementing semantics and styles? > > >>> > > >>> -----Original Message----- > > >>> From: John J. Boyer [mailto:john.boyer@xxxxxxxxxxxxxxxxx] > > >>> Sent: Tuesday, January 08, 2013 6:53 PM > > >>> To: Keith Creasy > > >>> Cc: John Gardner; 'Yuemei Sun'; 'Michael Whapples' > > >>> Subject: Re: Steering: Design outline > > >>> > > >>> Some comments. > > >>> > > >>> StyledText can handle graphics. There are methods and events especially > > >>> for this. > > >>> > > >>> Will one of the views display the document content, including images in > > >>> a format similar to a word processor, without synchronizing with the > > >>> Braille? Styles would be appropriate here. > > >>> > > >>> Since the Braille and tactile graphics are completely specified by the > > >>> UTDML styles are neither necessary or appropriate for the Braille view. > > >>> > > >>> I am implementing the suggestion for the > > >>> org.brailleblaster.document.Document wrapper class. It should be in the > > >>> repository soon. How will this fit in with the rest of the design? > > >>> > > >>> John B > > >>> > > >>> On Tue, Jan 08, 2013 at 08:20:27PM +0000, Keith Creasy wrote: > > >>>> For sure. It is a challenge. > > >>>> > > >>>> I think the advantage is that the views are "pluggable". Unless I'm > > >>>> mistaken the SWT styled text view won't do it. If we do a good job of > > >>>> designing the document/view interface then when Yuemei designs her > > >>>> awesome editable HTML view and you design your awesome interleaved > > >>>> text/braille view it's just a matter of adding it to the list of views > > >>>> that can be opened and attached to the document. In the meantime our > > >>>> pair of styled text views gets us at least to the point of having a > > >>>> functional XML braille editor. > > >>>> > > >>>> I think it is going to be very important to define our document > > >>>> interface carefully and as completely as possible. > > >>>> > > >>>> From: John Gardner [mailto:john.gardner@xxxxxxxxxxxx] > > >>>> Sent: Tuesday, January 08, 2013 3:04 PM > > >>>> To: Keith Creasy; 'Yuemei Sun' > > >>>> Cc: 'John J. Boyer'; 'Michael Whapples' > > >>>> Subject: RE: Steering: Design outline > > >>>> > > >>>> And the plot thickens. Most math equations are actually in-line, so > > >>>> the text view would need to handle in-line graphics. The Braille view > > >>>> could get by with "stand alone" graphics, because the in-line equation > > >>>> is Braille or at least something tactile. Formatting to show math in > > >>>> the text view to correspond in position with Braille math has been a > > >>>> big challenge for Yuemei. > > >>>> > > >>>> John G > > >>>> > > >>>> > > >>>> From: Keith Creasy [mailto:kcreasy@xxxxxxx] > > >>>> Sent: Tuesday, January 08, 2013 11:57 > > >>>> To: Yuemei Sun > > >>>> Cc: John Gardner; 'John J. Boyer'; 'Michael Whapples' > > >>>> Subject: RE: Steering: Design outline > > >>>> > > >>>> Hi Yuemei. > > >>>> > > >>>> You are running a bit ahead of me here. I sure hope we can allow for > > >>>> graphics to be displayed in the text view but I'm not sure how we can > > >>>> do it initially. So the answer is, I hope so. . It's just a matter of > > >>>> writing and plugging in a different view of the document. > > >>>> > > >>>> Having the graphic available isn't a problem but implementing a > > >>>> view that can handle graphics and OLE objects is not a trivial > > >>>> undertaking. > > >>>> I do think that, using this design, the styled text view could > > >>>> be replaced with one that can do more > > >>>> > > >>>> > > >>>> From: Yuemei Sun [mailto:yuemei.sun@xxxxxxxxxxxx] > > >>>> Sent: Tuesday, January 08, 2013 2:51 PM > > >>>> To: Keith Creasy > > >>>> Cc: John Gardner; 'John J. Boyer'; 'Michael Whapples' > > >>>> Subject: Re: Steering: Design outline > > >>>> > > >>>> Would Math/Graphics be reflected in the views? The math braille would > > >>>> already be in the braille view, but the original math input would > > >>>> likely be an OLE object or a graphics image, how would it be included > > >>>> in the text view? If the XML contains graphics, will it be included > > >>>> in both views? > > >>>> > > >>>> Yuemei > > >>>> On 1/8/2013 11:29 AM, Keith Creasy wrote: > > >>>> Yes, the text and braille views should display line-by-line and be > > >>>> synchronized. > > >>>> > > >>>> Since the braille view actually uses the braille output of > > >>>> LibLouisUTDML then it would indeed have to be processed through it > > >>>> first. If we can just get the XML document and views to work though I > > >>>> think we're well on the way. At some point we'll need to add > > >>>> functionality to re-render just segments of an XML document through > > >>>> LibLouisUTDML in order to update the braille view but we don't have to > > >>>> have that to get everything else working. We do, however, have to > > >>>> design this in such a way that we can do that when we are ready. > > >>>> > > >>>> > > >>>> > > >>>> Keith Creasy > > >>>> Software Developer > > >>>> American Printing House for the Blind > > >>>> KCreasy@xxxxxxx<mailto:KCreasy@xxxxxxx> > > >>>> Phone: 502.895.2405 > > >>>> Skype: keith537 > > >>>> > > >>>> From: Yuemei Sun [mailto:yuemei.sun@xxxxxxxxxxxx] > > >>>> Sent: Tuesday, January 08, 2013 2:23 PM > > >>>> To: John Gardner > > >>>> Cc: Keith Creasy; 'John J. Boyer'; 'Michael Whapples' > > >>>> Subject: Re: Steering: Design outline > > >>>> > > >>>> > > >>>> I have the same question as John G. about the "text" view. If the > > >>>> "text" view is to have the option of having it formatted to match the > > >>>> braille view line by line, the XML Dom Document will need to have > > >>>> UTDML info in it, which specifies the mapping from texts to braille. > > >>>> In that case, the semantic action would have been applied to the > > >>>> original XML document. > > >>>> > > >>>> Yuemei > > >>>> On 1/8/2013 10:31 AM, John Gardner wrote: > > >>>> Keith, what is the "text" view? Does one have the option of having it > > >>>> formatted by utdml to match the Braille view? My thought is that one > > >>>> could have a menu option to have text automatically formatted so that > > >>>> each line corresponds to the Braille, or to have it formatted > > >>>> normally/Braille-match on some toggle command. That would be elegant. > > >>>> I'm concerned about having too many views. So a non-expert would > > >>>> presumably be able to have only two views - braille and text. An > > >>>> expert could also look at the XML tree and/or a XML document formatted > > >>>> somehow to show structure. > > >>>> > > >>>> I don't understand most of the detail, so I'll ask Michael and Yuemei > > >>>> to comment. I'm copying Yuemei on this with the note that it is > > >>>> steering committee business, not for the full list. > > >>>> > > >>>> Thanks. > > >>>> John G > > >>>> > > >>>> > > >>>> From: Keith Creasy [mailto:kcreasy@xxxxxxx] > > >>>> Sent: Tuesday, January 08, 2013 09:32 > > >>>> To: John J. Boyer; John Gardner; 'Michael Whapples' > > >>>> Subject: Steering: Design outline > > >>>> > > >>>> Gentlemen. > > >>>> > > >>>> I'm not ready to make this public but thought I'd send it to you so > > >>>> that you are aware of where my thinking is going at this point. Also, > > >>>> you may have ideas and suggestions. It certainly isn't final or > > >>>> definite but we are at least discussing it among ourselves here at APH > > >>>> in order to get going in what hopefully is a helpful direction. See > > >>>> below. > > >>>> > > >>>> > > >>>> Braille Blaster document/view Design > > >>>> > > >>>> One of the factors in the design of BrailleBlaster is the ability to > > >>>> edit text, update an XML DOM document, and also update all the views > > >>>> of that document dynamically. When text or braille changes, or when a > > >>>> new node in the XML is selected the braille, text, tree view, and/or > > >>>> HTML view have to likewise be updated. To deal with this I am > > >>>> proposing a solution with these key elements: > > >>>> > > >>>> 1. The option of attaching at least four views to a single XML > > >>>> Document; braille, text, structure tree, and HTML. > > >>>> 2. A map that matches each text node in an XML document with the print > > >>>> offset where that text node starts. Synchronizing the braille version > > >>>> is also involved. > > >>>> 3. A queue of events that is managed by the document so that when a > > >>>> view changes the text or selection the document can be updated in the > > >>>> order in which these updates occurred. > > >>>> 4. a method in the document postEvent that is called by any view that > > >>>> needs to update the XML content. > > >>>> 5. Each view implements methods onUpdateText and onUpdateSelection as > > >>>> needed so that the contents of that view can update. > > >>>> > > >>>> The scope in this portion of the design is to deal with the process of > > >>>> editing and updating an XML dom and dynamically updating and > > >>>> synchronizing several views of that document. We won't deal with > > >>>> applying styles to braille or print or using the "semantic action > > >>>> files (.sem)". > > >>>> > > >>>> > > >>>> Document class > > >>>> > > >>>> The document class wraps the XOM XML document and implements, for the > > >>>> purposes of the user interface, methods that handle events fired from > > >>>> the various views. These might include: > > >>>> > > >>>> postEvent (We could add convenience methods like > > >>>> postUpdateTextEvent) updateViews setSelection updateText > > >>>> insertNode splitNode mergeNode deleteNode attachView detachView > > >>>> translateNode readXML writeXML > > >>>> > > >>>> > > >>>> > > >>>> Also, some "getter" methods that the views can use: > > >>>> > > >>>> getText > > >>>> getBraille > > >>>> getInsertionPoint > > >>>> getSelection > > >>>> getNode > > >>>> > > >>>> > > >>>> There may be others and the precise arguments required are yet to be > > >>>> defined. > > >>>> > > >>>> The document maintains a "cursor" that points to a specific > > >>>> text node in the XML document and a corresponding offset. There > > >>>> is also a map of the text nodes with a start offset so that one > > >>>> and only one position in the textual content is considered the > > >>>> current insertion point. The document also maintains a > > >>>> "selection" that represents a range of text either before or > > >>>> after the current cursor position > > >>>> > > >>>> > > >>>> EventHandler > > >>>> > > >>>> The event handler queues events received through the Document's > > >>>> postEvent method. Events are queued in a FIFO order and are tied to a > > >>>> timer that controls the rate at which events are processed. The > > >>>> handler calls the methods in the document that actually make changes > > >>>> to the XMLDOM. The event handler runs in its own thread and only it > > >>>> has access to change the document. > > >>>> > > >>>> The postEvent method does two things. It adds the event to the end of > > >>>> the queue and resets a timer that controls when the event queue gets > > >>>> processed. When the timer expires then a "processEvents method is > > >>>> called and all pending events are processed and applied. > > >>>> > > >>>> Processing events > > >>>> > > >>>> Events are processed in FIFO order. If a sequence of events can be > > >>>> merged; i.e. several individual character insertion events are > > >>>> consecutive, they are merged before being processed. In this way the > > >>>> number of actual changes to the XML DOM is reduced as well as the > > >>>> number of updates to the views. > > >>>> > > >>>> Events, after processing, include an identifier to indicate which view > > >>>> requested the change. The updateViews method in the document is called > > >>>> that notifies the views of a change and each view updates it's content > > >>>> accordingly. > > >>>> > > >>>> > > >>>> Proposal summary > > >>>> > > >>>> I propose that APH developers begin work on this architecture to > > >>>> dynamically edit XML documents including text, braille, and changes to > > >>>> the XML document structure. This is independent of Abilitiessoft' > > >>>> (John Boyer's) work on Semantic Action And styles. We plan to focus > > >>>> only on editing text and managing an XML DOM in a dynamic way using > > >>>> all four views. Our intention is to incorporate John's work at some > > >>>> future point when both efforts are developed further. > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> > > >>>> ________________________________________ > > >>>> > > >>>> Yuemei Sun | Software Engineer | > > >>>> > > >>>> 541.754.4002 ext. 229 | > > >>>> www.viewplus.com<http://www.viewplus.com> > > >>>> > > >>>> > > >>>> > > >>>> ________________________________________ > > >>>> > > >>>> PRIVILEGED AND CONFIDENTIAL: This message and any files transmitted > > >>>> with it may be proprietary and are intended solely for the use of the > > >>>> individual to whom they are addressed. If you are not the intended > > >>>> recipient, any use, copying, disclosure, dissemination or distribution > > >>>> is strictly prohibited; please notify the sender and delete the > > >>>> message. ViewPlus Technologies, Inc. accepts no liability for damage > > >>>> of any kind resulting from this email. > > >>>> > > >>>> > > >>>> -- > > >>>> > > >>>> ________________________________________ > > >>>> > > >>>> Yuemei Sun | Software Engineer | > > >>>> > > >>>> 541.754.4002 ext. 229 | > > >>>> www.viewplus.com<http://www.viewplus.com> > > >>>> > > >>>> > > >>>> > > >>>> ________________________________________ > > >>>> > > >>>> PRIVILEGED AND CONFIDENTIAL: This message and any files transmitted > > >>>> with it may be proprietary and are intended solely for the use of the > > >>>> individual to whom they are addressed. If you are not the intended > > >>>> recipient, any use, copying, disclosure, dissemination or distribution > > >>>> is strictly prohibited; please notify the sender and delete the > > >>>> message. ViewPlus Technologies, Inc. accepts no liability for damage > > >>>> of any kind resulting from this email. > > >>> -- > > >>> John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc. > > >>> http://www.abilitiessoft.com > > >>> Madison, Wisconsin USA > > >>> Developing software for people with disabilities > > >> -- > > >> John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc. > > >> http://www.abilitiessoft.com > > >> Madison, Wisconsin USA > > >> Developing software for people with disabilities > > > > -- > > ________________________________________ > > Yuemei Sun | Software Engineer | > > 541.754.4002 ext. 229 | www.viewplus.com > > > > ________________________________________ > > PRIVILEGED AND CONFIDENTIAL: This message and any files transmitted with it > > may be proprietary and are intended solely for the use of the individual to > > whom they are addressed. If you are not the intended recipient, any use, > > copying, disclosure, dissemination or distribution is strictly prohibited; > > please notify the sender and delete the message. ViewPlus Technologies, > > Inc. accepts no liability for damage of any kind resulting from this email. > > -- > John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc. > http://www.abilitiessoft.com > Madison, Wisconsin USA > Developing software for people with disabilities -- John J. Boyer; President, Chief Software Developer Abilitiessoft, Inc. http://www.abilitiessoft.com Madison, Wisconsin USA Developing software for people with disabilities