Re: The Syllable/Short Term Memory Problem
- From: "Andreas Stefik" <stefika@xxxxxxxxx>
- To: programmingblind@xxxxxxxxxxxxx
- Date: Mon, 11 Feb 2008 10:54:08 -0800
cdh,
Have you looked at Robert Steven's work on MathTalk as well? He went
to great lengths to map out the kind of prosodic cues people naturally
use when reading mathematical symbols, which he incorporated into
MathTalk, his reader. His dissertation maps it out in great detail if
you haven't read it, it's quite interesting.
With programming, your argument is spot on. In my compiler for the
blind, called the Sonified Omniscient Debugger(SOD), I break up all of
the compiler's operations into atomic units, then have a series
programmatic entities that can abstract these operations, in various
ways, in various contexts (Debugging, editing, etc). I've been doing
usability testing on the thing for years, and getting the auditory
cues just right is significantly difficult. Sometimes you can change
the sound that gets output by just a few words, or have a bad prosodic
cue, and it will cause total havoc for the user. The whole process is
beautifully complex and been a fun research project.
Andreas
On Feb 11, 2008 3:47 AM, Chris Hofstader <chris.hofstader@xxxxxxxxxxx> wrote:
> Hi,
>
> Mathew adds some reasonably good ideas but most have already been tried.
> The last one, using a Braille display is, unfortunately, the least practical
> in this part of the world because, in the US where such items are certainly
> "expensive" they are not so costly as to be out of reach for institutional
> buyers. The problem, though, is that according to the Braille Institute,
> less than 12% of US citizens with total to profound vision impairment can
> be described as having actual literacy in any of the more useful tactile
> writing systems. Many blinks, like me for instance, can "unspell" Braille
> information which is helpful for CD labels and such but not useful in any
> kind of system that requires actual literacy.
>
> In the math reader/editor project I'm working with Pratik and some others on
> for CUNY Research Foundation, we plan on adding something similar to
> Mathew's idea of reading by quadrant but (hopefully) it will be a bit more
> intelligent. Our MathML parser (which is currently hanging off of Jamal's
> ED Sharp which gave us a great head start) has a number of heuristics to
> turn certain mathematical constructs into "parts" and "groups" (a part is a
> container of a single chunk of an equation and a group contains one or more
> parts) so, in a long equation, if such grouping is turned on, the user can
> hear,"x = numerator over denominator + numerator over denominator and can,
> via keystroke expand just the denominators to to work out an LCD, make a
> scratch note, close the denominators and adjust the numerators accordingly.
> This bit is highly recursive so a denominator can contain more numinators
> and denominators and so on.
>
> More ambiguous information, like virtually anything between parenthesis in
> an equation can be broken up into "part1," "part2," "group1," and so one
> where parts 1 and 2 are contained by group 1. Thus, a user can hear
> numerator over denominator and, if the numerator has only one group it will
> open up into "part1 + part2 over denominator," if there are two or more
> groups and the user opens the numerator they will hear, "group1 + group2
> over denominator," and so on. We hope to add the ability to allow the user
> to name groups and parts in a manner that is more meaningful to them, sort
> of like C macros or C++ inlines, but we're still working on getting the
> parser to play nice with everything else.
>
> The major problems with doing this kind of thing with a programming language
> is that one often needs to look at more than one line at a time to
> understand some kind of appropriate context. I suppose someone could build
> a function or method translator that could reduce the items in said function
> into parts and groups but this would have some really troublesome situations
> regarding globals. This would sort of be a reverse compiler that turned
> code into pseudo code that could be expanded as described above or
> translated and restored entirely with a single click.
>
> Our work is not entirely unique and very large parts of it are based on
> Raman's paper on his Aster system for modeling math in speech to whom we are
> quite grateful. Our additions to the system use far more sound and 3D
> effects (which weren't really available when Raman first published his work)
> to (hopefully) even further improve the efficiency with which a non-visual
> interface can expose math.
>
> I hope this is on topic and of interest to someone other than me.
>
> Enjoy,
> cdh
>
>
>
> -----Original Message-----
> From: programmingblind-bounce@xxxxxxxxxxxxx
> [mailto:programmingblind-bounce@xxxxxxxxxxxxx] On Behalf Of Matthew2007
> Sent: Sunday, February 10, 2008 4:30 PM
> To: programmingblind@xxxxxxxxxxxxx
> Subject: Re: Some stories
>
> Couldn't you solve some of these working memory issues by creating a script
> or manner by which the programmer could execute a keyboard command that will
>
> repeat the current line of code at a desired speed? That is, let's say
> someone hands the blind programmer a lengthy line of code to review, to
> overcome the classic "Magical Number 7 + or - 3" and similar character
> limitations of human memory, why not allow the programmer to hit a hot key
> such as control+l to read the current line again, and if this keyboard
> command is pressed again, the line will read yet slower and on and on until
> the line of code is reviewed thoroughly and understood. After the line of
> code is read in this manner the screen reader will automatically return to
> its default reading speed. If you want a little more information you might
> want to look at the jaws spellcheck function in word which will allow the
> user to slow down the speed at which errors in spelling are announced.
>
> Now that its popped into my head....
>
> How about breaking-up the line of code into quadrants and using hotkeys such
>
> as control+1, control+2, control+3, and control+4 to allow independent
> review of 4 sections of a line of code rather than waiting for the entire
> line to be read. That is, if the programmer already knows the leading
> characters that might stay constant from line to line, he or she might not
> want to hear that section of the code as it is repetitive and unnecessary
> clutter in one's memory, and the programmer might opt to only review the
> second and third quadrant of code via hotkeys. This too should help overcome
>
> the old problems with primacy and recency effects in human memory.
>
> Another option would be to simply use a Braille display to verify one's
> work, but this solution is very costly and probably time consuming.
>
> Matthew
>
> ---- Original Message -----
> From: "Andreas Stefik" <stefika@xxxxxxxxx>
> To: <programmingblind@xxxxxxxxxxxxx>
> Sent: Sunday, February 10, 2008 11:27 AM
> Subject: Re: Some stories
>
>
> > Thanks cdh,
> >
> > It might be interesting to note that the problems you mention are
> > exactly what I've been working on in my debugger, minimizing syllables
> > and reducing short term memory requirements. I designed a whole
> > compiler underneath the surface that tries to intelligently handle
> > these things, and I've been running human studies to figure out what
> > works best under what situations.
> >
> > And, if I've learned anything during the process, it's that you are
> > right. In fact, I might go so far as to say that, from the data I've
> > been able to gather so far, that short term memory issues are one of
> > the most, if not the most, significant issues.
> >
> > Andreas
> > __________
> > View the list's information and change your settings at
> > //www.freelists.org/list/programmingblind
> >
> >
> > __________ NOD32 2861 (20080209) Information __________
> >
> > This message was checked by NOD32 antivirus system.
> > http://www.eset.com
> >
> >
>
> __________
> View the list's information and change your settings at
> //www.freelists.org/list/programmingblind
>
>
>
> __________ NOD32 2863 (20080211) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>
> __________
> View the list's information and change your settings at
> //www.freelists.org/list/programmingblind
>
>
__________
View the list's information and change your settings at
//www.freelists.org/list/programmingblind
Other related posts:
