It's worth noting that AudioScreen was primarily intended to be used
with a touch screen. You can sonify an entire object, but if you want to
explore it bit by bit, the most natural way to do this is using a touch
screen and AudioScreen provides for this.
On 13/04/2016 1:27 AM, Grzegorz Zlotowicz wrote:
Hello.
I know thevoice program, I tested it on PC and even old good Symbian version...
Regarding the zoom level, you have only two options in voice - zoom and no zoom.
The sonify navigator object in Audioscreen gives you the view of whole image, so no way of going into more or less detailed view depending on needs.
Ideal situation would be if user would have a possibility of seeing the whole image at first, and then explore it in more details as needed, and preferrably using different presentations (sound, speech and braille) - I described it yesterday.
About Seeing with sound, I read somewhere, that well trained users could "see" that on the image apple lies on the table; nice, but unanswerred question is, if they're able to differentiate apple from orange or watermelon, and if it was only one, or one of many other objects on this table.
By the way - I tested the earth plus program, and I'm impressed how such serious institution could produce such poor implementation; as I read, the project's leader, prof. Shelton, is himself blind, so I expected more usable solutions in the software.
User can define bookmarks, and organise it in ways, but no svg support, and no way of different zoom levels.
One way of navigating is pixel by pixel, or between grid lines, but the grid lines are placed in arbitrary positions, so I even don't know if after going to next grid, I missed some fragment of image, or only the background pixels - btw, interesting how many thousands of dollars this development was paid.
So, after about an hour, when I tried to find some usable element of image, but was only touching the background, I gave up.
Greetings, Greg.
W dniu 12.04.2016 02:16, Pranav Lal napisał/a:
Hi Greg,
All good ideas. Audio screen is based on the vOICe which you can download for free from http://www.seeingwithsound.com. You can zoom an image and manually walk through it using the arrow keys.
In what program are you viewing the picture? You can sonify the entire picture under the navigator object by hitting nvda key +alt+a.
Pranav
-----Original Message-----
From: nvda-addons-bounce@xxxxxxxxxxxxx [mailto:nvda-addons-bounce@xxxxxxxxxxxxx] On Behalf Of Grzegorz Zlotowicz
Sent: Monday, April 11, 2016 6:12 PM
To: nvda-addons@xxxxxxxxxxxxx
Subject: [nvda-addons] AudioScreen addon questions and suggestions
Hello.
I just listened to the AudioScreen [1] demonstration, and did some basic testing myself.
First of all, thanks to Mick for trying to solve the image-viewing issue, as we all know, it's important problem, but hard to solve.
My question: is it possible, after longer training, to view electronic diagrams? After my one hour testing, no success, but maybe I should be more patient.
For example, have a look at simple SR flip-flop diagram [2].
This diagram consists of two nand gates.
Each gate has two inputs and one output.
Each gate's output is connected to the input of other one.
Looking at this image as a whole, is impossible for me in audioScreen at the moment, but - as I said - maybe it's lack of training.
I looked at it using optacon, which I just succeded to configure to work with my laptop's screen in a nice way - due to lack of training in using optacon too, it took me a long time to view the image, but it was doable.
But, I thought about alternative approaches to image viewing, which I'd like to discuss here; maybe they could be developed as an alternative modes of audioscreen or a separate add-on.
Mode 1.
The image in question (e.g. navigator object) is loaded into add-on buffer, and in the view mode, user can use arrow keys to move inside of it by tiles (up/down/left/right).
One tile is initially some part of the image, let say - 10% of it.
This initial look around, can be helpful to identify the colour of image, if it's light on dark or dark on light.
Tile size can be increased (up to whole image) and decreased (up to one pixel accuracy).
Tile value, as a gray-scale colour, can be read by the synthesizer or played by the "seeingwithsound.com" algorithm.
If the tile consist of many pixels - an average of the colour is read as a number.
User can label some parts of image e.g. after checking that somewhere is the "r" letter, can label the area of this letter as "r".
When viewing the labelled part of image, given label is read aloud independent of the tile size (so on small tiles user can identify letter, and then have it read aloud when viewing image on greater tiles).
Mode 2. Using braille display:
When we select smallest representation of above SR flip-flop image, it has 200x125 pixels.
It means, that on a 32-cell braille monitor, it would occupy 3 lines (63
characters) horizontally and 32 lines vertically in full accuracy.
As in previous mode, one tile, represented here by one braille dot, could be resized.
After increasing it 3 times of original - everything fits in 21 characters and 11 lines - it seems enough to read the image in pretty short time.
4-times increase would give the size of 16 characters in 8 lines - possibly too much to catch small details, but nice for having a quick preview...
The same for higher levels.
Routing keys could be used to move virtual "cursor" to interesting part of image, e.g. when on high zoom I find the unidentified object, possibly letter, I can route the cursor to its' left up corner, then switch the zoom to less one, view the fragment in more details, and then label it (routing cursor keys and rewind of display could be helpful to mark the label area).
The histogram analysis could be used to determine which value should be used as an trigger of braille dot - the histogram statistics could be presented to the user in both modes, after loading of image.
Labeling of rectangular fragments is simple, but what about labelling the lines which aren't horizontal or vertical, and are intersected by other objects?
So at the beginning, maybe labels for points would be sufficient.
And as additional helper - a list of bookmarks function, where user can easilly switch to previously labelled point.
I used this approach in my Grmapa program [3], and can say, that combination of zooming and bookmarking allows for pretty efficient viewing of complicated objects.
Of course, the OpenStreetMap objects are described itself in machine-readable way, a raw image is not, so it would be user's work to do it.
Mode 3. The SVG view.
The SVG format seems a most accessible one when talking about images, because of fact, that everything there is described as a concept, not group of pixels.
So, we have areas, paths, and text.
When we open in notepad the keypad, or house images (included in audioscreen demo images archive), we can easily read the images.
For more complicated ones (map of Australia or the above mentioned SR flip-flop), notepad reading is not enough.
This is probably idea of separated program, not a NVDA add-on, but the svg viewing could became more accessible using the Grmapa approach.
Each object in SVG can be easily identified, so machine knows, that some fragment of image is a line going from point a to point b.
If there is some text, no ocr is required to recognize it, because the text is placed as text.
Some fragments of the image can have labels (Australia regions areas), some may require the user interaction after investigating the line (SR flipflop).
Some paths can encircle the area, without declaring it (nand gates on a schematic), so user should be able to manually convert line to area and label it.
Main problem is the SVG rendering into the programs' internal buffer, but the problem is well solved in other applications.
I'm seriously thinking about doing something like this as my engineering thesis, but I recall that I read about some tool developed by (NASA, Microsoft, or some other entity), which had to help blind to "see"
scientific diagrams, but can not find it.
Maybe it was the Earth plus, and math trax programs [4], which I just found.
Did somebody use it and can share some thoughts?
Oops, and I somehow went into off-topic, sorry for that, but from the other side, mentioned things are closely related to the main subject, and can bee an inspirration for further add-on development.
[1] https://github.com/nvaccess/audioScreen
[2] https://en.wikipedia.org/wiki/File:SR_Flip-flop_Diagram.svg
[3] http://grmapa.zlotowicz.pl
[4]
http://www.nasa.gov/audience/foreducators/helping-blind-see-math-science.html
Greetings, Greg.
----------------------------------------------------------------
NVDA add-ons: A list to discuss add-on code enhancements and for reporting bugs.
Community addons are available from: http://addons.nvda-project.org To send a message to the list: nvda-addons@xxxxxxxxxxxxx To change your list settings/unsubscribe: //www.freelists.org/list/nvda-addons
To contact list moderators: nvda-addons-moderators@xxxxxxxxxxxxx
----------------------------------------------------------------
NVDA add-ons: A list to discuss add-on code enhancements and for reporting bugs.
Community addons are available from: http://addons.nvda-project.org
To send a message to the list: nvda-addons@xxxxxxxxxxxxx
To change your list settings/unsubscribe: //www.freelists.org/list/nvda-addons
To contact list moderators: nvda-addons-moderators@xxxxxxxxxxxxx
----------------------------------------------------------------
NVDA add-ons: A list to discuss add-on code enhancements and for reporting bugs.
Community addons are available from: http://addons.nvda-project.org
To send a message to the list: nvda-addons@xxxxxxxxxxxxx
To change your list settings/unsubscribe: //www.freelists.org/list/nvda-addons
To contact list moderators: nvda-addons-moderators@xxxxxxxxxxxxx
