[jawsscripts] Re: Jaws reading mixed up info when doing SayAll

  • From: "Jon Squally" <jonsqually@xxxxxxxxx>
  • To: <jawsscripts@xxxxxxxxxxxxx>
  • Date: Tue, 16 Feb 2010 22:43:32 -0800

Hello,

Take a look t the following URL for an introduction to ARIA.
http://www.w3.org/WAI/intro/aria
HTH, JOn.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Mark A. DeMichele
Sent: Tuesday, February 16, 2010 04-52 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

What exactly is ARIA?  Is there somewhere I can read about it.  It's the
first time I've seen this acronym.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Travis Roth
Sent: Monday, February 15, 2010 11:32 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

I understand your issues, especially the not reading automatic updates.
That's a tough one for a screen reader to implement too, to be fair.
When a web page does a total refresh content from a Flash object will be
repopulated. But for dynamic updates its probably not going to read it
automatically. That's true of dynamic HTML as well with the exception of
live regions, browsers that support them and JAWS 11.

To that end, it may be an interesting experiment to see if a live region
compatible combination would trigger reading of a Flash/Flex object that's
contained within the region.
This would assume to that the entire flash object would want to be read, not
only part of it.
Not sure, but hey you never know...

(Live regions are part of ARIA and define regions on a web page that are
dynamically updated. They also define what should occur when an update
occurs, such as read it or ignore it. Firefox 3.x and IE 8 and JAWS 11 have
the best support for them thus far.)


-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Mark A. DeMichele
Sent: Monday, February 15, 2010 6:37 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

Thanks for this all the info.  I will try some more stuff and get back to
everyone.

Just so you know, I am using the latest flex scripts.  My scripts are on top
of those.   Things I'm trying to fix are mainly the following.

1.  Dialog boxes don't work out of the box even with the scripts.  They
aren't real windows and that's probably why.
2.  Jaws doesn't automatically read new content when I add it to my
interface.
3.  If I change "documents" in my interface, jaws doesn't read it
automatically.
4.  When my flex app loads it makes a URL request to get the document it
displays.  Since this happens asynchronously, jaws thinks that initially the
flash player has nothing in it and just says Flash Start Flash End.  My fix
for 3 above seems to somewhat address this.

These are the most important issues I'm facing right now.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Travis Roth
Sent: Sunday, February 14, 2010 3:08 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

Hmm I see that MoveToLandmark() only navigates to them first, next, previous
or last. Can't move to a specific one. Still if you only have one landmark
it'd work fine.
I.e. in your HTML:

<div role="main"> ... all stuff </div>

In JAWSScript
MoveToLandmark(S"NEXT) ; moves the virtual cursor to next landmark on page.

No, not all functions are defined in the script files, there are many that
are internal. Also there are functions that are available in .JSB files that
are otherwise not documented, others are more experts on finding those than
me. (Not all .jsb files have their corresponding .jss files included,
unfortunately.)

Yes I think that when JAWS builds the virtual buffer contents, it switches
to MSAA to gather Flash objectinformation. It needs to is that's how Flash
communicates to AT. The virtual buffer is a "flat" representation of what
JAWS sees on the webpage. It makes stuff "flat" vs. dealing with a
hierarchical situation which HTML web pages are (they're trees).

I don't think that the Flash window handle will be helpful here as that gets
JAWS thinking to much in OSM terms. Moving to the Flash object inside the
virtual buffer you're best off focusing the JAWS virtual cursor with one of
the MoveTo functions such as MoveToLandmark or MoveToTag().
The other way to focus on specific objects on a web page is to use
JavaScript in the web page. 
<body onLoad="document.getElementById('MyObjectID').setFocus();">

Or something similar. That should work regardless of if JAWS is in virtual
buffer mode, or not.

Btw yes SayObjectTypeAndText() is intended as an MSAA function. It is called
when focus is moved generally, such as when tabbing through an application.
I think in software that if MSAA isnt' found JAWS switches back to the OSM
methods it knows so well. But that should not be an issue in Flex on a web
page. (In case you're wondering, yes I do believe JAWS is not the best MSAA
consumer at times.)

Btw, Adobe and SSB teamed up to make some Flex scripts available. You can
find them with a Google search. I have not been able to determine any
difference in JAWS behavior with or without the Flex scripts, but that could
easily because I haven't understood what problem(s they're trying to fix.

Not sure this helps but maybe it'll cause an idea to occur for you.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Mark A. DeMichele
Sent: Sunday, February 14, 2010 10:25 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

I have another clue.  I just noticed that normally, my flash objects are
"said" from the function SayObjectTypeAndText().  However, when I do the
TopOfFile(), SayAll(), the objects are Said, but not through the
SayObjectTypeAndText().  This has a bad side affect in that it doesn't say
them the way I want since I override this function and do other things.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Mark A. DeMichele
Sent: Sunday, February 14, 2010 10:52 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

Actually, I think you're understanding of my issues are pretty good.

I have the window handle to the flash player object and I also have the MSAA
object pointer and child ID of the client object for the flash player.  Do
you think I can use that to "work with the virtual buffer".

Also, it's not clear to me how to make use of the virtual buffer.  I
searched for MoveToLandmark() and only found it reference in two places in
Virtual.jss.  I expected to find its definition in some JSS file, but I
didn't.  I was under the impression that all functions were defined in
script files somewhere, but I guess Jaws handles some internally?

Here's a theory I have, please let me know if you think my understanding of
things is off or not.

When I do JawsTopOfFile() and then SayAll(), IE is doing the "Saying" and
when it hits my flash player, it's somehow smart enough to switch to MSAA to
get the info and "say" that as well.

However, when I point directly to the flash window, I lose those smarts and
it tries to say it based on position of the items in it (but those items are
MSAA so I'm a little confused here).  Not only that, it seems to be getting
it's info from some buffer that was previously filled with stuff that's not
current.

The question is how to I get it to a) use MSAA to say stuff and b) clear
anything it might have had buffered.

I tried adding a Refresh(), Delay(2) (I saw this in another script), but it
didn't seem to help.  I also tried add in and MSAARefresh(), but I think
according to the documentation that only refreshes the currently focused
object.

Any help or insight, of just pointing me a good examples would be greatly
appreciated.  Thanks so much for the time you already spent answering my
questions.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Travis Roth
Sent: Sunday, February 14, 2010 10:20 AM
To: jawsscripts@xxxxxxxxxxxxx; 'Jamal Mazrui'
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

Hi Mark,

As for JAWS remembering stuff that isn't there: perhaps JAWS isn't updating
its off screen model (OSM) in a timely manner. Try a screen refresh to see
if that forces some improvement. (JAWSKey+Escape, or refresh() function.)

As for SayWindow() I suspect that JAWS is trying to do exactly what you ask
of it: read the contents of the window.This function is based on JAWS' idea
of what the window has in it based on its OSM. I don't believe that MSAA is
used. You are correct, JAWS has two modes of working: the window/OSM way,
and MSAA. And they do not seamlessly work together all of the time.
You will want your Flex application to use MSAA, so you will need to use
JAWS' MSAA functions. If you can get it to work with the virtual buffer
enabled, the best bet for moving to your Flex application is to use one of
the functions for navigating the virtual buffer, such as MoveToLandmark()
(if you have landmarks in your HTML.) Or perhaps MoveToTag() looking for the
"object" tag.

If I misunderstood your goal of navigation, please try again.

-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Mark A. DeMichele
Sent: Sunday, February 14, 2010 7:58 AM
To: Jamal Mazrui; jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Jaws reading mixed up info when doing SayAll

I did try them and it doesn't help.  It seems that jaws has two modes of
"Saying".  Sometimes it seems to do based on the MSAA structure and then
other times it seems to do it based on physical position on the screen.
One thing I noticed is that I have a navigation bar at the top.  I
purposely put that at the end of my MSAA tree.  That way, normally jaws
says it last.  However, when I do the SayWindow or MoveToWIndow and
SayAll, it will read the buttons in the navigation bar first.  Maybe
that's a clue. I don't know.

I looked through the IE scripts to see if anything funky is going on in
the TopOfFile, but I didn't see anything.  Why would Jaws still remember
items that are not there anymore.  It's kind of strange.

-----Original Message-----
From: Jamal Mazrui [mailto:empower@xxxxxxxxx] 
Sent: Sunday, February 14, 2010 8:53 AM
To: jawsscripts@xxxxxxxxxxxxx
Cc: Mark A. DeMichele
Subject: Re: [jawsscripts] Jaws reading mixed up info when doing SayAll

Try the Refresh and MSAARefresh functions in case they make a
difference.

Jamal


On 2/13/2010 6:39 PM, Mark A. DeMichele wrote:
> I have an odd issue.  I'm adding some custom script on top of the
latest
> Flex/Flash scripts to get a flex application to do some special stuff
in
> IE.  My flex app essentially displays a "document".  It has a
navigation
> area which allows the user to change to another "document".  I wrote
> some scripts to sense that change and when that happens I want jaws to
> say the new document.  Here's the odd part.  I have the window handle
to
> the flash window within IE.  If I do something like the following,
>
>
> MoveToWindow(hwnd)
>
> JawsPageUp()
>
> JawsHome()
>
> SayAll()
>
>
>
> Or just
>
>
>
> SayWindow(hwnd)
>
>
>
> It does two things I can't explain.  First, it reads the stuff from my
> app in "screen order" (i.e., top to bottom, left to right), not in the
> MSAA order I see the stuff in the MS Accessibility Explorer.  Also,
and
> this is worse, it's saying a garbled mess of stuff from all the
> documents I loaded since then mix in with the current one.  It's as if
> it's buffering stuff based on screen location and it's not ever
getting
> rid of it.  I used the AccEvent program to check that things were
> getting destroyed and it seems they are, but for some reason, Jaws
wants
> to read them.
>
>
>
> If, however, I use this code
>
>
>
> JawsTopOfFile()
>
> SayAll()
>
>
>
> Then jaws reads everything as expected (no old document info).
However,
> it says the entire IE window contents which I don't really want.  But
> the order is fine.
>
>
>
> Is there something I'm doing wrong?  I'm a seasoned experienced
> programmer, but I'm new to Jaws scripts so any technical information
or
> background would be greatly appreciated.  So far, I read both the
basic
> scripting manual and most of the FSDN manual, and to be honest, I'm
> learning the most by reading through the scripts that were delivered
> with jaws.  If anyone knows any other places to get good technical
info,
> that would be great.
>
>
>
> Thanks,
>
> Mark
>
>
> __________
> Visit and contribute to The JAWS Script Repository
http://jawsscripts.com
>
> View the list's information and change your settings at
> //www.freelists.org/list/jawsscripts
>
>
__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts


__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts

__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts

__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts


__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts

__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts


__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts

__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts

__________ 
Visit and contribute to The JAWS Script Repository http://jawsscripts.com

View the list's information and change your settings at 
//www.freelists.org/list/jawsscripts

Other related posts: