Udo & Brian,
Browser content is out of the UIA box. The HTML content is rendered by
JAWS to virtual document (which we navigate on) and its elements can be
accessed by JWS Script HTML commands like SelectHTMLElement,
StepToStartOfelement, GetDocumentXML, etc. Also the controls displayed
in a HTML page are part of these XML tree. So to navigate on it use
rather HTML/XML commands than UIA's.
HTH
Chaba
2020. 10. 20. 6:20 keltezéssel, Udo Egner-Walter írta:
Hi Bryan,__________�
I think this depends on the browser you use. In Edge (Chromium) and Chrome I
think you can find it in the UIA tree in Firefox you should use other methods.
Udo
Am 20.10.2020 um 00:12 schrieb Bryan Schulz (Redacted sender "b.schulz" for
DMARC) <dmarc-noreply@xxxxxxxxxxxxx>:
Hi,
Is a normal textbox on a website a FSUIA Element?
Bryan
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx <jawsscripts-bounce@xxxxxxxxxxxxx> On
Behalf Of Theodore Cooke
Sent: Monday, October 19, 2020 4:59 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Learning to find information with script utility
mode
Udo, why don't you use the FSUIAElement method getClickablePoint instead of
adding +5 to the left and top edges of the bounding rectangle? Is your
method more reliable?
On 7/23/20, Udo Egner-Walter <udo.egner-walter@xxxxxx> wrote:
Hi Theodore,__________
Glad I could help.
One more thing about the invoke command: Here's a litte code sample
that helped me most of the time. It makes a mouse click at the UIA
alement. The small disadvantage is a little delay.
<Code Start>
; click at UIA element with given AutomationID void Function
ClickAtAutomationID (string sAutomationID) var object oUIA, object
oElement, object oCondition, object oRect, int x, int y
oUIA = CreateObjectEx ("FreedomSci.UIA", 0, "UIAScriptAPI.x.manifest"
) oElement = oUIA.GetElementFromHandle (GetAppMainWindow(GetFocus()))
If !oElement then Return EndIf oCondition =
oUIA.CreateStringPropertyCondition (UIA_AutomationIDPropertyId,
sAutomationID)
If !oCondition then Return Endif
oElement = oElement.FindFirst (TreeScope_Descendants, oCondition)
If oElement then
oRect = oElement.BoundingRectangle
x = oRect.Left + 5
y = oRect.Top + 5
If x > 5 && y > 5 then
ClickAtPoint (x, y, True)
EndIf ; If x > 5 && y > 5 then
EndIf ; If oElement then
EndFunction
<Code end>
Udo
Am 23.07.2020 um 17:58 schrieb Theodore Cooke <theodorecooke@xxxxxxxxx>:
Before I forget, Udo, thank you for your code samples. Your script
from that other thread is what I used as a basis for my script, but I
trimmed it down so much that it now looks more like the script you
wrote in here. I appreciate all your effort to copy/write out those
code samples.
On 7/23/20, Theodore Cooke <theodorecooke@xxxxxxxxx> wrote:
Yes. That is exactly what I did. I tried calling the EnterKey__________
function, but that was opening the directery element with focus, for
some strange reason. I guess the EnterKey function activates the
element with the PC curser focus. Then I tried TypeKey("space"), and
that did the job just fine.
On 7/23/20, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:
Invoke() should activate it, but elements don’t' always handle the
events correctly either. Welcome to the perils of scripting. Sorry.
If it focuses it instead, for example then sending a Space or Enter
may get the button to activate.
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
<jawsscripts-bounce@xxxxxxxxxxxxx>
On
Behalf Of Theodore Cooke
Sent: Thursday, July 23, 2020 4:19 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Learning to find information with script
utility mode
I did read that reply, but at the time, I couldn't make sense of
the example he gave. I did not understand MSAA and UIAutomation,
among other difficulties. I thought you were referring to a reply
he sent in another thread, so I looked that up, and found his
script to copy an entire UIA tree to the clipboard. That turned out
to be useful as I was able to modify it to just copy the UIA data
for the element I want, then further modify it to invoke that
button instead of copying its data. I now have a script for Dropbox
that will use UIA to invoke the button named, "Open in File
Explorer".
A couple days ago I had coded a more basic script to open Dropbox
in file explorer, except that one simply tabs until it finds an
object named, "Open in File Explorer", then presses the space key.
Both these scripts have been a good start, and along with the
Microsoft pages on UIA and MSAA, I have finally been able to grasp
the basics of how to access and invoke UI elements in the jaws
scripting language. Thank you for all your assistance.
One last little question for now: Do you have any idea why the
invoke function does not simulate clicking this button? It seems to
just move the system focus to it, but I would expect a function
named invoke to be equivalent to pressing the space bar on the
given element. Could this just be a querk of this particular
element in this particular application, or does the invoke function
typically just move the focus to the element?
On 7/22/20, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:
__________What interface is the jaws script utility mode accessing when itf1 and f3 telling me? Is that the UIAutomation data, or something
tells me the control type, class, handle, etc of a UI element?
f9 and f10 have clear documentation that they're accessing the
MSAA, but what is
else?
It is the old window class information from the JAWS off screen
model back before apps had MSAA and UIAutomation. It is stil
useful in some apps, bu tnot useful in most modern apps like VS most of
the time.
If it does give something useful to you such as the window name,
then see the corresponding window functions such as GetWindowName().
2. I am trying to write this script using FS UIA, but the jaws
documentation lacks any clear examples. My little script
currently isn't doing anything whatsoever, and I am not even
sure how to debug it. Could you give me a short example of a
script that navigates to an element using FS UIA, so I have something
to work with?
Did you see Udo's email reply from this weekend 7/18/20? That has
some good starting points. (You may have to reformat it, in my
email client the lines are wrapped together.) Travis
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
<jawsscripts-bounce@xxxxxxxxxxxxx> On Behalf Of Theodore Cooke
Sent: Wednesday, July 22, 2020 8:52 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Learning to find information with
script utility mode
Alright, I'm getting to a point where I'm able to answer some of
these questions for myself, but I'm not out of the woods yet.
What interface is the jaws script utility mode accessing when it
tells me the control type, class, handle, etc of a UI element? f9
and f10 have clear documentation that they're accessing the MSAA,
but what is
f1 and f3 telling me? Is that the UIAutomation data, or something
else?
2. I am trying to write this script using FS UIA, but the jaws
documentation lacks any clear examples. My little script currently
isn't doing anything whatsoever, and I am not even sure how to
debug it. Could you give me a short example of a script that
navigates to an element using FS UIA, so I have something to work with?
On 7/20/20, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:
Sometimes knowing which kind of app it is is just checking to see__________
which method works.
If it is a recent Microsoft app though, it is likely using or
trying to use UIAutomation as that's Microsoft's technology.
I am unable to find the line number with JAWS scan cursor as
well; I am not sure if it just will not show up or not. If I was
doing this myself I'd open UIVerify and start reading through the tree.
In this case I "cheated" and asked a sighted colleague to put the
mouse on it and then look at the tree. As I mentioned the tools
will track focus and the mouse. I didn't have time to read the
tree node by node and he was available. <smile>
Oh, one thing I should mention as part of UIA is JAWS touch
cursor uses UIA.
So another way to see what information is available via UIA is to
turn on touch cursor. I think you may find the line number this
way, with lots of patience. I sincerely doubt the line number is
findable via the script utility mode.
As for if you want to override a default script, turn on JAWS
keyboard help
(JAWSKey+1) then pres the script key three times rapidly and JAWS
will announce the script name that is being called.
Travis
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
<jawsscripts-bounce@xxxxxxxxxxxxx> On Behalf Of Theodore Cooke
Sent: Monday, July 20, 2020 11:53 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Learning to find information with
script utility mode
Thank you for the tips, Travis. This does help, but I think I
still need a bit more assistance to get a foothold in jaws
scripting. I still don't know how to find where the piece of info
I want is hiding, and I am likewise very new to all the ways of
programatically finding a path to it.
1: how can I tell the difference between classic apps and
slightly more modern apps that use MSAA, as well as VS
UIAutomation and com interfaces?
2: These UIA tools, what are they? Can they help with all types
of apps, or only certain ones like MSAA?
3: I'll take a look at that location for the line number and see
what I can do. Perhaps one of the proximal windows has a
consistent ID number or some such that I could hook into.
4: As I have been getting at, one of my difficulties is finding
where in the UI that line number is displayed. I navigated a
bunch with script utility mode, but didn't find it. Could you
explain to me how you found it?
5: If I want to override the event that updates the line number,
how do I find that event to override it?
Thank you for all the time you put into your answers.
On 7/17/20, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:
Hi,__________
Understood. The thing to realize, in my opinion, is JAWS
scripting is at times an art as much as a science. The key I've
found is to figure out where the pice of info you want is
hiding, then figure a path to get to it programmatically.
Step 1 is to figure out where the info is and what technology
will get there. Is it a classic app so old windowing tools work?
Or slightly more modern and MSAA?
Or in the case of VS UIAutomation is how it exposes accessibility.
And as one fellow poster mentioned, if that doesn't work you can
dive into COMM interfaces, etc. For web apps, sometimes looking
at the HTML or DOM, although I find looking at the UIA from
Chrome or Firefox to be easier a lot of times anymore.
In the case of VS 2019 my advice is to find a tool you are
comfortable with that allows you to explore the UIA tree. I use
accVerify which at least used to be buried in the Windows SDK
somewhere. I believe Accessibility Insights has this info to but
I do not understand it's interface.
SO once you find your UIA tool of choice, you just have to start
exploring until you find the info you want, if it exists.
A shortcut is to have the tool track the focus or mouse and that
gets you to the place in the tree automatically, but I rarely
can get this to work and usually browse the tree by hand. (It
really helps if you have access to a
sightling.)
In the case of the line number for example, in a C# project with
program.cs open, it appears to be located at "Desktop 1" pane >
"test
- Microsoft Visual Studio" window > "" pane > ""
tab > "program.cs" tab item > "program.cs" pane > "program.cs"
pane
""
custom > "Ln: 6" text
From here you need to work out an algorithm to reliably get to
this object.
The text is in a rather benign text node, so this is probably
going to be a challenge.
Try to isolate it as much as you can, and as last resort walk
the tree and look for the value that matches what you're looking for.
For example in this instance, I'd start at the application root,
then search for the tab object, then from that object search for
pane, and then possibly the child pane of that pane, and then
eventually load up all the remaining text nodes and look through
them for something containing "ln". And possibly while doing
this I'd happen across a shortcut or helpful clue to make it simpler.
I'm not going to write a discord on navigating and using UIA,
there are various messages over the years in this list's archive
that help, and honestly a lot of time it is trial and error. At
least for me.
Others who do this more may have better methods, I do not script
all that much these days.
Now after you have all that you have to decide how you want JAWS
to read it.
DO you just want it on demand when you press a script key?
That's pretty easy. Just make a script and define a key.
Or if you want it to read automatically then you have to find
what event is being triggered and override it.
For example, this may be handy to be announced with the JAWS
status line reading command, so override that script
(SayBottomLineOfWindow).
Bottom line from me though is the hard part is finding where the
info is hiding and reliably getting at it.
Not sure if this is at all helpful.
Travis
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
<jawsscripts-bounce@xxxxxxxxxxxxx> On Behalf Of Theodore Cooke
Sent: Thursday, July 16, 2020 1:51 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Learning to find information with
script utility mode
I am using C# as well.
I am likewise able to show line numbers as you are, but I also
don't like having them be read every time I up or down arrow. I
swear the line number is always displayed somewhere on the
screen, but I am not sure where I read that. Either way, this is
just one example of the many things I have tried but failed to do
with jaws scripting.
The main thing I am asking for is general help with getting the
hang of scripting.
On 7/16/20, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:
Hi,__________
Visual Studio 2019 is certainly going to be a difficult thing
to script. I suspect the utility mode scripts will be of
limited use as they focus on old style windows and have some
MsAA, whereas the VS accessibility is primarily via UIAutomation.
What language in Visual Studio 2019 are you using?
I opened a C# project and when I enabled show line numbers
under Options > Text Editor > All Languages, the numbers are
shown at the left margin and JAWS reads them.
This does not seem ideal to me to have them constantly
announced, but where else are they shown? How do you have yours
configured?
Travis
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
<jawsscripts-bounce@xxxxxxxxxxxxx> On Behalf Of Theodore Cooke
Sent: Thursday, July 16, 2020 1:11 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Learning to find information with script
utility mode
Hello hello.
I have tried many times to get the hang of writing jaws
scripts, but it just has not clicked for me. I wouldn't say
this is because of any lack of technical ability. I am a
software developer by trade.
However, every time I have found a situation where jaws isn't
speaking all that I would like, I have not been able to figure
out how to get that needed information. maybe it is because
that by some terrible accident, I have only tried to write
scripts for applications that are inherently unaccessible, but
I have now tried to do this with so many applications that I
feel I must just be missing the boat on some important detail.
My current task at hand is creating a hotkey in visual studio
to read the current line number, but I can't find how to read
that using either script utility mode or by navigating around
using the cursors.
Would somebody be able to walk me through how they use the
script utility mode to find their desired information? I would
greatly appreciate that. I'm sure that just as soon as this
clicks for me, I'll be off to the races and writing all sorts of
scripts.
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________�
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________�
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
