Thanks so, so much Travis! This is very helpful. I'd looked hard at touch.jss,
and at the Win8Calc.jss, but couldn't figure out how to glue it all together.
I'll use your work to explore further. Thanks again!
Jeremiah Rogers
Cell: 704-996-5334
Email: jeremiahzrogers@xxxxxxxxx
Social Networking: /jzrogers
On Jul 19, 2016, at 15:08, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:__________�
Hi Jerimiah,
You ask a good, but complicated question. In my experience nothing with
UIAutomation ends up being easy.
I have done something similar so I will try to describe it.
First a warning: I have found that when navigating UIA elements via
scripting, it may not match the experience of using the touch cursor
manually. E.g., in my application navigating two times is enough, but in
script it requires three. So you will have to experiment with this. I have
no idea why this is. And its possible you may not run into the issue.
As a reference there is a lot of UIAutomation code in touch.jss. And it is
where I borrowed most of my code. Since touch.jss is loaded gloabally and I
didn't want to interfere with how JAWS is using it otherwise, I copied
functions I needed to my own application's script file and changed their
name. In my case I prepended "Sonos" to them because that was the name of
the app; I am leaving the code as is; you will be able to deduce the
original functions names that are in touch.jss.
In my example I was searching for the name of a slider that was in focus
with the PC cursor. The name was two prior UIA objects away when navigating
with touch cursor, and three UIAGoToPriorInTree() calls away via script (for
whatever reason see warning above).
The below function calls UIA functions to find it.
String Function GetSliderName()
; moves 3 objects to left and retrieves name
;assumes calling function has verified slider is in focus
var
int I;
InitUIA();
SonosSetCurrentElementToDeepestFocusElement(UIA, treeWalker);
;have to do 3 calls even though manual touch navigation takes 2
i = SonosUIAGoToPriorInTree (treeWalker);
i = SonosUIAGoToPriorInTree (treeWalker);
if SonosUIAGoToPriorInTree(treeWalker);
return treeWalker.currentElement.name;
EndIf;
return "" ;
EndFunction;
I will copy the rest of my script file for your reference. Because of email
formatting you may have to adjust the spacing and line endings; or this will
give you an idea of what to copy from touch.jss.
; rest of script file:
;include "HJGlobal.jsh"
include "hjConst.jsh"
include "UIA.jsh"
include "MSAAConst.jsh"
globals
object UIA,
object treewalker
int function InitUIA()
if ! UIA
UIA = CreateObjectEx ("FreedomSci.UIA", false,
"UIAScriptAPI.x.manifest" )
if ! UIA
SayString("Cannot get UIA")
return false
EndIf
treewalker = UIA.CreateTreeWalker(UIA.CreateRawViewCondition())
if ! treewalker
saystring("Cannot get tree walker")
return false
EndIf
EndIf
return true
EndFunction
void function SonosSetCurrentElementToDeepestFocusElement(object oUIA,
object temptreeWalker)
; from touch.jss
;More than one element in a process may show as having the focus,
;this tries to set the current element to the deepest focus element which is
keyboard focusable.
if !oUIA || !temptreeWalker return Null() endIf
var object focusElement = oUIA.GetFocusedElement().BuildUpdatedCache()
if !focusElement
;Even though there is focus, GetFocusedElement may not retrieve an
element
temptreeWalker.currentElement =
oUIA.GetElementFromHandle(GetFocus())
return
endIf
var object condition = oUIA.CreateAndCondition(
oUIA.CreateBoolPropertyCondition(UIA_HasKeyboardFocusPropertyId,
UIATrue),
oUIA.CreateBoolPropertyCondition(UIA_IsKeyboardFocusablePropertyId,
UIATrue))
if !condition
temptreeWalker.currentElement = FocusElement
return
endIf
;looking for the focus element which is keyboard focusable will cause a
crash if used in the context menu of an item in the system tray:
if focusElement.controltype == UIA_MenuBarControlTypeId
&& focusElement.isOffscreen
temptreeWalker.currentElement = oUIA.GetRootElement()
temptreeWalker.currentElement =
temptreeWalker.currentElement.FindFirst(TreeScope_Subtree,
oUIA.CreateBoolPropertyCondition(UIA_HasKeyboardFocusPropertyId, UIATrue))
return
endIf
temptreeWalker.currentElement = oUIA.GetRootElement()
while !!FocusElement &&
!oUIA.CompareElements(focusElement,temptreeWalker.currentElement)
temptreeWalker.currentElement = focusElement
focusElement = FocusElement.FindFirst(TreeScope_Subtree, condition
).BuildUpdatedCache()
endWhile
EndFunction
int function SonosUIAGoToPriorInTree( object temptreeWalker )
;from touch.jss
if ! temptreeWalker
return false
endIf
if temptreeWalker.GoToPriorSibling()
SonosUIAGoToLastDescendant( temptreeWalker );
return true
EndIf
if !temptreeWalker.GoToParent()
return false
EndIf
return true
EndFunction
int function SonosUIAGoToLastDescendant( object temptreeWalker )
if ! temptreeWalker return false endif
var int retVal = false
while SonosUIAGoToLastChild( temptreeWalker )
retVal = true
EndWhile
return retVal
EndFunction
int function SonosUIAGoToLastChild( object temptreeWalker )
if ! temptreeWalker return false endif
if ! temptreeWalker.GoToFirstChild()
return false
EndIf
while temptreeWalker.GoToNextSibling()
EndWhile
return true
EndFunction
Travis Roth
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Jeremiah Rogers
Sent: Tuesday, July 19, 2016 11:59 AM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Automating Reading Info Available Via Touch Cursor
Hello all. I'm hoping to get some help with the same challenge I posted on
July 14. Maybe the following alteration of the question might help.
I want to create a script that will read information that I can presently
find via the touch cursor.
I need to add a keystroke to an app used in a call center. The keystroke
will simply read a piece of information. Here's how I get to the information
manually.
Navigate using TAB or SHIFT+TAB to a control whose name is reported as "Call
Information" and whose LocalizedControlType is reported as "button".
Issue TouchNextElement 22 times to find a control whose name is reported as
"IVR Exit Reason:".
Any ideas how I can script this task? Any ideas for how I can consistently
turn the attention of JAWS to the Call Information button, then have it look
for the IVR Exit Reason text without disturbing system focus? Thanks as
always for your thoughts.
Thanks in advance for your guidance.
Jeremiah Rogers
Cell: 704-996-5334
Email: jeremiahzrogers@xxxxxxxxx
Social Networking: /jzrogers
__________o?=
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
__________�
View the list's information and change your settings at
//www.freelists.org/list/jawsscripts
