Get structure from dllcall result into struct variable

[junkew]

In the help I could not clearly understand how I can manage a dll returning a structure as a a result

Definition of the structure 

const $cTypeStructAccessibleContextInfo="WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];INT indexInParent;INT childrenCount;INT x;INT y;INT width;INT height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces"

I assume the dll function call returns me a pointer to a structure AccessibleContext

;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);
;~ _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.
$result =dllcall($bridgeDLL, "ULONG_PTR:cdecl", "getAccessibleChildFromContext","long", $vmId,"long", $acParent, "int", 1)

Now I have to cast back the pointer to the structure

Local $AccessibleContextInfo=DllStructCreate($cTypeStructAccessibleContextInfo)

This is not working ;-)

$AccessibleContextInfo=$result[0]

[Yashied]

#Include <WinAPIEx.au3>

Global Const $tagAccessibleContextInfo = _
        'WCHAR name[1024];' & _
        'WCHAR description[1024];' & _
        'WCHAR role[256];' & _
        'WCHAR role_en_US[256];' & _
        'WCHAR states[256];' & _
        'WCHAR states_en_US[256];' & _
        'INT indexInParent;' & _
        'INT childrenCount;' & _
        'INT x;' & _
        'INT y;' & _
        'INT width;' & _
        'INT height;' & _
        'BOOL accessibleComponent;' & _
        'BOOL accessibleAction;' & _
        'BOOL accessibleSelection;' & _
        'BOOL accessibleText;' & _
        'BOOL accessibleInterfaces'

$tAccessibleContextInfo = DllStructCreate($tagAccessibleContextInfo)
$Result = DllCall($bridgeDLL, 'ulong_ptr:cdecl', 'getAccessibleChildFromContext', 'ulong', $vmId, 'ulong_ptr', DllStructGetPtr($tAccessibleContextInfo), 'ulong', 1)

ConsoleWrite('Result: ' & $Result[0] & @CR)

_WinAPI_DisplayStruct($tAccessibleContextInfo, $tagAccessibleContextInfo)

[junkew]

function works a little different 

definition is: AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);

So it gets as parameters an parent accessiblecontext and ask for child n and it returns a new accessiblecontext as a structure. so parameter ac is not the context i am looking for.

The displaystruct copies not back I assume from result[0]

[Yashied]

$Result = DllCall($bridgeDLL, 'ulong_ptr:cdecl', 'getAccessibleChildFromContext', ...)
$tAccessibleContextInfo = DllStructCreate($tagAccessibleContextInfo, $Result[0])

[junkew]

Although the function doesn't break I do not get the accessible context from the child
 
Defined as

AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);
AccessibleContext = 
typedef JOBJECT64 AccessibleContext; = 
typedef jlong JOBJECT64;=
typedef __int64 jlong;

leads to

;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.

                local $myChild=1
                $result =dllcall($bridgeDLL, "UINT64:cdecl", "getAccessibleChildFromContext","long", $vmId,"long", $acParent, "int", $myChild)
                if @error=0 Then
consolewrite("We have a child " & @error & @CRLF)
consolewrite("Result is <" & $result[0] & "> ubound: " & ubound($result) & @crlf)

Local $tAccessibleContextInfo2=DllStructCreate($tagAccessibleContextInfo, $Result[0])
;ConsoleWrite('Result getAccessibleChildFromContext: <' & $Result[0] & ">" & @CRLF)
_WinAPI_DisplayStruct($tAccessibleContextInfo, $tagAccessibleContextInfo)
_WinAPI_DisplayStruct($tAccessibleContextInfo2, $tagAccessibleContextInfo)

Whole script
1. needs JDK 1.8 (or above 1.7)
2. jabswitch /enable has to be run
3. to test java window needs to be there. I start demojfcSwingSet2SwingSet2.Jar for testing
4. test with javamonkey if the tree is shown in the monkey
5. test below script if similar things can be reproduced as in javamonkey
 
edit: script removed, see later post

Edited January 7, 2015 by junkew

[junkew]

slowly progressing the function getTopLevelObject also returns an accessiblecontext maybe a better candidate to investigate

 

Could be function GetAccessibleChildFromContext works but that I am logically not looking at a visible child

;~      * Returns the Accessible Context for the top level object in
;~      * a Java Window.  This is same Accessible Context that is obtained
;~      * from GetAccessibleContextFromHWND for that window.  Returns
;~      * (AccessibleContext)0 on error.
;~      */
;~     AccessibleContext getTopLevelObject (const long vmID, const AccessibleContext accessibleContext);
;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac);
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)
local $topAC
$result=dllcall($accessBridgeDll, "long:cdecl", "getTopLevelObject", "long", $vmId, "UINT64", $acparent)
if @error=0 Then
    consolewrite(" *** getTopLevelObject Result is <" & $result[0] & "> ubound: " & ubound($result) & @crlf)
    $vmID=$result[1]
    $topAC=$result[0]
    consolewrite("We have a VMid " & $vmID & " toplevel object ac " & $topAC & @CRLF)
Else
    consolewrite("getTopLevelObjecterror:" & @error  & @CRLF)
endif

[LarsJ]

Did you manage to get getAccessibleContextFromHWND (post 5, second codebox) to work? I'll guess that the two last parameters should be

"struct*", $tAccessibleContextInfo

And you get the documentation in Java Access Bridge API?

[junkew]

only problem is the struct that should come back from the dll as I assume it will be in result[0] but unable to translate the returnvalue back to the elements of the structure.

• Does anyone have an example of a win32 / user32 function that returns a struct as a result (most give an integer back) and how that should be written in AutoIT.
  should I write  
  a. $result =dllcall($accessBridgeDll,"ptr:cdecl",
  b. $result =dllcall($accessBridgeDll,"struct:cdecl",
  c. $result =dllcall($accessBridgeDll,"ULONG_PTR:cdecl",
  d. .......

1. getAccessibleContextFromHWND works

JAVA window found <49> Java Window Title=SwingSet2 Handle=0x00000000000109C4     res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
We have a VMid 68012 ac 1538008736

2. Yes. about all information I could find with google I have been reading including c++ source and header files and monkey and ferret source and jabhandler.py
 
now trying on getTopLevelObject function as that should return also an accessible context and I seem to get something back but unable to translate to a structure so far
*** getTopLevelObject Result is <1538008728> ubound: 3
We have a VMid 68012 toplevel object ac 0
 
Output I get

expandcollapse popup

We are using X64 at cpu X64 Autoit 64 bit version 1
  Windows accessbridge 32 opened 1
Windows_run passed :
Before checking all Windows: 464
  JAVA window found <57> Java Window Title=SwingSet2 Handle=0x00000000000109C4  res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 68012 ac 1538006120
Call version info: PASSED VMID found : 68012
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <1538006112> ubound: 3
We have a VMid 68012 toplevel object ac 0
  name: <0>
  description: <0>
  role: <0>
  role_en_US: <0>
  states: <0>
  states_en_US: <0>
  indexInParent: <0>
  childrenCount: <0>
  x: <0>
  y: <0>
  width: <0>
  height: <0>
0
0
0
0
0
0
0
0
0
0
0
0
  We still have a VMid 68012 acParent 1538006120
Result is <> ubound: 4
  name: <SwingSet2>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,resizable>
  states_en_US: <enabled,focusable,visible,showing,resizable>
  indexInParent: <-1>
  childrenCount: <1>
  x: <1143>
  y: <219>
  width: <736>
  height: <0>
SwingSet2
 
frame
frame
enabled,focusable,visible,showing,resizable
enabled,focusable,visible,showing,resizable
-1
1
1143
219
736
0
  We still have before getAccessible Child FromContext a VMid 68012 acParent 1538006120
rect:11431879219920We have a child 0
Result is <0> ubound: 4
  name: <0>
  description: <0>
  role: <0>
  role_en_US: <0>
  states: <0>
  states_en_US: <0>
  indexInParent: <0>
  childrenCount: <0>
  x: <0>
  y: <0>
  width: <0>
  height: <0>
0
0
0
0
0
0
0
0
0
0
0
0
+>17:41:49 AutoIt3.exe ended.rc:0
+>17:41:49 AutoIt3Wrapper Finished.
 

Latest source for above output (trying to clean, make a simple JAB spy based on simple spy of iuia if I get things to work).
Cleaning and restructuring at the moment to see If I can clarify in more detail and If I am looking at the right objects to get ac from
edit: script removed, see later post

Edited January 7, 2015 by junkew

[LarsJ]

Have you tried something like this to get the struct from result[0]:

$tPtr = DllStructCreate( "ptr", $result[0] )
$ptr = DllStructGetData( $tPtr )
$tAccessibleContextInfo = DllStructCreate( $tagAccessibleContextInfo, $ptr )

I think a) is the proper code.

[junkew]

Still not working
 
1. Made it easier with below script to reproduce only requirement is having java runtime 1.7 or 1.8 installed
2. runs java window : C:Program FilesJavajre1.8.0_25binjavacpl.exe as a window to analyse
3. tried to put it in a function wrapper

func __getTopLevelObject($vmId,$acParent)
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getTopLevelObject", "long", $vmId, "UINT64", $acparent)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc

edit: script removed, see later post

Edited January 7, 2015 by junkew

[LarsJ]

I'll install the software and test the scripts myself. Probably not before weekend. I'll come back with the results here whether success or failure.

[junkew]

Maybe I should change the topic name, struct returning seems not to be the issue and I made some positive progress but not fully where I want to be (recursive dump of information of a Java Window)
 
Now managed to get  
getTopLevelObject working
It was not returning a struct but an AccessibleContext which is a 

typedef JOBJECT64 AccessibleContext;

typedef jlong JOBJECT64

typedef __int64 jlong;

in doubt if this is working yes/no seems to be child counting starting at 0 so 0 is first child and not 1
func __getAccessibleChildFromContext($vmId, $ac, $index) not working (or not tested fully) and also ActiveDescent not working (not tested fully)
 
wrapped stuff a little in smaller helper functions (__getTopLevelObject is working the others are not)
 

;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac);
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)
func __getTopLevelObject($vmId,$ac)
;~ Seems not to be allowed to call with null so passing $acParent
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getTopLevelObject", "long", $vmId, $c_JOBJECT64, $ac)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc

;~ typedef AccessibleContext (*GetAccessibleChildFromContextFP) (long vmID, AccessibleContext ac, jint i);
;~ typedef AccessibleContext (*GetAccessibleParentFromContextFP) (long vmID, AccessibleContext ac);
;~ typedef AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.
func __getAccessibleChildFromContext($vmId, $ac, $index)
;~ Seems not to be allowed to call with null so passing $acParent
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getAccessibleChildFromContext", "long", $vmId, $c_JOBJECT64, $ac, "int", $index)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc

;~ _fixBridgeFunc(JOBJECT64,'getActiveDescendent',c_long,JOBJECT64)
func __getActiveDescendent($vmId, $ac)
;~ Seems not to be allowed to call with null so passing $acParent
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getActiveDescendent", "long", $vmId, $c_JOBJECT64, $ac)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc

full script
1. Only Java Runtime needed (no need for samples / JDK to install)
2. JABSWITCH /ENABLE should have been run from commandline
3. Then script can be run (will run JavaCPL with shellexecute("C:Program FilesJavajre1.8.0_25binjavacpl.exe")) so needs change if other JRE version
Optional (mainly for definitions in header files but main info copied in au3 script in comments)
4. Java Accessibility Bridge 2.0.2. for CPP source and headers and javamonkey (java spy tool)
5. JDK installed for other JAVA sample applications
6. NVDA screen reader source JABHANDLER.PY as a reference
 

logic

1. getTopLevelObject

2. Assume it has at least 1 child

3. get that child and write information

spyscreen is not doing anything at the moment

expandcollapse popup

#AutoIt3Wrapper_UseX64=Y

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <WinAPIEx.au3>
#include <debug.au3>
#include <memory.au3>

Global $accessBridgeDll  ;~ reference to the accessbridge dll

Const $tagAccessBridgeVersionInfo="WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]"

const $c_JOBJECT64="UINT64"   ;~ JOBJECT64
const $cP_JOBJECT64="UINT64*" ;~ POINTER(JOBJECT64)


Local $tAccessBridgeVersionInfo=DllStructCreate($tagAccessBridgeVersionInfo)

const $tagAccessibleContextInfo="WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];INT indexInParent;INT childrenCount;INT x;INT y;INT width;INT height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces"
Local $tAccessibleContextInfo=DllStructCreate($tagAccessibleContextInfo)

HotKeySet("{ESC}", "Close") ; Set ESC as a hotkey to exit the script.
HotKeySet("^w", "GetElementInfo") ; Set Hotkey Ctrl+W to get some basic information in the GUI

const $AutoSpy=0 ;2000 ; SPY about every 2000 milliseconds automatically, 0 is turn of use only ctrl+w

;~ global $oldUIElement ; To keep track of latest referenced element
global $frmSimpleSpy, $edtCtrlInfo , $lblCapture, $lblEscape, $lblRecord, $edtCtrlRecord, $msg
global $i              ; Just a simple counter to measure time expired in main loop

shellexecute("C:\Program Files\Java\jre1.8.0_25\bin\javacpl.exe")
sleep(3000)

_JABInit()
_Example1_IterateJavaWindows()

#Region ### START Koda GUI section ### Form=
$frmSimpleSpy = GUICreate("Simple UIA Spy", 801, 601, 181, 4)
$edtCtrlInfo = GUICtrlCreateEdit("", 18, 18, 512, 580)
GUICtrlSetData(-1, "")
$lblCapture = GUICtrlCreateLabel("Ctrl+W to capture information", 544, 10, 528, 17)
$lblEscape = GUICtrlCreateLabel("Escape to exit", 544, 53, 528, 17)
$edtCtrlRecord = GUICtrlCreateEdit("", 544, 72, 233, 520)
GUICtrlSetData(-1, "//TO DO edtCtrlRecord")
;~ $lblRecord = GUICtrlCreateLabel("Ctrl + R to record code", 544, 32, 527, 17)
GUISetState(@SW_SHOW)
#EndRegion ### END Koda GUI section ###

;~ _UIA_Init()

;~ loadCodeTemplates() ; //TODO: To use templates per class/controltype

; Run the GUI until the dialog is closed
While true
    $msg = GUIGetMsg()
    sleep(100)
    ;~ if _ispressed(01) Then
    ;~ getelementinfo()
    ;~ endif

    ;Just to show anyway the information about every n ms so ctrl+w is not interfering / removing window as unwanted side effects
    $i=$i+100
    if ($autoSpy<>0) and ($i>= $autoSpy) then
        $i=0
        getelementinfo()
    EndIf

    If $msg = $GUI_EVENT_CLOSE Then ExitLoop
WEnd
_JABShutDown()

Func GetElementInfo()
;~  _fixBridgeFunc(BOOL,'getAccessibleContextAt',c_long,JOBJECT64,jint,jint,POINTER(JOBJECT64),errcheck=True)
;~  $result =dllcall($accessBridgeDll,"BOOL:cdecl", "getAccessibleContextAt", "long*", $vmID, "UINT64*", $acParent,"int", $mouseX, "int", $mousey,"UINT64*", $acNew)


EndFunc
Func Close()
_JABShutDown()
Exit
EndFunc   ;==>Close

Func _JABInit()
    ;~ Make sure Java Access Bridge is turned on
    runwait("C:\Program Files\Java\jre1.8.0_25\bin\jabswitch /enable","",@SW_MAXIMIZE)

    ;~ TODO: Check if it works with both dll's
    consolewrite("We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF)

    if @AutoItX64 = 1 then
        $accessBridgeDLL="WindowsAccessBridge-64.dll"
    Else
        $accessBridgeDLL="WindowsAccessBridge-32.dll"
    EndIf

    ;~ Open the Dll for accessibility
    $accessBridgeDll=DLLOPEN($AccessBridgeDLL)
    If $accessBridgeDll=true Then
        consolewrite("  Windows accessbridge 32 opened " & $accessBridgeDll & @CRLF)
    Else
        consolewrite("  DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows")
    endIf

    ;~ TODO: Handle messages received from initialize
    $result =dllcall($accessBridgeDll,"NONE", "Windows_run")
    consolewrite($result & " " & @error & " initializeAccessBridge is finished")

    sleep(250)

    consolewrite("Windows_run passed :" & $result & @CRLF)
EndFunc




;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs
;~   So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed
;~
;~ Somehow it sends a message unclear if this is to the JVM to respond to
;~   push   SSZ6E73E320_AccessBridge_FromJava_Hello
;~   push   SSZ6E73E300_AccessBridge_FromWindows_Hello
;~   db 'AccessBridge-FromWindows-Hello',0
;~   db 'AccessBridge-FromJava-Hello',0

;~ JABHANDLER.PY is a usefull source to study on internet

;~ *******
;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf
;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file.
;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY
;~      typedef jobject JOBJECT64;
;~      typedef HWND ABHWND64;
;~      #define ABHandleToLong
;~      #define ABLongToHandle
;~ #else
;~      typedef jlong JOBJECT64;
;~      typedef long ABHWND64;
;~      #define ABHandleToLong HandleToLong
;~      #define ABLongToHandle LongToHandle
;~ #endif
;~ a jobject is a 32 bit pointer for 32 bit builds
;~ 'bool' is a built-in C++ type while 'BOOL' is a Microsoft specific type that is defined as an 'int'. You can find it in 'windef.h'
;~
;~ ** jni_md.h **
;~ typedef long jint;
;~ typedef __int64 jlong;
;~ typedef signed char jbyte;
;~ *******
;~ accessbridgecalls.h
;~     typedef JOBJECT64 AccessibleContext;
;~     typedef JOBJECT64 AccessibleText;
;~     typedef JOBJECT64 AccessibleValue;
;~     typedef JOBJECT64 AccessibleSelection;
;~     typedef JOBJECT64 Java_Object;
;~     typedef JOBJECT64 PropertyChangeEvent;
;~     typedef JOBJECT64 FocusEvent;
;~     typedef JOBJECT64 CaretEvent;
;~     typedef JOBJECT64 MouseEvent;
;~     typedef JOBJECT64 MenuEvent;
;~     typedef JOBJECT64 AccessibleTable;
;~     typedef JOBJECT64 AccessibleHyperlink;
;~     typedef JOBJECT64 AccessibleHypertext;

;~ #Appropriately set the return and argument types of all the access bridge dll functions
;~ if bridgeDll:
;~  _fixBridgeFunc(None,'Windows_run')
;~  _fixBridgeFunc(None,'setFocusGainedFP',c_void_p)
;~  _fixBridgeFunc(None,'setPropertyStateChangeFP',c_void_p)
;~  _fixBridgeFunc(None,'setPropertyCaretChangeFP',c_void_p)
;~  _fixBridgeFunc(None,'setPropertyActiveDescendentChangeFP',c_void_p)
;~  _fixBridgeFunc(None,'releaseJavaObject',c_long,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'getVersionInfo',POINTER(AccessBridgeVersionInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'isJavaWindow',HWND)
;~  _fixBridgeFunc(BOOL,'isSameObject',c_long,JOBJECT64,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextFromHWND',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True)
;~  _fixBridgeFunc(HWND,'getHWNDFromAccessibleContext',c_long,JOBJECT64,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextAt',c_long,JOBJECT64,jint,jint,POINTER(JOBJECT64),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextWithFocus',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextInfo',c_long,JOBJECT64,POINTER(AccessibleContextInfo),errcheck=True)
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleParentFromContext',c_long,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'getAccessibleRelationSet',c_long,JOBJECT64,POINTER(AccessibleRelationSetInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextInfo',c_long,JOBJECT64,POINTER(AccessibleTextInfo),jint,jint,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextItems',c_long,JOBJECT64,POINTER(AccessibleTextItemsInfo),jint,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextSelectionInfo',c_long,JOBJECT64,POINTER(AccessibleTextSelectionInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextAttributes',c_long,JOBJECT64,jint,POINTER(AccessibleTextAttributesInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextLineBounds',c_long,JOBJECT64,jint,POINTER(jint),POINTER(jint),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextRange',c_long,JOBJECT64,jint,jint,POINTER(c_wchar),c_short,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getCurrentAccessibleValueFromContext',c_long,JOBJECT64,POINTER(c_wchar),c_short,errcheck=True)
;~  _fixBridgeFunc(BOOL,'selectTextRange',c_long,JOBJECT64,c_int,c_int,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getTextAttributesInRange',c_long,JOBJECT64,c_int,c_int,POINTER(AccessibleTextAttributesInfo),POINTER(c_short),errcheck=True)
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)
;~  _fixBridgeFunc(c_int,'getObjectDepth',c_long,JOBJECT64)
;~  _fixBridgeFunc(JOBJECT64,'getActiveDescendent',c_long,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'requestFocus',c_long,JOBJECT64,errcheck=True)
;~  _fixBridgeFunc(BOOL,'setCaretPosition',c_long,JOBJECT64,c_int,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getCaretLocation',c_long,JOBJECT64,POINTER(AccessibleTextRectInfo),jint,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleActions',c_long,JOBJECT64,POINTER(AccessibleActions),errcheck=True)
;~  _fixBridgeFunc(BOOL,'doAccessibleActions',c_long,JOBJECT64,POINTER(AccessibleActionsToDo),POINTER(jint),errcheck=True)


;~ Primitive Types and Native Equivalents
;~ Java Type    Native Type Description
;~ boolean      jboolean    unsigned 8 bits
;~ byte         jbyte       signed 8 bits
;~ char         jchar       unsigned 16 bits
;~ short        jshort      signed 16 bits
;~ int          jint        signed 32 bits
;~ long         jlong       signed 64 bits
;~ float        jfloat      32 bits
;~ double       jdouble     64 bits
;~ void         void        not applicable

;~ Copied from this NVDA reference and translated to AutoIT
;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html
;~
;~ def initialize():
;~         global isRunning
;~         if not bridgeDll:
;~                 raise NotImplementedError("dll not available")
;~         bridgeDll.Windows_run()
;~         #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages
;~***         ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None)
;~***         if ChangeWindowMessageFilter:
;~***                 if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1):
;~***                         raise WinError()
;~***                 for msg in xrange(winUser.WM_USER+1,65535):
;~***                         if not ChangeWindowMessageFilter(msg,1):
;~***                                 raise WinError()
;~         #Register java events
;~         bridgeDll.setFocusGainedFP(internal_event_focusGained)
;~         bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange)
;~         bridgeDll.setPropertyNameChangeFP(event_nameChange)
;~         bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange)
;~         bridgeDll.setPropertyValueChangeFP(event_valueChange)
;~         bridgeDll.setPropertyStateChangeFP(internal_event_stateChange)
;~         bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange)
;~         isRunning=True

;create the structs
;~
;~  http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm
;~
;~ #define MAX_STRING_SIZE     1024
;~ #define SHORT_STRING_SIZE    256
;~                  struct AccessBridgeVersionInfo {
;~  wchar_t VMversion[SHORT_STRING_SIZE];              // version of the Java VM
;~  wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class
;~  wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE];   // version of JavaAccessBridge.dll
;~  wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE];    // version of WindowsAccessBridge.dll
;~ };


;~ struct AccessibleContextInfo {
;~  wchar_ name[MAX_STRING_SIZE];        // the AccessibleName of the object
;~  wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object
;~  wchar_ role[SHORT_STRING_SIZE];      // localized AccesibleRole string
;~  wchar_ states[SHORT_STRING_SIZE];    // localized AccesibleStateSet string
;~                                       //   (comma separated)
;~  jint indexInParent                   // index of object in parent
;~  jint childrenCount                   // # of children, if any
;~  jint x;                              // screen x-axis co-ordinate in pixels
;~  jint y;                              // screen y-axis co-ordinate in pixels
;~  jint width;                          // pixel width of object
;~  jint height;                         // pixel height of object
;~  BOOL accessibleComponent;            // flags for various additional
;~  BOOL accessibleAction;               // Java Accessibility interfaces
;~  BOOL accessibleSelection;            // FALSE if this object doesn't
;~  BOOL accessibleText;                 // implement the additional interface
;~  BOOL accessibleInterfaces;           // new bitfield containing additional
;~                                       //   interface flags
;~ };
;~     typedef struct AccessibleContextInfoTag {
;~         wchar_t name[MAX_STRING_SIZE];          // the AccessibleName of the object
;~         wchar_t description[MAX_STRING_SIZE];   // the AccessibleDescription of the object

;~         wchar_t role[SHORT_STRING_SIZE];        // localized AccesibleRole string
;~         wchar_t role_en_US[SHORT_STRING_SIZE];  // AccesibleRole string in the en_US locale
;~         wchar_t states[SHORT_STRING_SIZE];      // localized AccesibleStateSet string (comma separated)
;~         wchar_t states_en_US[SHORT_STRING_SIZE]; // AccesibleStateSet string in the en_US locale (comma separated)

;~         jint indexInParent;                     // index of object in parent
;~         jint childrenCount;                     // # of children, if any

;~         jint x;                                 // screen coords in pixels
;~         jint y;                                 // "
;~         jint width;                             // pixel width of object
;~         jint height;                            // pixel height of object

;~         BOOL accessibleComponent;               // flags for various additional
;~         BOOL accessibleAction;                  //  Java Accessibility interfaces
;~         BOOL accessibleSelection;               //  FALSE if this object doesn't
;~         BOOL accessibleText;                    //  implement the additional interface
;~                                                 //  in question

;~         // BOOL accessibleValue;                // old BOOL indicating whether AccessibleValue is supported
;~         BOOL accessibleInterfaces;              // new bitfield containing additional interface flags

;~     } AccessibleContextInfo;


local $hwnd
local $i
local $result
local $vmID
local $ac

;~ //TODO: Check if needed
;~ Make messages elevated
;~ _ChangeWindowMessageFilter($WM_COPYDATA,1)
;~ for $i=$WM_USER to $WM_USER+65536
;~  _ChangeWindowMessageFilter($i, 1)
;~ Next

;~ Func _ChangeWindowMessageFilter($iMsg, $iAction)
;~     Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction)
;~     If @error Or Not $aCall[0] Then Return SetError(1, 0, 0)
;~     Return 1
;~ EndFunc

;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac);
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)
func __getTopLevelObject($vmId,$ac)
;~ Seems not to be allowed to call with null so passing $acParent
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getTopLevelObject", "long", $vmId, $c_JOBJECT64, $ac)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc

;~ typedef AccessibleContext (*GetAccessibleChildFromContextFP) (long vmID, AccessibleContext ac, jint i);
;~ typedef AccessibleContext (*GetAccessibleParentFromContextFP) (long vmID, AccessibleContext ac);
;~ typedef AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.
func __getAccessibleChildFromContext($vmId, $ac, $index)
;~ Seems not to be allowed to call with null so passing $acParent
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getAccessibleChildFromContext", "long", $vmId, $c_JOBJECT64, $ac, "int", $index)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc

;~ _fixBridgeFunc(JOBJECT64,'getActiveDescendent',c_long,JOBJECT64)
func __getActiveDescendent($vmId, $ac)
;~ Seems not to be allowed to call with null so passing $acParent
    $result=dllcall($accessBridgeDll, "ptr:cdecl", "getActiveDescendent", "long", $vmId, $c_JOBJECT64, $ac)
    If @error Then Return SetError(1, 0, 0)
    Return $result[0]
EndFunc
func _Example1_IterateJavaWindows()


;~ Just check all windows if its a java window
Local $var = WinList()
consolewrite("Before checking all Windows: " & $var[0][0] & @CRLF)

For $i = 1 To $var[0][0]
; Only display visble windows that have a title
    If IsVisible($var[$i][1]) Then
        local $handle=wingethandle($var[$i][0])
        $result =dllcall($accessBridgeDll,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle)
        if @error=0 Then
            if $result[0]=1 Then
                consolewrite( "  JAVA window found <" & $i & "> Java Window Title=" & $var[$i][0] &  " Handle=" & $var[$i][1] & @TAB & " res: " &  $result[0] & @CRLF)

            local $acParent=0
            local $vmID=0

;~AccessibilityContext (acParent above), which I had incorrectly mapped as an IntPtr, is actually an Int32 when using the "legacy" WindowsAccessBridge.dll library (used under x86),
;~and an Int64 when using the WOW64 WindowsAccessBridge-32.dll library.
;~WindowsAccessBridge-32.dll bool getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int64 acParent);
;~WindowsAccessBridge.dll    bool _getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int32 acParent);
;~  _fixBridgeFunc(BOOL,'getAccessibleContextFromHWND',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True)
            $result =dllcall($accessBridgeDll,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "UINT64*", $acParent)
;~          $result =dllcall($accessBridgeDll,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $acParent)
;~          $result =dllcall($accessBridgeDll,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent)


                if @error=0 Then
                    consolewrite("Result getAccessibleContextFromHWND is <" & $result & "> ubound: " & ubound($result) & @crlf)
                    $vmID=$result[2]
                    $acParent=$result[3]
                    consolewrite("  We have a VMid " & $vmID & " ac " & $acParent & @CRLF)

;~                  $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo)
;~   typedef BOOL (*GetVersionInfoFP) (long vmID, AccessBridgeVersionInfo *info);
                    $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct*", DllStructGetPtr($tAccessBridgeVersionInfo))

                    if @error=0 Then
                        $s1=dllstructgetdata($tAccessBridgeVersionInfo, "VMVersion")
                        $s2=dllstructgetdata($tAccessBridgeVersionInfo, "bridgeJavaClassVersion")
                        $s3=dllstructgetdata($tAccessBridgeVersionInfo, "bridgeJavaDLLVersion")
                        $s4=dllstructgetdata($tAccessBridgeVersionInfo, "bridgeWinDLLVersion")

                        consolewrite("Call version info: PASSED VMID found : " & $vmID & @CRLF)
                        consolewrite("  VMVersion: <" & $s1 & ">" & @CRLF)
                        consolewrite("  bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF)
                        consolewrite("  bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF)
                        consolewrite("  bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF)
                    Else
                        consolewrite("getVersionInfo error:" & @error  & @CRLF)
                    EndIf

;~ /**
;~      * Returns the Accessible Context for the top level object in
;~      * a Java Window.  This is same Accessible Context that is obtained
;~      * from GetAccessibleContextFromHWND for that window.  Returns
;~      * (AccessibleContext)0 on error.
;~      */
;~     AccessibleContext getTopLevelObject (const long vmID, const AccessibleContext accessibleContext);
;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac);
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)


local $topAC=__getTopLevelObject($vmId, $acparent)
if @error=0 Then
    consolewrite(" *** getTopLevelObject Result is <" & $topAC & "> " & @crlf)

;~  $vmID=$result[1]

;~  local $tPtr = DllStructCreate( "ptr", $result[0] )
;~  local $ptr = DllStructGetData( $tPtr,1 )
;~  local $topAC = $result
;~  DllStructCreate( $tagAccessibleContextInfo, $ptr )

;~  Local $topAC=DllStructCreate($tagAccessibleContextInfo,DllStructGetPtr($result[0]))
;~  Local $topAC=DllStructCreate($tagAccessibleContextInfo,$result[0])

;~  dllstructsetdata($topAC,-1,$result[0]
    consolewrite("We have a VMid " & $vmID & " toplevel object ac " & $topAC & @CRLF)
    $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $topAC, "struct*", $tAccessibleContextInfo)
    $AccessibleContextInfo=$result[3]

    consolewrite(getDescription($AccessibleContextInfo))

    consolewrite("End of top level object info" & @CRLF)

    $childAC=__getAccessibleChildFromContext($vmId, $topAC, 0)
    consolewrite(" *** getAccessibleChildFromContext Result is <" & $childAC & "> " & @crlf)

    $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $childAC, "struct*", $tAccessibleContextInfo)
    $AccessibleContextInfo=$result[3]

    consolewrite(getDescription($AccessibleContextInfo))

    consolewrite("End of getAccessibleChild object info" & @CRLF)

    $descentAC=__getActiveDescendent($vmId, $topAC)
    consolewrite(" *** getActiveDescendent Result is <" & $descentAC & "> " & @crlf)

    $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $descentAC, "struct*", $tAccessibleContextInfo)
    $AccessibleContextInfo=$result[3]

    consolewrite(getDescription($AccessibleContextInfo))

    consolewrite("End of getActiveDescendent object info" & @CRLF)

Else
    consolewrite("getTopLevelObjecterror:" & @error  & @CRLF)
    _WinAPI_DisplayStruct($topAC, $tagAccessibleContextInfo)

endif



                    consolewrite("  We still have a VMid " & $vmID & " acParent " & $acParent & @CRLF)
;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac.
;~                  $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo))
;~ typedef BOOL (*GetAccessibleContextInfoFP) (long vmID, AccessibleContext ac, AccessibleContextInfo *info);
;~  _fixBridgeFunc(BOOL,'getAccessibleContextInfo',c_long,JOBJECT64,POINTER(AccessibleContextInfo),errcheck=True)

;~                  $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "int", $ac, "ptr*", DllStructGetPtr($AccessibleContextInfo))
;~                  $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo),DllStructGetSize($AccessibleContextInfo))
;~                  $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo))
                    $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $acParent, "struct*", $tAccessibleContextInfo)

                    if @error<>0 Then
                        consolewrite("We have an error " & @error & @CRLF)
                    EndIf
;~ We should pass it by reference
;~ Exception code c0000005 is an access violation, also known as general protection fault. The program is reading from, or writing to, an address which is not part of the virtual address space.
;~ A very common cause is that you are de-referencing a stale pointer. In other words, the pointer was once valid, but you have subsequently freed it. Then later when you attempt to access it,
;~ an exception is raised.

;~ The problem is don't use out parameter. Use the IntPtr.
;~ So it should be getAccessibleContextInfo(Int32 vmID, IntPtr accessibleContext, IntPtr acInfo)


                    consolewrite("Result is <" & $result & "> ubound: " & ubound($result) & @crlf)
                    $AccessibleContextInfo=$result[3]
                _WinAPI_DisplayStruct($tAccessibleContextInfo, $tagAccessibleContextInfo)

                    consolewrite(getDescription($AccessibleContextInfo))

                    $s9=dllstructgetdata($AccessibleContextInfo, "x")
                    $s10=dllstructgetdata($AccessibleContextInfo, "y")
                    $s11=dllstructgetdata($AccessibleContextInfo, "width")
                    $s12=dllstructgetdata($AccessibleContextInfo, "height")

consolewrite("  We still have before getAccessible Child FromContext a VMid " & $vmID & " acParent " & $acParent & @CRLF)
consolewrite("rect:" &$s9 & $s9+$s11 & $s10 & $s10+$s12)
 _UIA_DrawRect($s9, $s9+$s11, $s10, $s10+$s12)
sleep(3000)


;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.
    $childAC=__getAccessibleChildFromContext($vmId, $topAC, 0)
    consolewrite(" *** getAccessibleChildFromContext Result is <" & $childAC & "> " & @crlf)

    $result =dllcall($accessBridgeDll, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $childAC, "struct*", $tAccessibleContextInfo)
    $AccessibleContextInfo=$result[3]

    consolewrite(getDescription($AccessibleContextInfo))

    consolewrite("End of getAccessibleChild object info" & @CRLF)

                _WinAPI_DisplayStruct($tAccessibleContextInfo, $tagAccessibleContextInfo)

;~ memcpy(tData->ResponceHTML, m_pResponse, INT_BUFFERSIZE);

                    consolewrite(getDescription($AccessibleContextInfo))


                Else
                    consolewrite( @error & " No context found" & @CRLF)
                endif
            EndIf
        Else
        EndIf
    endif
Next

EndFunc

;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/
func _JABShutDown()
    local $result
    $result=dllcall($accessBridgeDll,"BOOL", "shutdownAccessBridge")
EndFunc

Func IsVisible($handle)
    If BitAND(WinGetState($handle), 2) Then
        Return 1
    Else
        Return 0
    EndIf

EndFunc   ;==>IsVisible

; Draw rectangle on screen.
Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
    Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2
    $x1 = $tLeft
    $x2 = $tRight
    $y1 = $tTop
    $y2 = $tBottom
    $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop)
    $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color)
    $obj_orig = _WinAPI_SelectObject($hDC, $hPen)

    _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right
    _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right
    _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right
    _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left

    ; clear resources
    _WinAPI_SelectObject($hDC, $obj_orig)
    _WinAPI_DeleteObject($hPen)
    _WinAPI_ReleaseDC(0, $hDC)
EndFunc   ;==>_UIA_DrawRect

func getDescription($ac)

                    $s1=dllstructgetdata($ac, "name")
                    $s2=dllstructgetdata($ac, "description")
                    $s3=dllstructgetdata($ac, "role")
                    $s4=dllstructgetdata($ac, "role_en_US")
                    $s5=dllstructgetdata($ac, "states")
                    $s6=dllstructgetdata($ac, "states_en_US")
                    $s7=dllstructgetdata($ac, "indexInParent")
                    $s8=dllstructgetdata($ac, "childrenCount")
                    $s9=dllstructgetdata($ac, "x")
                    $s10=dllstructgetdata($ac, "y")
                    $s11=dllstructgetdata($ac, "width")
                    $s12=dllstructgetdata($ac, "heigth")

                    consolewrite("  name: <" & $s1 & ">" & @CRLF)
                    consolewrite("  description: <" & $s2 & ">" & @CRLF)
                    consolewrite("  role: <" & $s3 & ">" & @CRLF)
                    consolewrite("  role_en_US: <" & $s4 & ">" & @CRLF)
                    consolewrite("  states: <" & $s5 & ">" & @CRLF)
                    consolewrite("  states_en_US: <" & $s6 & ">" & @CRLF)
                    consolewrite("  indexInParent: <" & $s7 & ">" & @CRLF)
                    consolewrite("  childrenCount: <" & $s8 & ">" & @CRLF)
                    consolewrite("  x: <" & $s9 & ">" & @CRLF)
                    consolewrite("  y: <" & $s10 & ">" & @CRLF)
                    consolewrite("  width: <" & $s11 & ">" & @CRLF)
                    consolewrite("  height: <" & $s12 & ">" & @CRLF)

    return $s1 & @crlf & $s2 & @crlf & $s3 & @crlf & $s4 & @crlf & $s5 & @crlf & $s6 & @crlf & $s7 & @crlf & $s8 & @crlf & $s9 & @crlf & $s10 & @crlf & $s11 & @crlf & $s12 & @crlf

EndFunc

Edited January 7, 2015 by junkew

[LarsJ]

junkew, Your code seems to be working fairly good. But I have not looked much at the details.

The textbox is output in Scite on my old XP, where I had JDK 8 installed, and started the SwingSet2 demo. The textbox contains output from two Java windows: the Java Control Panel (which was opened by your script) and the SwingSet2 window.

 

expandcollapse popup

>"E:\Program Files\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "D:\Programmering\AutoIt\Samples\Java Automation\tst01.au3" /UserParams    
+>13:32:23 Starting AutoIt3Wrapper v.2.1.4.4 SciTE v.3.3.7.0 ;  Keyboard:00000406  OS:WIN_XP/Service Pack 3  CPU:X64 OS:X86    Environment(Language:0409  Keyboard:00000406  OS:WIN_XP/Service Pack 3  CPU:X64 OS:X86)
>Running AU3Check (3.3.10.2)  from:E:\Program Files\AutoIt3
+>13:32:23 AU3Check ended.rc:0
>Running:(3.3.10.2):E:\Program Files\AutoIt3\autoit3.exe "D:\Programmering\AutoIt\Samples\Java Automation\tst01.au3"    
--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop
We are using X86 at cpu X64 Autoit 64 bit version 0
  Windows accessbridge 32 opened 1
Windows_run passed :
Before checking all Windows: 265
  JAVA window found <10> Java Window Title= Handle=0x00030048    res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 984398 ac 67533404
Call version info: PASSED VMID found : 984398
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <0x04067A58> 
We have a VMid 984398 toplevel object ac 0x04067A58
  name: <Java Control Panel>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,active>
  states_en_US: <enabled,focusable,visible,showing,active>
  indexInParent: <-1>
  childrenCount: <1>
  x: <538>
  y: <227>
  width: <523>
  height: <0>
Java Control Panel

frame
frame
enabled,focusable,visible,showing,active
enabled,focusable,visible,showing,active
-1
1
538
227
523
0
End of top level object info
 *** getAccessibleChildFromContext Result is <0x04067A34> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <541>
  y: <256>
  width: <517>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
541
256
517
0
End of getAccessibleChild object info
 *** getActiveDescendent Result is <0x00000000> 
  name: <>
  description: <>
  role: <>
  role_en_US: <>
  states: <>
  states_en_US: <>
  indexInParent: <0>
  childrenCount: <0>
  x: <0>
  y: <0>
  width: <0>
  height: <0>






0
0
0
0
0
0
End of getActiveDescendent object info
  We still have a VMid 984398 acParent 67533404
Result is <> ubound: 4
  name: <Java Control Panel>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,active>
  states_en_US: <enabled,focusable,visible,showing,active>
  indexInParent: <-1>
  childrenCount: <1>
  x: <538>
  y: <227>
  width: <523>
  height: <0>
Java Control Panel

frame
frame
enabled,focusable,visible,showing,active
enabled,focusable,visible,showing,active
-1
1
538
227
523
0
  We still have before getAccessible Child FromContext a VMid 984398 acParent 67533404
rect:5381061227787 *** getAccessibleChildFromContext Result is <0x048A8444> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <541>
  y: <256>
  width: <517>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
541
256
517
0
End of getAccessibleChild object info
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <541>
  y: <256>
  width: <517>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
541
256
517
0
  JAVA window found <52> Java Window Title=Java Control Panel Handle=0x00080490  res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 984398 ac 76186628
Call version info: PASSED VMID found : 984398
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <0x03F29A08> 
We have a VMid 984398 toplevel object ac 0x03F29A08
  name: <Java Control Panel>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,active>
  states_en_US: <enabled,focusable,visible,showing,active>
  indexInParent: <-1>
  childrenCount: <1>
  x: <538>
  y: <227>
  width: <523>
  height: <0>
Java Control Panel

frame
frame
enabled,focusable,visible,showing,active
enabled,focusable,visible,showing,active
-1
1
538
227
523
0
End of top level object info
 *** getAccessibleChildFromContext Result is <0x03F299D4> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <541>
  y: <256>
  width: <517>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
541
256
517
0
End of getAccessibleChild object info
 *** getActiveDescendent Result is <0x00000000> 
  name: <>
  description: <>
  role: <>
  role_en_US: <>
  states: <>
  states_en_US: <>
  indexInParent: <0>
  childrenCount: <0>
  x: <0>
  y: <0>
  width: <0>
  height: <0>






0
0
0
0
0
0
End of getActiveDescendent object info
  We still have a VMid 984398 acParent 76186628
Result is <> ubound: 4
  name: <Java Control Panel>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,active>
  states_en_US: <enabled,focusable,visible,showing,active>
  indexInParent: <-1>
  childrenCount: <1>
  x: <538>
  y: <227>
  width: <523>
  height: <0>
Java Control Panel

frame
frame
enabled,focusable,visible,showing,active
enabled,focusable,visible,showing,active
-1
1
538
227
523
0
  We still have before getAccessible Child FromContext a VMid 984398 acParent 76186628
rect:5381061227787 *** getAccessibleChildFromContext Result is <0x03F27A2C> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <541>
  y: <256>
  width: <517>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
541
256
517
0
End of getAccessibleChild object info
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <541>
  y: <256>
  width: <517>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
541
256
517
0
  JAVA window found <79> Java Window Title=SwingSet2 Handle=0x0002071E   res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 198344 ac 68737820
Call version info: PASSED VMID found : 198344
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <0x0418DB18> 
We have a VMid 198344 toplevel object ac 0x0418DB18
  name: <SwingSet2>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,resizable>
  states_en_US: <enabled,focusable,visible,showing,resizable>
  indexInParent: <-1>
  childrenCount: <1>
  x: <411>
  y: <202>
  width: <728>
  height: <0>
SwingSet2

frame
frame
enabled,focusable,visible,showing,resizable
enabled,focusable,visible,showing,resizable
-1
1
411
202
728
0
End of top level object info
 *** getAccessibleChildFromContext Result is <0x0418DAEC> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <415>
  y: <232>
  width: <720>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
415
232
720
0
End of getAccessibleChild object info
 *** getActiveDescendent Result is <0x00000000> 
  name: <>
  description: <>
  role: <>
  role_en_US: <>
  states: <>
  states_en_US: <>
  indexInParent: <0>
  childrenCount: <0>
  x: <0>
  y: <0>
  width: <0>
  height: <0>






0
0
0
0
0
0
End of getActiveDescendent object info
  We still have a VMid 198344 acParent 68737820
Result is <> ubound: 4
  name: <SwingSet2>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,resizable>
  states_en_US: <enabled,focusable,visible,showing,resizable>
  indexInParent: <-1>
  childrenCount: <1>
  x: <411>
  y: <202>
  width: <728>
  height: <0>
SwingSet2

frame
frame
enabled,focusable,visible,showing,resizable
enabled,focusable,visible,showing,resizable
-1
1
411
202
728
0
  We still have before getAccessible Child FromContext a VMid 198344 acParent 68737820
rect:4111139202899 *** getAccessibleChildFromContext Result is <0x0412ABFC> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <415>
  y: <232>
  width: <720>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
415
232
720
0
End of getAccessibleChild object info
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <415>
  y: <232>
  width: <720>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
415
232
720
0
  JAVA window found <89> Java Window Title=SwingSet2 Handle=0x001E0040   res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 198344 ac 68332488
Call version info: PASSED VMID found : 198344
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <0x0412ABC0> 
We have a VMid 198344 toplevel object ac 0x0412ABC0
  name: <SwingSet2>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,resizable>
  states_en_US: <enabled,focusable,visible,showing,resizable>
  indexInParent: <-1>
  childrenCount: <1>
  x: <411>
  y: <202>
  width: <728>
  height: <0>
SwingSet2

frame
frame
enabled,focusable,visible,showing,resizable
enabled,focusable,visible,showing,resizable
-1
1
411
202
728
0
End of top level object info
 *** getAccessibleChildFromContext Result is <0x0412AB8C> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <415>
  y: <232>
  width: <720>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
415
232
720
0
End of getAccessibleChild object info
 *** getActiveDescendent Result is <0x00000000> 
  name: <>
  description: <>
  role: <>
  role_en_US: <>
  states: <>
  states_en_US: <>
  indexInParent: <0>
  childrenCount: <0>
  x: <0>
  y: <0>
  width: <0>
  height: <0>






0
0
0
0
0
0
End of getActiveDescendent object info
  We still have a VMid 198344 acParent 68332488
Result is <> ubound: 4
  name: <SwingSet2>
  description: <>
  role: <frame>
  role_en_US: <frame>
  states: <enabled,focusable,visible,showing,resizable>
  states_en_US: <enabled,focusable,visible,showing,resizable>
  indexInParent: <-1>
  childrenCount: <1>
  x: <411>
  y: <202>
  width: <728>
  height: <0>
SwingSet2

frame
frame
enabled,focusable,visible,showing,resizable
enabled,focusable,visible,showing,resizable
-1
1
411
202
728
0
  We still have before getAccessible Child FromContext a VMid 198344 acParent 68332488
rect:4111139202899 *** getAccessibleChildFromContext Result is <0x036E0F40> 
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <415>
  y: <232>
  width: <720>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
415
232
720
0
End of getAccessibleChild object info
  name: <>
  description: <>
  role: <root pane>
  role_en_US: <root pane>
  states: <enabled,focusable,visible,showing,opaque>
  states_en_US: <enabled,focusable,visible,showing,opaque>
  indexInParent: <0>
  childrenCount: <2>
  x: <415>
  y: <232>
  width: <720>
  height: <0>


root pane
root pane
enabled,focusable,visible,showing,opaque
enabled,focusable,visible,showing,opaque
0
2
415
232
720
0
+>13:33:56 AutoIt3.exe ended.rc:0
+>13:33:56 AutoIt3Wrapper Finished..
>Exit code: 0    Time: 93.511

I spend some time to try to get the JavaFerret and JavaMonkey tools to work. But I didn't succeed. I found the tools in an old version of the Jave Access Bridge from april 2011. But the tools are probably not working with the newest version of the Access Bridge. Are these tools working for you? Do you have newer versions? Are you aware of other tools to inspect Java windows?

[junkew]

@LarsJ: thx for confirming it works

I installed for the tools to work version 2.0.2 (but they are not needed for running but indeed ferret and monkey give you just like inspect more then with a simple spy):

accessbridge-2_0_2-fcs-bin-b06accessbridge2_0_2

Tools JavaFerret and JavaMonkey are only in the java accessibility bridge sdk 2.0.2. and tools still work with Java 1.7 and Java 1.8

I was unable to compile them from source due to missing include files but still .h and .c files gave a lot of usefull information

Yes the tools work well under jdk1.8/jre1.8 after you turn on JAB with: JABSWITCH /ENABLE 

I did install older versions for more source reference like accessbridge-2_0_1-manual_install

other tools not I am aware of Google gives some directions: JAVA Spy++

So far I think with Java Accessibility bridge you can get the basic information from SWT/AWT Java ui objects

The open source screenreader NVDA with JABHANDLER.PY gives a source direction 

Not much information on JAB and hard to find the details and a lot of source and trial I did and this thread is the usefull endresult.

Will make a JAB Simple Spy and post it in examples section together with the basic JAB Functions

[LarsJ]

You are right, JavaFerret and JavaMonkey works after JABSWITCH /ENABLE. Especially JavaMonkey which shows the structure of a Java window can be useful. Is there anything that prevents extracting the same structure with AutoIt code?

[junkew]

No as far as i can see all can work with the basics in this thread hopefully coming weeks i can find some time to wrap up in an udf