Could not find Java Application control name

[caramen]

Well that mean you got nothing.

I whould just mouseclick in this case.

[KhalidAnsari]

Hi,

Thanks for reply @caramen

This is just my sample screen. Which I am trying to automate before implementing to actual screen.

Mouse click I have already achieved through mouse pointer position i can able to set value or sending Tab key I need to set the value as finding name of control and click the button.  Any other option i can try. 

 

[caramen]

I dont understand can you repeat with what are you trying to do ?

[KhalidAnsari]

I am trying to find control of java screen and set value in textbox and click button based on the name or class name. Instead of mouseclick. I could not find textbox, button name from simplespy window.  As i said before if i place my mouse pointer on control then press Crtrl+w instead of showing me java screen detail. It shows me desktop screen detail.

My major issue i cannot able to find any detail of child window.  First  i am trying to automate with my sample screen. 

Hope i can able to make you understand.

Thanks

[junkew]

Java app will only be possible with jab bridge

[KhalidAnsari]

Hi 

My JabSimpleSpy is errorring  out.  I have enable Jabswitch.  My JabSimpleSpy stop responding when i move focus on my java application. My Java version is 1.8.05.  When I open JabSimpleSpy Javacpl.exe is also open with it. I am using 64 bit. 

Details of Java accessibility bridge dll 

Edited November 20, 2018 by KhalidAnsari
need to add another screen detail

[caramen]

Out of my knowledge sry. Hope you gonna success. Bye

[junkew]

Try 32 bits version

[KhalidAnsari]

Hi 

Thanks for the reply. @junkew and @caramen

I can able find the detail of java app. Once Thanks for the help.  I can able the set the value in my textbox below is my code through which i can set value 

Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=RichEdit20W", $treescope_children)
_UIA_Action($oP1,"setfocus")
_UIA_action($oP1,"setvalue", "ABCDE9999F")

But I cannot able to set value and click button of other control like "Submit" button.

ocal $oP0=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=JavaApplication;controltype:=UIA_WindowControlTypeId;class:=SunAwtFrame", $treescope_children)    
_UIA_Action($oP0,"setfocus")
;_UIA_setVar("Submit.mainwindow","title:=Submit;classname:=Button")
;_UIA_action("Submit.mainwindow","setfocus")

_UIA_setVar($oP0,"title:=Submit;classname:=Button")
_UIA_action($oP0,"setfocus")

_UIA_action($oP0,"click")

------------------------------

Below is the detail from simplespy window along with screenshot

*** Detailed properties of the highlighted element ***
UIA_title:= <Submit>
UIA_text:= <Submit>
UIA_regexptitle:= <Submit>
UIA_class:= <Button>
UIA_regexpclass:= <Button>
UIA_iaccessiblechildId:= <0>
UIA_id:= <5>
UIA_handle:= <329040>
UIA_RuntimeId:= <42;329040>
UIA_BoundingRectangle:= <8;211;192;90>
UIA_ProcessId:= <4592>
UIA_ControlType:= <50000>
UIA_LocalizedControlType:= <button>
UIA_Name:= <Submit>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <5>
UIA_ClassName:= <Button>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <329040>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <Win32>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Submit>
UIA_LegacyIAccessibleRole:= <43>
UIA_LegacyIAccessibleState:= <1048576>
UIA_LegacyIAccessibleDefaultAction:= <Press>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:3676,hwnd:0x50550 Main:Nested [pid:4592,hwnd:0x50550 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

Edited November 23, 2018 by KhalidAnsari

[caramen]

You are more close day after day dont give up !  

[junkew]

Ok you now seem to be using uia that uses the iaccessible interface. If that works for your java app its fine. Be aware in general to get more you have to use java accessibility bridge. Jabsimplespy can help but i didnt fix all structures to be 32 and 64 bit compatible that probably why spy breaks on 64 bits.

[junkew]

@KhalidAnsari Whats the JABSimpleSpy giving in the output window?
It should start with something like below (normally all 64 bits)

C:\Program Files\Java\jre1.8.0_181
We are using X64 at cpu X64 Autoit 64 bit version @AutoItX64=1
Opening C:\Program Files\Java\jre1.8.0_181\bin\WindowsAccessBridge-64.dll  Windows accessbridge 32 opened 1
 0 initializeAccessBridge is finishedWindows_run passed :

 

[KhalidAnsari]

@junkew thanks for reply. I will check it. My JabSimplyspy was erroring out before. i need to recheck it. I

[KhalidAnsari]

Hi @junkew

For JabSimpleSpy i tried following this install  java  version jdk1.8.0_191. Install Jaws 64. Download Java Access Bridge dll. place the dll on jre\bin\ folder. After this Jabsimple spy is not erroring out. But It does not display any details of Java app screen. JabSimpleSpy screen is blank and empty. 

 

[junkew]

Just to double check. Did you hover over about button and pressed ctrl w

Running it from scite?

Edited November 27, 2018 by junkew

[KhalidAnsari]

@junkew Thanks for reply.  I have done hover over button and ctrl w as well. I am running JabSimpleSpy as exe.  This is how i run

1. Open my Java Screen.

2. Open JabSimpleSpy.exe.  Place cursor on Java application then Ctrl + w  

3. Move cursor over it as well.

JabSpy screen is empty.

 

 

[KhalidAnsari]

Hi @junkew

I tried to debug JabSimpleSpy.au3 to find out the reason why it is not showing detail. I tried to have msgbox print for me. You can able to guide me more looking into info. 

;~ Make sure Java Access Bridge is turned on
    runwait($sJavaHome & "\bin\jabswitch /enable","",@SW_MAXIMIZE)

    ;~ TODO: Check if it works with both dll's
    consolewrite("We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF)
    MsgBox(0,"JabSpy","We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF)

    if @AutoItX64 = 1 then
        $sAccessBridgeDLL="WindowsAccessBridge-64.dll"
    Else
        $sAccessBridgeDLL="WindowsAccessBridge-32.dll"
    EndIf

    ;~ Open the Dll for accessibility
    $hAccessBridgeDll=DLLOPEN($sJavaHome & "\bin\" & $sAccessBridgeDLL)

    If $hAccessBridgeDll=true Then
        consolewrite("  Windows accessbridge 32 opened " & $hAccessBridgeDll & @CRLF)
        MsgBox(0," Windows accessbridge 32 opened ","  Windows accessbridge 32 opened " & $hAccessBridgeDll & @CRLF)
    Else
        consolewrite("  DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows")
        MsgBox(0," Windows accessbridge 32 opened ","  DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows")
    endIf

[junkew]

yes, i know the jabspy screen edit box will not show anything on what is loaded.

You should run it from scite and see what comes in the output window of scite. 

What you now try to do will work also but is not needed if you run from scite editor.

 

But it basically says dll is *** NOT *** loaded ;-) 

try to load dll in different ways with full path and check both dll 32 and 64 bits versions

https://www.autoitscript.com/autoit3/docs/functions/DllOpen.htm

 

 

 

[KhalidAnsari]

Hi @junkew

I am running the script from SciTe. I complie JapsimpleSpy.au3 script then run JabsimpleSpy.exe. Regarding Dll full path. I will make necessary changes as suggested by you.

Sorry for stupid question how can i check the detail in console window. 

[junkew]

Just choose run in scite instead of compiling