Sign in to follow this  
Followers 0
Akashrai

autoit info tool is not detecting GUI components

20 posts in this topic

I want to automate a client application but auto it info tool is not detecting the GUI components. I don't know what to do. please help me out. Screenshot  of first login page of application is in attachment 

 

visa.png

Share this post


Link to post
Share on other sites



Please help It is very important. Alternate of auto it is also welcome.  

Share this post


Link to post
Share on other sites

There are lots of alternative GUI inspection tools, many of them free. I use Ranorex myself, but that one is now proprietary.

1 person likes this

Share this post


Link to post
Share on other sites

Yes but what about auto it why it is not detecting the GUI components and what and way to achieve the solution i am attaching page after login it also has same problem.  

visa1.png

Share this post


Link to post
Share on other sites

The "AutoIt Window Info" tool generally only detects standard Windows controls implemented with the functions in ComCtl32.dll. These controls have existed since Windows XP. You can find a list of the controls in the Control Library.

Many modern controls including modern MicroSoft controls are not detected by the "AutoIt Window Info" tool. These controls can be detected with Inspect.exe in Windows SDK or the "Simple Spy Tool" in the UI Automation framework.

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

I have fetched info by Autoit info tool and attached please help. You can see it is not detecting any thing.

Please help

Login.png

Edited by Akashrai

Share this post


Link to post
Share on other sites

Where I will get simplespy.au3 file . I am new so please help. I have searched on google as well as in auto it setup but did not got .

Share this post


Link to post
Share on other sites

Yes, Gone through your given link and also searched on google but :sweating:. Could you provide link to download the same.

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

I have tried it . It is also not detecting .

The VISA application which i am automating is a java (Applet ) application. I have searched for that and got "FEST" but do not know how to get id/class name of GUI component of the VISA application. Is there any spy tool in fest.

Other that FEST any other open source tool to automate applet desktop application?

Please help.

Edited by Akashrai

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

Thanks for support.

Here it says ctrl+w. After pressing Ctrl+w  it gives me following code -

Mouse position is retrieved 133-54
  JAVA window found <> Java Window Title= Handle=0x00000000002A0740     res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 789588 ac 369080852
Call version info: PASSED VMID found : 789588
  VMVersion: <1.8.0_65>
  bridgeJavaClassVersion: <1.8.0_65>
  bridgeJavaDLLVersion: <1.8.0_65>
  bridgeWinDLLVersion: <AccessBridge 2.0.4>
 *** getAccessibleContextAT result is <369081228>
 *** getAccessibleContextAT result is <369081228>
Paste

push button
push button
enabled,focusable,visible,showing
enabled,focusable,visible,showing
2
0
98
35
46
0
End of getAcceccibleContextAt info

 

How to use above info in my autoit script. By which Detail of button i cab use.

The application on which i am testing is bellow

 

 

applet.PNG

Edited by Akashrai

Share this post


Link to post
Share on other sites

For that you have to read a lot on java accessibility bridge and study source of jaws and other open source implementations of java accessibility bridge. It all starts with building a ui tree and traversing that to get the right item. And if its internally developed explain developers that they have to fill the accessibility properties.

1 person likes this

Share this post


Link to post
Share on other sites

Thanks for reply.

But the application i want to automate is on client side and and we do not know any thing except GUI. In the case it seems impossible by auto it.

I have searched for any other tool/spy(FREE) but did not get much. I went through Fest but do not know how to start. Any way thanks again.        

Share this post


Link to post
Share on other sites

search harder ;-)

 

https://github.com/ckundo/nvda/tree/master/source

http://www.oracle.com/technetwork/java/javase/downloads/downloads-jsp-138220.html    ferret and monkey

it is possible from AutoIT and the JAB Simplespy shows you the basics

1. Get an element : BOOL GetAccessibleContextAt(long vmID, AccessibleContext acParent, jint x, jint y, AccessibleContext *ac)

2. Get an info reference BOOL GetAccessibleContextInfo(long vmID, AccessibleContext ac, AccessibleContextInfo *info);

3. Get the text by using the reference you get in step 2 BOOL GetAccessibleTextInfo(long vmID, AccessibleText at, AccessibleTextInfo *textInfo, jint x, jint y);

 

 

Share this post


Link to post
Share on other sites

I searched a little further and played with JABSimpleSpy

Its all working with demo's as can be found in JDK ..\jdk-8u25-windows-i586-demos\jdk1.8.0_25\demo\jfc\SwingSet2

And as you have shown you get with JABSimpleSpy the text of your button the concept fully works

Combined with this where we iterate the tree from top to bottom

 

 

another AWT monitor found using same Java Access Bridge

https://docs.oracle.com/cd/E17802_01/j2se/javase/technologies/accessibility/docs/jaccess-1.3/doc/AWTMonitor.html

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • VaishnaviBUtpat
      By VaishnaviBUtpat
      <!DOCTYPE html> <html lang="en" xml:lang="en" style="height: 100%;" xmlns="http://www.w3.org/1999/xhtml"> <head> <title></title> <style> * { margin: 0; padding: 0; } .th-lk { color: #3665d0; font-family: Arial; font-size: small; text-decoration: none; } .th-lk { vertical-align: 0px; } .th-menu2 .th-lk { line-height: 2em; margin-bottom: 0px; margin-right: 0px; overflow: hidden; padding: 0; text-decoration: none; text-overflow: ellipsis; white-space: nowrap; width: 100%; } .th-menu2 .th-lk { color: black; font-weight: bold; } .th-menu2 > li > .th-lk { display: block; padding-left: 8px; width: auto; } .th-menu2 .th-menu2-sub-item .th-lk, .th-menu2 .th-menu2-sub-item-hov .th-lk { margin-right: 20px; } .th-menu2-sub-item { position: relative !important; } .th-menu2 .th-menu2-item, .th-menu2 .th-menu2-item-hov, .th-menu2 .th-menu2-sub-item, .th-menu2 .th-menu2-sub-item-hov { background-repeat: repeat-x; border-left-style: solid; border-left-width: 1px; border-right-style: solid; border-right-width: 1px; border-top-style: solid; border-top-width: 1px; height: 2em; list-style: none; margin-bottom: 0px; padding: 0; width: 100%; } .th-menu2 .th-menu2-item, .th-menu2 .th-menu2-item-hov, .th-menu2 .th-menu2-sub-item, .th-menu2 .th-menu2-sub-item-hov { background-color: #ECECEC; background-image: url(sap_skins/default/styling/lshape/chg_butt_det_nav.gif); border-left-color: #d3d1ce; border-right-color: #d3d1ce; border-top-color: #d3d1ce; border-top-width: 0px; } .th-menu2 { border: 0 solid black; left: 0px; list-style: none; margin: 0; padding: 0; position: relative; } .th-menu2 { z-index: 10006; } .th-menu2 { background-color: white; } div { zoom: 1; } .th-sc-content { left: 0px; position: absolute; top: 0px; } .th-sc-container { left: 0px; overflow: hidden; position: relative; top: 0px; } .th-sc-top { position: relative; } .th-sc-top, .th-sc-content, .th-sc-container, .th-sc-buttondown, .th-sc-buttonup { width: 172px; } .th-sc-buttonup, .th-sc-container { z-index: 10101; } .th-sc-top { z-index: 10100; } body, td, th { font-family: Arial,Helvetica,sans-serif; font-size: small; } .th-l-navcontainer, .th_l_downcontainer { border-right-style: solid; border-right-width: 1px; width: 172px; } .th-l-navcontainer, .th_l_downcontainer { background-color: white; border-right-color: #d3d1ce; } body, html { margin: 0px; border: 0; margin: 0; } </style> </head> <body><form name="myFormId" id="myFormId" action="/sap(ZT1TVVJEWDFWVFVsOWZYMTlmTWpNNU9UWmZXWTlwZG5telZ1RGhBSUFBQ3Nyc2tBPT0=)/bc/bsp/sap/crm_ui_frame/BSPWDApplication.do?sap-client=100&amp;sap-language=EN&amp;sap-domainrelax=min" method="post" target="WorkAreaFrame2"><div class="th-ajax-area" id="rootAreaDiv"><div id="C1_W1_V2" tgt="" dhe="false"><table width="100%" style="table-layout: fixed;" cellspacing="0" cellpadding="0"><tbody><tr><td><table width="100%" style="table-layout: fixed;" cellspacing="0" cellpadding="0"><tbody><tr valign="top"><td class="th-l-navcontainer" id="th_l_navcontainer"><div class="th-sc-top" id="C1_W1_V2_thescroll" style="height: 786px;"><div class="th-sc-container" id="C1_W1_V2_thescroll_scbox" style="height: 786px;"><div class="th-sc-content" id="C1_W1_V2_thescroll_sccontent"><div class="th-ajax-area" id="C1_W1_V2_$navbar"><div id="C7_W35_V36" tgt="" dhe="true" excevt="" intevt="c:C7_W35_V36:C1_W1_V2_C7_W35_V36_MainNavigationLinks.do;" automode="true"><div class="th-ajax-area" id="C1_W1_V2_C7_W35_V36_MainNavigationLinks.do"><ul class="th-menu2" id="C7_W35_V36_mainmenu" style="width: 171px;"><li class="th-menu2-sub-item"><a title="Sales Cycle" class="th-lk" id="C7_W35_V36_UTL-SLS" onclick="htmlbSubmitLib('htmlb',this,'thtmlb:link:click:0','myFormId','C7_W35_V36_UTL-SLS','UTL\x2dSLS\x2dWC',0);return false" onfocus="thSaveKbFocus(this);" oncontextmenu="return false;" href="javascript:void(0)">Sales Cycle</a></li></ul></div></div></div></div></div></div></td></tr></tbody></table></td></tr></tbody></table></div></div></form></body> </html> How to capture above HTML element using AutoIT
    • cu0x
      By cu0x
      Hello guys,
       
      im trying to solved a problem that I have.
       
      Need to get some chinese text from an old Wise script, and in the wise file says f.e. Ù×÷ϵͳ¡£ ÇëÉý¼¶Ä. Is there any way to convert it to traditional chinese?
       
      Already tryied the following code...
       
      #include <MsgBoxConstants.au3> Example() Func Example() ; Define the string that will be converted later. ; NOTE: This string may show up as ?? in the help file and even in some editors. ; This example is saved as UTF-8 with BOM. It should display correctly in editors ; which support changing code pages based on BOMs. Local Const $sString = "Ù×÷ϵͳ¡£ ÇëÉý¼¶Ä" ; Temporary variables used to store conversion results. $dBinary will hold ; the original string in binary form and $sConverted will hold the result ; afte it's been transformed back to the original format. Local $dBinary = Binary(""), $sConverted = "" ; Convert the original UTF-8 string to an ANSI compatible binary string. $dBinary = StringToBinary($sString) ; Convert the ANSI compatible binary string back into a string. $sConverted = BinaryToString($dBinary) ; Display the resulsts. Note that the last two characters will appear ; as ?? since they cannot be represented in ANSI. DisplayResults($sString, $dBinary, $sConverted, "ANSI") ; Convert the original UTF-8 string to an UTF16-LE binary string. $dBinary = StringToBinary($sString, 2) ; Convert the UTF16-LE binary string back into a string. $sConverted = BinaryToString($dBinary, 2) ; Display the resulsts. DisplayResults($sString, $dBinary, $sConverted, "UTF16-LE") ; Convert the original UTF-8 string to an UTF16-BE binary string. $dBinary = StringToBinary($sString, 3) ; Convert the UTF16-BE binary string back into a string. $sConverted = BinaryToString($dBinary, 3) ; Display the resulsts. DisplayResults($sString, $dBinary, $sConverted, "UTF16-BE") ; Convert the original UTF-8 string to an UTF-8 binary string. $dBinary = StringToBinary($sString, 4) ; Convert the UTF8 binary string back into a string. $sConverted = BinaryToString($dBinary, 4) ; Display the resulsts. DisplayResults($sString, $dBinary, $sConverted, "UTF8") EndFunc ;==>Example ; Helper function which formats the message for display. It takes the following parameters: ; $sOriginal - The original string before conversions. ; $dBinary - The original string after it has been converted to binary. ; $sConverted- The string after it has been converted to binary and then back to a string. ; $sConversionType - A human friendly name for the encoding type used for the conversion. Func DisplayResults($sOriginal, $dBinary, $sConverted, $sConversionType) MsgBox($MB_SYSTEMMODAL, "", "Original:" & @CRLF & $sOriginal & @CRLF & @CRLF & "Binary:" & @CRLF & $dBinary & @CRLF & @CRLF & $sConversionType & ":" & @CRLF & $sConverted) EndFunc ;==>DisplayResults Thanks a lot!
    • nacerbaaziz
      By nacerbaaziz
      Hi dear
      I want create retractable bar using autoit
      I tried creating slider, but there's a problem with screen reader for the blind, so is there another retractable tape?
      It is advisable to not accept dragging with the keybord only with  mouse
      note:
      This bar is needed in the process of raising and lowering the volume
      I hope that there is a solution to do that
      i waiting your responses.
      Thanks in advance to all members and administrators
    • XanzyX
      By XanzyX
      Is there a function out there that will edit an existing file witha a "Save" and "Cancel" at the bottom?
      Example: Funcrion("TestFile.txt")
    • rte7
      By rte7
      Hello. We work with the BMC Remedy site to manage incidents and would like to automate the processes of clicking through this button code:
      <a href="javascript:" class="btn btn3d menu" style="top:21; left:184; width:21; height:21;"> <img class="btnimg" src="../../../../resources/images/mt_sprites.gif" alt="menu category issue" title=""> </a> I use firefox with mozrepl and autoit with include FF.au3
      I need your help
      I read the topic https://www.autoitscript.com/forum/topic/95595-ffau3-v0601b-10/ but I could not run the code from the listed commands
      ...
      It is necessary to press the button and from the list of the menu of cahorsations to select the item = "zone of trust abonent"
      Screen # 1 http://joxi.ru/52aJK7bh4a7ypA or below.
      1) Press button
      <a href="javascript:" class="btn btn3d menu" style="top:21; left:184; width:21; height:21;"> <img class="btnimg" src="../../../../resources/images/mt_sprites.gif" alt="menu category issue" title=""> </a>
      2) After pressing the button, I need to choose in this menu of 4 items to select an item called "zone of trust abonent", Screen # 2 - http://joxi.ru/p27gwQKHo71VQ2
      html code table items
      <table class="MenuTable" cellpadding="0" cellspacing="0" style="width: 179px;"> <tbody class="MenuTableBody"> <tr class="MenuTableRow"> <td class="MenuEntryName" nowrap="">zone of trust abonent</td> <td class="MenuEntryNoSub" arvalue="zone of trust abonent"></td> </tr> <tr class="MenuTableRow"><td class="MenuEntryName" nowrap="">zone of trust operator</td> <td class="MenuEntryNoSub" arvalue="zone of trust operator"></td> </tr> <tr class="MenuTableRow"><td class="MenuEntryName" nowrap="">zone of trust third operator</td> <td class="MenuEntryNoSub" arvalue="zone of trust third operator"></td></tr> <tr class="MenuTableRow"><td class="MenuEntryNameHover" nowrap="">No settings</td> <td class="MenuEntryNoSubHover" arvalue="No settings"></td> </tr> </tbody> </table>
       
       
      More in detail, I choose from the menu item "zone of trust abonent" Screen # 3 - http://joxi.ru/GrqVq4ZIQEPW4m

       
      3) The choice is made, Screen # 4 http://joxi.ru/v297w6aC3PovLr
      <textarea class="text sr " wrap="off" id="arid_WIN_0_1000002488" cols="20" maxlen="60" style="top:21; left:0; width:179; height:21;" armenu="CFG:PRC:CategoryHPD-Res-T1-Q" mstyle="2" arautoc="1" arautocmb="1" arautocak="0" arautoctt="400" rows="1"> </textarea>
       
      It's all. I would be very grateful to you =) !!!!!!!!!!
      Here are more detailed codes in the attachment. 
      button1.txt
      menu - table.txt
      all.txt