Jump to content
Sign in to follow this  
Akashrai

autoit info tool is not detecting GUI components

Recommended Posts

I want to automate a client application but auto it info tool is not detecting the GUI components. I don't know what to do. please help me out. Screenshot  of first login page of application is in attachment 

 

visa.png

Share this post


Link to post
Share on other sites

There are lots of alternative GUI inspection tools, many of them free. I use Ranorex myself, but that one is now proprietary.

Share this post


Link to post
Share on other sites

Yes but what about auto it why it is not detecting the GUI components and what and way to achieve the solution i am attaching page after login it also has same problem.  

visa1.png

Share this post


Link to post
Share on other sites

The "AutoIt Window Info" tool generally only detects standard Windows controls implemented with the functions in ComCtl32.dll. These controls have existed since Windows XP. You can find a list of the controls in the Control Library.

Many modern controls including modern MicroSoft controls are not detected by the "AutoIt Window Info" tool. These controls can be detected with Inspect.exe in Windows SDK or the "Simple Spy Tool" in the UI Automation framework.

Share this post


Link to post
Share on other sites

please give a little more details on supplier or on the mainwindow component that is recognized

Check with Au3Inf and Simplespy

Based on that you get an indication if its build in C/C++, Java, .NET, QTxx or other frameworks

Share this post


Link to post
Share on other sites

I have fetched info by Autoit info tool and attached please help. You can see it is not detecting any thing.

Please help

Login.png

Edited by Akashrai

Share this post


Link to post
Share on other sites

I have tried it . It is also not detecting .

The VISA application which i am automating is a java (Applet ) application. I have searched for that and got "FEST" but do not know how to get id/class name of GUI component of the VISA application. Is there any spy tool in fest.

Other that FEST any other open source tool to automate applet desktop application?

Please help.

Edited by Akashrai

Share this post


Link to post
Share on other sites

Thanks for support.

Here it says ctrl+w. After pressing Ctrl+w  it gives me following code -

Mouse position is retrieved 133-54
  JAVA window found <> Java Window Title= Handle=0x00000000002A0740     res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 789588 ac 369080852
Call version info: PASSED VMID found : 789588
  VMVersion: <1.8.0_65>
  bridgeJavaClassVersion: <1.8.0_65>
  bridgeJavaDLLVersion: <1.8.0_65>
  bridgeWinDLLVersion: <AccessBridge 2.0.4>
 *** getAccessibleContextAT result is <369081228>
 *** getAccessibleContextAT result is <369081228>
Paste

push button
push button
enabled,focusable,visible,showing
enabled,focusable,visible,showing
2
0
98
35
46
0
End of getAcceccibleContextAt info

 

How to use above info in my autoit script. By which Detail of button i cab use.

The application on which i am testing is bellow

 

 

applet.PNG

Edited by Akashrai

Share this post


Link to post
Share on other sites

For that you have to read a lot on java accessibility bridge and study source of jaws and other open source implementations of java accessibility bridge. It all starts with building a ui tree and traversing that to get the right item. And if its internally developed explain developers that they have to fill the accessibility properties.

Share this post


Link to post
Share on other sites

Thanks for reply.

But the application i want to automate is on client side and and we do not know any thing except GUI. In the case it seems impossible by auto it.

I have searched for any other tool/spy(FREE) but did not get much. I went through Fest but do not know how to start. Any way thanks again.        

Share this post


Link to post
Share on other sites

search harder ;-)

 

https://github.com/ckundo/nvda/tree/master/source

http://www.oracle.com/technetwork/java/javase/downloads/downloads-jsp-138220.html    ferret and monkey

it is possible from AutoIT and the JAB Simplespy shows you the basics

1. Get an element : BOOL GetAccessibleContextAt(long vmID, AccessibleContext acParent, jint x, jint y, AccessibleContext *ac)

2. Get an info reference BOOL GetAccessibleContextInfo(long vmID, AccessibleContext ac, AccessibleContextInfo *info);

3. Get the text by using the reference you get in step 2 BOOL GetAccessibleTextInfo(long vmID, AccessibleText at, AccessibleTextInfo *textInfo, jint x, jint y);

 

 

Share this post


Link to post
Share on other sites

I searched a little further and played with JABSimpleSpy

Its all working with demo's as can be found in JDK ..\jdk-8u25-windows-i586-demos\jdk1.8.0_25\demo\jfc\SwingSet2

And as you have shown you get with JABSimpleSpy the text of your button the concept fully works

Combined with this where we iterate the tree from top to bottom

 

 

another AWT monitor found using same Java Access Bridge

https://docs.oracle.com/cd/E17802_01/j2se/javase/technologies/accessibility/docs/jaccess-1.3/doc/AWTMonitor.html

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • By nacerbaaziz
      hello all, and welcome to this tool
      the NB-Password_generator is a small tool which allow you to create a strong passwords
      with this tool you can create a random passwords using :
      1. capital letters
      2. small letters
      3. numbers
      4. symbols
      be sure that you can check any option that  you want and uncheck what you don't want to use
      this tool allow you to create a password from 6 letters to 150 lettersNB-Password_generator.zip
      at the end please accept my greetings
      am waiting for your commants
    • By Codenameglassy
      Hello so i need help, so i have a while(1) Loop but i want to restart the loop form the begining if a specific event occurs or what i want on screen is occured.. i want  to restart the loop from beginning , leaving the rest of the script.
      But if the event didnt occur it follows the rest of the script and goes on while loop..
    • By TrashBoat
      Im trying to iterate through a 2d array created by 
      ProcessList() function that returns a 2d array containing names and pid's, i wanted to filter through the names and delete entries that have "svchost.exe" and this is how i did it
      Func _Sanitize($array) For $x = 1 To UBound($array) - 1 ConsoleWrite($array[$x][0] & @CRLF) If $array[$x][0] == "svchost.exe" Then _ArrayDelete($array, $x) EndIf Next EndFunc ;==>_Sanitize And i get this error message
      What is wrong?
    • By gahhon
      Global Const $DIR_WA_FOLDER = @MyDocumentsDir & '\Whatsapp Extension' Global Const $DIR_SHORTCUT = @DesktopDir & '\WhatsappWeb.exe' Func _Create_Shortcut() Local $sMessage = "Creating shortcut " _Metro_SplashTextScreen(1, "", $sMessage) AdlibRegister("_Metro_LoopInProgress") If FileExists($DIR_WA_FOLDER) Then $iDelete = FileDelete($DIR_SHORTCUT) If $iDelete Then _FileWriteLog($LOG_INSTALLATION, "Debug: Desktop's shortcut is deleted.") Else _FileWriteLog($LOG_INSTALLATION, "Debug: Desktop's shortcut is not found. But, desktop's shortcut is still created.") EndIf If FileExists($DIR_WA_FOLDER & '\WhatsappWeb.exe') Then $iCreate = FileCreateShortcut($DIR_WA_FOLDER & '\WhatsappWeb.exe', $DIR_SHORTCUT) If $iCreate Then _Metro_MsgBox(0, "", "Desktop shortcut is now created!") _FileWriteLog($LOG_INSTALLATION, "Debug: Desktop's shortcut is created.") Else _Metro_MsgBox(0, "", "Desktop shortcut is failed to created!") _FileWriteLog($LOG_INSTALLATION, "Error: Fail to create shortcut.") EndIf Else _Metro_MsgBox(0, "", "Source not found!") _FileWriteLog($LOG_INSTALLATION, "Error: WhatsappWeb.exe is not found.") EndIf Else _Metro_MsgBox(0, "", "Source not found!") _FileWriteLog($LOG_INSTALLATION, "Error: Whatsapp Extension folder is not found.") EndIf AdlibUnRegister("_Metro_LoopInProgress") _Metro_SplashTextScreen(0) EndFunc So I do have a button to trigger this create shortcut to desktop.
      But after it created shortcut at desktop, I cannot execute the shortcut as it has no any response.
      Then I go try trigger the file at the source path @MyDocumentDir & '\Whatsapp Extension\WhatsappWeb.exe' and it work as expected.
      Then I go to check my log, and it shows:- (P/S: I already did trigger the function twice)
      2019-02-28 23:27:32 : Debug: Desktop's shortcut is not found. But, desktop's shortcut is still created. 2019-02-28 23:27:33 : Debug: Desktop's shortcut is created. 2019-02-28 23:28:40 : Debug: Desktop's shortcut is not found. But, desktop's shortcut is still created. 2019-02-28 23:28:41 : Debug: Desktop's shortcut is created. So based on my log shows, it failed to locate the shortcut, but it did exists there.
       
      P/S: This project is done developed weeks ago, every testing is passed. Is just somehow the executable cannot be execute.
       
       
      Any advise?
    • By chrweav86
      Here is an existing AS400 PCOM script that I have: 
      #include-once
      #cs ----------------------------------------------------------------------------
       UDF Header Title: _PCOMM
       UDF Header Version: 0.1.0
       AutoIt Version: 3.3.8.1
       Author:         Jason S. - Jewelry Supply, Inc.
       Script Function: PCOMM UDF Header
                          For use with IBM AS/400 - IBM OS/400 - IBM System i - IBM i application system.
                          This PCOMM UDF Header targets ASW - an ERP application suite written in IBM's RPG language (yuck!),
                          although it will work for any AS/400 system and is not ASW specific.
                          PCOMM allows injection of keystrokes into the datastream of
                          a running terminal/console application ("Green Screen" Console Emulator).
                              - Assumes Session 'A' but can be overriden using the aswConnect() function.
                          PCOMM allows observing of characters in the datastream of a running
                          terminal/console application ("Green Screen" Console Emulator).
                              - Assumes Session 'A' but can be overriden using the aswConnect() function.
      #ce ----------------------------------------------------------------------------
      Global $Session = "A"
      Global $Ps
      Global $Oia
      ; Specify session to connect to
      Func aswSetSession($nSession)
          $Session = $nSession
      EndFunc
      ; Connect to ASW
      ; @param $nSession - Specify session to connect to. Ex: "A"
      Func aswConnect($nSession = $Session)
          $Obj_ConnMgr = ObjCreate("PCOMM.autECLConnMgr")
          $OBJ_EmulSession = ObjCreate("PCOMM.autECLSession")
          $OBJ_EmulSession.SetConnectionByName($nSession)
          $Ps = $OBJ_EmulSession.autECLPS
          $Oia = $OBJ_EmulSession.autECLOIA
      EndFunc
      ; Send Data to ASW
      ; if $rownum and $colnum are not specified, defaults to the cmd line
      Func aswSend($keys, $rownum = 20, $colnum = 7, $inhibit = 1)
          $Ps.SendKeys($keys, $rownum, $colnum)
          if $inhibit = 1 Then
              While $Oia.InputInhibited <> 0
              WEnd
          EndIf
          $Oia.WaitForInputReady()
      EndFunc
      ; Get Data from ASW
      Func aswGet($row, $col, $length)
          $info = $Ps.GetText($row,$col,$length)
          return $info
      Endfunc
      ; Check which page we are on
      ; @param $page - Page to see if we are on. Ex: "DMR30101" == Sales Order Maintenance page
      Func checkScreen($page, $rownum = 1, $colnum = 72, $length = 8)
          If StringInStr(aswGet($rownum, $colnum, $length), $page) Then
              Return 1
          Else
              Return 0
          EndIf
      EndFunc
      I would like to use it launch a connection to the AS400. I have an executable that I run with an hod file. Do I just need to run the exe and hod file then run the session connect function and send keys function to pass credentials?
       
×
×
  • Create New...