Jump to content
Sign in to follow this  
Akashrai

autoit info tool is not detecting GUI components

Recommended Posts

Akashrai

I want to automate a client application but auto it info tool is not detecting the GUI components. I don't know what to do. please help me out. Screenshot  of first login page of application is in attachment 

 

visa.png

Share this post


Link to post
Share on other sites
Akashrai

Please help It is very important. Alternate of auto it is also welcome.  

Share this post


Link to post
Share on other sites
RTFC

There are lots of alternative GUI inspection tools, many of them free. I use Ranorex myself, but that one is now proprietary.

  • Like 1

Share this post


Link to post
Share on other sites
Akashrai

Yes but what about auto it why it is not detecting the GUI components and what and way to achieve the solution i am attaching page after login it also has same problem.  

visa1.png

Share this post


Link to post
Share on other sites
LarsJ

The "AutoIt Window Info" tool generally only detects standard Windows controls implemented with the functions in ComCtl32.dll. These controls have existed since Windows XP. You can find a list of the controls in the Control Library.

Many modern controls including modern MicroSoft controls are not detected by the "AutoIt Window Info" tool. These controls can be detected with Inspect.exe in Windows SDK or the "Simple Spy Tool" in the UI Automation framework.

Share this post


Link to post
Share on other sites
Akashrai

I have fetched info by Autoit info tool and attached please help. You can see it is not detecting any thing.

Please help

Login.png

Edited by Akashrai

Share this post


Link to post
Share on other sites
Akashrai

Where I will get simplespy.au3 file . I am new so please help. I have searched on google as well as in auto it setup but did not got .

Share this post


Link to post
Share on other sites
Akashrai

Yes, Gone through your given link and also searched on google but :sweating:. Could you provide link to download the same.

Share this post


Link to post
Share on other sites
Akashrai

I have tried it . It is also not detecting .

The VISA application which i am automating is a java (Applet ) application. I have searched for that and got "FEST" but do not know how to get id/class name of GUI component of the VISA application. Is there any spy tool in fest.

Other that FEST any other open source tool to automate applet desktop application?

Please help.

Edited by Akashrai

Share this post


Link to post
Share on other sites
Akashrai

Thanks for support.

Here it says ctrl+w. After pressing Ctrl+w  it gives me following code -

Mouse position is retrieved 133-54
  JAVA window found <> Java Window Title= Handle=0x00000000002A0740     res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 789588 ac 369080852
Call version info: PASSED VMID found : 789588
  VMVersion: <1.8.0_65>
  bridgeJavaClassVersion: <1.8.0_65>
  bridgeJavaDLLVersion: <1.8.0_65>
  bridgeWinDLLVersion: <AccessBridge 2.0.4>
 *** getAccessibleContextAT result is <369081228>
 *** getAccessibleContextAT result is <369081228>
Paste

push button
push button
enabled,focusable,visible,showing
enabled,focusable,visible,showing
2
0
98
35
46
0
End of getAcceccibleContextAt info

 

How to use above info in my autoit script. By which Detail of button i cab use.

The application on which i am testing is bellow

 

 

applet.PNG

Edited by Akashrai

Share this post


Link to post
Share on other sites
junkew

For that you have to read a lot on java accessibility bridge and study source of jaws and other open source implementations of java accessibility bridge. It all starts with building a ui tree and traversing that to get the right item. And if its internally developed explain developers that they have to fill the accessibility properties.

  • Like 1

Share this post


Link to post
Share on other sites
Akashrai

Thanks for reply.

But the application i want to automate is on client side and and we do not know any thing except GUI. In the case it seems impossible by auto it.

I have searched for any other tool/spy(FREE) but did not get much. I went through Fest but do not know how to start. Any way thanks again.        

Share this post


Link to post
Share on other sites
junkew

search harder ;-)

 

https://github.com/ckundo/nvda/tree/master/source

http://www.oracle.com/technetwork/java/javase/downloads/downloads-jsp-138220.html    ferret and monkey

it is possible from AutoIT and the JAB Simplespy shows you the basics

1. Get an element : BOOL GetAccessibleContextAt(long vmID, AccessibleContext acParent, jint x, jint y, AccessibleContext *ac)

2. Get an info reference BOOL GetAccessibleContextInfo(long vmID, AccessibleContext ac, AccessibleContextInfo *info);

3. Get the text by using the reference you get in step 2 BOOL GetAccessibleTextInfo(long vmID, AccessibleText at, AccessibleTextInfo *textInfo, jint x, jint y);

 

 

Share this post


Link to post
Share on other sites
junkew

I searched a little further and played with JABSimpleSpy

Its all working with demo's as can be found in JDK ..\jdk-8u25-windows-i586-demos\jdk1.8.0_25\demo\jfc\SwingSet2

And as you have shown you get with JABSimpleSpy the text of your button the concept fully works

Combined with this where we iterate the tree from top to bottom

 

 

another AWT monitor found using same Java Access Bridge

https://docs.oracle.com/cd/E17802_01/j2se/javase/technologies/accessibility/docs/jaccess-1.3/doc/AWTMonitor.html

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • ufukreis1212
      By ufukreis1212
      Hello, I am new members. Help me please. I want vbs convert to au3 .
      This vbs code : 
      'deneme Set SystemSet = GetObject("winmgmts:").InstancesOf ("Win32_OperatingSystem") strOSArch = GetObject("winmgmts:root\cimv2:Win32_OperatingSystem=@").OSArchitecture Set objNetwork = CreateObject("Wscript.Network") Set wshShell = CreateObject( "WScript.Shell" ) strComputerName = wshShell.ExpandEnvironmentStrings( "%COMPUTERNAME%" ) Set oShell = WScript.CreateObject("WScript.Shell") proc_arch = oShell.ExpandEnvironmentStrings("%PROCESSOR_ARCHITECTURE%") Set oEnv = oShell.Environment("SYSTEM") strComputer = "." Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\CIMV2") Set colMB = objWMIService.ExecQuery("Select * from Win32_BaseBoard") Set colCSes = objWMIService.ExecQuery("SELECT * FROM Win32_ComputerSystem") Set colProcessors = objWMIService.ExecQuery("Select * from Win32_Processor") Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\CIMV2") Set colItemsx = objWMIService.ExecQuery( _ "SELECT * FROM Win32_VideoController",,48) '------------------------------------------------------------------- Set obj = GetObject("winmgmts:").InstancesOf("Win32_PhysicalMemory") i = 1 For Each obj2 In obj memTmp1 = obj2.capacity / 1024 / 1024 TotalRam = TotalRam + memTmp1 i = i +1 Next '-------------------------------------------------------------------- Dim objWMIService : Set objWMIService = GetObject("winmgmts:\\.\root\cimv2") Set colItems = objWMIService.ExecQuery("Select Architecture from Win32_Processor") For Each objItem in colItems if objItem.Architecture = 0 then strArchitecture = "x86" end if if objItem.Architecture = 9 then strArchitecture = "x64" end if next '-------------------------------------------------------------------- strComputer = "." ' Local computer strMemory = "" i = 1 set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2") set colItems = objWMIService.ExecQuery("Select * from Win32_PhysicalMemory") For Each objItem In colItems if strMemory <> "" then strMemory = strMemory & vbcrlf strMemory = strMemory & "Bank" & i & " : " & (objItem.Capacity /1024 /1024) & " Mb" i = i + 1 Next installedModules = i - 1 Set colItems = objWMIService.ExecQuery("Select * from Win32_PhysicalMemoryArray") For Each objItem in colItems totalSlots = objItem.MemoryDevices Next '---------------------------------------------------------------------- Set objWMISvc = GetObject( "winmgmts:\\.\root\cimv2" ) Set colItems = objWMISvc.ExecQuery( "Select * from Win32_ComputerSystem" ) For Each objItem in colItems strComputerDomain = objItem.Domain Next for each System in SystemSet For Each objItem in colItemsx For Each objProcessor in colProcessors For Each bbType In colMB MbVendor = bbType.Manufacturer MbModel = bbType.Product MsgBox "İşletim Sistemi : " & System.Caption & vbNewLine & _ "İşletim Sistemi Versionu : " & + System.Version & vbNewLine & _ "Windows Mimari Yapısı: " & strOSArch & vbNewLine & _ "Kullanıcı isminiz: " & objNetwork.UserName & vbNewLine & _ "Bilgisayar ismi: " & strComputerName & vbNewLine & _ "Çalışma Grubu: " & strComputerDomain & vbNewLine & _ "--------------------------------------" & vbNewLine & _ "Anakart: " & MbVendor & " " & "[" & MbModel & "]" & vbNewLine & _ "--------------------------------------" & vbNewLine & _ "Grafik Kartı: " & objItem.Caption & vbNewLine & _ "Driver Version: " & objItem.DriverVersion & vbNewLine & _ "--------------------------------------" & vbNewLine & _ "İşlemci Üreticisi: " & objProcessor.Manufacturer & vbNewLine & _ "İşlemci İsmi: " & objProcessor.Name & vbNewLine & _ "CPU Mimarisi: " & strArchitecture & vbNewLine & _ "İşlemci Çekirdek sayısı: " & oEnv("NUMBER_OF_PROCESSORS") & vbNewLine & _ "--------------------------------------" & vbNewLine & _ "Toplam RAM: " & TotalRam & " MB" & vbNewLine & _ "Toplam Slot: " & totalSlots & vbNewLine & _ "Boş Slot: " & (totalSlots - installedModules) & vbNewLine & _ "Ramlerin bulunduğu slotlar:" & vbcrlf & strMemory,0,"deneme" Next Next Next Next please help me , thanks.
    • JuanFelipe
      By JuanFelipe
       
      Hello friends, I have a query, it happens that I have a matrix nxn, the first column of this contains each cell a number with 21 digits, when I pass this matrix to Excel I obviously write that number as 1,10016E + 20, I solved it by adding a * at the end of each cell, but I want it to be written as a text so that I do not have to remove that *, I searched in Google but could not find an effective solution, thanks in advance. 
    • Puls3
      By Puls3
      Hi, This is my first post on the AutoIt Forums, I most of the time was able to get help from this community by merely searching though google and have been able to solve hundreds of problems this way, though in this case I was incapable of finding a solution to my problem. I am quite confident that it is a user error (And my fault) but I haven't been able to determine what I did wrong, any and all help would be greatly appreciated! I have uploaded my whole script pardon passwords to provide as much information as possible.
      I declared a global array at the top of the script and am having issues when I try to set it's value, and I fail to understand why.
      Error Message:
      "D:\Projects\AutoIt\Arena\Base.au3" (270) : ==> Subscript used on non-accessible variable.:
      $ExpMax[$i] = IniRead($TempSavePath, "Combatant" & $i, "ExpMax", $ExpMax[0])
      $ExpMax^ ERROR
      ConsoleWrite Log:
      Ran.
      Checking launch parameters.
      No launch parameters.
      Initiating load.
      Checking for save file.
      Save file not found.
      Load canceled.
      Loading or creating Combatant 1
      Base.au3
    • RamDemon
      By RamDemon
      Hi,
      Did anyone tried to  use Redis  with AutoIT as store DB ?
       Redis <-- more about it
      Mladen
    • nooneclose
      By nooneclose
      Hey. I'm working on a new project and was wondering if there is a better way to "update" my Column E array. 
      Here is my code: 
      Local $nI  = 0                                                            ;Creates a name index of 0: nI = Name index Local $nII = 1                                                            ;Creates a name index of 1 for second loop: nII = Name Index 2 For    $iN = 0 To $IndexRows Step 1                                       ;Checks the roster for any names that appear twice      For $iN2 = 0 To $IndexRows Step 1          if $d_Names[$nI] == $d_Names[$nII] And $d_Names[$nII] <> "" Then              Local $timeSheetName = _ArraySearch($e_Names, $d_Names[$nI], 0, 0, 0, 0, 1)              ;MsgBox($MB_SYSTEMMODAL, "Found it", $d_Names[$nI] & " In column E on Row " & $timeSheetName)              Local $eI  = $timeSheetName + 1              ;ConsoleWrite($timeSheetName & @CRLF)              ;ConsoleWrite($eI & @CRLF)              ;ConsoleWrite(@CRLF)              _Excel_RangeInsert($OpenWorkbook.ActiveSheet, "E" & $eI & ":F" & $eI, $xlShiftDown)                                                                          ;Inserts a empty cell in columns E and F.              _Excel_RangeWrite($OpenWorkbook, $OpenWorkbook.ActiveSheet, $d_Names[$nII], "E" & $eI)                                                                         ;Fills the empty cell in columns E with the doubled name              $aArray_Index = 2                                           ;Array element counter              For $Index = 2 To $IndexRows Step 1                        ;Loops through every row in the Excel file unto no rows are found or a null row is found                  $Array_Value_E = _Excel_RangeRead($OpenWorkbook, Default, "E"&$Index)                  $e_names[$aArray_Index] = $Array_Value_E                ;While the code loops every value in column E is stored in the E array (updating the array)                  $aArray_Index += 1              Next              ExitLoop          EndIf      Next      $nI  += 1      $nII += 1 Next Basically, It checks a roster for people whose name appears twice then inserts a new "row" for that person because they work in two different departments.
      I have to find that name however in Column E if two appear in column D. My code works but I think it is not as efficient as it could be. 
      Any ideas on how to improve the "update" for my array?
      (once it finds the double names in Column D it then searches for that name by going name by name in the Column E array and once it finds it inserts a new row. However, the E array doesn't have that new row stored in it so I have to "update" the array to properly find the next name)
      Any and all tips would be greatly appreciated. 
       
      NOTE: Just assume I'm opening the excel file properly please do not add that code in, it only complicates your answer. 
×