Trying to click buttons that AutoIt Window Info doesn't recognize

[jaeger52]

Hello all! I'm trying to automate a repetitive process for work. I use a program called MAILER+4 that imports, processes, and exports address data for our databases. I've been able to successfully automate the entire process for myself, but because of reliance on Mouseclick it's unreliable for other users running it on their desktops. The problem is that there are multiple buttons and menus that AutoIt Window Info won't recognize as individual buttons. I've attached a screen capture of the program I'm working with. The buttons on the left under "MAILERS+4" and the menu buttons on top are the ones I'm looking at. I have a couple of questions on how best to approach this. First, what is the best approach for clicking the buttons? Second, is there a reliable method for moving the mouse to a spot on the page itself rather than using screen coordinates? If I could just have some suggestions to start with I'd really appreciate it. 

[MikahS]

Couple things I have to ask. Have you tried using Control functions before?

Have you used the Au3 window info tool? If it did not give enough info, did you try and use maybe a class or ID (you can use multiple properties like so: "[CLASS:window; INSTANCE:2]"). This is a list of all title definitions for controls.

Let us know.

[jaeger52]

MikahS, thanks for the reply! I am familiar with the window info tool, and in the case of the buttons on the left side of the page they all show up as part of the same window. None of them show up in Control mode. The only place they show up at all is in the Hidden Text mode for that window. The top menu bar buttons grey out when the focus shifts away from the MAILERS+4 window and don't show up at all in the window info tool.I use ControlClick often in the script for buttons I can identify, but these are not showing up with individually identifiable information that I can use to select them specifically. I can post a script snippet if that would help, but it's pretty much a series of ControlClicks, Sleeps and MouseClicks. Simple stuff. Another thought- is there some way I could use the text on the buttons to navigate to? 

[alienclone]

this should help you find the control id for those pesky button groups.

'?do=embed' frameborder='0' data-embedContent>>

[JohnOne]

'?do=embed' frameborder='0' data-embedContent>>

[jaeger52]

Thanks for the replies, alienclone and JohnOne. Lots to look into! I'll see what I can learn from these and post a solution or more questions.

[jaeger52]

Just got back to this after spending time on another project. I got the Simple Spy tool and have been experimenting with it and a few other techniques as well.

Keying to the pixel colors of the symbols next to the menu items hasn't worked because the symbols aren't part of the buttons, so clicking on them is useless. The code is solid but the mouse isn't over the button portion of the graphic. 

Using ConbtrolGetText and StatusbarGetText, for the sake of experimentation, has also not worked because the text I need to key to is hidden. I haven't made any headway retrieving the text at all.

Here is what Simple Spy returns for the menu window:

Mouse position is retrieved 79-162
At least we have an element [Quicktools][Static]

Having the following values for all properties: 
Title is: <Quicktools>    Class   := <Static>    controltype:= <UIA_ImageControlTypeId>    ,<50006>    , (0000C356)    4;69;141;429
*** Parent Information top down ***
4: Title is: <Desktop>    Class   := <#32769>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;0;1920;1080
"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"    
3: Title is: <MAILERS+4>    Class   := <Afx:00400000:8:00010007:00000000:0B8C026F>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -8;-8;1936;1056
"Title:=MAILERS+4;controltype:=UIA_WindowControlTypeId;class:=Afx:00400000:8:00010007:00000000:0B8C026F"    
2: Title is: <Workspace>    Class   := <MDIClient>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;65;1920;956
"Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient"    
1: Title is: <>    Class   := <AfxFrameOrView110>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    2;67;148;954
"Title:=;controltype:=UIA_WindowControlTypeId;class:=AfxFrameOrView110"    
0: Title is: <>    Class   := <#32770>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    2;67;148;954
"Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770"    

;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=MAILERS+4;controltype:=UIA_WindowControlTypeId;class:=Afx:00400000:8:00010007:00000000:0B8C026F", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AfxFrameOrView110", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar("Quicktools.mainwindow","title:=Quicktools;classname:=Static")
_UIA_action("Quicktools.mainwindow","setfocus")

*** Detailed properties of the highlighted element ***
UIA_title:= <Quicktools>
UIA_text:= <Quicktools>
UIA_regexptitle:= <Quicktools>
UIA_class:= <Static>
UIA_regexpclass:= <Static>
UIA_iaccessiblechildId:= <0>
UIA_id:= <918>
UIA_handle:= <263678>
UIA_RuntimeId:= <42;263678>
UIA_BoundingRectangle:= <4;69;141;429>
UIA_ProcessId:= <4424>
UIA_ControlType:= <50006>
UIA_LocalizedControlType:= <image>
UIA_Name:= <Quicktools>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <False>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <918>
UIA_ClassName:= <Static>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <263678>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <Win32>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Quicktools>
UIA_LegacyIAccessibleRole:= <40>
UIA_LegacyIAccessibleState:= <64>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:4900,hwnd:0x405FE Main:Nested [pid:4424,hwnd:0x405FE Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

Plugging "918" in for control ID does not work. Any ideas or thoughts would be greatly appreciated!

[junkew]

First of all the simple.spy tells you it easily can work as objects/elements are properly recognized by title property.

With a little tweaking of the simple spy source output you should be able to get the control and do the actions you want

Edited May 17, 2015 by junkew

[jaeger52]

I got the script finished, but I ended up using hot keys and Send commands to circumvent the button controls. Some of them I just could not get to work, and in the course of troubleshooting I found other ways to solve the problems. Thanks for the tips and resources, I learned a lot! Here's the script I finished in case anybody finds it useful.

 

expandcollapse popup

;                          `                Run MP4

; Automated command to open the Melissa Data MP4 program.
;   Runs the MP4 Process.
Run ( "C:\MP4\mp4.exe" )
; wait
Sleep ( 1000 )

;
;                           *** Melissa Data automated processing procedure ***
;

; Hotkey for File menu
;   Sends 'Alt+F' to open the File dropdown menu.
Send ( "{LALT}{F}" )
; wait
Sleep ( 1000 )

; Hotkey for Open Files
;   Sends 'Ctrl+O' to open the 'Open' menu
Send ( "{LCTRL}{O}" )
; wait
Sleep ( 1000 )

; Open "MP4-Standard for Mail Table.csv" file
;   Selects "MP4-Standard for Mail Table.csv" file in "Open File" window.                                           *
Send ( "MP4 Standard.ms4" )
; wait
Sleep ( 1000 )

; Click "Open" button
;     Clicks the "Open" button to open the file.
ControlClick ( "Open", "&Open", 1 )
; wait
Sleep ( 1000 )

;
;                                           Delete All
;

; Click "Database" text button
;     Clicks the "Database" button to open the dropdown menu.
Send ( "{LALT}{D}" )
; wait
Sleep ( 1000 )

; Select "Delete"
;    Selects "Delete" from the "Database" dropdown menu
Send ( "{D}" )
; wait
Sleep ( 1000 )

; Click "All" button
;    Selects the "All" option in the "Delete" window.
ControlClick ( "Delete", "&All", 5040 )
; wait
Sleep ( 1000 )

; Click "Delete" button
;    Selects the "Delete" button in the "Delete" window.
ControlClick ( "Delete", "&Delete", 1 )
Sleep ( 1000 )

; Wait for Delete process
;    Waits for Delete processing to finish before allowing the program to proceed.
WinWaitActive ( "MAILER'S+4" )
Sleep ( 1000 )

; Select "OK"
;    Selects "OK" button in the pop-up window after deleting selected records.
ControlClick ( "MAILER'S+4", "OK", 2 )
; wait
Sleep ( 1000 )

;
;                                           Pack Records
;

; Click "Database" text button
;    Clicks the "Database" button to open the dropdown menu.
Send ( "{LALT}{D}" )
; wait
Sleep ( 1000 )

; Select "Pack"
;    Selects "Pack" from the "Database" dropdown menu.
Send ( "{P}" )
; wait
Sleep ( 1000 )

; Click "Pack" button
;    Clicks the "Pack" button in the "Pack" window.
ControlClick ( "Pack", "&Pack", 1 )
; wait
Sleep ( 1000 )

;
;                               Import "Standard for Mail Table" file
;

; Click "Database" text button
;    Clicks the "Database" button to open the dropdown menu.
Send ( "{LALT}{D}" )
; wait
Sleep ( 1000 )

; Select "Import"
;    Selects "Import" from the "Database" dropdown menu.
Send ( "{I}" )
; wait
Sleep ( 1000 )

; Select "MP4 Standard for Mail Table" file
;   Enters the file path and name of the "MP4 Standard for Mail Table" file.
Send ( "C:\MP4\MP4-STANDARD FOR MAIL TABLE.CSV" )
; wait
Sleep ( 1000 )

; Click "Open" button
;    Clicks the "Open" button in the "Open" menu.
ControlClick ( "Open", "&Open", 1 )
; wait
Sleep ( 1000 )

; Click "OK" button
;    Clicks the "OK" button in the "Import Options" menu.
ControlClick ( "Import Options", "&OK", 5059 )
; wait
Sleep ( 1000 )

;
;                                           Address Check
;

; Select the 'List' menu from command bar
;   Hotkeys to the 'List' menu to bypass the 'Address Check' button feature.
Send ( "{LALT}{L}" )
; wait
Sleep ( 1000 )

; Select 'List' menu from command bar
;   This series of commands selects the "List" menu, then "Address Check," then "Options."
Send ( "{A}" )
; wait
Sleep ( 1000 )

; Select 'List' menu from command bar
;   This series of commands selects the "List" menu, then "Address Check," then "Options."
Send ( "{O}" )
; wait
Sleep ( 1000 )

; Click "Go" button
;   Clicks on the "Go" button.
ControlClick ( "Address Options", "&Go", 2021 )
Sleep ( 500 )

; Click out if expiration pop-up
;   If the USPS 90 day warning window pops up, click "OK."
If WinExists ( "MAILER'S+4" ) Then Send ( "{ENTER}" )

; Wait for "Address Results" window
;   Waits for the address process to finish.
WinWaitActive ( "Address Results" )

; Click "Close" button
;   Clicks the "Close" button in the "Address Results" window.
ControlClick ( "Address Results", "&Close", 2008 )

;
;                           Export "Standard for Mail Table-Errors" file
;

; Click "Database" text button
;    Clicks the "Database" button to open the dropdown menu.
Send ( "{LALT}{D}" )
; wait
Sleep ( 1000 )

; Select "Export"
;    Selects "Export" from the "Database" dropdown menu.
Send ( "{E}" )
; wait
Sleep ( 1000 )

; Click "Add All" button
;    Clicks the "Add All" button in the "Export Records" button.
ControlClick ( "Export Records", "&Add All", 5074 )
; wait
Sleep ( 1000 )

; Click "Save"
;    Clicks the "Save" button in the "Export Records" menu.
ControlClick ( "Export Records", "&Save", 5080 )
; wait
Sleep ( 1000 )

; Bypass the dropdown menu and select .CSV filetype
;   Selects the filetype dropdown menu, skips to .CSV, and selects it. and shifts focus back to the text box.
Send ( "{TAB}" )
Send ( "{DOWN}" )
Send ( "{DOWN}" )
Send ( "{ENTER}" )

; Selects the text box
;   Cycles through objects until the text box is selected.
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
Send ( "{TAB}" )
; wait
Sleep ( 1000 )

; Specify 'MP4 Standard for Mail Table-Errors.CSV'
;   Inputs the 'MP4 Standard for Mail Table-Errors.CSV' filename as the file name.
Send ( "MP4 Standard for Mail Table-Errors.CSV" )

; Click "Save" button
;    Clicks the "Save" button.
ControlClick ( "Save As", "&Save", 1 )
; wait
Sleep ( 1000 )

; Click "Yes" button
;    Clicks the "Yes" button in the "Confirm Save As" pop-up window.
Send ( "{LEFT}" )
Send ( "{ENTER}" )
Sleep ( 1000 )

; Wait for Save As process
;    Waits for Save As process to finish before allowing the program to proceed.
WinWaitActive ( "Results" )

; Click "OK" button
;    Clicks the "OK" button in the "Results" pop-up window.
ControlClick ( "Results", "&OK", 5083 )
; wait
Sleep ( 1000 )

;
;                                           Close MP4 process
;

; Close MAILERS+4 program
;    Sends a close command and ends the instance.
WinClose ( "MAILERS+4" )

; Exit
;   Closes the AutoIt script.
Exit

 

[Rupa]

Hi , I am new to AutoIt and I have some similar kind of issue, I need to check in a particular grid is there any data , if the data exists select the data and delete it.

The data , I am not able to get from Auto Info tool. Here is the summary of the AutoIt info Tool-

>>> Window <<<<
Title:    P99SMOKE - [Smoke Test Find/Browse Entry Point Form]
Class:    Afx:6EE00000:3:00010003:0110005E:0EDB03BF
Position:    355, 128
Size:    1200, 625
Style:    0x14CF0000
ExStyle:    0x00000100
Handle:    0x00000000003204FC

>>>> Control <<<<
Class:    GXWND
Instance:    2
ClassnameNN:    GXWND2
Name:    
Advanced (Class):    [CLASS:GXWND; INSTANCE:2]
ID:    1001
Text:    
Position:    13, 148
Size:    1151, 388
ControlClick Coords:    581, 68
Style:    0x50910000
ExStyle:    0x00000000
Handle:    0x00000000003B0478

>>>> Mouse <<<<
Position:    957, 394
Cursor ID:    0
Color:    0x000000

>>>> StatusBar <<<<
1:    
2:    
3:    
4:    

>>>> ToolsBar <<<<

>>>> Visible Text <<<<
Smoke Test Find/Browse Entry Point Form
Smoke Test Find/Browse Entry Point Form
100.000000
The Processing Option Text that was entered was:
JD Edwards EnterpriseOne Errors & Warnings
Exit Bar

>>>> Hidden Text <<<<
More Options
123
Stop
JD Edwards EnterpriseOne Errors & Warnings
Exit Bar
Tools
Form
Row
Report
 

I need to check for 123 data which is hidden Text. Attaching the screenshot. In the screenshot displayed there is a data 123 which i need to get and then select it and delete.

[Jos]

@Rupa,

Why create a second post in a very old thread while you already have your own thread with this exact same question opened?
Please stick to that one!

Jos