Automation, what are my options?

[MightyWeird]

Hi,

 

I am trying to automate some programs, but some of them prevent automation.

When I use the window info tool or the "GetAllWindowsControls" script, no data / info  about the controls comes up. (for example the latest adwcleaner). (I know  adwcleaner supports CLI, but this is a general example). 

I found more and more programs comes with a gui where I cant use the window info tool.

Is  there is another tool available to get those control data?

 

Thank you.

 

 

 

[Nine]

 

[JLogan3o13]

55 minutes ago, MightyWeird said:

(for example the latest adwcleaner). (I know  adwcleaner supports CLI, but this is a general example)

Without knowing more than "a general example" of what you're trying to do, it is just a bit difficult to give decent suggestions.

[leuce]

1 hour ago, MightyWeird said:

When I use the window info tool ..., no data / info about the controls comes up. (For example the latest AdwCleaner).

I just installed Malwarebytes AdwCleaner, and I confirm that the Window Info tool can't see any of the buttons etc. in that utility.  Some windows are just like that, unfortunately.  When this happens to me, I try to find some predicable way of getting to the relevant buttons etc.  You can sometimes click blindly in certain places if you are sure that the button is there in that location, and you can sometimes use e.g. Tab, Space, Enter, arrow keys, Alt menu shortcuts etc. to get where you want to be, too.  If I want to automate a very repetitive action inside a single widow, I let the script sleep for e.g. 2 seconds while I hover over the relevant button etc., and then I get that button's position with MouseGetPos (however, then I need to do this all over again next time I use this particular program).

In the case of AdwCleaner, I can tell you that the window does not always launch on the same location on the screen, but it is always the same size when it launches, even if you resized it previously before closing.  This is a good thing: it means that you can figure out where to click, relative to the top left corner of the window.  For example, the "Log File" button is located at +69 X +167 pixels from the top left corner.  So, the script would use WinGetPos to determine where the window is, and then use MouseClick with coordinates that add 69 to the X position and 167 to the Y position.  It's a bit dangerous, I know.  Another way to get to the Log Files in AdwCleaner is to launch it and then go Tab three times and Space once.  But even in AdwCleaner, there is no way to select a log file without using the mouse.

[WilliamasKumeliukas]

2 hours ago, leuce said:

I just installed Malwarebytes AdwCleaner, and I confirm that the Window Info tool can't see any of the buttons etc. in that utility.  Some windows are just like that, unfortunately.  When this happens to me, I try to find some predicable way of getting to the relevant buttons etc.  You can sometimes click blindly in certain places if you are sure that the button is there in that location, and you can sometimes use e.g. Tab, Space, Enter, arrow keys, Alt menu shortcuts etc. to get where you want to be, too.  If I want to automate a very repetitive action inside a single widow, I let the script sleep for e.g. 2 seconds while I hover over the relevant button etc., and then I get that button's position with MouseGetPos (however, then I need to do this all over again next time I use this particular program).

In the case of AdwCleaner, I can tell you that the window does not always launch on the same location on the screen, but it is always the same size when it launches, even if you resized it previously before closing.  This is a good thing: it means that you can figure out where to click, relative to the top left corner of the window.  For example, the "Log File" button is located at +69 X +167 pixels from the top left corner.  So, the script would use WinGetPos to determine where the window is, and then use MouseClick with coordinates that add 69 to the X position and 167 to the Y position.  It's a bit dangerous, I know.  Another way to get to the Log Files in AdwCleaner is to launch it and then go Tab three times and Space once.  But even in AdwCleaner, there is no way to select a log file without using the mouse.

Using UIAutomation UDF like @Nine suggested to you is the best approach for automating programs when Info Tool does not detect controls and UIASpy is essential for creating UI Automation code.

(P.S if your goal is to automate program silently then I must say as far as I know, UIAutomation is not Silent friendly.)

Edited July 20, 2021 by WilliamasKumeliukas
Clarifying a bit more

[MightyWeird]

Thank you all!

 

On 7/20/2021 at 7:48 PM, WilliamasKumeliukas said:

Using UIAutomation UDF like @Nine suggested to you is the best approach for automating programs when Info Tool does not detect controls and UIASpy is essential for creating UI Automation code.

(P.S if your goal is to automate program silently then I must say as far as I know, UIAutomation is not Silent friendly.)

 

Well, that is why I started this post I guess. I just wanted to be sure there isn't another option I did not  know about.

I looked at the UIautomation UDF before, but it looked kinda... euh .. not easy!?

Just wanted to make sure, before I spend a lot of time figuring this out.

 

 

 

Edited July 21, 2021 by MightyWeird

[WilliamasKumeliukas]

32 minutes ago, MightyWeird said:

Thank you all!

 

 

Well, that is why I started this post I guess. I just wanted to be sure there isn't another option I did not  know about.

I looked at the UIautomation UDF before, but it looked kinda... euh .. not easy!?

Just wanted to make sure, before I spend a lot of time figuring this out.

 

 

 

I agree, it does look complicated at first because of how much stuffs it can do so you can easily get lost if you don't know how to do it, I did recently had to automate Microsoft Store to automate installation of Lenovo Vantage and took me awhile to figure out how to use it just because the way it is explained is not... huh.. simple.

If you don't mind about execution speed (2~5 seconds per action) then, its possible to be a little minimalist like this:

_UIA_setVar("AppBarButton","Title:=Rechercher;controltype:=UIA_ButtonControlTypeId;class:=AppBarButton")
                _UIA_action("AppBarButton","highlight")
                _UIA_action("AppBarButton","leftclick")

                _UIA_setVar("TextBox","Title:=Rechercher;controltype:=UIA_EditControlTypeId;class:=TextBox")
                _UIA_action("TextBox","settextvalue" , "Lenovo Vantage")

                _UIA_setVar("ListViewItem3","ListItem:=Lenovo Vantage Application;controltype:=UIA_ListItemControlTypeId;class:=ListViewItem")
                _UIA_action("ListViewItem3", "leftclick")


                _UIA_setVar("GetButton", "Button:=Obtenir;controltype:=UIA_ButtonControlTypeId;automationid:=buttonPanel_AppIdentityBuyButton")
                _UIA_action("GetButton", "leftclick")

                _UIA_setVar("PlayButton", "Button:=Lancer;controltype:=UIA_ButtonControlTypeId;automationid:=PlayBar_AppIdentityOpenButton")
                _UIA_action("PlayButton", "leftclick")

this is my working code using UIAutomation to install Lenovo Vantage App via Microsoft Store Application

it does:

---------------------------------------------------------

-Left click on Search Bar

-Type Lenovo Vantage in Search bar

-Left click Lenovo Vantage from Search results

-Left Click Install button

-Left click start button

------------------------------------------------------

(assuming you want to write into inputs and click on buttons) so all you'll mostly need is to collect informations using UIASpy and do this each controls you want to interact with.

so you'll need the following informations from each controls (if available):

• Title or Control name ex: Button
• ControlType
• Class

from the given code you should be able to figure pretty easily how to use _UIA_action

~WilliamasKumeliukas