Jump to content

Automation, what are my options?


Recommended Posts

Hi,

 

I am trying to automate some programs, but some of them prevent automation.

When I use the window info tool or the "GetAllWindowsControls" script, no data / info  about the controls comes up. (for example the latest adwcleaner). (I know  adwcleaner supports CLI, but this is a general example). 

I found more and more programs comes with a gui where I cant use the window info tool.

Is  there is another tool available to get those control data?

 

Thank you.

 

 

 

Link to post
Share on other sites
  • Moderators
55 minutes ago, MightyWeird said:

(for example the latest adwcleaner). (I know  adwcleaner supports CLI, but this is a general example)

Without knowing more than "a general example" of what you're trying to do, it is just a bit difficult to give decent suggestions.

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Link to post
Share on other sites
1 hour ago, MightyWeird said:

When I use the window info tool ..., no data / info about the controls comes up. (For example the latest AdwCleaner).

I just installed Malwarebytes AdwCleaner, and I confirm that the Window Info tool can't see any of the buttons etc. in that utility.  Some windows are just like that, unfortunately.  When this happens to me, I try to find some predicable way of getting to the relevant buttons etc.  You can sometimes click blindly in certain places if you are sure that the button is there in that location, and you can sometimes use e.g. Tab, Space, Enter, arrow keys, Alt menu shortcuts etc. to get where you want to be, too.  If I want to automate a very repetitive action inside a single widow, I let the script sleep for e.g. 2 seconds while I hover over the relevant button etc., and then I get that button's position with MouseGetPos (however, then I need to do this all over again next time I use this particular program).

In the case of AdwCleaner, I can tell you that the window does not always launch on the same location on the screen, but it is always the same size when it launches, even if you resized it previously before closing.  This is a good thing: it means that you can figure out where to click, relative to the top left corner of the window.  For example, the "Log File" button is located at +69 X +167 pixels from the top left corner.  So, the script would use WinGetPos to determine where the window is, and then use MouseClick with coordinates that add 69 to the X position and 167 to the Y position.  It's a bit dangerous, I know.  Another way to get to the Log Files in AdwCleaner is to launch it and then go Tab three times and Space once.  But even in AdwCleaner, there is no way to select a log file without using the mouse.

Link to post
Share on other sites
2 hours ago, leuce said:

I just installed Malwarebytes AdwCleaner, and I confirm that the Window Info tool can't see any of the buttons etc. in that utility.  Some windows are just like that, unfortunately.  When this happens to me, I try to find some predicable way of getting to the relevant buttons etc.  You can sometimes click blindly in certain places if you are sure that the button is there in that location, and you can sometimes use e.g. Tab, Space, Enter, arrow keys, Alt menu shortcuts etc. to get where you want to be, too.  If I want to automate a very repetitive action inside a single widow, I let the script sleep for e.g. 2 seconds while I hover over the relevant button etc., and then I get that button's position with MouseGetPos (however, then I need to do this all over again next time I use this particular program).

In the case of AdwCleaner, I can tell you that the window does not always launch on the same location on the screen, but it is always the same size when it launches, even if you resized it previously before closing.  This is a good thing: it means that you can figure out where to click, relative to the top left corner of the window.  For example, the "Log File" button is located at +69 X +167 pixels from the top left corner.  So, the script would use WinGetPos to determine where the window is, and then use MouseClick with coordinates that add 69 to the X position and 167 to the Y position.  It's a bit dangerous, I know.  Another way to get to the Log Files in AdwCleaner is to launch it and then go Tab three times and Space once.  But even in AdwCleaner, there is no way to select a log file without using the mouse.

Using UIAutomation UDF like @Nine suggested to you is the best approach for automating programs when Info Tool does not detect controls and UIASpy is essential for creating UI Automation code.

(P.S if your goal is to automate program silently then I must say as far as I know, UIAutomation is not Silent friendly.)

Edited by WilliamasKumeliukas
Clarifying a bit more

Sorry if it's quite challenging to understand me sometimes, there is 2 possible reasons to this:

Spoiler

#1. I am a native French speaker and learned English mainly from chatting with others players in online games.

#2. I have a developmental disorder mainly affecting my social abilities, way of thinking, understanding stuffs and explaining myself in which it can seem ironic of seeing me here, but I been working on getting better every days for an dozens of years now ^_^

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Do not like my dirty code? It's fine, but in my opinion, the dirty codes are unique and I believe it's a good glimpse of how the mind of the person who wrote it thinks, and that, that's what I call a psychological deduction step by step in all its splendor.

~WilliamasKumeliukas

Link to post
Share on other sites

Thank you all!

 

On 7/20/2021 at 7:48 PM, WilliamasKumeliukas said:

Using UIAutomation UDF like @Nine suggested to you is the best approach for automating programs when Info Tool does not detect controls and UIASpy is essential for creating UI Automation code.

(P.S if your goal is to automate program silently then I must say as far as I know, UIAutomation is not Silent friendly.)

 

Well, that is why I started this post I guess. I just wanted to be sure there isn't another option I did not  know about.

I looked at the UIautomation UDF before, but it looked kinda... euh .. not easy!?

Just wanted to make sure, before I spend a lot of time figuring this out.

 

 

 

Edited by MightyWeird
Link to post
Share on other sites
32 minutes ago, MightyWeird said:

Thank you all!

 

 

Well, that is why I started this post I guess. I just wanted to be sure there isn't another option I did not  know about.

I looked at the UIautomation UDF before, but it looked kinda... euh .. not easy!?

Just wanted to make sure, before I spend a lot of time figuring this out.

 

 

 

I agree, it does look complicated at first because of how much stuffs it can do so you can easily get lost if you don't know how to do it, I did recently had to automate Microsoft Store to automate installation of Lenovo Vantage and took me awhile to figure out how to use it just because the way it is explained is not... huh.. simple.

If you don't mind about execution speed (2~5 seconds per action) then, its possible to be a little minimalist like this:

_UIA_setVar("AppBarButton","Title:=Rechercher;controltype:=UIA_ButtonControlTypeId;class:=AppBarButton")
                _UIA_action("AppBarButton","highlight")
                _UIA_action("AppBarButton","leftclick")

                _UIA_setVar("TextBox","Title:=Rechercher;controltype:=UIA_EditControlTypeId;class:=TextBox")
                _UIA_action("TextBox","settextvalue" , "Lenovo Vantage")

                _UIA_setVar("ListViewItem3","ListItem:=Lenovo Vantage Application;controltype:=UIA_ListItemControlTypeId;class:=ListViewItem")
                _UIA_action("ListViewItem3", "leftclick")


                _UIA_setVar("GetButton", "Button:=Obtenir;controltype:=UIA_ButtonControlTypeId;automationid:=buttonPanel_AppIdentityBuyButton")
                _UIA_action("GetButton", "leftclick")

                _UIA_setVar("PlayButton", "Button:=Lancer;controltype:=UIA_ButtonControlTypeId;automationid:=PlayBar_AppIdentityOpenButton")
                _UIA_action("PlayButton", "leftclick")

this is my working code using UIAutomation to install Lenovo Vantage App via Microsoft Store Application

it does:

---------------------------------------------------------

-Left click on Search Bar

-Type Lenovo Vantage in Search bar

-Left click Lenovo Vantage from Search results

-Left Click Install button

-Left click start button

------------------------------------------------------

(assuming you want to write into inputs and click on buttons) so all you'll mostly need is to collect informations using UIASpy and do this each controls you want to interact with.

so you'll need the following informations from each controls (if available):

  • Title or Control name ex: Button
  • ControlType
  • Class

from the given code you should be able to figure pretty easily how to use _UIA_action ;)

~WilliamasKumeliukas

Sorry if it's quite challenging to understand me sometimes, there is 2 possible reasons to this:

Spoiler

#1. I am a native French speaker and learned English mainly from chatting with others players in online games.

#2. I have a developmental disorder mainly affecting my social abilities, way of thinking, understanding stuffs and explaining myself in which it can seem ironic of seeing me here, but I been working on getting better every days for an dozens of years now ^_^

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Do not like my dirty code? It's fine, but in my opinion, the dirty codes are unique and I believe it's a good glimpse of how the mind of the person who wrote it thinks, and that, that's what I call a psychological deduction step by step in all its splendor.

~WilliamasKumeliukas

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...