Jump to content

Controlling a GUI menu or drop down box by displayed textual descriptions


Recommended Posts

I am new to AutoIt. I have been reading the FAQ and working through training videos. But I haven't seen a demo of controlling a GUI menu or drop down box by displayed textual descriptions.  By that I mean, not using GUI X,Y coordinates, not using tab key to navigate between fields, but rather being more like a "simulated human".  Something like "Go to Tools" menu, search for something exactly like "textX" or preferably by matching a regex that describes "textX" which I supply.  If that selection is not found, then look for it under the "Config" menu. Same thing for a Drop Down box. Find a specific entry and click on that.  The GUI of the application that I will be controlling changes frequently.  An option may move from one high level menu to another menu, or a new option could appear in a drop down box as a result of a weekly program updates. 

The "program to be configured" runs on 6-12 networked computers. Re-configuration of this network happens every couple of weeks via a human logging into each computer. I can handle the Perl or Python networking code. I could perhaps reverse engineer this program's SQLite DB, but my suspicion is that I'm better off simulating what the humans do now and use this program's GUI interface to set the various DB parameters.  I am hoping that AutoIt is up to that task!  Each computer in the network runs a slightly different O/S (Win 7 vs Win10) and differing display hardware and resolutions. In addition each user can configure different font sizes, etc. on a per workstation basis.    A simple minded: "click on coordinate X,Y" or "select the 14th thing in the drop down box", just won't work.  As an aside, I know that this "program to be configured" is written in MS VB and without evidence to the contrary, I am assuming that the GUI objects that it uses/displays are "well behaved" in a Microsoft sense - I think that means that the text which I seek will be obtainable.

I intend to write a "master config' program with its own GUI, probably in Perl or Python (although I understand Autoit has significant GUI capabilities of its own!).  The result of this program is a text file describing what needs to be done on each networked machine.  My network code will distribute that file to the various nodes and cause AutoIt to use it. I am hoping that the Autoit executable on each networked machine is relatively stable (changes a 1-2x per year instead of every couple of week) . The file generated by the master config program changes frequently according to what has to be done this week.  

I welcome and seek advice.  I have heard that "human simulation" instead of "blind macro replay" is possible with AutoIt.  

 

Link to comment
Share on other sites

9 hours ago, Marshall said:

But I haven't seen a demo of controlling a GUI menu or drop down box by displayed textual descriptions.

Since you’re talking about controlling a program that you didn’t write, I should warn you that there is no guarantee that the program will respond to ANY messages you send it, even if you can figure out the controlIDs.  Consider yourself lucky if it works ;)

Code hard, but don’t hard code...

Link to comment
Share on other sites

3 hours ago, Marshall said:

ALL of us here are using AutoIt to control programs which we didn't write! 

Sure, myself included.

But my response was about a very specific way of controlling a program, to wit, WinMenuSelectItem(), since you specifically eschew (rightly) the brittle method of keystrokes and mouse clicks:

13 hours ago, Marshall said:

By that I mean, not using GUI X,Y coordinates, not using tab key to navigate between fields, but rather being more like a "simulated human".

And although I applaud you, using higher level methods of GUI control on a third party app often doesn’t work.

For instance, take a Chrome window and try WinMenuSelectItem() and see how well it works.

This is not to dissuade you or dispute @BigDaddyO’s excellent suggestion.  Just to say that if it doesn’t work don’t tear your hair out trying to figure out what you did wrong.

Now if it is your own app that doesn’t respond to WinMenuSelectItem() then definitely, tear your hair out :)

Edited by JockoDundee

Code hard, but don’t hard code...

Link to comment
Share on other sites

Sounds like I am on some "uncharted territory" here. I was hoping to hear from somebody who has actually gotten this to work in an application.

I am still hopeful and haven't given up yet.  The thing that I want to control is actually freeware so I know people who know folks on the dev team.  I do know that it is written using VB in the Microsoft  .NET environment.  I am hoping that this will allow AutoIt to work in the way that I need.  A multi platform thing like Chrome wouldn't have been written that way and there are some serious Java script issues. A button displayed in a Chrome browser window is a far different thing than a "native" Windows button.

I wouldn't use AutoIt to control Chrome.  The way to do that is via Chrome's telnet I/F.  I've written a number of web automation programs in Perl.  It is possible to emulate a browser and interpret the HTML that comes back.   This works fine as long as the website is serving up HTML pages to be displayed.  Some websites send a Java Script program to the browser and the result of that program's execution is what is displayed. That changes things a lot!  For a bunch of complicated technical reasons it is just not feasible to emulate the browser's Java environment.  So you make a TCPIP connection to Chrome, send Chrome to the website, ask Chrome to run the Java Script and then report back to you what the results are, whether its a table or whatever.  You get the names of all of the buttons, all of the text options for drop down menus, etc.  When controlling Chrome in this way, there is no concept of X,Y coordinates or tabbing between fields. If the thing I want to control had a browser I/F, there is no doubt that I would be able to control it in the way that I want to.   Perhaps most folks don't know that Chrome has a programmatic I/F to control that "bad-boy" but, it does.  To access this method of control, significant programming skills are required.   However, the result is about as "robust" as is possible.  If I want to click on the "Next" button, it doesn't matter where it is on the screen or how many other buttons there are.  Usually I can do this by emulating a browser, but in extreme cases, it is necessary to control a browser (eg Chrome) and have it do the work.

About 5+ years ago, I had another GUI automation task and I remember writing some Perl code to tell me about all of the stuff displayed on the main window.  I gave up because that is not for the feeble hearted!  AutoIt understands that stuff, but starting from "ground zero" is not an easy thing.

I am looking for advice on how to proceed.  

 

Link to comment
Share on other sites

29 minutes ago, Marshall said:

A button displayed in a Chrome browser window is a far different thing than a "native" Windows button.

I’m not talking about a button in a Chrome browser window, I am talking about the menu of Chrome itself.  

32 minutes ago, Marshall said:

I was hoping to hear from somebody who has actually gotten this to work in an application.

You have heard from someone who has actually got “this to work”.  Me.  Many times.

What I am telling you is this - no one can tell you it will or will not work unless you provide the third party app.
 

Code hard, but don’t hard code...

Link to comment
Share on other sites

Ok. I think you are asking me what app that I am actually using? 
This is the 3rd party app:

N1MM Logger Plus – Free Software for Phone, CW & Digital

This is a ham radio thing. This is freeware. My use of this is non-commercial - this is just a hobby.
 
There are blizzard of options for configuration. 
I only need to mess around with maybe 5 things per contest, mostly in the FILE menu,
But some in the CONFIG menu.
Create a new DB, open a new file in that DB, select a contest (drop down menu), enter
a new callsign for this particular contest.
I am working on a definitive list of parms that need to configured.
The configuration process has to be "harmless" if it has been run before.
 
I didn't understand that you were offering to look at my app and see what
could be done. THANKS!
 
 
Edited by Marshall
Link to comment
Share on other sites

Jocko, the sky isn't falling, please stop making it into some serious complicated issue to scare the newbie.  

Marshall, have you tried to use the function yet? if you have problems with that, then respond, else mark as solved.  WinMenuSelectItem()

Link to comment
Share on other sites

55 minutes ago, BigDaddyO said:

have you tried to use the function yet?

Actually I have tried WinMenuSelectItem() on the program in question.  However, when I was doing so, I noticed that the program was intended to be used during a ham radio contest.  Not wanting to violate TOS, I flagged it for the mods to decide.

1 hour ago, BigDaddyO said:

Jocko, the sky isn't falling, please stop making it into some serious complicated issue to scare the newbie.

I did little more than reiterate what the help says:

Quote

WinMenuSelectItem() will only work on standard menus. Unfortunately, many menus in use today are actually custom written or toolbars "pretending" to be menus. This is true for most Microsoft applications.

 

Code hard, but don’t hard code...

Link to comment
Share on other sites

  • Moderators

Hi,

1 hour ago, JockoDundee said:

I flagged it for the mods to decide

And we are happy for the thread to stay open.

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to comment
Share on other sites

3 hours ago, Melba23 said:

And we are happy for the thread to stay open.

Unfortunately, for me at least, WinMenuSelectItem() did not cause any noticable action from the GUI, I tried several screens and commands, for instance:

WinWaitActive("14200.00 USB Manual - VFO A")

Sleep(200)
WinMenuSelectItem("14200.00 USB Manual - VFO A", "", "&File")

Sleep(200)
WinMenuSelectItem("14200.00 USB Manual - VFO A", "", "&File", "&Exit")

Sleep(200)
WinMenuSelectItem("[CLASS:Notepad]", "", "&File", "E&xit")

In this case, only Notepad was affected.

Suggest using ControlSend() instead.

 

Code hard, but don’t hard code...

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...