Leaderboard

Automate all windows and browser applications with one UDF function library. Based on the microsoft automation API this library high level supports Recognition of conttrols from EDGE, Chrome, FF, Opera, Safari and Windows native apps Small testing framework to split object repository from coding away Introduction Quickstart - Getting started quickly Simple scripts With this module you can automate all applications/programs that support ui automation and/or accesibility api from microsoft you can recognize more controls than AutoIT can recognize "out of the box" you can use concepts from other testing frameworks like http://download.freedesktop.org/ldtp/doc/ldtp-tutorial.pdf http://safsdev.sourceforge.net/Default.htm coded ui testing from microsoft Some of those controls / applications are chrome browser (partly mainwindow has to be done with MSAA for navigating) chrome://accessibility in the adress bar of chrome or start with "--force-renderer-accessibility" silverlight controls Ribbon control controlbars of Excel/Word IE and FF browsers Windows Media Player Windows clock AFX .. controls (partly) .... Based on the initial AIO Object I now have made the interface file to work with objCreateInterface function which is in the latest beta's automate clicking and querying basic information It gives you a lot of basic information to be able to automate clicking, querying basic information where it goes further in certain situations than AutoIt is identifying Starting threads for background on the ui automation api of microsoft (not for starters) http://en.wikipedia.org/wiki/Microsoft_UI_Automation http://msdn.microsoft.com/en-us/library/ms747327.aspx Previous threads in general help/support Interface AutoItObject IUIAutomation ObjCreateInterface and struct tagPoint in method ElementFromPoint Be aware that API is not allways installed under XP/Vista see http://support.microsoft.com/kb/971513 Within Windows 7 and Windows 8 it should be preinstalled by default. Be aware on 32 and 64 bits way of running your script #AutoIt3Wrapper_UseX64=Y or N Basic example of usage / showing and retrieving the default information, will post multiple examples later Hover your mouse to an area of interest and press ctrl+w and information will be shown in the edit box of the form Simple spy demo (see simplespy.au3 or use latest ZIP attachment for latest version) Main features Recognize windows and html controls for the major browsers Logical and physical description for controls (UI mapping, Application map) Simple repository logic to abstract logical and physical descriptions Store Runtime Type Information in RTI. variables Rubberbanding/highlighting of objects Simple spy to help in making / identifying the physical description Support of regular expression(s) in identifying objects recognize objects on multiple properties supported properties: name ,title, automationid, classname, class, iaccessiblevalue, iaccessiblechildId, controltype, processid, acceleratorkey The actions provided so far "leftclick", "left", "click", "leftdoubleclick", "leftdouble", "doubleclick", _ "rightclick", "right", "rightdoubleclick", "rightdouble", _ "middleclick", "middle", "middledoubleclick", "middledouble", "mousemove", "movemouse" "setvalue","settextvalue" "setvalue using keys" "setValue using clipboard" "getvalue" "sendkeys", "enterstring", "type", "typetext" "invoke" "focus", "setfocus", "activate" "close" "move","setposition" "resize" "minimize", "maximize", "normal", "close", "exist", "exists" "searchcontext", "context" "highlight" "getobject","object" "attach" "capture","screenshot", "takescreenshot" "dump", "dumpthemall" "propertyvalue", "property" match on multiple properties like: name:=((Zoeken.*)|(Find.*)); ControlType:=Button; acceleratorkey:=Ctrl+F Support for 117 different properties see $UIA_propertiesSupportedArray in uiawrappers like for example title, regexptitle, class, regexpclass, iaccessiblevalue, iaccessiblechildid, name, accesskey, automationid, classname IAccessible, IAccessible2, ISimpleDom interfaces debuglogging to a file log.txt (no output in scitewindow) Examples Example 1 Iterating thru the different ways of representing the objects in the tree (#comment-1105548) Example 2 Finding the taskbar and clicking on the start menu button (#comment-1105680) Example 3 Clicking a litlle more and in the end displaying all items from the clock (thats not directly possible with AU3Info) (#comment-1108849) Example 4 that demonstrates the calculator Example 5 Automating chrome Example 6 Demonstrates all stuff within chrome to navigate html pages, find hyperlink, click hyperlink, find picture, click picture, enter data in inputbox Example 7 The chrome example modified to a firefox example Example 8 The other major browser Internet Explorer automated (made on Example 6 and 7) Example 9 Windows media player Example 10 Automating mach 3 (AFX windows and other hard to get recognized by AutoIT) Lot of links are broken due to forum upgrade just search for the text like "Example 11 Demonstrate Word, Notepad and Calculator actions" Example 11 Demonstrate Word, Notepad and Calculator actions ... Example 13 Details 1 about the right pane of the windows explorer Example 14 Details 2 about the right pane of the windows explorer Example 15 Details 3 about the right pane of the windows explorer Example 16 Details 4 about the right pane of the windows explorer Example 17 Details 5 about the right pane of the windows explorer WITH CACHING Example 18 Details 6 about the right pane of the windows explorer WITH VIRTUAL ITEMS Example 19 Eventhandling examples Example 20 Eventhandling examples Example 21a Eventhandling examples Internet Explorer Example 21b Eventhandling examples Internet Explorer Example 22 Eventhandling examples Follow focus Example 23 Eventhandling examples structure changed Example 24 Eventhandling examples IUIAutomationEventHandler Example 25 SAFEARRAYS Example 26 IACCESSIBLE / MSAA Example 27 IACCESSIBLE2 / MSAA Example 28 IACCESSIBLE / MSAA events Example 29 IACCESSIBLE2 events Example 30 ISimpleDOM Example 31 Notepad window move, maximize, minimize Example 32 Three browsers doing the same stuff with small differences in scripting only .. TODO Build recorder Enhance the spy with a nicer UI UI for the repository (now in the script with dot notation) Enhance mapping / identifying on multiple properties instead of 1 combined with index If speed becomes an issue use the caching logic of the MS UIA framework Add the other patterns later Generalize the concept of System Under Test of starting the SUT (for testing framework purposes) Remote running of scripts Fix issue on finding within dynamic context ... edit august 18th 2013 initial post Only zip files are needed to download , just unzip in 1 directory edit july 2016 Made V0_63 and examples works with AutoIt v3.3.14 Windows 10 tested Simple spy gives some basic code as a present Chrome latest versions seems to be having issues with IUIAutomation on tabs/buttons of mainwindow use MSAA for accessing tabsheets / buttons more cleanup to be in UDF style More comments in the source see changelog.txt for previous changes edit september 2017 All examples fixed for the IE, Firefox and Chrome browser Some small but essential fixes in UIAWrappers edit april 2018 Enhanced logic on fallback / dynamic search, still not perfect, to slow Retested with latest Chrome, FF, Edge and IE11 and some extensions to show how to get text from the webpage (examples 5,6,7) Some small bugfixes Some comments as given in forum incorporated edit may 2019 Speed enhancements on especially fallback searching UIA.CFG works now in a better way to turn on/off debug, highlighting, debug2file More stable and consistent behavior Internal cleanup and refactoring of bigger functions Checked with W10 (not tested on W7) Added some W10 properties Run with 3.3.14.5 on W10 UIA_V0_51.zip EXAMPLES_V0_5.zip UIA_V0_63.zip EXAMPLES_V0_63.zip UIA_V0_64.zip EXAMPLES_V0_64.zip EXAMPLES_V0_66.zip UIA_V0_66.zip EXAMPLES_V0_70.zip UIA_V0_70.zip

BLOG None of what I am saying in this post, is a criticism of CyberSlug, as I feel I am all too aware of how and why GuiBuilder was just whipped up the way it was. In fact, it feels like something slapped together from various bits and pieces and rearranged often, both adding & removing elements as the experiment advanced. I don't think it was ever meant to live for long. Neither is it a criticism of others who have also tinkered with the code. --------------------------------------------------------------------- Well, after having a break from coding for a few days, I felt sufficiently refreshed today to dig deep, and that is what I did. After a few hours of good solid work, I eventually became so confused with several elements, that I had to stop and consider my options. In the end, I decided to integrate what I had done so far, with the original code, cleaning up the original code as I went. Then in a day or five, I would try again, from a better position. The re-integration & cleanup initially went well, but I was increasingly becoming more and more confused, as I realized just how bad a state the original code was in. There are several reasons why I don't like working with other people's code, and top of that list would be mistakes and bad practices that hamper your understanding, as well as concepts beyond my skill-set. I fear this ticks all the boxes in that regard. You have variables that appear in both upper and lower case, and the same or others declared in both the Local and Global scope. You have elements that have been declared, but are not in use, looking important but not actually doing anything. I generally like short variable names, but single letters that are used for one thing one moment and something very different the next, that would only work (if some of them are indeed working), because they are in the Local scope. Then you have double letter ones, etc. Often it seems to me, you could get rid of most of them, because many seem to be doing the same thing, but following the logic to determine this, is an agonizing process. With very little comments and variable names not descriptive enough, it has become a nightmare, that would take a mountain of work to fix. Add to that the degree of difficulty from my lack of understanding of some things, and I fear to proceed lest some untoward thing happens, when I run the code eventually. And then there is the good chance of failure after all that work, and then because I have committed that far, continuing to reach for a solution, perhaps futilely. So wasting my time is also of great concern. I definitely don't know enough, to be a good troubleshooter of other people's code, especially when I come across seeming mysteries and contradictions or conflicts in the code. Honestly, it is a wonder the program actually works as well as it does. I think I can see why CyberSlug hasn't returned to it in many years ... and he understood it. I must be remembered too, that when CyberSlug started work on this GUI builder, it was very much experimental ... being early days for GUI's with AutoIt, with many things we take for granted now, still being developed. So in a sense, GuiBuilder (AutoBuilder) was very much an exercise in proof of concept, as well as a very helpful tool at that time. CyberSlug, I am guessing, essentially abandoned his experiment, when others started to work on alternates and when Koda came along. I guess he was never counting on some like me, becoming passionately attached to his effort. Where to from here? Tomorrow perhaps, I will review things again, but it seems to me right now, that I should start again with the integration, and do no more than that. With any luck, that might mean the program is at least usable with latest AutoIt. If someone braver and smarter than me, then wants to take on the challenge of improving it further, then that would be great. I do like a good challenge, but it has to be an enjoyable one ... I like being creative and I like learning ... to a point. For me, it is as much about the journey as it is the result. If the journey becomes too unpleasant, then you do something else ... Life is too short to waste it on things you don't like doing.

Welcome to AutoIt and the forum! Please have a look at my OutlookEX UDF, function _OL_ItemCreate, to create this mail.

This trap is catchy indeed! Thanks for pulling me out of it.

@pnyfat After a brief glance at A Dark Room I would say yes, you could make something similar with AutoIt. As far as the GUI and button limitations, if you do some searching on the forum you'll find different contributions in this area. This one, for example, shows how to make more rounded buttons:

@pnyfat - I see you have fallen for the trap that some do, with the AutoIt name. AutoIt stands for 'automate it'. The 'It' at the end does NOT stand for IT, as in IT department.