Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Really no need to quote the previous post.

The parts to post the output of are the parts that show the controls in question.


Spoiler

Renamer - Rename files and folders, remove portions of text from the filename etc.

GPO Tool - Export/Import Group policy settings.

MirrorDir - Synchronize/Backup/Mirror Folders

BeatsPlayer - Music player.

Params Tool - Right click an exe to see it's parameters or execute them.

String Trigger - Triggers pasting text or applications or internet links on specific strings.

Inconspicuous - Hide files in plain sight, not fully encrypted.

Regedit Control - Registry browsing history, quickly jump into any saved key.

Time4Shutdown - Write the time for shutdown in minutes.

Power Profiles Tool - Set a profile as active, delete, duplicate, export and import.

Finished Task Shutdown - Shuts down pc when specified window/Wndl/process closes.

NetworkSpeedShutdown - Shuts down pc if download speed goes under "X" Kb/s.

IUIAutomation - Topic with framework and examples

Au3Record.exe

Share this post


Link to post
Share on other sites

If its hanging 99.99% sure you do not have clear your UI hierarchy and you do not have a proper identificationstring

you mix au3 functions control* with _UIA* functions.
Thats in general a bad approach which will lead to issues.

  • To learn this udf I advice to start in post 1 and with example 1 and make it work on calculator and notepad.
  • Understand the first 10 examples and then start on your own application.
  • Simplespy will help you by giving code as a present which you can continue to build with.
     

First understand the base and use highlight action before doing any other actions

#include "UIAWrappers.au3" 

_UIA_SETVAR("Description","classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad118")

_UIA_Action("Description","highlight")

;~_UIA_Action("Description","focus")

And read the first few posts in detail otherwise I am repeating what is already in start of this thread

Share this post


Link to post
Share on other sites

 

tTrying the first example but i get Error: Unknown Fuction Name.  I can't find these functions in autoit help either. What do these mean?

samplewinlist()

sampleTW(1)
sampleTW(2)
sampleTW(3)

Nvm found the function in the example file.

 

I am using simplespy to define the vars but It seems that the program is getting the info for a different part of the window. Is there another way to fetch the info to define the var?

I'm notiding it says Enteredby: as the title but that i beleive is a different part of the window. The part im trying to select is the Description and that is not being highlighted when i select Description: or Description as the title

_UIA_setVar("oP1","Title:=View Activity;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")  ;View Activity
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1") ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1") ;
_UIA_setVar("oP4","Title:=Delete;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")   ;Delete

_UIA_setVar("EnteredBy:.mainwindow","title:=Entered By:;classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad1")

This is the whole code.

AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=View Activity;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;View Activity
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oP4","Title:=Delete;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Delete

_UIA_setVar("Description:.mainwindoww","title:=Entered By:;classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad1")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")

;~_UIA_Action("Description:","highlight")
_UIA_Action("Description:.mainwindow","setfocus")

Local $oDescription = _UIA_Action("Description:.mainwindow","getvalue")

_UIA_setVar("oFIADescription1","Title:=Activity / Suspense;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Activity / Suspense
_UIA_setVar("oFIADescription2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oFIADescription3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oFIADescription4","Title:=Delete;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Delete
_UIA_setVar("oFIADescription5","Title:=Company:;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Company:
_UIA_setVar("oFIADescription6","Title:=Activity;controltype:=UIA_GroupControlTypeId;class:=WindowsForms10.BUTTON.app.0.30495d1_r9_ad1")    ;Activity

_UIA_setVar("FIADescription:.mainwindow","title:=Description:;classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad1")

;~_UIA_Action("oFIADescription1","highlight")
_UIA_Action("oFIADescription1","setfocus")
;~_UIA_Action("oFIADescription2","highlight")
_UIA_Action("oFIADescription2","setfocus")
;~_UIA_Action("oFIADescription3","highlight")
_UIA_Action("oFIADescription3","setfocus")
;~_UIA_Action("oFIADescription4","highlight")
_UIA_Action("oFIADescription4","setfocus")
;~_UIA_Action("oFIADescription5","highlight")
_UIA_Action("oFIADescription5","setfocus")
;~_UIA_Action("oFIADescription6","highlight")
_UIA_Action("oFIADescription6","setfocus")

;~_UIA_Action("FIADescription:.mainwindow","highlight")
_UIA_Action("FIADescription:.mainwindow","setfocus")
_UIA_Action("FIADescription:.mainwindow","settextvalue",$oDescription)

 

Edited by milkmoron

Share this post


Link to post
Share on other sites

Use inspect.exe and see faq31 on tools that will tell you the hierarchy of your elements. You will have to read a lot and try things out. Its explained really in the beginning and for starters first advice is to automate calculator, notepad of windows. As you already see you think its a different element for that to know its true only inspect.exe can tell you. For sure nobody can tell based on what you post. We do not have access to your application so we can only guess what the actual hierarchy is.

Share this post


Link to post
Share on other sites

@milkmoron Yes, just open the included wrapper script and and look at the functions :). Pro Tip: If you have full SciTE4AutoIt3 installed then you can use Alt + Q to open SciTE Jump which lists all of the functions!


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

@milkmoron please tell me why first post is not enough on explaining the functions and actions available then i can improve first post. And as given you can allways look in the udf module itself.

Share this post


Link to post
Share on other sites
23 hours ago, junkew said:

@milkmoron please tell me why first post is not enough on explaining the functions and actions available then i can improve first post. And as given you can allways look in the udf module itself.

we'll it's a little hard to know what functions are available but I guess looking into the wrapper itself helps alot. A list of functions and a brief description of what they do would be nice. Like the Autoit help file.

Share this post


Link to post
Share on other sites

I can't figure out how to use partial Window-Title. I need it because it constantly changes.

So this works 

Local $oP4 = _UIA_getObjectByFindAll($UIA_oDesktop,"Title:=Untitled - Notepad")

However, this doesn't

Opt("WinTitleMatchMode",2)
WinActivate("Note")
Local $sText = WinGetTitle("[ACTIVE]")
Local $sText2 = '"Title:=' & $sText & '"'
Local $oP4 = _UIA_getObjectByFindAll($UIA_oDesktop,$sText2)

Above are just examples. I'm trying to get it to work for Trade-Ideas program that has multiple windows of same class with constantly changing titles

Share this post


Link to post
Share on other sites

Hello,

I'm trying to figure out how to open a menu item in a program that has a non-traditional menu structure.  

I had the script working in an old version of the software like this - 

WinMenuSelectItem($handle, "", "&FILE", "Save As...")

Which obviously did a simple file -> save as.  

But the program has since been updated by the vendor, and now menu is custom-built.  Using the simply spy tool, when I highlight the "save as" menu item and press CTRL + W, I get this output (standard code flexible) - 

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Logix Designer - HP_RCS_VSC in HP_VSC.ACD [1756-L82ES 31.11];controltype:=UIA_WindowControlTypeId;class:=Afx:00400000:8:00010003:00000000:019F1155", $treescope_children) 
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=File;controltype:=UIA_MenuControlTypeId;class:=Afx:00400000:800:00010003:00000010:00000000", $treescope_children)  
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=FILE;controltype:=UIA_ToolBarControlTypeId;class:=BCGPToolBar:400000:8:10003:10", $treescope_children) 
_UIA_Action($oP0,"setfocus")
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("SaveAs....mainwindow", "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

As you can see, it's qutie large, and I can't figure out how to clean up that output to make it usable in my script.  Can someone help me please?

Share this post


Link to post
Share on other sites

Is the code working? In general you have to follow gui hierarchy and first set focus to parent before invoking new childsearch.

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Logix Designer - HP_RCS_VSC in HP_VSC.ACD [1756-L82ES 31.11];controltype:=UIA_WindowControlTypeId;class:=Afx:00400000:8:00010003:00000000:019F1155", $treescope_children) 
_UIA_Action($oP2,"setfocus")

;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("SaveAs....mainwindow", "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP2, "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

Sometimes you can leave out some elements like above only topwindow p2 and search in subtree of p2.

Share this post


Link to post
Share on other sites

Hi Junkew. I like your library. I took a look at it and used similar ideas to make a GUI automation package in python. It works pretty much the same as how your code works but its a bit faster and available on PIP. Anyone can try it out, and I would love feedback. I also am using an expansion of the recorder-spy in the package.

I made a getting started guide on medium for those interested.

https://medium.com/@lundbird/how-to-automate-windows-applications-with-autogui-626c7b452eed 

Share this post


Link to post
Share on other sites
On 11.08.2013 at 12:04 AM, junkew said:

First link to examples have wrong title.
Could you edit OP ?

 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API Forum Rules *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

11

Signature last update: 2018-10-31

Share this post


Link to post
Share on other sites

I just start to check your examples, a little..... and in first place I change the names to have better ordering in sorting by name.
I change only file names, but hope this will help others, and I hope you will find this as a good contribution :)

 

 

Examples.zip

Edited by mLipok

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API Forum Rules *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

11

Signature last update: 2018-10-31

Share this post


Link to post
Share on other sites

I've been using this to automate C# screens for a while, so I'm very thankful for all your work junkew. 

My question is that there seem to be some sort of inconsistency. I'm not sure if I'm using a more outdated approach, but the way I target C# objects is by first identifying the window I want to click in, and then making recursive functional calls to iterate through the tree of elements using a RawViewWalker to search for the element which matches the automation ID I'm targeting. 

I then have an order of operations that clicks various buttons and boxes on the screen. But sometimes the RawViewWalker doesn't seem to be able to find the elements I want to click despite them very clearly being there. Is there a known reason this could be the case? I get the vibe that it's because the screen is partially dynamic and some elements are added/updated after a click, and the walker is using a cached version of the window instead of the updated on.
 

Share this post


Link to post
Share on other sites

You can only check with inspect.exe but thats probably a fresh treecache in a different process.

maybe MS documented it somewhere

https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/caching-in-ui-automation-clients

 

if you know the automation id then probably findone or findfirst method will help you to find it with a filter condition on automationid instead of walking around the tree yourself

 

Share this post


Link to post
Share on other sites

Thanks, where can I find the documentation/usage of findall/findone? Is there a wiki page for the entire framework anywhere?

Did a search/scan of the thread and no results. Tried to find info on the wiki and couldn't either.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...