Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

your second window is a different popup (or a hidden one you do not see) so thats why the event fires twice (2 windows are closing)

I am not sure but this if
If $iEventId <> $UIA_Window_WindowClosedEventId Then

and later this if
If _UIA_getPropertyValue($oSender, $UIA_ControlTypePropertyId) = "50032" Then ;50032 popup

seems to be in the wrong order.

adding sleep 5000 will not help in any way

not sure if this $TreeScope_Subtree should be $TreeScope_Children

Copied your example and works on my system for Class=#32770 and X top right has name Sluiten (is in dutch)

Triggered popup in IE with javascript:alert("hello"); in addressbar

PopUp Visible - Windows Name: Bericht van webpagina
Title: Bericht van webpagina Pos: 879;484;179;171 Center = Class=#32770
Title: OK Pos: 954;610;88;26 Center = Class=Button
Title: Pos: 912;540;32;32 Center = Class=Static
Title: hello Pos: 952;548;61;17 Center = Class=Static
Title: Pos: 887;487;163;27 Center = Class=
Title: Sluiten Pos: 1003;485;49;20 Center = Class=
Closing Now
Func oIUIAutomationEventHandler_HandleAutomationEvent($pSelf, $pSender, $iEventId) ; Ret: long  Par: ptr;int
    Dim $t
    If $iEventId <> $UIA_Window_WindowClosedEventId Then
        Local $oSender = ObjCreateInterface($pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        $oSender.AddRef()
;~         If _UIA_getPropertyValue($oSender, $UIA_ControlTypePropertyId) = "50032" Then ;50032 popup
        if _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId )= "#32770" then
            ConsoleWrite("PopUp Visible - Windows Name: " & _UIA_getPropertyValue($oSender, $UIA_NamePropertyId) & @CRLF)
            findThemAll($oSender, $TreeScope_Subtree)
            ; sleep(5000) ; used to avoid 2nd loop
        EndIf
    EndIf
    Return $S_OK
EndFunc   ;==>oIUIAutomationEventHandler_HandleAutomationEvent

Func findThemAll($oElementStart, $TreeScope)
    Local $oCondition, $oAutomationElementArray, $oUIElement
    Dim $pCondition, $pTrueCondition
    Dim $pElements, $iLength
    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)
    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
    $oAutomationElementArray.Length($iLength)
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        ConsoleWrite("Title: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Pos: " & _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId) & @TAB & "Center =" & _UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)
            if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) = "Sluiten" Then
            ConsoleWrite("Closing Now" & @CRLF)
            Sleep(500)
;~             dim $t2
;~             $t2=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),";")
;~             MouseClick("left",$t2[1],$t2[2],1,0)

            _UIA_Action($oUIElement,"left")

;~             Sleep(5000)
            ExitLoop
        EndIf
    Next
EndFunc   ;==>findThemAll

and i just use _UIA_Action($oUIElement,"left")

Share this post


Link to post
Share on other sites

marko001, How many event handlers have you created? If you are only watching for window open events, you should not create event handlers to handle window close, menu open or menu close. If a window is opened and the window contains a menu, you'll probably get a window open event and a menu open event. That could be the reason why you are getting two events.

Share this post


Link to post
Share on other sites

@Marko001: I think you have the buttons top right beeing minimize, maximize, CLOSE

and a button CLOSE

I suspect when closing the popup gets hidden and as such

the 2nd button CLOSE window you receive the closing event (but then is already hidden)

Share this post


Link to post
Share on other sites

@larsJ, @junkew

Hi guys and sorry for delay in  answer, i was out with family.

i removed all unuseful handlers:

Func MainFunc()

    $oIUIAutomationEventHandler = ObjectFromTag("oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True)

    If Not IsObj($oIUIAutomationEventHandler) Then Return



    $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)

    If Not IsObj($oUIAutomation) Then Return



    Local $pUIElement

    $oUIAutomation.GetRootElement($pUIElement) ; Desktop

    If Not $pUIElement Then Return



    ; Window open/close events

    If $oUIAutomation.AddAutomationEventHandler($UIA_Window_WindowOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler) Then Exit



    HotKeySet("{ESC}", "Quit")



    While Sleep(10)

    WEnd



EndFunc   ;==>MainFunc

There are no others button (visible or not) except

Title: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Center =374;698    Class=ToolWindow

Title: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Center =374;698    Class=ToolWindow  <---- WHY IT CALLS IT TWICE??

Title:     Pos: 257;424;24;24    Center =269;436    Class=Image

Title: Network Message    Pos: 286;428;69;15    Center =320;435    Class=TextBlock

Title: (private)    Pos: 365;428;43;15    Center =386;435    Class=TextBlock

Title: Minimize    Pos: 425;424;24;24    Center =437;436    Class=Button

Title: Maximize    Pos: 449;424;24;24    Center =461;436    Class=Button

Title: Close    Pos: 473;424;24;24    Center =485;436    Class=Button

my class is

_UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId )= "ToolWindow"

so I had to change it accordingly.

If I use _UIA_Action()

Func findThemAll($oElementStart, $TreeScope)

    Local $oCondition, $oAutomationElementArray, $oUIElement

    Dim $pCondition, $pTrueCondition

    Dim $pElements, $iLength

    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)

    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)

    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)

    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)

    $oAutomationElementArray.Length($iLength)

    For $i = 0 To $iLength - 1; it's zero based

        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)

        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

        ConsoleWrite("Title: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Pos: " & _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId) & @TAB & "Center =" & _UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)

          if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) = "Close" Then

            ConsoleWrite("Closing Now" & @CRLF)

            Sleep(1000)

            _UIA_Action($oUIElement,"left")

             Sleep(5000)

            ExitLoop

        EndIf

    Next

EndFunc   ;==>findThemAll

I get the following:

PopUp Visible - Windows Name: Shiny.Chat.ViewModels.PrivateChatSessionViewModel

Title: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Center =374;698    Class=ToolWindow

Title: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Center =374;698    Class=ToolWindow

Title:     Pos: 257;424;24;24    Center =269;436    Class=Image

Title: Network Message    Pos: 286;428;69;15    Center =320;435    Class=TextBlock

Title: (private)    Pos: 365;428;43;15    Center =386;435    Class=TextBlock

Title: Minimize    Pos: 425;424;24;24    Center =437;436    Class=Button

Title: Maximize    Pos: 449;424;24;24    Center =461;436    Class=Button

Title: Close    Pos: 473;424;24;24    Center =485;436    Class=Button

Closing Now

F:\AUTO-IT Projects\___Health Software Manager\_________________________2.0 __\ex3\UIAWrappers.au3 (1020) : ==> Variable used without being declared.:

$controlType=_UIA_getPropertyValue($UIA_oUIElement,$UIA_ControlTypePropertyId)

^ ERROR

PopUp Visible - Windows Name: Shiny.Chat.ViewModels.PrivateChatSessionViewModel

Title: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Center =374;698    Class=ToolWindow

Title: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Center =374;698    Class=ToolWindow

Title:     Pos: 257;424;24;24    Center =269;436    Class=Image

Title: Network Message    Pos: 286;428;69;15    Center =320;435    Class=TextBlock

Title: (private)    Pos: 365;428;43;15    Center =386;435    Class=TextBlock

Title: Minimize    Pos: 425;424;24;24    Center =437;436    Class=Button

Title: Maximize    Pos: 449;424;24;24    Center =461;436    Class=Button

Title: Close    Pos: 473;424;24;24    Center =485;436    Class=Button

Closing Now

So it still replicates the loop twice and set this error using UIA_Action().

Moreover it doesn't click and it loose focus

[EDIT]:

Found that Local $controlType needed to be declared in UAIWrappers.au3.

Now it works correctly and clicks.

No matter for the 2nd pop-up. It's not a problem since when called for the 2nd time it now looks like:

Title:     Pos: 262;413;250;550    Center =387;688    Class=ToolWindow
Title:     Pos: 262;413;250;550    Center =387;688    Class=ToolWindow
Title:     Pos: 305;496;17;17    Center =313;504    Class=Image
Title: Minimize    Pos: 423;496;17;17    Center =431;504    Class=Button
Title: Maximize    Pos: 0;0;0;0    Center =-2147483648;-2147483648    Class=Button
Title: Close    Pos: 0;0;0;0    Center =-2147483648;-2147483648    Class=Button

So consider that completely solved!

I will create a template .au3 and put it as a new thread as help. I believe this could be very useful for people always looking trough "_imagesearch()" that could now use these solutions to achieve their goals.

Thanks again guys for the support.

Marco

Edited by marko001

Share this post


Link to post
Share on other sites

please try the same for another popup with entering in addressbar

javascript:alert('Hello world');

Difficult to see what happens. Could be a program error in your application if it sends twice a WM_CLOSE message in the system.

Share this post


Link to post
Share on other sites

Infact is what I did, I tried and it didn't duplicate. That's for now I consider it closed, probably is due to what you said some posts before

your second window is a different popup (or a hidden one you do not see) so thats why the event fires twice (2 windows are closing)

Thanks,

Marco

Share this post


Link to post
Share on other sites

Hi all again guys,

i'm trying now to update my applications with this technology just used as seen before.

I have an event I can catch:

        if StringInStr(_UIA_getPropertyValue($oSender, $UIA_NamePropertyId),"Diagnostic: ") <> 0 Then
            ConsoleWrite("Diagnostic Window: " & _UIA_getPropertyValue($oSender, $UIA_NamePropertyId) & @CRLF)
            findThemAll($oSender, $TreeScope_Subtree)
        EndIf

 

Among all datas collected I need to focus on a grid, containing samples to be analyzed.

This grid, 6 rows visible (up to 400), at beginning (i.e. now, as soon as I catch the event) is empty and can be filled in real time by analysis lab  personnel. So I can see the rows filling the grid. I always can see 6 rows so i need to scrolldown to see the rest.

Using simplespy.au3 I see, clicking in the middle of the grid (still empty)

*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=DetailsViewDataGrid
UIA_BoundingRectanglePropertyId :=10;564;491;181
UIA_ClassNamePropertyId :=ReqExamListViewDataGrid

 

If I now click on the filled grid, clicking on an item I see:

*** Parent Information ***
Title is: <Exam_Req: Toxoplasmosis >    Class   := <DataGridRow>    controltype:= <UIA_DataItemControlTypeId>    ,<50029>    , (0000C36D)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=
UIA_BoundingRectanglePropertyId :=50;588;126;19
UIA_ClassNamePropertyId :=DataGridCell

 

And I see arrows to go further 6th row.

Question is: is there a way to catch when the list is updated (rows added/removed/modified) and get the datas? (also the one hidden)?

Every row contains (of my interest)

1) Number of dosis

2) Exame requested

3) Lot #

Thanks,

Marco

Share this post


Link to post
Share on other sites

May be you can catch updates with the StructureChangedEventHandler. Example 3 in post 112. The example is looking for events in all windows in the Desktop. You should only be looking for events in your own window. But for a test you can probably use the example as it is.

Share this post


Link to post
Share on other sites

@LarsJ

yes, probably I could use it.

How can I embeed the code in the previous one so I can get the event for the specific window *only* once the window is active?

(or maybe I can first check if winexists($mywindowpartialname) and then check for it?)

[Edit] I see I can update event but, for example, how can I catch text sent in the pop-up network message? (similar to netsend, just to understand)

Thanks,

M.

Edited by marko001

Share this post


Link to post
Share on other sites

To get an automation element for the entire desktop use this code:

Local $pDesktop, $oDesktop
$oUIAutomation.GetRootElement( $pDesktop )
$oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "Creating Desktop automation element failed" & @CRLF )

To get an automation element for a window use this code:

Local $pWindow, $oWindow
$hWindow = WinGetHandle( "String to identify window" )
$oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Creating Window automation element failed" & @CRLF )

You can only get a pop-up network message (with the UI automation framework) from the pop-up window. And you would probably use $oDesktop to identify the window.

Edited by LarsJ

Share this post


Link to post
Share on other sites

@LarsJ

Ok I got it, now I can focus on that window, when available, to monitor events.

As I wrote in post 247, there is another window(*) i need to work with. That contains several rows, one for every exam:

Every row is read, from simplespy.au3, as

Title is: <Element: ExamSlot: Toxoplasmosis , index visualization column: 6>    Class   := <DataGridCell>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    

*** Parent Information ***

Title is: <Element: ExamSlot: Toxoplasmosis >    Class   := <DataGridRow>    controltype:= <UIA_DataItemControlTypeId>    ,<50029>    , (0000C36D)    

*** Detailed properties of the highlighted element ***

UIA_AcceleratorKeyPropertyId :=

UIA_AccessKeyPropertyId :=

UIA_AriaPropertiesPropertyId :=

UIA_AriaRolePropertyId :=

UIA_AutomationIdPropertyId :=

UIA_BoundingRectanglePropertyId :=822;626;137;20

UIA_ClassNamePropertyId :=DataGridCell

Every row has (exactly like Excel) 3 interesting cells I need to monitor (not necessary visibile in the grid (may be over the 6th row, or over the width of the window (that means scrollbar...))):

1) Number of dosis

2) Exam requested (Exam Name)

3) Lot #

I can't get with simplespy.au3 the 1) and 3). I mean, every mouse pos between the row give me same result (the one posted) except the different mouse position and index visualization column:

Is there a way to catch all the info for every cell of every row? (at least the ones i need to store in a var?)

Thanks,

Marco

(*): honestly this PART of a main window, but I can get it's infos by just monitoring that part with

     If _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) = "ExamListViewDataGrid" Then

Share this post


Link to post
Share on other sites

This code should find all rows in your data grid. What does it print in the console?

Local $pCondition
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_DataItemControlTypeId, $pCondition )
If Not $pCondition Then Return ConsoleWrite( "CreatePropertyCondition Err" & @CRLF )

Local $pUIElementArray, $oUIElementArray, $iElements
$oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
$oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
$oUIElementArray.Length( $iElements )
If Not $iElements Then Return ConsoleWrite( "FindAll Err" & @CRLF )
ConsoleWrite( "$iElements = " & $iElements & @CRLF )

Local $pUIElement, $oUIElement, $name
For $i = 0 To $iElements - 1
  $oUIElementArray.GetElement( $i, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
  ConsoleWrite( $name & @CRLF )
Next
$oWindow is the window that contains the data grid. Edited by LarsJ

Share this post


Link to post
Share on other sites

Thanks for this!

I'm trying to automate clicking on a string in Chrome, can't make it work.

The thing is that it is not a link, as in that you can't see a link to a different HTML file. It just sends a different parameter via GET.

Anyone knows if there's a way to find a text and then click there?

This is a bit advanced for me, so any guidance would be greatly apreciated.

Share this post


Link to post
Share on other sites

This example prints the items of the Windows Explorer listview in Scite console. The selected state of the items is printed too.

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )
  Next

EndFunc

This code works for me. But I am facing one problem. Suppose I have many files such that I have to scroll to see all files. In that case $iElements will have the count of files visible only in the window and not of those beyond the scroll. How can I have a list of all the files i.e including files existing below the scroll

Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

dilipped, The example you are referring to (post 78, example 15 in the list in first post), does not handle virtual items in Windows Explorer listview.

To handle virtual items see post 94, example 18 in the list in first post. I have updated the code. This example also prints the items in Windows Explorer listview and the selected state.

You should take a look at this example: Automating Windows Explorer.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...