Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

I use the following code to get text in a cell, but it select all grid view and copy all to clipboard. What should i do to get value in a cell?

_UIA_setVar("oP7","Title:=abc Row 0;controltype:=UIA_CustomControlTypeId;class:=")
_UIA_Action("oP7","getvalue")

 

 

6qxeDxX.png

Share this post


Link to post
Share on other sites

you should read the first few posts that tell you to try with simplespy and post the output so we can see

  1. What kind of controltype we are dealing with
  2. What patterns are supported natively by UIA
  3. More details if you ask a question as there are dozens of ways to automate application tables
  4. ....

If getvalue did work then your text is on the clipboard and you should be able to get it. But I assume that after you find your text you want to turn on/of checkboxeds on that line.

 

Share this post


Link to post
Share on other sites

I need to Copy all the text from a Chrome Browser page. But the underlying page is protected against copying.

I can unprotect the page with extensions, but I'd like to avoid that. Unfortunately Chrome is required for the web application we are using.

Before I go down the road of learning this UDF I'd like to know if I can get access to content of the Chrome page without an extension.

All I really need to accomplish is find a specific Chrome Browser tab (base on user input into autoit) and then copy all the text from the page so that I can extract specific information from it.

Thank you.

Edited by dchaosw360

Share this post


Link to post
Share on other sites

You can in any browser type javascript in addressbar

Javascript:<your jscode>;void,(0);

So either do

1.alert(document.body.innertext)

or

2. more complex scripting like adding textbox and copy pasting from that textbox.

Or

3. make use of adding websocket thru addressbar and make use of tcpip commands autoit

So as soon as regular functions cannot help you you can run any javascript thru addressbar. Sometimes only small chunks as number of characters is limited in addressbar.

 

Be aware javascript and html is very case sensitive.

Edited by junkew

Share this post


Link to post
Share on other sites

Unfortunately the tool doesn't work out of the box on Windows 7 64-bit SP1 :(

I tried only V0.70 and V0.66 and they can't get proper control information from a WPF Application - they are querying the desktop window instead.

I guess, it is caused by the UAC, which elevates the spy (what for?), and the target application isn't elevated.

V0.51 works better - it can see the control (a button) in a WPF application, but crashes when pressing CTRL+R.

Here its log:

>"F:\Programs\autoit-v3\SciTe\..\autoit3.exe" /ErrorStdOut "E:\KA\...\UIA_V0_51\simplespy.au3"    
//TODO codetemplates.txt not available An error occurred when reading the file. & @CRLF"E:\KA\...\UIA_V0_51\simplespy.au3" (229) : ==> Subscript used on non-accessible variable.:
$tLine=$UIA_CodeArray[$i]
$tLine=$UIA_CodeArray^ ERROR
>Exit code: 1    Time: 10.01

 

Share this post


Link to post
Share on other sites
  • See FAQ 31
  • Try inspect.exe as thats out of the box Microsoft tool based on the UIA library they provide.

V0.51 and V0.70 are based on same MS DLL so objects that are recognized should be the same.

  1. Best to run first in admin mode 
  2. Try 32 bits and 64 bits AutoIt and identify what type your application is regarding 32 or 64 bit applications.

Share this post


Link to post
Share on other sites

I am trying automation CALC, for study how use UIAutomation. this error is happening i don't know why

#include "CUIAutomation2.au3"

Func Num($number)

    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)

    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement($pDesktop)
    $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $pCondition
    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, "ApplicationFrameWindow", $pCondition)

    Local $pCalc, $oCalc
    $oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pCalc)
    $oCalc = ObjCreateInterface($pCalc, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, 'Windows.UI.Core.CoreWindow', $pCondition)

    Local $pCalc1, $oCalc1
    $oCalc.FindFirst($TreeScope_Descendants, $pCondition, $pCalc1)
    $oCalc1 = ObjCreateInterface($pCalc1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, 'LandmarkTarget', $pCondition)

    Local $pGroup, $oGroup
    $oCalc1.FindFirst($TreeScope_Descendants, $pCondition, $pGroup)
    $oGroup = ObjCreateInterface($pGroup, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, 'NumberPad', $pCondition)

    Local $pGroup2, $oGroup2
    $oGroup.FindFirst($TreeScope_Descendants, $pCondition, $pGroup2)
    $oGroup2 = ObjCreateInterface($pGroup2, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, $number, $pCondition)

    Local $pButton, $oButton
    $oGroup2.FindFirst($TreeScope_Descendants, $pCondition, $pButton)
    $oButton = ObjCreateInterface($pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $pInvoke, $oInvoke
    $oButton.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
    $oInvoke.Invoke()
EndFunc   ;==>Num

Func Operators($Operators)

    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)

    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement($pDesktop)
    $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $pCondition
    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, "ApplicationFrameWindow", $pCondition)

    Local $pCalc, $oCalc
    $oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pCalc)
    $oCalc = ObjCreateInterface($pCalc, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, 'Windows.UI.Core.CoreWindow', $pCondition)

    Local $pCalc1, $oCalc1
    $oCalc.FindFirst($TreeScope_Descendants, $pCondition, $pCalc1)
    $oCalc1 = ObjCreateInterface($pCalc1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, 'LandmarkTarget', $pCondition)

    Local $pGroup, $oGroup
    $oCalc1.FindFirst($TreeScope_Descendants, $pCondition, $pGroup)
    $oGroup = ObjCreateInterface($pGroup, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, 'StandardOperators', $pCondition)

    Local $pGroup2, $oGroup2
    $oGroup.FindFirst($TreeScope_Descendants, $pCondition, $pGroup2)
    $oGroup2 = ObjCreateInterface($pGroup2, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, $Operators, $pCondition)

    Local $pButton, $oButton
    $oGroup2.FindFirst($TreeScope_Descendants, $pCondition, $pButton)
    $oButton = ObjCreateInterface($pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $pInvoke, $oInvoke
    $oButton.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
    $oInvoke.Invoke()
EndFunc   ;==>Operators

Func GetResult()

    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)

    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement($pDesktop)
    $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $pCondition
    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, "ApplicationFrameWindow", $pCondition)

    Local $pCalc, $oCalc
    $oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pCalc)
    $oCalc = ObjCreateInterface($pCalc, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, 'Windows.UI.Core.CoreWindow', $pCondition)

    Local $pCalc1, $oCalc1
    $oCalc.FindFirst($TreeScope_Descendants, $pCondition, $pCalc1)
    $oCalc1 = ObjCreateInterface($pCalc1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, 'LandmarkTarget', $pCondition)

    Local $pGroup, $oGroup
    $oCalc1.FindFirst($TreeScope_Descendants, $pCondition, $pGroup)
    $oGroup = ObjCreateInterface($pGroup, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, 'CalculatorResults', $pCondition)

    Local $pGroup2, $oGroup2
    $oGroup.FindFirst($TreeScope_Descendants, $pCondition, $pGroup2)
    $oGroup2 = ObjCreateInterface($pGroup2, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

;~  $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, 'TextContainer', $pCondition)
    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId,'ScrollViewer', $pCondition)

    Local $pTextContainer, $oTextContainer, $Value
    $oGroup2.FindFirst($TreeScope_Descendants, $pCondition, $pTextContainer)
    $oTextContainer = ObjCreateInterface($pTextContainer, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $pInvokePattern, $oInvokePattern
    $oTextContainer.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
    $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
    $oInvokePattern.Invoke()
    $oTextContainer.GetCurrentPropertyValue($UIA_NamePropertyId, $Value)

    Return $Value

EndFunc   ;==>GetResult

Run("calc")
Sleep(2000)
WinWaitActive("Calculadora", "", 5)

Num("num1Button")
Num("num2Button")
Num("num3Button")
Num("num4Button")
Num("num5Button")
Num("num6Button")
Num("num7Button")
;~ Num("num8Button")
Num("num9Button")

Operators("multiplyButton")

Num("num9Button")

Operators('equalButton')

MsgBox("", "", GetResult())
>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\resource\Desktop\AutoIt Library\exemploUIAutomate\TesteUIA2.au3" /UserParams    
+>20:24:31 Starting AutoIt3Wrapper v.19.102.1901.0 SciTE v.4.1.2.0   Keyboard:00010416  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0416)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\resource\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\resource\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\Users\resource\Desktop\AutoIt Library\exemploUIAutomate\TesteUIA2.au3
+>20:24:31 AU3Check ended.rc:0
>Running:(3.3.14.5):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\Users\resource\Desktop\AutoIt Library\exemploUIAutomate\TesteUIA2.au3"    
+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
"C:\Users\resource\Desktop\AutoIt Library\exemploUIAutomate\TesteUIA2.au3" (134) : ==> Variable must be of type "Object".:
$oTextContainer.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oTextContainer^ ERROR
->20:24:37 AutoIt3.exe ended.rc:1
+>20:24:37 AutoIt3Wrapper Finished.
>Exit code: 1    Time: 6.717

Share this post


Link to post
Share on other sites

Check with inspect or simplespy if the pattern is supported on that object and make sure you have object found. I frequently highlight my objects during development to see if it finds the correct object or I position mousecursor to see if things are identified properly. Errors like variable is not an object is either

No element found

Pattern not supported

Share this post


Link to post
Share on other sites

You can do it this way:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "ApplicationFrameWindow", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "CalculatorResults", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pText1, $oText1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition1, $pText1 )
  $oText1 = ObjCreateInterface( $pText1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oText1 ) Then Return ConsoleWrite( "$oText1 ERR" & @CRLF )
  ConsoleWrite( "$oText1 OK" & @CRLF )

  ; --- Element Properties ---

  ConsoleWrite( "--- Element Properties ---" & @CRLF )

  Local $sName1
  $oText1.GetCurrentPropertyValue( $UIA_NamePropertyId, $sName1 )
  ConsoleWrite( "$sName1 = " & $sName1 & @CRLF )
EndFunc

Output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oWindow1 OK
--- Find window/control ---
$pCondition1 OK
$oText1 OK
--- Element Properties ---
$sName1 = Skærm er 3,1415926535897932384626433832795

Note that the text control is not an edit control, but a static text like a label in AutoIt. Therefore, use the name property to read the text value.


Share this post


Link to post
Share on other sites
On 8/8/2019 at 3:50 PM, Heinzl said:

Did anybody recently try to run the Firefox automation example? After doing some modifications, I have been able to access tree elements like tabs and click to them, so basic things work. But I can't access the document itself, I always get ""Action 6 failed". Any Ideas?

audacity temp mail origin

Thanks for any help.

Heinz

Attempt if simplespy features your record zone or potentially components. In all likelihood during upgrades  they change component request and accordingly your record distinguishing proof or component tree changed. Which ff rendition are you running.

Edited by Fleurpink

Share this post


Link to post
Share on other sites

I'm seeing a weird issue where if the application I'm automating freezes and goes into (Not Responding) then the UIA code seems to freeze up the AutoIt script.

If a screen doesn't open within the 120 second timeout, it calls a function that resets the program back to the main screen.

Part of that is to reset a tree view, that code below.  That seems to be the part that freezes up the script but as it's very random I can't seem to figure out a more specific line to look at.

Any ideas on which of these could get stuck in an infinite loop?

Local $oP4 = _UIA_getObjectByFindAll($UIA_oDesktop, "processid:=" & $iPID & ";controltype:=UIA_WindowControlTypeId;class:=Afx", $treescope_children)
_UIA_Action($oP4, "setfocus")
Local $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=Applications;controltype:=UIA_PaneControlTypeId;class:=Afx:ControlBar", $treescope_children)
_UIA_Action($oP3, "setfocus")
Local $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=Afx:TabWnd", $treescope_children)
_UIA_Action($oP2, "setfocus")
Local $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=Applications;controltype:=UIA_PaneControlTypeId;class:=Afx:ControlBar", $treescope_children)
_UIA_Action($oP1, "setfocus")
Local $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_TreeControlTypeId;class:=SysTreeView32", $treescope_children)
_UIA_Action($oP0, "setfocus")

Dim $pTrueCondition, $pElements, $iLength                                                       ;Create the variables that will be populated by the functions below

$UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
Local $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
$oP0.FindAll($treescope_subtree, $oCondition, $pElements)                                       ;Find all objects under the $oP5 and populate the variables with them

Local $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) ;Create a UIA Array Element

$oAutomationElementArray.Length($iLength)                                                       ;Get the count of items in the UIA Array
For $U = $iLength - 1 To 0 Step -1                                                              ; it's zero based   Loop through the Array in reverse looking for each item in the Tree view, End at first as we want focus set to the top when done
    $oAutomationElementArray.GetElement($U, $UIA_pUIElement)                                    ;Get the element of Array Item $i
    Local $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $sItemText = _UIA_action($oUIElement, "property", $UIA_NamePropertyId)                      ;This Can be used to identify the item to select by Visible Text for opening/closing specific items

    $iState = _UIA_action($oUIElement, "property", $UIA_ExpandCollapseExpandCollapseStatePropertyId) ;Identify if this is expanded
    If $iState = 1 Then
        _UIA_Action($oUIElement, "setfocus")                                                    ;Select the item
        ControlSend($hFacets, "Applications", "SysTreeView323", "{Left}")                       ;Now that the item is selected, send the Left key to the containing Control to collapse it.
    EndIf

    If $sItemText = "Accounting                    " Then                                       ;Since we moved through the list in reverse, the item 0 should be the first item in the list.
        _UIA_Action($oUIElement, "setfocus")                                                    ;Select the item
    EndIf

Next

 

Update:  Got a bit more info from COM error trapping

We intercepted a COM Error at line: 1079!
    Number: 0x00000003
    Description: NULL Pointer assignment
 

In my UIAWrapper.au3 file it is for the Func _UIA_getObjectByFindAll on this line

$oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)

 

 

Thanks

Edited by BigDaddyO

hmm... I guess I have to have a signature...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...