Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Share this post


Link to post
Share on other sites

Thanks for all the hard work junkew!  I've tried a couple of times to wrap my brain around how this works.  But it is really confusing.  I'm sure once I understand it a bit more it will click and I'll be able to figure it out. LOL!

Basically what I am testing with now is Malwarebytes 3.0.6.  When it's first opened up a box pops up with a "Learn More" button and a "Close" button.

 

I tried SimpleSpy and it pulled some info from the window... but I don't see any info about the "Close" button which is what I had highlighted when I clicked CTRL + W.  I'll post the info it grabbed below.

When I used Inspect.exe I was able to see the "Close" info.  I'll post that info below as well.  Utalizing the info from SimpleSpy and Inspect... what autoit code would I utalize to left click on the "Close" button?

 

SimpleSpy:

Mouse position is retrieved 738-585
At least we have an element [Dashboard Page][]
Having the following values for all properties: 
Title is: <Dashboard Page>    Class   := <>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    
*** Parent Information ***
Title is: <Malwarebytes Premium Trial 3.0.6 Malwarebytes Premium Trial 3.0.6>    Class   := <Qt5QWindowIcon>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=
UIA_BoundingRectanglePropertyId :=474;218;586;431
UIA_ClassNamePropertyId :=
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50033
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=SetFocus
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Dashboard Page
UIA_LegacyIAccessibleRolePropertyId :=16
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=1048576
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=pane
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Dashboard Page
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=6764
UIA_ProviderDescriptionPropertyId :=[pid:6764,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

Share this post


Link to post
Share on other sites

Inspect:

How found:    Selected from tree...
Name:    "Close"
ControlType:    UIA_ButtonControlTypeId (0xC350)
LocalizedControlType:    "button"
BoundingRectangle:    {l:887 t:708 r:997 b:746}
IsEnabled:    true
IsOffscreen:    false
IsKeyboardFocusable:    true
HasKeyboardFocus:    false
AccessKey:    ""
ProcessId:    6764
ProviderDescription:    "[pid:6764,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]"
IsPassword:    false
HelpText:    ""
LegacyIAccessible.ChildId:    0
LegacyIAccessible.DefaultAction:    "Press"
LegacyIAccessible.Description:    ""
LegacyIAccessible.Help:    ""
LegacyIAccessible.KeyboardShortcut:    ""
LegacyIAccessible.Name:    "Close"
LegacyIAccessible.Role:    push button (0x2B)
LegacyIAccessible.State:    focusable (0x100000)
LegacyIAccessible.Value:    ""
IsAnnotationPatternAvailable:    false
IsDragPatternAvailable:    false
IsDockPatternAvailable:    false
IsDropTargetPatternAvailable:    false
IsExpandCollapsePatternAvailable:    false
IsGridItemPatternAvailable:    false
IsGridPatternAvailable:    false
IsInvokePatternAvailable:    true
IsItemContainerPatternAvailable:    false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable:    false
IsObjectModelPatternAvailable:    false
IsRangeValuePatternAvailable:    false
IsScrollItemPatternAvailable:    false
IsScrollPatternAvailable:    false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsSpreadsheetItemPatternAvailable:    false
IsSpreadsheetPatternAvailable:    false
IsStylesPatternAvailable:    false
IsSynchronizedInputPatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTablePatternAvailable:    false
IsTextChildPatternAvailable:    false
IsTextEditPatternAvailable:    false
IsTextPatternAvailable:    false
IsTextPattern2Available:    false
IsTogglePatternAvailable:    false
IsTransformPatternAvailable:    false
IsTransform2PatternAvailable:    false
IsValuePatternAvailable:    false
IsVirtualizedItemPatternAvailable:    false
IsWindowPatternAvailable:    false
IsCustomNavigationPatternAvailable:    false
FirstChild:    [null]
LastChild:    [null]
Next:    "Malwarebytes" text
Previous:    "Learn More" button
Other Props:    Object has no additional properties
Children:    Container has no children
Ancestors:    "Malwarebytes Premium Trial 3.0.6 Malwarebytes Premium Trial 3.0.6" window
    "Desktop" pane
    [ No Parent ]

Share this post


Link to post
Share on other sites

looks like you have not used latest simplespy.

It will give you a source code example and also important your whole tree hierarchy of all (grand)parents

The basic concept is

  1. Setfocus to desktop, mainwindow
  2. Optional setfocus/activate then depending on tree (sub)elements, 
  3. Now your element should be visible and only when visible you can click/gettext/settext/sendkeys etc. actions whereas the starter is action highlight so you can at least see you reached your element

Frequently its confusing hierarchy that messes up finding the element you want to act on or elements with similar id's to differenticate from.

I also see you spy on QtWidgets and then developers should use QT Accessibility which is mainly iaccessible and iaccessible2. The examples for that are also in the thread made by LarsJ on how to get to the iaccessible/iaccessible2 interfaces. But as you try an QT App not developed by you I am not sure how accessible it will be.

maybe try ranorex spy: http://www.ranorex.com/test-automation-tools/ranorex-spy.html

If ranorex spy reveals the information you can do it also with the iuiautomation stuff but most likely then the iaccessible examples you should study

Edited by junkew

Share this post


Link to post
Share on other sites

See post 142 and that source should work I tried it just a moment ago with Malware. QTWidgets are recognizable with the iaccessible/iaccessible2 interface.

Checked with Ranorex spy which nicely recognizes directly. I will enhance simplespy in future with iaccessible.

Tool testcomplete is not recognizing the objects out of the box

 

Share this post


Link to post
Share on other sites

Hi all.

In the post #139 LarsJ said: With IUIAutomationLegacyIAccessiblePattern.GetIAccessible you can get a pointer for IAccessible. I have tested this. It works.

I tried that

#include "CUIAutomation2.au3"

$hHandle = ControlGetHandle("Program Manager", "", "SysListView321") ; desktop Win7

Global $pElement, $oElement, $pIAccess, $pIAccess, $oIAccess, $oAcc, $iCount

$oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
$oUIAutomation.ElementFromHandle($hHandle, $pElement)
$oElement = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

$oElement.GetCurrentPattern($UIA_LegacyIAccessiblePatternId, $pIAccess)
$oIAccess = ObjCreateInterface($pIAccess, $sIID_IUIAutomationLegacyIAccessiblePattern, $dtagIUIAutomationLegacyIAccessiblePattern)

$oIAccess.GetIAccessible($oAcc)
ConsoleWrite(VarGetType($oAcc) & ":" & IsObj($oAcc) & @CRLF)

$oAcc.get_accChildCount($iCount) ; The requested action with this object has failed

$oAcc looks like object (VarGetType), but not is object (IsObj). And I get an error.

Then I changed description in CUIAutomation2.au3 to "GetIAccessible hresult(ptr*);" and tried that

$pAcc = 0
$oIAccess.GetIAccessible($pAcc)
$oAcc = ObjCreateInterface($pAcc, $sIID_IAccessible, $dtagIAccessible)
$oAcc.get_accChildCount($iCount) ; Variable must be of type "Object"

And get an error again.

How can I get access to IAccessible methods from UIAutomation element?

Thanks.

Share this post


Link to post
Share on other sites

The easiest way to create an IAccessible object is to use the IAccessible functions:

#include "CUIAutomation2.au3"
#include "MSAccessibility.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  Local $hHandle = ControlGetHandle("Program Manager", "", "SysListView321")
  If Not $hHandle Then Return ConsoleWrite( "$hHandle ERR" & @CRLF )
  ConsoleWrite( "$hHandle OK" & @CRLF )

  Local $pAccess
  AccessibleObjectFromWindow( $hHandle, $OBJID_CLIENT, $tIID_IAccessible, $pAccess )
  If Not $pAccess Then Return ConsoleWrite( "$pAccess ERR" & @CRLF )
  ConsoleWrite( "$pAccess OK" & @CRLF )

  Local $oAccess
  $oAccess = ObjCreateInterface( $pAccess, $sIID_IAccessible, $dtagIAccessible )
  If Not IsObj( $oAccess ) Then Return ConsoleWrite( "$oAccess ERR" & @CRLF )
  ConsoleWrite( "$oAccess OK" & @CRLF )

  Local $iChildCount
  $oAccess.get_accChildCount( $iChildCount )
  If Not $iChildCount Then Return ConsoleWrite( "$iChildCount ERR" & @CRLF )
  ConsoleWrite( "$iChildCount = " & $iChildCount & @CRLF )
EndFunc

Note that it's usually much easier to use UIAutomation objects than IAccessible objects.

In post 139 I was testing with Chrome which supports the IAccessible interface very well.

MSAccessibility.7z

Edited by LarsJ

Share this post


Link to post
Share on other sites

@LarsJ

I know about functions from MSAccessibility UDF. And usually I use LegacyIAccessiblePattern methods because it really is easier. But I can't understand how to use GetIAccessible method for create IAccessible object. Can you explain me?

Thanks.

Share this post


Link to post
Share on other sites

You do it this way:

; LegacyIAccessible object
Local $pLegacyIAccessible, $oLegacyIAccessible
$oUIAutomationElement.GetCurrentPattern( $UIA_LegacyIAccessiblePatternId, $pLegacyIAccessible )
$oLegacyIAccessible = ObjCreateInterface( $pLegacyIAccessible, $sIID_IUIAutomationLegacyIAccessiblePattern, $dtagIUIAutomationLegacyIAccessiblePattern )
If Not IsObj( $oLegacyIAccessible ) Then Return ConsoleWrite( "$oLegacyIAccessible ERR" & @CRLF )
ConsoleWrite( "$oLegacyIAccessible OK" & @CRLF )

; IAccessible object
Local $pIAccessible, $oIAccessible
$oLegacyIAccessible.GetIAccessible( $pIAccessible )
$oIAccessible = ObjCreateInterface( $pIAccessible, $sIID_IAccessible, $dtagIAccessible )
If Not IsObj( $oIAccessible ) Then Return ConsoleWrite( "$oIAccessible ERR" & @CRLF )
ConsoleWrite( "$oIAccessible OK" & @CRLF )

Here is an example with Chrome:

Open Chrome
Enter this in address bar: chrome://accessibility
Click off in this line: Global accessibility mode: off
off should switch to on: Global accessibility mode: on

Run this code:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Chrome window handle
  Local $hChrome = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]", "" )  ; Chrome
  If Not $hChrome Then Return ConsoleWrite( "$hChrome ERR" & @CRLF )
  ConsoleWrite( "$hChrome OK" & @CRLF )

  ; UI Automation object
  Local $oUIAutomation
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; UI Automation element
  Local $pChrome, $oChrome
  $oUIAutomation.ElementFromHandle( $hChrome, $pChrome )
  $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
  ConsoleWrite( "$oChrome OK" & @CRLF )

  ; LegacyIAccessible object
  Local $pLegacyIAccessible, $oLegacyIAccessible
  $oChrome.GetCurrentPattern( $UIA_LegacyIAccessiblePatternId, $pLegacyIAccessible )
  $oLegacyIAccessible = ObjCreateInterface( $pLegacyIAccessible, $sIID_IUIAutomationLegacyIAccessiblePattern, $dtagIUIAutomationLegacyIAccessiblePattern )
  If Not IsObj( $oLegacyIAccessible ) Then Return ConsoleWrite( "$oLegacyIAccessible ERR" & @CRLF )
  ConsoleWrite( "$oLegacyIAccessible OK" & @CRLF )

  ; IAccessible object
  Local $pIAccessible, $oIAccessible
  $oLegacyIAccessible.GetIAccessible( $pIAccessible )
  $oIAccessible = ObjCreateInterface( $pIAccessible, $sIID_IAccessible, $dtagIAccessible )
  If Not IsObj( $oIAccessible ) Then Return ConsoleWrite( "$oIAccessible ERR" & @CRLF )
  ConsoleWrite( "$oIAccessible OK" & @CRLF )
EndFunc

Output in SciTE:

$hChrome OK
$oUIAutomation OK
$oChrome OK
$oLegacyIAccessible OK
$oIAccessible OK

The code works for Chrome. But there is no guarantee that it works for other windows/controls. If it doesn't work, there is probably not much to do.

Share this post


Link to post
Share on other sites
4 hours ago, LarsJ said:

The code works for Chrome. But there is no guarantee that it works for other windows/controls.

Yes, it works for Chrome. And for Firefox. Strange, but I couldn't find any Microsoft applications that work with this code. Thank you for explanation.

Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

Hello please can you kindly help me with one UI script as I am struggling . I have one need urgently for a project . Please help me kindly . Sequence is like this as mentioned below .

1) you need to select the Pricing Tab from upper bar left
2) Then Select a option from the page ( upper left hand side )
3) Then Click the Search button from right button
4) Choose the correct Pricing ( Various Currency option 13 USD , EUR,YEN,etc..)option which is with hyperlink ( Left hand side )
5) Click hyperlink from the left hand pane
6) Then type the Product name and click search
7) Once found simply choose
8) Click " Add Selected Item " option box from middle upper bar .

That all . I need script help for that can be run with 64 bit windows machine

Share this post


Link to post
Share on other sites

you should ask that in general help and support and provide more details. Advice is to first read frequently asked questions question number 31.

Without information from Au3Inf or simplespy people will not be able to guide you to a solution.

Share this post


Link to post
Share on other sites

Hi, first of all, great framework! :thumbsup:

I am in a scenario where I have multiple buttons with the "same properties" (atleast the same "title" property). They all look the same... but I have to click on a specific button. The simplespy.au3 script's generated code clicks on the first button... and I have to click on the 3rd button (let's assume). How do I accomplish this? :)

I can provide more information if wanted, thanks in advance!


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

property indexrelative                   just goes from a found element n positions further in the elementarray (usefull for textbox without a name after a known label)

property index or instance           just finds the n-th element with that specific match (similar behavior as AutoIt has)

Share this post


Link to post
Share on other sites

Thank you very much! :D. I just appended ;indexrelative:=3 to the _UIA_getObjectByFindAll's $str parameter on $oUIElement's declaration, it worked like a charm! Now it clicks on the 3rd button instead of the first :). simplespy.au3 should contain functionality to add indexrelative to the $str parameter


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

Hello, my English is not very good, can only use google to translate, my question is to use your code to get the object.


Can you control the object in this way?

$oExcel = ObjCreate("Excel.Application")

$oExcel.Visible = 1 
$oExcel.WorkBooks.Add

Local $aArray[16][16] 
For $i = 0 To 15 
    For $j = 0 To 15 
        $aArray[$i][$j] = $i 
    Next ;
Next

 


It's not good to translate., please forgive me

Edited by eqjtlm

Share this post


Link to post
Share on other sites

Not exactly but UIA uses objects so you can use different methods associated with them to manipulate UI elements. This might be a good start if you are experienced: https://msdn.microsoft.com/en-us/library/ms747327(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/ee684009(v=vs.85).aspx

Edited by TheDcoder

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

eqjtlm, If you want to automate Excel spreadsheets then look at the functions in the Excel.au3 UDF. See under "Excel Management" in the AutoIt help file.


TheDcoder, This is not the correct link to documentation. As you can see in the Note box in top of the page the documentation is about UI Automation implemented in the .NET framework. But this is not what this thread is about.

This thread is about UI Automation implemented in the Windows API. And you can find the documentation here.

Maybe the chit-chatting documentation is more or less the same. But the Reference documentation (the important documentation) is definitely not the same.

Edited by LarsJ

Share this post


Link to post
Share on other sites
31 minutes ago, LarsJ said:

eqjtlm,如果要自动化Excel电子表格,请查看Excel.au3 UDF中的函数。请参阅AutoIt帮助文件中的“Excel管理”。


TheDcoder,这不是文档的正确链接。正如您可以在页面顶部的“注释”框中看到的,文档是关于UI Automation实现的。.NET框架但这不是这个线程。

这个线程是关于在Windows API中实现的UI自动化您可以在这里找到文档

也许聊天聊天文档或多或少是相同的。但参考文件(重要文件)绝对不一样。

Hello, my question is an example, I want to automate the two controls of "vsflex" and "ttf16", and I want to manipulate him like my example

 

 

We have the time here is 23:30, and I want to sleep, see you tomorrow

Edited by eqjtlm

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...