maniootek

Get e-mail address from selected e-mail message in Thunderbird

26 posts in this topic

#1 ·  Posted (edited)

Autoit Window Info gives no information about Thunderbird window exept title and class which is not enough for me. I would like to read some controls from it to get more info like e-mail address from selected message. I have found some script which extract all information from the handle of specified window to txt file, which is based on UIAutomation UDF from this thread:

https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/

this is my part of code:

#include "CUIAutomation2.au3"

Opt("MustDeclareVars", 1)
Opt("WinTitleMatchMode", 2)

Global $oUIAutomation
Global $file = "result.txt"

MainFunc()

Func MainFunc()

    ;Local $hWindow = 0x0006045C                                                             ; Infragistics
    ;Local $hWindow = WinGetHandle( "[CLASS:WindowsForms10.Window.8.app.0.bb8560_r19_ad1]" ) ; Infragistics
    ;Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )                                 ; Windows Explorer, Windows 7
    ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )                                ; Windows Explorer, Windows XP
    ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" )                          ; Windows Explorer right pane
    ;Local $hWindow = WinGetHandle( "[TITLE:My GUI Checkbox; CLASS:AutoIt v3 GUI]" )         ; AutoIt GUI window
    ;Local $hWindow = WinGetHandle( "[CLASS:AutoIt v3 GUI]" )                                ; AutoIt script
    ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                                      ; Internet Explorer
    
    Local $hWindow = WinGetHandle("Thunderbird")
    If Not $hWindow Then Return

    $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return

    Local $pWindow
    ;$oUIAutomation.GetRootElement( $pWindow )             ; Desktop
    $oUIAutomation.ElementFromHandle($hWindow, $pWindow) ; Window
    If Not $pWindow Then Return

    Local $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oWindow) Then Return

    ;ListDescendants( $oWindow, 0, 1 ) ; Desktop
    ListDescendants($oWindow, 0, 0) ; Window

EndFunc   ;==>MainFunc


Func ListDescendants($oParent, $iLevel, $iLevels = 0)

    If Not IsObj($oParent) Then Return
    If $iLevels And $iLevel = $iLevels Then Return

    Local $pRawWalker, $oRawWalker
    $oUIAutomation.RawViewWalker($pRawWalker)
    $oRawWalker = ObjCreateInterface($pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)

    Local $pUIElement, $oUIElement
    $oRawWalker.GetFirstChildElement($oParent, $pUIElement)
    $oUIElement = ObjCreateInterface($pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $sIndent = ""
    For $i = 0 To $iLevel - 1
        $sIndent &= "    "
    Next

    While IsObj($oUIElement)

        FileWrite($file, $sIndent & "Title     = " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @CRLF & _
                $sIndent & "Class     = " & _UIA_getPropertyValue($oUIElement, $UIA_ClassNamePropertyId) & @CRLF & _
                $sIndent & "Ctrl type = " & _UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId) & @CRLF & _
                $sIndent & "Ctrl name = " & _UIA_getPropertyValue($oUIElement, $UIA_LocalizedControlTypePropertyId) & @CRLF & _
                $sIndent & "Value     = " & _UIA_getPropertyValue($oUIElement, $UIA_LegacyIAccessibleValuePropertyId) & @CRLF & _
                $sIndent & "Handle    = " & Hex(_UIA_getPropertyValue($oUIElement, $UIA_NativeWindowHandlePropertyId)) & @CRLF & @CRLF)

        ListDescendants($oUIElement, $iLevel + 1, $iLevels)

        $oRawWalker.GetNextSiblingElement($oUIElement, $pUIElement)
        $oUIElement = ObjCreateInterface($pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    WEnd

EndFunc   ;==>ListDescendants


Func _UIA_getPropertyValue($obj, $id)
    Local $tVal
    $obj.GetCurrentPropertyValue($id, $tVal)
    If Not IsArray($tVal) Then Return $tVal
    Local $tStr = $tVal[0]
    For $i = 1 To UBound($tVal) - 1
        $tStr &= "; " & $tVal[$i]
    Next
    Return $tStr
EndFunc   ;==>_UIA_getPropertyValue

in the result.txt file I have found a lot of information extracted from thunderbird window like this:

Quote

                Title     = Od: Malgorzata Nowak <malgorzatanowak@domain.com>
                Class     =
                Ctrl type = 50025
                Ctrl name =
                Value     =
                Handle    = 00000000

which states "e-mail address from field" from selected message

Now, how can I get this specified type of information directly? Is there some function to read specified ctrl type? I just want my script working faster without proces this huge part of returned information.

Edit:

I just realised that there is already function to read these values "_UIA_getPropertyValue", but it still too slow to get it. Full window read takes about 4.5sec, when i modify the script to look for ctrl type "50025" and title which contains "od: " then it takes 1.5sec which is still not enought for me.

Edited by maniootek

Share this post


Link to post
Share on other sites



Save email to EML file

And you can parse it or use COM object to access EML properties

here is some related stuff

 

but it need to do some work with this.

 

 

 

 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

this is more about read from the window rather than read from the file

Share this post


Link to post
Share on other sites

The code in post 1 should only be used to print the entire tree structure. It should not be used to search for a specific control.

To search for a specific control for example the Inbox you should use the "Simple spy demo" from the UI Automation framework or Inspect.exe from Windows SDK to identify the control. When you have identified the control you can set up a property condition to search directly for this control. The Inbox is probably some kind of a list control. Then you can get information for all or selected items in the list.

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Thank you for your message.

I have tried Inspect.exe and simplespy.au3 and this is result

Inspect.exe:

 

screenshot3.thumb.jpg.76df1d244e7452cd1f

simplespy.au3:

Mouse position is retrieved 262-584
At least we have an element [Od: Dysk Google <drive-noreply@google.com>][]

Having the following values for all properties: 
Title is: <Od: Dysk Google <drive-noreply@google.com>>  Class   := <>   controltype:= <UIA_EditControlTypeId>   ,<50004>    , (0000C354)    207;566;250;20
*** Parent Information top down ***
6: Title is: <Pulpit>   Class   := <#32769> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    0;0;1920;1200
"Title:=Pulpit;controltype:=UIA_PaneControlTypeId;class:=#32769"    
5: Title is: <Odebrane - email@gmail.com - Mozilla Thunderbird> Class   := <MozillaWindowClass> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    -8;-8;1936;1176
"Title:=Odebrane - email@gmail.com - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass"    
4: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    0;54;1562;1084
"Title:=;controltype:=UIA_CustomControlTypeId;class:="  
3: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    0;54;1562;1084
"Title:=;controltype:=UIA_CustomControlTypeId;class:="  
2: Title is: <Od>   Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    207;565;250;23
"Title:=Od;controltype:=UIA_CustomControlTypeId;class:="    
1: Title is: <> Class   := <>   controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    207;565;250;22
"Title:=;controltype:=UIA_TextControlTypeId;class:="    
0: Title is: <Od: Dysk Google <drive-noreply@google.com>>   Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    207;566;250;20
"Title:=Od: Dysk Google <drive-noreply@google.com>;controltype:=UIA_CustomControlTypeId;class:="    


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP5=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Odebrane - email@gmail.com - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass", $treescope_children)    
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Od;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) 
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_TextControlTypeId;class:=", $treescope_children) 
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Od: Dysk Google <drive-noreply@google.com>;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) 
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Od:DyskGoogle<drive-.mainwindow", "title:=Od: Dysk Google <drive-noreply@google.com>;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Od: Dysk Google <drive-noreply@google.com>;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_text:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_regexptitle:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_iaccessiblevalue:= <Dysk Google <drive-noreply@google.com> Dodaj do książki adresowej>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <207;566;250;20>
UIA_ProcessId:= <9188>
UIA_ControlType:= <50004>
UIA_LocalizedControlType:= <edytuj>
UIA_Name:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <True>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueValue:= <Dysk Google <drive-noreply@google.com> Dodaj do książki adresowej>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_LegacyIAccessibleValue:= <Dysk Google <drive-noreply@google.com> Dodaj do książki adresowej>
UIA_LegacyIAccessibleRole:= <42>
UIA_LegacyIAccessibleState:= <1074790464>
UIA_LegacyIAccessibleDefaultAction:= <click>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:9188,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

now, how can I access and get title from that control?

Edited by maniootek

Share this post


Link to post
Share on other sites

If you look at the output from "Simple spy demo" the chain of parent controls contains only one UIA_TextControlTypeId. Try to set up a property condition (with CreatePropertyCondition method) to search for this text control.

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

This code seems to work:

#include "UIAWrappers.au3"

Local $oP5=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:= - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass", $treescope_children)
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Od;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_TextControlTypeId;class:=", $treescope_children)
_UIA_action($oP1, "click")

but how to get title of $oP1?

tried this but does not work

MsgBox(0,0,_UIA_getPropertyValue($oP1, $UIA_NamePropertyId))

Also is there other way to process this code faster? I have changed UIA.cfg file to do not mark on red but it still do it. Any chance to get this title value even when Thunderbird is not in focus (from the background) ?

Edited by maniootek

Share this post


Link to post
Share on other sites

If you've looked at UIAWrappers.au3 you've seen that the functions in this UDF executes a lot of code. To execute a smaller amount of code and make it faster search for CreatePropertyCondition in some of my examples. The Thunderbird window does not need to have focus.

I'll give you a little time to do some reading and testing. I'll take a new look at this thread next sunday.

Share this post


Link to post
Share on other sites

Speed is a relative problem

A. Faster cpu

B. Switch to compiled language

C. Study all examples.zip to not use uiawrappers but loose flexibility

D. Wait till I make a firebird example but that takes longer

In general first make an object highlight and after that make another action. Your msgbox example normally should work.

1 person likes this

Share this post


Link to post
Share on other sites

Not on my pc so hard to make example but try this

1. Get top level window as treescope child from desktop

2. Get element as treescope subtree from top level window based on title property

3. Highlight element

Speed can be faster by finding with 1 property using function _UIA_getFirstObjectOfElement

 

 

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

I have this part of code so far:

#include "CUIAutomation2.au3"

Opt("WinTitleMatchMode", 2)

Local $hWindow = WinGetHandle(" - Mozilla Thunderbird")
$oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
Local $pWindow
$oUIAutomation.ElementFromHandle($hWindow, $pWindow) ; Window
Local $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

what metod for $oWindow object should I use to find element with controltype "UIA_TextControlTypeId" ?

 

Edited by maniootek

Share this post


Link to post
Share on other sites

Sorry if i misunderstood your intent here. I belive you need to get the text of a specific part of the Thunderbird, and that is retrived by the "title" right?

Try something like this:

Notice that you will need to get the $oUIElement first.

_UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId)

Look if this retrieve the data you want.

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

 

20 minutes ago, MichaelHB said:

Notice that you will need to get the $oUIElement first.

the question now is how to get this element first, i know that title contains text started with "Od: " and controltype is 50025 (which I think means "UIA_CustomControlTypeId") and that parent element controltype is "UIA_TextControlTypeId" (which I think means 50020)

I have noticed that this element (controltype 50020) only occur 2 times before my target. First 50020 element has title "Od" and the second one has empty title. The second one is parent of my target. My target is first child of this parent.

 

 

Przechwytywanie.PNG

Edited by maniootek

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

Use simplespy to get the element near/parent to the data you want to extract and then use something like this. This is the same function that is in the simplespy.

#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"


MsgBox(0, "", Retrieve())

Func Retrieve()
    Local $oParentHandle[1]

    Local $Thunderbird = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Odebrane - email@gmail.com - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass", $treescope_children)    
    _UIA_Action($oP5,"setfocus")
    Local $oUIElement = _UIA_getObjectByFindAll($Thunderbird, "*** TITLE AND TYPE ***", $treescope_subtree) ; <= you need to fill this part

    $UIA_oUIAutomation.RawViewWalker($UIA_pTW)
    Local $oTW = ObjCreateInterface($UIA_pTW, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)
    If IsObj($oTW) = 0 Then
        MsgBox(0, "UI automation treewalker failed", "UI Automation failed failed")
    EndIf

    $oTW.GetParentElement($oUIElement, $oParentHandle[0])
    $oParentHandle[0] = ObjCreateInterface($oParentHandle[0], $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If IsObj($oParentHandle[0]) = 0 Then
        MsgBox(0, "No parent", "UI Automation failed could be you spy the desktop")
    Else
        Return _UIA_getPropertyValue($oParentHandle[0], $UIA_NamePropertyId)
    EndIf

EndFunc

Exit

 

Edited by MichaelHB
Changed the code.

Share this post


Link to post
Share on other sites

My goal is to save the time when working on UIA that's why I don't want to use UIAWrappers (as @junkew advice). I don't want to use

_UIA_getObjectByFindAll

or 

_UIA_Action

 

Share this post


Link to post
Share on other sites

You can do some tests with this code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )
Opt( "WinTitleMatchMode", 2 )

Example()

Func Example()
  ; Mozilla Thunderbird window
  Local $hWindow = WinGetHandle( " - Mozilla Thunderbird" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  ; Note that $oWindow is an AutomationElement object
  ; Search for $dtagIUIAutomationElement in CUIAutomation2.au3 to see which methods can be used
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  #cs
  ; Condition to find text controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )
  #ce

  ; Condition to find custom controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CustomControlTypeId, $pCondition )
  ; You can use any of the ControlTypeIds in CUIAutomation2.au3 (search for ControlTypeId)
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find custom controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  ; You can use any of the TreeScope constants in CUIAutomation2.au3 (search for TreeScope)
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find controls ERR" & @CRLF )
  ConsoleWrite( "Find controls OK" & @CRLF )

  ; Print custom control info
  Local $pUIElement, $oUIElement
  For $i = 0 To $iElements - 1
    ConsoleWrite( @CRLF )
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    ConsoleWrite( "Index     = " & $i & @CRLF & _
                  "Name      = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  "Rectangle = " & GetCurrentPropertyValue( $oUIElement, $UIA_BoundingRectanglePropertyId ) & @CRLF & _
                  "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF )
                  ; You can add any of the PropertyIdIds in CUIAutomation2.au3 (search for PropertyId)
  Next

EndFunc

Func GetCurrentPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

 

1 person likes this

Share this post


Link to post
Share on other sites

Thank you so much for your example, it is very clear. Anyway, I still have some questions:

1. Where can I find list of methods for $oWindow (AutomationElement object), so far i know .FindAll and .FindFirst

2. What would be fastest way to find element which $UIA_NamePropertyId contains "Reply to" value and which is $UIA_CustomControlTypeId (50025)

3. My target element is a child of second occurrence of $UIA_TextControlTypeId (50020) - would be that helpful to speed up searching my target element?

This code is processed in Time: 0.7706 which is much faster than my previous solution but I think it could be still improved:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )
Opt( "WinTitleMatchMode", 2 )

Example()

Func Example()
  ; Mozilla Thunderbird window
  Local $hWindow = WinGetHandle( " - Mozilla Thunderbird" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  ; Note that $oWindow is an AutomationElement object
  ; Search for $dtagIUIAutomationElement in CUIAutomation2.au3 to see which methods can be used
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find custom controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CustomControlTypeId, $pCondition );$UIA_CustomControlTypeId
  ; You can use any of the ControlTypeIds in CUIAutomation2.au3 (search for ControlTypeId)
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find custom controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  ; You can use any of the TreeScope constants in CUIAutomation2.au3 (search for TreeScope)
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find controls ERR" & @CRLF )
  ConsoleWrite( "Find controls OK" & @CRLF )

  ; Print custom control info
  Local $pUIElement, $oUIElement, $name
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $name = GetCurrentPropertyValue($oUIElement,$UIA_NamePropertyId)
    if StringInStr($name, "Reply to:") Then
        ConsoleWrite($name & @CRLF)
        Exit
    EndIf
  Next

EndFunc

Func GetCurrentPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

 

Share this post


Link to post
Share on other sites

From CUIAutomation2.au3. This is all the methods and properties:

Global Const $sIID_IUIAutomationElement="{D22108AA-8AC5-49A5-837B-37BBB3D7591E}"
Global $dtagIUIAutomationElement = "SetFocus hresult();" & _
"GetRuntimeId hresult(ptr*);" & _
"FindFirst hresult(long;ptr;ptr*);" & _
"FindAll hresult(long;ptr;ptr*);" & _
"FindFirstBuildCache hresult(long;ptr;ptr;ptr*);" & _
"FindAllBuildCache hresult(long;ptr;ptr;ptr*);" & _
"BuildUpdatedCache hresult(ptr;ptr*);" & _
"GetCurrentPropertyValue hresult(int;variant*);" & _
"GetCurrentPropertyValueEx hresult(int;long;variant*);" & _
"GetCachedPropertyValue hresult(int;variant*);" & _
"GetCachedPropertyValueEx hresult(int;long;variant*);" & _
"GetCurrentPatternAs hresult(int;none;none*);" & _
"GetCachedPatternAs hresult(int;none;none*);" & _
"GetCurrentPattern hresult(int;ptr*);" & _
"GetCachedPattern hresult(int;ptr*);" & _
"GetCachedParent hresult(ptr*);" & _
"GetCachedChildren hresult(ptr*);" & _
"CurrentProcessId hresult(int*);" & _
"CurrentControlType hresult(int*);" & _
"CurrentLocalizedControlType hresult(bstr*);" & _
"CurrentName hresult(bstr*);" & _
"CurrentAcceleratorKey hresult(bstr*);" & _
"CurrentAccessKey hresult(bstr*);" & _
"CurrentHasKeyboardFocus hresult(long*);" & _
"CurrentIsKeyboardFocusable hresult(long*);" & _
"CurrentIsEnabled hresult(long*);" & _
"CurrentAutomationId hresult(bstr*);" & _
"CurrentClassName hresult(bstr*);" & _
"CurrentHelpText hresult(bstr*);" & _
"CurrentCulture hresult(int*);" & _
"CurrentIsControlElement hresult(long*);" & _
"CurrentIsContentElement hresult(long*);" & _
"CurrentIsPassword hresult(long*);" & _
"CurrentNativeWindowHandle hresult(hwnd*);" & _
"CurrentItemType hresult(bstr*);" & _
"CurrentIsOffscreen hresult(long*);" & _
"CurrentOrientation hresult(long*);" & _
"CurrentFrameworkId hresult(bstr*);" & _
"CurrentIsRequiredForForm hresult(long*);" & _
"CurrentItemStatus hresult(bstr*);" & _
"CurrentBoundingRectangle hresult(struct*);" & _
"CurrentLabeledBy hresult(ptr*);" & _
"CurrentAriaRole hresult(bstr*);" & _
"CurrentAriaProperties hresult(bstr*);" & _
"CurrentIsDataValidForForm hresult(long*);" & _
"CurrentControllerFor hresult(ptr*);" & _
"CurrentDescribedBy hresult(ptr*);" & _
"CurrentFlowsTo hresult(ptr*);" & _
"CurrentProviderDescription hresult(bstr*);" & _
"CachedProcessId hresult(int*);" & _
"CachedControlType hresult(int*);" & _
"CachedLocalizedControlType hresult(bstr*);" & _
"CachedName hresult(bstr*);" & _
"CachedAcceleratorKey hresult(bstr*);" & _
"CachedAccessKey hresult(bstr*);" & _
"CachedHasKeyboardFocus hresult(long*);" & _
"CachedIsKeyboardFocusable hresult(long*);" & _
"CachedIsEnabled hresult(long*);" & _
"CachedAutomationId hresult(bstr*);" & _
"CachedClassName hresult(bstr*);" & _
"CachedHelpText hresult(bstr*);" & _
"CachedCulture hresult(int*);" & _
"CachedIsControlElement hresult(long*);" & _
"CachedIsContentElement hresult(long*);" & _
"CachedIsPassword hresult(long*);" & _
"CachedNativeWindowHandle hresult(hwnd*);" & _
"CachedItemType hresult(bstr*);" & _
"CachedIsOffscreen hresult(long*);" & _
"CachedOrientation hresult(long*);" & _
"CachedFrameworkId hresult(bstr*);" & _
"CachedIsRequiredForForm hresult(long*);" & _
"CachedItemStatus hresult(bstr*);" & _
"CachedBoundingRectangle hresult(struct*);" & _
"CachedLabeledBy hresult(ptr*);" & _
"CachedAriaRole hresult(bstr*);" & _
"CachedAriaProperties hresult(bstr*);" & _
"CachedIsDataValidForForm hresult(long*);" & _
"CachedControllerFor hresult(ptr*);" & _
"CachedDescribedBy hresult(ptr*);" & _
"CachedFlowsTo hresult(ptr*);" & _
"CachedProviderDescription hresult(bstr*);" & _
"GetClickablePoint hresult(struct*;long*);"

And the MicroSoft documentation is here.

With $UIA_NamePropertyId you can only search for the entire name.

If you can find an object or element with a single FindAll or FindFirst command this is normally the fastest.

Did you notice that I've added the index in the console output in the code in post 16. This index is usually the same all the time. When you know the index you don't need the For loop. This can save a lot of time.

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

This is good advice about index of array. Anyway I did more research about speed up my script and look my results

matches.PNG.ca06198b71e695725f6095b35e23

Ctrl type 50025 occurs 268 times when 50020 (which contains my target title)  only 20 times. I tried to change

$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CustomControlTypeId, $pCondition ) ;50025 (find controls ok)

to:

$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition ) ;50020 (find controls err)

or 

$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition ) ;50004 (find controls err)

 

but console output says:

Quote

Find controls ERR

What could be wrong? I have noticed that you already used this part of new code in your example (between comments tags #cs and #ce)

Edited by maniootek

Share this post


Link to post
Share on other sites

#20 ·  Posted (edited)

I don't think there is anything wrong. But there is no direct edit or text control descendants of the window object. The edit and text controls are probably child elements of the custom controls.

How many custom controls have been found with the code in post 17? What's the value of $iElements? The figure should be quite high before it pays to make more partial searches. More partial searches requires more AutoIt code, while the FindAll command is running in the UI Automation dll with the speed of compiled C++ code.

Edited by LarsJ

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now