UIA Text control get Background Color

[BigDaddyO]

I'm looking for help with getting the background color of a text control using UI Automation.  I can't figure out how to use the $UIA_BackgroundColorAttributeId

The end goal is when submitting a new record into a website, if there are any problems with the submission, text will be displayed with a red background color. The only identifying options for that error message is that the ControlType = $UIA_TextControlTypeId and the background is red.  So, if the record doesn't save i'll scan the screen and pull the text out of the error message and save it to my log file.  "Currently i'm forced to take screenshots to know what happened which takes much longer to review and can't be easily grouped by error"

 

I found the Wikipedia main page has a similar text control with a green background that I built a sample for.  So, anybody know how to get the green background color from that wiki page?

expandcollapse popup

#include ".\Include\UIAWrappers.au3"
;   GetAttributeValue   $UIA_BackgroundColorAttributeId


;Sample written to get the background color of the "From today's featured article" text control on "https://en.wikipedia.org/wiki/Main_Page"
;This example only works if Chrome is open to Wiki Main page and no other tabs are open

; Ultimatly, I want to scan an entire page looking for any text control that has a specific background color (Error Messages)

WinWait("Wikipedia, the free encyclopedia", "", 30)

;Create UI Automation object that will be used by all the tests
    $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
    If Not IsObj( $oUIAutomation ) Then
        If Not @Compiled Then ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
        Exit
    EndIf
    If Not @Compiled Then ConsoleWrite( "$oUIAutomation OK" & @CRLF )

;Setup the UIAutomation
    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement( $pDesktop )
    $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If IsObj( $oDesktop ) Then
        If Not @Compiled Then ConsoleWrite( "$oDesktop OK" & @CRLF )

        ;Get the Main Chrome window
        Local $pCondition ; Note that $UIA_ClassNamePropertyId is CASE SENSITIVE
        $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition )
        Local $pIE, $oUIaIE
        $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pIE )
        $oIEdialog = ObjCreateInterface( $pIE, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
        If IsObj( $oIEdialog ) Then
            If Not @Compiled Then ConsoleWrite( "Found Chrome_WidgetWin_1" & @CRLF )


            Local $pCondition
            $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, "content", $pCondition )
            Local $pIE, $oUIaIE
            $oIEdialog.FindFirst( $TreeScope_Descendants, $pCondition, $pIE )
            $oIEframe = ObjCreateInterface( $pIE, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
            If IsObj( $oIEframe ) Then
                If Not @Compiled Then ConsoleWrite( "Found content AutomationID" & @CRLF )
            EndIf

        EndIf
    EndIf

;Get the text control [From today's featured article] that has a green background
    Local $pCondition, $pCondition1, $pCondition2
    $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition1 )
    $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "From today's featured article", $pCondition2 )
    $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )

    Local $pElements, $oElement
    $oIEframe.FindFirst( $TreeScope_Descendants, $pCondition, $pElements )
    $oElement = ObjCreateInterface($pElements, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If IsObj( $oElement ) Then
        If Not @Compiled Then ConsoleWrite("Found the text message we were looking for!" & @CRLF )
    EndIf


;JUST THROWING STUFF AT THE WALL FROM HERE.  How do I get the Green background color of the text?
    Local $pTextPattern, $sBkColor
    $oElement.GetCurrentPattern($UIA_TextPattern2Id, $pTextPattern)
    $oPattern = ObjCreateInterface($pTextPattern, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If IsObj($oPattern) Then
        ConsoleWrite("Found Current Pattern" & @CRLF)
    Else
        ConsoleWrite("Unable to find Pattern" & @CRLF)
    EndIf

;~  $oPattern.DocumentRange($Document_Range)
;~  $oPattern.GetAttributeValue($pTextPattern.BackgroundColorAttribute)

;~  $oElement.DocumentRange.GetAttributeValue($pTextPattern.BackgroundColorAttribute)



    $oElement.GetCurrentPropertyValue($UIA_BackgroundColorAttributeId, $sBkColor)
    ConsoleWrite("Background color = (" & $sBkColor & ")" & @CRLF)

Any help would be GREATLY appreciated!

Thanks,

Mike

Edited June 6, 2023 by BigDaddyO

[20Ice18]

Try use the UIA_BackgroundColorAttributeId attribute and the GetCurrentPropertyValue method.

 

#include ".\Include\UIAWrappers.au3"

Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
If Not IsObj($oUIAutomation) Then
    MsgBox(16, "Error", "$oUIAutomation creation failed.")
    Exit
EndIf

Local $oElement, $pCondition, $pElements, $sBkColor

; Find the text control with a green background
$oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition)
$oUIAutomation.ElementFromHandle($HWND, $oElement)
$oElement.FindFirst($TreeScope_Descendants, $pCondition, $pElements)
$oElement = ObjCreateInterface($pElements, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oElement) Then
    MsgBox(16, "Error", "Failed to find the text control.")
    Exit
EndIf

; Get the background color attribute
If $oElement.GetCurrentPropertyValue($UIA_BackgroundColorAttributeId, $sBkColor) <> $S_OK Then
    MsgBox(16, "Error", "Failed to retrieve the background color.")
    Exit
EndIf

MsgBox(64, "Background Color", "The background color of the text control is: " & $sBkColor)

Replace $HWND with the handle of the target window or control. You can obtain the handle using AutoIt's built-in functions such as WinGetHandle or ControlGetHandle.

Please note that UI Automation relies on the accessibility features of the target application, so not all applications may provide the necessary information to retrieve the background-color attribute.

You can try the PixelGetColor function as an alternative when UI Automation is unable to retrieve the background color of a control.

 

#include ".\Include\UIAWrappers.au3"

Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
If Not IsObj($oUIAutomation) Then
    MsgBox(16, "Error", "$oUIAutomation creation failed.")
    Exit
EndIf

Local $oElement, $pCondition, $pElements, $sBkColor

; Attempt to retrieve the background color using UI Automation
$oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition)
$oUIAutomation.ElementFromHandle($HWND, $oElement)
$oElement.FindFirst($TreeScope_Descendants, $pCondition, $pElements)
$oElement = ObjCreateInterface($pElements, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

If IsObj($oElement) Then
    ; UI Automation succeeded, get the background color attribute
    If $oElement.GetCurrentPropertyValue($UIA_BackgroundColorAttributeId, $sBkColor) = $S_OK Then
        MsgBox(64, "Background Color", "The background color of the text control is: " & $sBkColor)
        Exit
    EndIf
EndIf

; UI Automation failed, attempt to retrieve the background color using PixelGetColor
Local $x = 100 ; X-coordinate of the pixel
Local $y = 100 ; Y-coordinate of the pixel

$sBkColor = PixelGetColor($x, $y)

MsgBox(64, "Pixel Color", "The color of the pixel at (" & $x & ", " & $y & ") is: 0x" & Hex($sBkColor, 6))

 

PixelGetColor function retrieves the color of a pixel on the screen, not from a specific control or window. If you need to retrieve the color of a pixel within a specific window or control, you may need to combine PixelGetColor with other AutoIt functions such as WinGetPos or ControlGetPos to obtain the coordinates relative to the desired window or control.

Edited June 15, 2023 by 20Ice18