Get color of text from checkbox control

[Michiel]

Here are the results, there are 39 different colors and white in the text -> '>

 

I want to work on it more but my brain is tired.

 

  Holy ... ! I was afraid of that... thanks for all the effort though!

Something like this should work (although it lists the items in reverse order)

modules from

?do=embed' frameborder='0' data-embedContent>

 

Okay, that looks promising, I need to take a close look at that and get back to you. Thanks!

[junkew]

you should get something like

2 X and Y are: 217,308Title is: (Select All)    Handle=00010A9C Class=WindowsForms10.STATIC.app.0.378734a   Rectangle=201;303;66;14 black
Title is: CCleaner Slim 5.00    Handle=00010A9E Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;588;128;18 red
1 X and Y are: 87,570Title is: Chrome 39.0.2171.71  Handle=00010AA0 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;566;137;18 green
Title is: Skype 7.0.73.100  Handle=00010AA2 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;544;114;18 red
1 X and Y are: 87,526Title is: Silverlight 5.1.30514.0  Handle=00010AA4 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;522;141;18 green
1 X and Y are: 88,504Title is: iTunes 12.0.1    Handle=00010AA6 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;500;96;18  green
1 X and Y are: 87,482Title is: QuickTime 7.76   Handle=00010AA8 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;478;109;18 green
Title is: Java 7 Update 72 x64  Handle=00010AAA Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;456;136;18 red
Title is: Java 7 Update 72  Handle=00010AAC Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;434;115;18 red
Title is: Firefox 34.0.5    Handle=00010AAE Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;412;97;18  red
2 X and Y are: 87,394Title is: Shockwave 12.1.4.154 Handle=00010AB0 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;390;143;18 black
2 X and Y are: 111,372Title is: Air 15.0.0.356  Handle=00010AB2 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;368;96;18  black
Title is: Flash AX 15.0.0.239   Handle=00010AB4 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;346;127;18 red
Title is: Flash Plugin 15.0.0.239   Handle=00010AB6 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;324;146;18 red
Title is: Reader 11.0.09    Handle=00010AB8 Class=WindowsForms10.BUTTON.app.0.378734a   Rectangle=65;302;103;18 red

[Michiel]

@junkew:
 
I've been slowly educating myself about the Windows UI Automation framework (it's a whole world in itself, isn't it?) so my apologies if my responses have been slow. There's a lot to take in. Just a question in the mean time: is the fact that Patch My PC was written in VB.Net the reason why AutoIt can't handle automating it properly? And this applies to most applications written in Java with JWT, C#, QT, WPF and so on, too?

I think that means that your UI Automation library should eventually become a core AutoIt language feature, shouldn't it?

[Michiel]

@junkew: when I look at: "Get the Toggle State of a Check Box Using UI Automation" from Microsoft, I see they are using a "TogglePattern" object, about which SimpleSpy reports the following regarding one of the checkboxes used in PatchMyPc:

 

UIA_IsTogglePatternAvailable:= <False>

I'm screwed, aren't I? I'll have to tinker with pixel checksums?

[Michiel]

Cancel that, it seems my detection problems were related to not being administrator in either my script, or in SimpleSpy, etc. etc.

When running as Administrator, I also need to run SciTE as Administrator, otherwise ConsoleWrite won't work, etc. etc.

So, I'm back in business!

Will report back when I have some results. Seems I can read $UIA_ToggleToggleStatePropertyId !

[Michiel]

@junkew

Okay, so I've got something running which works, and is a bit of a fugly hybrid between your helpful tips and my older attempts. Check it out:

expandcollapse popup

#RequireAdmin

#include <GuiConstants.au3>
#include "include\UIAWrappers.au3"

AutoItSetOption("MustDeclareVars", 1)

Local $aChkBoxes[] = ['Chk_AdobeReader', 'Chk_FlashPlugin', 'Chk_FlashPlayer', 'Chk_AdobeAir', 'Chk_AdobeShockwave', 'Chk_MSSilverlight']

; Use ShellExecute when not running as Administrator, so that PatchMyPc has a chance to present the UAC prompt.
Run('patchmypc.exe')
If @error <> 0 Then
   MsgBox(0, 'Error', 'Error="' & @error & '"')
   Exit
Endif

Local $hWnd = WinWaitActive('[REGEXPTITLE:Patch My PC.*]')
Local $hTab = WinWaitActive('[REGEXPTITLE:Patch My PC.*; NAME:TabPage1]')

Sleep(10000)

AutoItSetOption ('PixelCoordMode', 1)

Local $aCoord, $color, $hChkBox, $name, $rectangle, $t, $toggle, $UIA_myObject, $oUIElement

For $i = 0 To UBound($aChkBoxes) - 1
    ConsoleWrite('Checking: ' & $aChkBoxes[$i] & @CRLF)

    $hChkBox = ControlGetHandle($hWnd, '', '[NAME:' & $aChkBoxes[$i] & ']')

    _UIA_setVar('$oP0','id:=' & $aChkBoxes[$i])
    $UIA_myObject = _UIA_Action('$oP0','getobject')
    If Not IsObj($UIA_myObject) Then MsgBox(0, 'Debug', 'Failure #1')
    $oUIElement = $UIA_myObject
    ;$UIA_oTW.GetFirstChildElement($UIA_myObject, $UIA_pUIElement)
    ;$oUIElement = ObjCreateInterface($UIA_myObject, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    ;If Not IsObj($oUIElement) Then MsgBox(0, 'Debug', 'Failure #2')

    $rectangle = _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId)
    $name = _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId)
    $t = StringSplit($rectangle, ";")

    Local $aCoord = PixelSearch($t[1]+16, $t[2], $t[3] + $t[1], $t[4] + $t[2], 0x006400)
    If Not @error Then
        consolewrite("1 X and Y are: " & $aCoord[0] & "," & $aCoord[1] & @TAB)
        $color="green"
    Else
        ; Find a pure black pixel
        Local $aCoord = PixelSearch($t[1]+16, $t[2], $t[3] + $t[1], $t[4] + $t[2], 0x000000)

        If Not @error Then
            consolewrite("2 X and Y are: " & $aCoord[0] & "," & $aCoord[1] & @TAB)
            $color="black"
            ControlCommand($hWnd, "", $hChkbox, "Check", "")
        Else
            $color="red"
            ControlCommand($hWnd, "", $hChkbox, "Check", "")
        EndIf
    EndIf

    $toggle = _UIA_getPropertyValue($oUIElement, $UIA_ToggleToggleStatePropertyId)
    ConsoleWrite("Title is: " & $name & @TAB & "Handle=" & Hex(_UIA_getPropertyValue($oUIElement, $UIA_NativeWindowHandlePropertyId)) & @TAB & "Class=" & _
    _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @TAB & "Rectangle=" & $rectangle & @TAB & $color & @TAB & "Toggle=" & $toggle & @CRLF)
Next

What do you think?

[computergroove]

#include "include\UIAWrappers.au3"

Can be wrtten as

#include <UIAWrappers.au3>

[SmOke_N]

#include "include\UIAWrappers.au3"

Can be wrtten as

#include <UIAWrappers.au3>

Not if it doesn't reside in the same folder as your script you're running/compiling or you put it in the main/beta AutoIt include folder.

Edited December 15, 2014 by SmOke_N

[junkew]

_UIA_setVar('$oP0','id:=' & $aChkBoxes[$i])
    $UIA_myObject = _UIA_Action('$oP0','getobject')

is equal to

$UIA_myObject = _UIA_Action('id:=' & $aChkBoxes[$i],'getobject')

just a matter of style

Regarding AutoIT I can only guess as I do not have insight in the AutoIT sources

Could be due to fact applications build in .NET and java are not natively recognized by AutoIT

When AutoIT was build there was no IUIAutomation around and it only works on Win XP or higher whereas AutoIT handles internally all window/control messages and I think it would be a huge task to replace inner workings of AutoIT with IUIAutomation (probably would mean a rebuild) but

yes it could be a core for inner workings of AutoIT just as Java Accessibility Bridge and iOS instruments and Android UIAutomation could be a core but so far nobody has written a tool that abstracts all of them in a flexible and extensible way to support at once Windows, Browsers, iOS and Android and Windows Phone. So far last 10-15 years I have not found the perfect tool. AutoIT comes close but you need to understand the available UDF's to complete all your wishes.

Yes, you are screwed as a checkbox means on/off/indeterminate and some Patch My PC developer is not understanding that there are people with disabilities and automation testers around and thought lets use three colors and forget to give meaningfull state and information in accessibility properties.