CUIAutomation - not working when run on hidden virtual desktop?

[mpreis]

Hello there

below you can see a script that works really well:

expandcollapse popup

#include <CUIAutomation2.au3>
Opt("MustDeclareVars", 1)

; CAD-Fenster: erstes Fenster, dessen Titel mit "Gruppe: " beginnt
Global Const $g_sCADWin = "[REGEXPTITLE:^Gruppe:\s]"

; COM-Error-Handler (für sicheres Array-Iterieren)
Global $g_bComError = False
Global $g_oComErr = ObjEvent("AutoIt.Error", "_ComErrHandler")

Func _ComErrHandler($oError)
    $g_bComError = True
EndFunc

; ================== Beispielaufruf =====================
ClickGeschossButton("EG")
; =======================================================


; Klickt "Toolbar Options" und danach den Geschoss-Button $sName
Func ClickGeschossButton($sName)

    ; 1) CAD-Fenster holen
    Local $hCAD = WinGetHandle($g_sCADWin)
    If @error Or $hCAD = 0 Then
        MsgBox(16, "Fehler", "CAD-Fenster nicht gefunden: " & $g_sCADWin)
        Return
    EndIf
    WinActivate($hCAD)

    ; 2) UIAutomation initialisieren
    Local $oUIA = ObjCreateInterface($sCLSID_CUIAutomation, _
                                     $sIID_IUIAutomation, _
                                     $dtagIUIAutomation)
    If Not IsObj($oUIA) Then
        MsgBox(16, "Fehler", "UIAutomation konnte nicht initialisiert werden.")
        Return
    EndIf

    ; 3) UIA-Element für CAD-Fenster
    Local $pWindow, $oWindow
    $oUIA.ElementFromHandle($hCAD, $pWindow)
    If Not $pWindow Then
        MsgBox(16, "Fehler", "UIA-Element für CAD-Fenster nicht gefunden.")
        Return
    EndIf
    $oWindow = ObjCreateInterface($pWindow, _
                                  $sIID_IUIAutomationElement, _
                                  $dtagIUIAutomationElement)

    ; 4) Zuerst MenuItem "Toolbar Options" klicken
    If Not _ClickMenuItemByName($oUIA, $oWindow, "Toolbar Options") Then
        MsgBox(16, "Fehler", "'Toolbar Options' wurde nicht gefunden / konnte nicht geklickt werden.")
        Return
    EndIf

    Sleep(150) ; kleines Delay, bis die Toolbar reagiert

    ; 5) Danach den gewünschten Button (z.B. "OG") klicken
    If Not _ClickButtonByNameOnElement($oUIA, $oWindow, $sName) Then
        MsgBox(16, "Fehler", "Button '" & $sName & "' wurde nicht gefunden / konnte nicht geklickt werden.")
        Return
    EndIf
EndFunc


; ======= Hilfsfunktion: MenuItem nach Name suchen & klicken =========
Func _ClickMenuItemByName($oUIA, $oRoot, $sName)
    Local $oElem = _FindElementByNameEnum($oUIA, $oRoot, _
                                          $UIA_MenuItemControlTypeId, _
                                          $sName)
    If Not IsObj($oElem) Then Return False

    Local $pInvoke, $oInvoke
    $oElem.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    If Not $pInvoke Then Return False

    $oInvoke = ObjCreateInterface($pInvoke, _
                                  $sIID_IUIAutomationInvokePattern, _
                                  $dtagIUIAutomationInvokePattern)
    $oInvoke.Invoke()
    Return True
EndFunc


; ======= Hilfsfunktion: Button nach Name suchen & klicken ===========
Func _ClickButtonByNameOnElement($oUIA, $oRoot, $sButtonName)
    Local $oElem = _FindElementByNameEnum($oUIA, $oRoot, _
                                          $UIA_ButtonControlTypeId, _
                                          $sButtonName)
    If Not IsObj($oElem) Then Return False

    Local $pInvoke, $oInvoke
    $oElem.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    If Not $pInvoke Then Return False

    $oInvoke = ObjCreateInterface($pInvoke, _
                                  $sIID_IUIAutomationInvokePattern, _
                                  $dtagIUIAutomationInvokePattern)
    $oInvoke.Invoke()
    Return True
EndFunc


; ======= Generische Suche: erstes Element eines ControlTypes, dessen Name passt =======
Func _FindElementByNameEnum($oUIA, $oRoot, $iControlType, $sNeedleName)

    ; Alle Elemente vom gewünschten ControlType holen
    Local $pCond, $pArray
    $oUIA.CreatePropertyCondition($UIA_ControlTypePropertyId, _
                                  $iControlType, _
                                  $pCond)

    $oRoot.FindAll($TreeScope_Descendants, $pCond, $pArray)
    If Not $pArray Then Return SetError(1, 0, 0)

    Local $oArr = ObjCreateInterface($pArray, _
                                     $sIID_IUIAutomationElementArray, _
                                     $dtagIUIAutomationElementArray)
    If Not IsObj($oArr) Then Return SetError(2, 0, 0)

    Local $sTarget = StringUpper(StringStripWS($sNeedleName, 3))

    Local $i = 0
    While True
        $g_bComError = False
        Local $pElem = 0
        $oArr.GetElement($i, $pElem)

        If $g_bComError Or Not $pElem Then ExitLoop ; Ende des Arrays

        Local $oElem = ObjCreateInterface($pElem, _
                                          $sIID_IUIAutomationElement, _
                                          $dtagIUIAutomationElement)
        If Not IsObj($oElem) Then
            $i += 1
            ContinueLoop
        EndIf

        Local $vName, $sName
        $oElem.GetCurrentPropertyValue($UIA_NamePropertyId, $vName)
        $sName = StringUpper(StringStripWS(String($vName), 3))

        ; exakter Vergleich, case-insensitive
        If $sName = $sTarget Then
            Return $oElem
        EndIf

        $i += 1
    WEnd

    Return SetError(3, 0, 0)
EndFunc

This code is used to click a button within our CAD application that is nested in a XTPToolBar. 

The script above is a part of a bigger software that is used to run tasks in the CAD app without any user interaction.

As long as I run the script and watch what is happening it works.

The intention of the script is that the main application should run on a virtual desktop where there is no display output - but in this case it doesn't work. When I connect to the virtual desktop (Citrix Receiver environment) and watch then it works fine again.

I have plenty of AutoIT scripts without UIA that work since years in such an environment - this tool uses UIA and it does not. Is there any requirement for graphical output to a screen or something else when using UIA?

"Normal" AutoIT commands like "ControlClick" don't work for that button, the UIA method is the only one I found that is working.

Does anyone have an idea for me how to solve that issue?

 

[junkew]

CUIAutomation was designed for UI automation not for headless automation. There are to many technologies around to have "one rules them all". 
You should in detail describe/understand in which technology your to be remote controlled application is written and see which technologies will work on how to controll them (this could mean AutoIt cannot handle it and you need commercial solutions that sometimes can go further). Probably cheaper to have a room thats locked where you can keep the desktops running.
Are you sure your virtual desktop continues after you disconnect? You could try to run scripts to "keep session alive" or "kill screenlock" like sendkeys numlock or other scripts where the VDI keeps active.