Running my Script on remote desktop machine when minimized

[MakzNovice]

Hello Experts,

I have a script which deals with an interactive application ( say notepad / Firefox ). I need to run this program on remote machine through task scheduler.

The compiled exe does everything fine while I have the remote screen open and I am looking at execution. 

But as soon as I minimize the screen ( which is going to be case when using it in actual scenario) , the script does nothing.

Can anyone guide me how can keep the program executing on a remote minimized screen?

Sample code that I would execute is as below:

expandcollapse popup

#include <File.au3>

$answer = MsgBox(4, "Pwnage Notepad", "This script will run Notepad type in save file names, save file and then quit.", 3)

Run("notepad.exe")

WinWaitActive("Untitled - Notepad")


 Local $aFileList = _FileListToArray("C:\Temp", "*.txt")
If @error = 1 Then
MsgBox(0, "", "Path was invalid.")
Exit
EndIf
If @error = 4 Then
MsgBox(0, "", "No file(s) were found.")
Exit
EndIf
; Display the results returned by _FileListToArray.
;_ArrayDisplay($aFileList, "$aFileList")

WinActivate("Untitled - Notepad")
For $i = 1 To $aFileList[0]
Local $sFileName = $aFileList[$i]

;MsgBox(0, "Writing", $sFileName, 1)
Send($sFileName & @CRLF)
Sleep(500)
Next

Send("!f")
Send("x")

WinWaitActive("Notepad")
Send("{ENTER}")

; And concatenate in the date/time macros
$sNewFile = "LogFile_" & @YEAR & "_" & @MON & "_" & @MDAY  & " " & @HOUR & "_" & @MIN & "_" & @SEC & ".txt"

; And here we are
MsgBox(0, "New name", $sNewFile, 5)
Sleep(500)
Send($sNewFile)
Sleep(5000)
Send("{ENTER}")
WinWaitClose("Untitled - Notepad")

 

Edited April 3, 2020 by MakzNovice

[BrewManNH]

Are you running the script on the remote machine, or your local machine?

[MakzNovice]

@BrewManNH I am running the script on remote machine..

[BrewManNH]

What type of automation are you doing? Are you using Send instead of ControlSend, MouseClick instead of ControlClick? 

 

https://www.autoitscript.com/wiki/FAQ#Why_doesn.27t_my_script_work_on_a_locked_workstation.3F

 

[Jos]

1 hour ago, MakzNovice said:

Pwnage

?

[MakzNovice]

@BrewManNH- Yes I use Tons of WinActive, WinActivate, WinWait and Send ... I saw the articles you mentioned, will try out those ControlSend() ControlSetText() ControlClick() WinWait() WinExists() as required and confirm.

By the way, what is alternative of WinActivate?

@Jos - Just had copied a simple script from some forum and modified as my requirement.

[Jos]

15 minutes ago, MakzNovice said:

Just had copied a simple script from some forum and modified as my requirement.

Yea right .... just some game script from some game forum as an example. Sometimes I really wonder what people think when they type these kind of avoiding crazy answers.

Jos

[MakzNovice]

@BrewManNH I tried using ControlSend and ControlFocus but it does not work perfectly for me. I use @Junkew developed UIA automation in my code.

Is there an alternative , the below snippet is from my code and mostly does similar steps

Please let me know what can I do differently to make this working on 'minimized remote machine'

expandcollapse popup

#include <FileConstants.au3>
#include <StringConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"


#include <Array.au3>
#include <File.au3>
#include <MsgBoxConstants.au3>

; used for chcking is the script is Singleton
#include <Misc.au3>

#include <ColorConstantS.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>

Global $ACROBAT_DC_EXE_PATH = "C:\Program Files (x86)\Adobe\Acrobat DC\Acrobat\Acrobat.exe" 
Global $PdfFileName = "AppDevGuide.pdf"
MyFunction

Func MyFunction()
    
    Local $Run = $ACROBAT_DC_EXE_PATH & " " & @ScriptDir & "\" & $PdfFileName
    ConsoleWrite("$Run : " & $Run & @CRLF)
    Local $PID = Run($Run, "", @SW_MAXIMIZE)
    ConsoleWrite("$PID : " & $PID & @CRLF)
    Local $list = WinList()
    For $i = 1 To $list[0][0]
        If WinGetProcess($list[$i][0]) = $PID And $list[$i][0] Then ;we need activate only window that matches our PID and has a title
            ConsoleWrite("Found $PID : " & $PID & @CRLF)
            WinSetState($list[$i][1], "", @SW_SHOW)
            WinActivate($list[$i][1])
            ExitLoop
        EndIf
    Next

    Local $szDrive, $szDir, $szFName, $szExt
    _PathSplit($fileName, $szDrive, $szDir, $szFName, $szExt)

    ConsoleWrite("$szDrive : " & $szDrive & @CRLF)
    ConsoleWrite("$szDir : " & $szDir & @CRLF)
    ConsoleWrite("$szFName : " & $szFName & @CRLF)
    ConsoleWrite("$szExt : " & $szExt & @CRLF)

;~ Make the PDF file as active and on top
    WinWait($szFName)
    WinActivate($szFName)

    ConsoleWrite("Activated : " & $szFName & @CRLF)
    _FileWriteLog($LOG_FILE_PATH, "Activated : " & $szFName)

;~ Wait for all the contents to be loaded
    Sleep(5000)
    ConsoleWrite("Sleep(5000)" & @CRLF)
    
    ConsoleWrite("Trying to get Find tools command" & @CRLF)

    Local $oP5 = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AcrobatSDIWindow", $treescope_children)
    _UIA_Action($oP5, "setfocus")
    Local $oP4 = _UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP4, "setfocus")
    Local $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP3, "setfocus")
    Local $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=Right Hand Tools Pane;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP2, "setfocus")
    Local $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
    Local $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
    Local $oUIElement = _UIA_getObjectByFindAll($oP0, "title:=Find Tools;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
    _UIA_action($oUIElement, "click")

    ConsoleWrite("I clicked Find tool " & @CRLF)

;~ Add the Send For Shared Commenting text in editor
    Send("Send For Review")
    ConsoleWrite("I Entered text in search Send For Review" & @CRLF)
    

    ; MOST important wait for some time after sending $COMMAND_STRING
    Sleep(5000)
    ConsoleWrite("Sleep(5000)" & @CRLF)
    
    $oP5 = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AcrobatSDIWindow", $treescope_children)
    _UIA_Action($oP5, "setfocus")
    $oP4 = _UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP4, "setfocus")
    $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP3, "setfocus")
    $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=Right Hand Tools Pane;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP2, "setfocus")
    $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP1, "setfocus")
    $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP0, "setfocus")
    _UIA_setVar(".mainwindow", "title:=;classname:=AVL_AVView")
    _UIA_action(".mainwindow", "setfocus")

    ConsoleWrite("Highliting the Send For Review command" & @CRLF)
    
    ; Send
    ;Send("{TAB 1}")

    ;Send("{ENTER}")
    Local $oP8 = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AcrobatSDIWindow", $treescope_children)
    _UIA_Action($oP8, "setfocus")
    Local $oP7 = _UIA_getObjectByFindAll($oP8, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP7, "setfocus")
    Local $oP6 = _UIA_getObjectByFindAll($oP7, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP6, "setfocus")
    $oP5 = _UIA_getObjectByFindAll($oP6, "Title:=Right Hand Tools Pane;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP5, "setfocus")
    $oP4 = _UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP4, "setfocus")
    $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP3, "setfocus")
    $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP2, "setfocus")
    $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP1, "setfocus")
    $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
    $oUIElement = _UIA_getObjectByFindAll($oP0, "title:=Send for Shared Commenting;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)

    _UIA_action($oUIElement, "click")

    ConsoleWrite("Must have clicked Send for Review " & @CRLF)

    Sleep(500)

    WinActivate($szFName)
;~ Close the opened file
    Send("!f")
    Send("c")
    Sleep(2000)
    ConsoleWrite("Alt + F then C to close file.... " & @CRLF)
    _FileWriteLog($LOG_FILE_PATH, "Alt + F + C to close file.... " )
    _FileWriteLog($LOG_FILE_PATH, "Sleep(2000)")

;~ Close the Acrobat Application
    Send("!f")
    Send("x")
    Sleep(3000)
    ConsoleWrite("Alt + F then  X to Exit Acrobat .... " & @CRLF)
    _FileWriteLog($LOG_FILE_PATH, "Alt + F then  X to Exit Acrobat .... " )
    _FileWriteLog($LOG_FILE_PATH, "Sleep(3000)")

;~ Now close the pdf application forcefully if it still exists
    If ProcessExists($PID) Then
        ProcessClose($PID)
        ConsoleWrite("Acrobat found, Killing it.... " & @CRLF)
        _FileWriteLog($LOG_FILE_PATH, "Acrobat found, Killing it.... " )
    EndIf

EndFunc

 

[BigDaddyO]

You could try to use this.  It will leave you logged into the remote system.  I never tested it with the task scheduler but it may work.

 

[MakzNovice]

@BigDaddyO - Thank you for your input, but the problem is the user do not wish to disconnect the RDP session. 

They just want it to be minimize screen and still wish to continue using the my program for PDF manipulation.

To keep the session alive, I can write a small program that keeps moving mouse 1 pixel to and fro and avoid any remote session closure.

my main problem now is, on a minimized remote machine Adobe not being on top when my script is running, as soon as I open the session again it brings Adobe back on top.

[BigDaddyO]

to keep the session alive, just put this at the top of your normal script.

Local $aRet = DllCall('kernel32.dll', 'long', 'SetThreadExecutionState', 'long', 0x80000003)         ;0x80000003 to disable windows sleep mode
 

Use this before you exit your script

Local $aRet = DllCall('kernel32.dll', 'long', 'SetThreadExecutionState', 'long', 0x80000000)    ;Remove the always on so it will go to sleep

 

As for your automation, the only option is to modify as BrewManNH suggested link says:

Why doesn't my script work on a locked workstation?

On a locked station any window will never be active (active is only dialog with text "Press Ctrl+Alt+Del"). In Windows locked state applications run hidden (behind that visible dialog) and do not have focus and active status. So generally don't use Send() MouseClick() WinActivate() WinWaitActive() WinActive() etc. Instead use ControlSend() ControlSetText() ControlClick() WinWait() WinExists() WinMenuSelectItem() etc. Doing so allows you to interact with an application regardless of whether it is active or not. It's possible to run such a script from scheduler on locked Windows stations.

[MakzNovice]

@BigDaddyO Thank you for your answer, I apologize for late response was occupied with some personal work.

But as I mentioned earlier problem is not with keeping session alive, it about running Adobe on 'minimized remote machine' such way that Adobe is launched in foreground.

The code you gave only keeps my session alive, now as I mentioned I use @junkew developed UIA automation API to identify objects/controls and perform action on them. The method in UIA Automation ( _UIA_Action ) script internally calls 'Send' which I guess is problem ( May be @junkew can confirm..)

Also, I use a lot of WinActive, WinActivate, WinWait in my script, so it also is problem, I am re-pasting the code if you guys ( @BrewManNH, @junkew ) can help me suggest alternatives.

expandcollapse popup

#include <FileConstants.au3>
#include <StringConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"


#include <Array.au3>
#include <File.au3>
#include <MsgBoxConstants.au3>

; used for chcking is the script is Singleton
#include <Misc.au3>

#include <ColorConstantS.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>

Global $ACROBAT_DC_EXE_PATH = "C:\Program Files (x86)\Adobe\Acrobat DC\Acrobat\Acrobat.exe" 
Global $PdfFileName = "AppDevGuide.pdf"
MyFunction

Func MyFunction()
    
    Local $Run = $ACROBAT_DC_EXE_PATH & " " & @ScriptDir & "\" & $PdfFileName
    ConsoleWrite("$Run : " & $Run & @CRLF)
    Local $PID = Run($Run, "", @SW_MAXIMIZE)
    ConsoleWrite("$PID : " & $PID & @CRLF)
    Local $list = WinList()
    For $i = 1 To $list[0][0]
        If WinGetProcess($list[$i][0]) = $PID And $list[$i][0] Then ;we need activate only window that matches our PID and has a title
            ConsoleWrite("Found $PID : " & $PID & @CRLF)
            WinSetState($list[$i][1], "", @SW_SHOW)
            WinActivate($list[$i][1])
            ExitLoop
        EndIf
    Next

    Local $szDrive, $szDir, $szFName, $szExt
    _PathSplit($fileName, $szDrive, $szDir, $szFName, $szExt)

    ConsoleWrite("$szDrive : " & $szDrive & @CRLF)
    ConsoleWrite("$szDir : " & $szDir & @CRLF)
    ConsoleWrite("$szFName : " & $szFName & @CRLF)
    ConsoleWrite("$szExt : " & $szExt & @CRLF)

;~ Make the PDF file as active and on top
    WinWait($szFName)
    WinActivate($szFName)

    ConsoleWrite("Activated : " & $szFName & @CRLF)
    _FileWriteLog($LOG_FILE_PATH, "Activated : " & $szFName)

;~ Wait for all the contents to be loaded
    Sleep(5000)
    ConsoleWrite("Sleep(5000)" & @CRLF)
    
    ConsoleWrite("Trying to get Find tools command" & @CRLF)

    Local $oP5 = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AcrobatSDIWindow", $treescope_children)
    _UIA_Action($oP5, "setfocus")
    Local $oP4 = _UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP4, "setfocus")
    Local $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP3, "setfocus")
    Local $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=Right Hand Tools Pane;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP2, "setfocus")
    Local $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
    Local $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
    Local $oUIElement = _UIA_getObjectByFindAll($oP0, "title:=Find Tools;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
    _UIA_action($oUIElement, "click")

    ConsoleWrite("I clicked Find tool " & @CRLF)

;~ Add the Send For Shared Commenting text in editor
    Send("Send For Review")
    ConsoleWrite("I Entered text in search Send For Review" & @CRLF)
    

    ; MOST important wait for some time after sending $COMMAND_STRING
    Sleep(5000)
    ConsoleWrite("Sleep(5000)" & @CRLF)
    
    $oP5 = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AcrobatSDIWindow", $treescope_children)
    _UIA_Action($oP5, "setfocus")
    $oP4 = _UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP4, "setfocus")
    $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP3, "setfocus")
    $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=Right Hand Tools Pane;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP2, "setfocus")
    $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP1, "setfocus")
    $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP0, "setfocus")
    _UIA_setVar(".mainwindow", "title:=;classname:=AVL_AVView")
    _UIA_action(".mainwindow", "setfocus")

    ConsoleWrite("Highliting the Send For Review command" & @CRLF)
    
    ; Send
    ;Send("{TAB 1}")

    ;Send("{ENTER}")
    Local $oP8 = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=;controltype:=UIA_WindowControlTypeId;class:=AcrobatSDIWindow", $treescope_children)
    _UIA_Action($oP8, "setfocus")
    Local $oP7 = _UIA_getObjectByFindAll($oP8, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP7, "setfocus")
    Local $oP6 = _UIA_getObjectByFindAll($oP7, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP6, "setfocus")
    $oP5 = _UIA_getObjectByFindAll($oP6, "Title:=Right Hand Tools Pane;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP5, "setfocus")
    $oP4 = _UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP4, "setfocus")
    $oP3 = _UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP3, "setfocus")
    $oP2 = _UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP2, "setfocus")
    $oP1 = _UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_GroupControlTypeId;class:=AVL_AVView", $treescope_children)
    _UIA_Action($oP1, "setfocus")
    $oP0 = _UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
    $oUIElement = _UIA_getObjectByFindAll($oP0, "title:=Send for Shared Commenting;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)

    _UIA_action($oUIElement, "click")

    ConsoleWrite("Must have clicked Send for Review " & @CRLF)

    Sleep(500)

    WinActivate($szFName)
;~ Close the opened file
    Send("!f")
    Send("c")
    Sleep(2000)
    ConsoleWrite("Alt + F then C to close file.... " & @CRLF)
    _FileWriteLog($LOG_FILE_PATH, "Alt + F + C to close file.... " )
    _FileWriteLog($LOG_FILE_PATH, "Sleep(2000)")

;~ Close the Acrobat Application
    Send("!f")
    Send("x")
    Sleep(3000)
    ConsoleWrite("Alt + F then  X to Exit Acrobat .... " & @CRLF)
    _FileWriteLog($LOG_FILE_PATH, "Alt + F then  X to Exit Acrobat .... " )
    _FileWriteLog($LOG_FILE_PATH, "Sleep(3000)")

;~ Now close the pdf application forcefully if it still exists
    If ProcessExists($PID) Then
        ProcessClose($PID)
        ConsoleWrite("Acrobat found, Killing it.... " & @CRLF)
        _FileWriteLog($LOG_FILE_PATH, "Acrobat found, Killing it.... " )
    EndIf

EndFunc

 

Edited April 17, 2020 by MakzNovice

[junkew]

For testing you could try with calc.exe. as uia recognizes more controls then autoit it really depends if you can use controlsend. As uia setvalue is not available for all controls I use send more often as that works nice certainly when screens are not locked. To prevent screens from locking I just send numlock twice every n minutes.

1. Use controlsend where possible
2. Use send where possible
3. Use UIA setvalue depending if control supports it
4. Use keybd_event or sendinput from win32 api
5. Use postmessage/sendmessage where possible (WM_CHAR, WM_KEYDOWN, WM_KEYUP, ...)
6. Go deeper with hooking events
   https://sites.google.com/site/janbeck/interception-of-win32-and-user-dll-api-calls-using-autoit-and-deviare

In general I stop at step 3 and start talking to developers of the application to make there application accessible for blind people and automation.

Levels 1,2 make use of 4 and 5. Level 3 makes use of internal windows api's and are partly based on hooking into the other process the uiautomation dlls
 

Edited April 19, 2020 by junkew

[MakzNovice]

@junkew : I had to take break from this script for some time

Can I ask have you tried using ControlSend or ControlClick instead of Send and MouseClick in your _UIA_action methods?

I tried doing so but it does nothing , even uncommented your logs using UIA_log . its just says object is non-existent.

Attached is your xml log file .. last two lines show the are from UIA_Action method.

I have tried below two options but same result.

1. Used $oElement

_UIA_LOG("Title is: <" &  _UIA_getPropertyValue($UIA_oUIElement,$UIA_NamePropertyId) &  ">" & $clickcount & ":" & $clickaction & ":" & $x & ":" & $y & ":" & @CRLF, $UIA_Log_Wrapper) 

;~ TODO: Check if setting focus should happen as it influences behavior before clicking
;~ Tricky when using setfocus on menuitems, seems to do the click already
;~ $obj.setfocus()

;~ Mouse should move to keep it as userlike as possible
;MouseMove($x, $y, 0)
;~ mouseclick($clickAction,Default,Default,$clickCount,0)
If Not StringInStr($strAction, "move") Then
;MouseClick($clickAction, $x, $y, $clickCount, 0)
ControlClick($oElement,"","",$clickAction,$clickcount,$x,$y)
ControlSend($oElement,"","","{ENTER}")
$error = @error
_UIA_LOG("Error is: <" &  $error & @CRLF, $UIA_Log_Wrapper)
EndIf
Sleep($UIA_DefaultWaitTime)

and 2. Used input $obj2ActOn

_UIA_LOG("Title is: <" &  _UIA_getPropertyValue($UIA_oUIElement,$UIA_NamePropertyId) &  ">" & $clickcount & ":" & $clickaction & ":" & $x & ":" & $y & ":" & @CRLF, $UIA_Log_Wrapper) 

;~ TODO: Check if setting focus should happen as it influences behavior before clicking
;~ Tricky when using setfocus on menuitems, seems to do the click already
;~ $obj.setfocus()

;~ Mouse should move to keep it as userlike as possible
;MouseMove($x, $y, 0)
;~ mouseclick($clickAction,Default,Default,$clickCount,0)
If Not StringInStr($strAction, "move") Then
;MouseClick($clickAction, $x, $y, $clickCount, 0)
ControlClick($obj2ActOn,"","",$clickAction,$clickcount,$x,$y)
ControlSend($obj2ActOn,"","","{ENTER}")
$error = @error
_UIA_LOG("Error is: <" &  $error & @CRLF, $UIA_Log_Wrapper)
EndIf
Sleep($UIA_DefaultWaitTime)

Note: The passed object to method is $obj2ActOn that I get from _UIA_getObjectByFindAll method call done, which is retrieved successfully as you can see in log file

 

 

20200914-072602771.XML

[MakzNovice]

@junkew any sharable information?

Any other expert who can help me out here?