Jump to content
griefman

Sending keys and mouse clicks inside a VMWare VM

Recommended Posts

griefman

Hi everyone,

i am writing to you after a very long struggle i had while trying to figure out how to send a simple click inside a virtual machine running in vmware workstation 14.

i have an autoit script running on my host machine watching for the UAC prompt to be displayed in a running vm. Both the host and the guest OS are Windows 10. This script worked perfectly with virtual box. It recognized the UAC prompt and clicked inside and the UAC was accepted. Since i switched to VMware Workstation 14, the script no longer clicks inside the VM successfully. It acts as if it clicks, but it doesn't. 

I tried sending key combinations instead of a click, so that the VM can grab the input, but it also did not work. Every attempt that i made to send clicks or keys from the host inside the VM did not work. I tried using:

MouseClick

ControlClick

MouseMove

_WinAPI_Mouse_Event

_WinAPI_Keybd_Event

 

I also noticed that while the cursor moves to the target which has to be cilcked when my vmware worstation window is not focused, it even doesn't do that when i WinActivate the vmware workstation window first.

 

Did anyone experience such an issue, or maybe could give me a hint, what else i could use to send a key combination or a mouse click in a vmware workstation 14 pro guest window?

 

here is my code, which works with virtualbox:

 

#AutoIt3Wrapper_Icon=".\uac.ico"
#include <ImageSearchSubrogated.au3>

FileInstall(".\ImageSearchDLL.dll", ".\ImageSearchDLL.dll", 0)
FileInstall(".\UAC_ginloSetup.bmp", ".\UAC_ginloSetup.bmp", 0)
FileInstall(".\UAC_Yes.bmp", ".\UAC_Yes.bmp", 0)

; set global variables for the coordinates, which should be delivered
global $x1 = 0, $y1 = 0
global $x2 = 0, $y2 = 0

global $counter1 = 0
global $counter2 = 0

global $sleep = 10000
global $smallSleep = 5000

; execute the script in a loop, so that it will hopefully recover from some unexpected errors
While $counter1 < 1
  checkForImage()
WEnd

#cs ------------

    Functions

#ce ------------

Func checkForImage()
  While $counter2 < 1
    ; search for the UAC in the entire screen - 2 screens supported
    local $searchUac = _ImageSearchArea('UAC_ginloSetup.bmp', 1, -2568, -8, 5136, 1440, $x1, $y1, 0)
    If $searchUac = 1 Then
      ; if the UAC was found search for the Yes button in a an area 200 x 200 from the middle of the found UAC image
      local $searchYes = _ImageSearchArea('UAC_Yes.bmp', 1, $x1, $y1, $x1 + 200, $y1 + 200, $x2, $y2, 0)
      If $searchYes = 1 Then
        ; if the Yes button was found click it and pause the script for $sleep seconds
        MouseClick("left", $x2, $y2, 1,0)
        Sleep($sleep)
      Else
        ; if the Yes button was not found retry from the beginning in $smallSleep seconds
        MsgBox(0, "UAC found error", "UAC was found but the 'Yes' button was not found. Script will retry in " & $smallSleep & " seconds.", $smallSleep)
      EndIf
        ; another way to accept the UAC - via shortcut
        ;Send("{TAB}{TAB}{TAB}{TAB}{TAB}{TAB}")
        ;Send("!y")
    Else
      ; if UAC was not found try again in $sleep seconds
      Sleep($sleep)
    EndIf
  WEnd
  ; if some error occured which expired the loop, pause the script for $sleep seconds
  MsgBox(0, "Error", "Some Error expired the timer and the script could not recover. The script will restart in " & $sleep & " seconds.", $sleep)
EndFunc

 

Share this post


Link to post
Share on other sites
JLogan3o13

@griefman we're not big on helping to automate a way around the UAC, as it is a security feature there for a reason. As this sounds like your own machine running on VMware Workstation, why not temporarily disable UAC? Perhaps if you explain what you are trying to accomplish, we can offer more suggestions (looks like a software install of some sort?)


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
griefman

Hi @JLogan3o13,

Yes, it is a software install. I am a QA Engineer and am testing our installation. For what i think are understandable reasons, we want to make sure that in our tests we cover scenarios as close to the reality as possible and since nobody really deactivates UAC and it does matter to the installer, if it is deactivated or not, we decided to work around it in our tests. However in this particular use case, i think, it is not really about automating the UAC, it is just an example. It is about the sudden impossibility to automate any click or key send from the host to a guest system in vmware workstation 14 pro, which works perfectly on virtualbox. I really hope that somebody else gas experienced such an issue and has a workaround for it. 

Hope i was able to give you the answer, you were looking for ;)

Share this post


Link to post
Share on other sites
Earthshine

Run autoit script inside the vm. Why are you trying to send commands to a vm 


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
griefman

Hey @Earthshine,

There is already automation in the vm. The problem is that not every scenario in our test can be covered with tests inside the vm, so cases like uac or, clicking things, when windows is not runing, or similar scenarios which you can't automate in the guest, have to be automated ot the host. Usually with image recognition.

 

So running automation for these cases is not only not a valid solution, but it us simply impossible. It is a very unusual approach, i know. People would usually disable UAC, or use auto-login or similar things, but there tests, where you just can't do that, otherwise you are not covering everything or you will not be testing the product or the system which you want to support. It makes a lot of sense, when you consider certification, auditing, etc reasons.

Edited by griefman

Share this post


Link to post
Share on other sites
Bilgus

It might that VMware blocks simulated mouse inputs two ideas,

1. run vnc software on the vmware and try interaction through that

2. create 2 small small scripts one on the host that presses a hotkey when mouse clicks and one on the guest the that when it gets a hotkey clicks the mouse at current position assuming your vm accepts keyboard input

  • Like 1

Share this post


Link to post
Share on other sites
JLogan3o13

VMware does indeed make it difficult to do mouse manipulation. If you absolutely must use mouseclicks, I would suggest something like this, which I have had to resort to in the past:

;pseudocode

WinActivate(<VMware Workstation Window>)
Send("^g") ;CTRL+g = Grab Input
<Do whatever else>

 

  • Like 1

√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
Earthshine

never mind

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
JLogan3o13

There have been a couple. But to my knowledge they both deal with the underlying registry to lower/deactivate it. Not sending keystrokes to bypass it.

  • Thanks 1

√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
Earthshine

i just read M23 say it can't be done. that's good enough for me.

however, with .net you can do it apparently

 

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
griefman

Hey everyone, 

 

thanks for the responses! 

@JLogan3o13: this is one of the first thing, which i tried, however as soon as the focus is on the vmware application, sending these keys does not work. i was only able to get the focus in the VM, when i selected this particular control and send the hotkeys for every menu item there. but afterwards i can't send anything in the VM. 

@Earthshine: i know about this solution, and since it is very severely changing the registry of the OS, i would prefer not to do it, unless there is no other way. Thanks!

I tried today to use VMware Workstation 12(the previous version) and it works like a charm. I thought that it might be the enhanced keyboard driver which vmware installs, but it is not the reason. I will stick with VMware Workstation 12 for now in the hope, that somebody will maybe shed some light what is going on and i will also post on the vmware forum.

 

Still keep it coming! I could not have exhausted every idea

Share this post


Link to post
Share on other sites
JLogan3o13

Apologies, @griefman I missed that you were on v. 14. Yes, that version has given me a number of problems; I ended up backing down to 12.5.8


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
Earthshine

well, after all  UAC prompt doesn't open on the current desktop, but rather in a Secure Desktop, which suspends any currently open desktops. It's this way intentionally. However, Trusted processes running as the SYSTEM user can still interact with the Secure Desktop

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine

there is a workaround I have read about. Using C# project using MSTest framework you can make agents and install those to your VMs and they listen on the network for instructions. You can make it so they run at SYSTEM user so it can talk to the Secure Desktop

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
JLogan3o13

That is exactly what you're doing when you run something through PSExec.

  • Like 1

√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
Earthshine

i was just thinking about using PSExec for this too... lol, thanks. I get his pain. I feel it too. I'm not going back to VMware 12, sorry. Onward and forward. I like to test all our stuff with it on as well so we don't get those gotcha's

the ONLY time our apps have trouble by our customers is 99.9% their machine is hosed or misconfigured. We never get any support tickets that things did not install. I mean they always blame me and when I look at all the logs, I see it's usually their system. One rare case a customer had to chcdsk /f and reboot because his drive was messed up. then repaired our install and it worked like a champ. An MSI log informed me of the drive error. It pays to look at ALL the logs in %TEMP% if they are willing to share them. My MSI installers also copy the MSI logs to a product folder under our %ProgramData% area for ease of access when they call us for support. I was going to copy all %temp% *.log and *.txt but I guess that would be illegal, even though I usually need it for the rare circumstances where our stuff does not run after install.

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
griefman

Hi everyone,

 

thanks a lot for your input! The solution with the agents is not a bad one, but too much of an overhead for something that simple. I hope that VMware will fix the issue soon and until then i am sticking with workstation 12 on the machine i need this or if there is a definitive need of 14, i can go over VNC(thanks @Bilgus). 

 

i started a discussion on the vmware forums and the only response i got so far is from somebody, who says that this is a security feature. I just don't agree with him. 

https://communities.vmware.com/message/2757661

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • davidacrozier
      By davidacrozier
      Hello all ~
      I am running an autoit script on Windows 10 inside VMware Workstation 12 Pro version 12.5.2.  Technically I am remoting into ESXi which has a Domain Controller (DC), WebServer, FilServer, Windows 10, etc.  Using the GUI (i.e. running explorer.exe) I am able to open several different folders successfully.  The desktop, documents, USB external all open without issue.  The network share opening gives me issues.  Whenever I attempt to open \\filserver\users\user\sharedfolder I get the documents folder instead.  I understand that the documents folder is the default for explorer.  I have also attempted to use the letter drive mapped to the network share (Z:) and receive the same result.  When I run this script on Windows 10 alone without  the VM or the ESXi I am able to open the network share without problems.  I have tried to use the net use command to designate a letter M: to the network share folder prior to running the script.  This did not work for me.  
      One additional avenue I think might work is to use the systreeview321 and _GUICtrlTreeView_FindItem to step through the tree looking for the network share.  Once found,  double click on it and see if that opens the shared network folder.  I can click inside the VM with my mouse on the network share and it opens just fine.  Not sure if running up against GUI issues, or permission issues, or what?
      Thanks in advance,
      Davida Crozier
      TestNetworkShare.au3
      This script is a subset of a much larger program, but it illustrates what I am dealing with.

    • CygnusX1
      By CygnusX1
      Hello, 
      I have a dialog box that pops up during an install on a foreign OS. In this case Spanish. The dialog displays a message the program being installed needs to install .Net, CRT, and C++. When inspected the dialog it seems normal. I can get control IDs, Text, Position, everything. But when I try to have AutoIt interact with it e.g. move it, click a button, click on a control, ControlGetFocus, Send("{TAB}"), ControlClick(). Nothing works. What even weirder is I can WinActivate the dialog but then can't move the mouse with MouseMove. If I don't WinActivate(), I can use the MouseMove() function. I can not figure out how to interact with the dialog box.
      Any thoughts on a way to interact with this dialog box?
      Thanks.
       
       
       

    • tcurran
      By tcurran
      Here's a short UDF that will, at least in most cases, detect whether a window can be copied from or pasted to programmatically--for example, by Send()ing ctl-c, ctl-v. This is often disabled when programs (like your AutoIt script) run at a lower UAC integrity level than the application they are trying to operate on.
      #include <WinAPI.au3> Func _WindowIsPasteable($handle) ;accepts window handle; returns true or false whether a window will accept Ctl-C, Ctl-V Local $bCanPaste = True Local $hTestWindowPID = 0 Local $hTestWindowTID = _WinAPI_GetWindowThreadProcessId($handle, $hTestWindowPID) _WinAPI_AttachThreadInput(_WinAPI_GetCurrentThreadId(), $hTestWindowTID, True);attach to window we want to paste into $bCanPaste = _WinAPI_GetFocus() ;Test whether window is paste-able--returns False if it is not _WinAPI_AttachThreadInput(_WinAPI_GetCurrentThreadId, $hTestWindowTID, False);detach from window thread Return $bCanPaste EndFunc Pass it a window handle; it returns true or false whether a window will accept programmatic pasting. The function may not work on the CMD window, since it handles the clipboard uniquely.
      This function works by attaching to the program thread of the window whose handle it receives, then attempting to perform a GetFocus on that thread. In most cases, the attempt will fail if the window will not accept programmatic copy-paste.
    • dreivilo47
      By dreivilo47
      When I use the following code I receive an UAC message:
       
      #RequireAdmin RunWait("msiexec /i winzip205-64.msi /quiet") Exit How can I hide (bypass) the UAC message?
    • ur
      By ur
      With the below command I am able to revert a virtual machine in vmware workstation to a snapshot mentioned.
      $sVMPath = <path of the vmx file of virtual machine> $sSnapshot = <snapshot name> $Ret = RunWait(@ComSpec & " /c VMRun.exe -T WS revertToSnapshot ""$sVMPath$"" ""$sSnapshot$"" nogui", "", @SW_HIDE) But the problem here is the virtual machine may contain multiple snapshots with same name because multiple users use same workstation and eventually they are keeping same name sometimes like below or in attachment.
      Both snapshots are with same name and in GUI we can use them with no issue.

       
      But with autoit or with vmrun command line it is showing error as 
      Error: The name does not uniquely identify one snapshot Could anyone please help me on this.
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.