Jump to content

Sending keys and mouse clicks inside a VMWare VM


Recommended Posts

Hi everyone,

i am writing to you after a very long struggle i had while trying to figure out how to send a simple click inside a virtual machine running in vmware workstation 14.

i have an autoit script running on my host machine watching for the UAC prompt to be displayed in a running vm. Both the host and the guest OS are Windows 10. This script worked perfectly with virtual box. It recognized the UAC prompt and clicked inside and the UAC was accepted. Since i switched to VMware Workstation 14, the script no longer clicks inside the VM successfully. It acts as if it clicks, but it doesn't. 

I tried sending key combinations instead of a click, so that the VM can grab the input, but it also did not work. Every attempt that i made to send clicks or keys from the host inside the VM did not work. I tried using:

MouseClick

ControlClick

MouseMove

_WinAPI_Mouse_Event

_WinAPI_Keybd_Event

 

I also noticed that while the cursor moves to the target which has to be cilcked when my vmware worstation window is not focused, it even doesn't do that when i WinActivate the vmware workstation window first.

 

Did anyone experience such an issue, or maybe could give me a hint, what else i could use to send a key combination or a mouse click in a vmware workstation 14 pro guest window?

 

here is my code, which works with virtualbox:

 

#AutoIt3Wrapper_Icon=".\uac.ico"
#include <ImageSearchSubrogated.au3>

FileInstall(".\ImageSearchDLL.dll", ".\ImageSearchDLL.dll", 0)
FileInstall(".\UAC_ginloSetup.bmp", ".\UAC_ginloSetup.bmp", 0)
FileInstall(".\UAC_Yes.bmp", ".\UAC_Yes.bmp", 0)

; set global variables for the coordinates, which should be delivered
global $x1 = 0, $y1 = 0
global $x2 = 0, $y2 = 0

global $counter1 = 0
global $counter2 = 0

global $sleep = 10000
global $smallSleep = 5000

; execute the script in a loop, so that it will hopefully recover from some unexpected errors
While $counter1 < 1
  checkForImage()
WEnd

#cs ------------

    Functions

#ce ------------

Func checkForImage()
  While $counter2 < 1
    ; search for the UAC in the entire screen - 2 screens supported
    local $searchUac = _ImageSearchArea('UAC_ginloSetup.bmp', 1, -2568, -8, 5136, 1440, $x1, $y1, 0)
    If $searchUac = 1 Then
      ; if the UAC was found search for the Yes button in a an area 200 x 200 from the middle of the found UAC image
      local $searchYes = _ImageSearchArea('UAC_Yes.bmp', 1, $x1, $y1, $x1 + 200, $y1 + 200, $x2, $y2, 0)
      If $searchYes = 1 Then
        ; if the Yes button was found click it and pause the script for $sleep seconds
        MouseClick("left", $x2, $y2, 1,0)
        Sleep($sleep)
      Else
        ; if the Yes button was not found retry from the beginning in $smallSleep seconds
        MsgBox(0, "UAC found error", "UAC was found but the 'Yes' button was not found. Script will retry in " & $smallSleep & " seconds.", $smallSleep)
      EndIf
        ; another way to accept the UAC - via shortcut
        ;Send("{TAB}{TAB}{TAB}{TAB}{TAB}{TAB}")
        ;Send("!y")
    Else
      ; if UAC was not found try again in $sleep seconds
      Sleep($sleep)
    EndIf
  WEnd
  ; if some error occured which expired the loop, pause the script for $sleep seconds
  MsgBox(0, "Error", "Some Error expired the timer and the script could not recover. The script will restart in " & $sleep & " seconds.", $sleep)
EndFunc

 

Link to post
Share on other sites
  • Moderators

@griefman we're not big on helping to automate a way around the UAC, as it is a security feature there for a reason. As this sounds like your own machine running on VMware Workstation, why not temporarily disable UAC? Perhaps if you explain what you are trying to accomplish, we can offer more suggestions (looks like a software install of some sort?)

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Link to post
Share on other sites

Hi @JLogan3o13,

Yes, it is a software install. I am a QA Engineer and am testing our installation. For what i think are understandable reasons, we want to make sure that in our tests we cover scenarios as close to the reality as possible and since nobody really deactivates UAC and it does matter to the installer, if it is deactivated or not, we decided to work around it in our tests. However in this particular use case, i think, it is not really about automating the UAC, it is just an example. It is about the sudden impossibility to automate any click or key send from the host to a guest system in vmware workstation 14 pro, which works perfectly on virtualbox. I really hope that somebody else gas experienced such an issue and has a workaround for it. 

Hope i was able to give you the answer, you were looking for ;)

Link to post
Share on other sites

Hey @Earthshine,

There is already automation in the vm. The problem is that not every scenario in our test can be covered with tests inside the vm, so cases like uac or, clicking things, when windows is not runing, or similar scenarios which you can't automate in the guest, have to be automated ot the host. Usually with image recognition.

 

So running automation for these cases is not only not a valid solution, but it us simply impossible. It is a very unusual approach, i know. People would usually disable UAC, or use auto-login or similar things, but there tests, where you just can't do that, otherwise you are not covering everything or you will not be testing the product or the system which you want to support. It makes a lot of sense, when you consider certification, auditing, etc reasons.

Edited by griefman
Link to post
Share on other sites

It might that VMware blocks simulated mouse inputs two ideas,

1. run vnc software on the vmware and try interaction through that

2. create 2 small small scripts one on the host that presses a hotkey when mouse clicks and one on the guest the that when it gets a hotkey clicks the mouse at current position assuming your vm accepts keyboard input

Link to post
Share on other sites
  • Moderators

VMware does indeed make it difficult to do mouse manipulation. If you absolutely must use mouseclicks, I would suggest something like this, which I have had to resort to in the past:

;pseudocode

WinActivate(<VMware Workstation Window>)
Send("^g") ;CTRL+g = Grab Input
<Do whatever else>

 

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Link to post
Share on other sites
  • Moderators

There have been a couple. But to my knowledge they both deal with the underlying registry to lower/deactivate it. Not sending keystrokes to bypass it.

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Link to post
Share on other sites

Hey everyone, 

 

thanks for the responses! 

@JLogan3o13: this is one of the first thing, which i tried, however as soon as the focus is on the vmware application, sending these keys does not work. i was only able to get the focus in the VM, when i selected this particular control and send the hotkeys for every menu item there. but afterwards i can't send anything in the VM. 

@Earthshine: i know about this solution, and since it is very severely changing the registry of the OS, i would prefer not to do it, unless there is no other way. Thanks!

I tried today to use VMware Workstation 12(the previous version) and it works like a charm. I thought that it might be the enhanced keyboard driver which vmware installs, but it is not the reason. I will stick with VMware Workstation 12 for now in the hope, that somebody will maybe shed some light what is going on and i will also post on the vmware forum.

 

Still keep it coming! I could not have exhausted every idea

Link to post
Share on other sites
  • Moderators

Apologies, @griefman I missed that you were on v. 14. Yes, that version has given me a number of problems; I ended up backing down to 12.5.8

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Link to post
Share on other sites

well, after all  UAC prompt doesn't open on the current desktop, but rather in a Secure Desktop, which suspends any currently open desktops. It's this way intentionally. However, Trusted processes running as the SYSTEM user can still interact with the Secure Desktop

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to post
Share on other sites

there is a workaround I have read about. Using C# project using MSTest framework you can make agents and install those to your VMs and they listen on the network for instructions. You can make it so they run at SYSTEM user so it can talk to the Secure Desktop

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to post
Share on other sites

i was just thinking about using PSExec for this too... lol, thanks. I get his pain. I feel it too. I'm not going back to VMware 12, sorry. Onward and forward. I like to test all our stuff with it on as well so we don't get those gotcha's

the ONLY time our apps have trouble by our customers is 99.9% their machine is hosed or misconfigured. We never get any support tickets that things did not install. I mean they always blame me and when I look at all the logs, I see it's usually their system. One rare case a customer had to chcdsk /f and reboot because his drive was messed up. then repaired our install and it worked like a champ. An MSI log informed me of the drive error. It pays to look at ALL the logs in %TEMP% if they are willing to share them. My MSI installers also copy the MSI logs to a product folder under our %ProgramData% area for ease of access when they call us for support. I was going to copy all %temp% *.log and *.txt but I guess that would be illegal, even though I usually need it for the rare circumstances where our stuff does not run after install.

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to post
Share on other sites

Hi everyone,

 

thanks a lot for your input! The solution with the agents is not a bad one, but too much of an overhead for something that simple. I hope that VMware will fix the issue soon and until then i am sticking with workstation 12 on the machine i need this or if there is a definitive need of 14, i can go over VNC(thanks @Bilgus). 

 

i started a discussion on the vmware forums and the only response i got so far is from somebody, who says that this is a security feature. I just don't agree with him. 

https://communities.vmware.com/message/2757661

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By ThePappas
      I am feeling really stupid. I cannot figure out what I am doing wrong.
      I followed the "Winzip" example, and I cannot get autoit to simply send an enter when the first window appears.  There are 2 windows like that, and neither one gets the enter.
      ShellExecute("VMware-EnhancedAuthenticationPlugin-6.7.0.exe") WinWaitActive("VMware Plug-in Service Installer.","instances will be closed") Send("{ENTER}") WinWaitActive("VMware Plug-in Service Installer.","run in succession") Send("{ENTER}") WinWaitActive("VMware Enhanced Authentication Plug-in 6.7.0","Welcome to the installation wizard") Send("!n") WinWaitActive("VMware Enhanced Authentication Plug-in 6.7.0","End-User License Agreement") Send("!a!n!n!i") WinWaitActive("VMware Enhanced Authentication Plug-in 6.7.0","Change") Send("!n") WinWaitActive("VMware Enhanced Authentication Plug-in 6.7.0","Install") Send("!i") WinWaitActive("VMware Enhanced Authentication Plug-in 6.7.0","Finish") Send("!f") WinWaitActive("VMware Plug-in Service","Welcome to the installation wizard") Send("!n") WinWaitActive("VMware Plug-in Service", "End-User License Agreement") Send("!a!n") WinWaitActive("VMware Plug-in Service", "Install the VMware Plug-in Service in") Send("!n") WinWaitActive("VMware Plug-in Service", "Install") Send("!i") WinWaitActive("VMware Plug-in Service","Finish") Send("f") I used the Window ID tool to verify that I am using the correct title, and those substrings appear in the dialog.
      I really did not want to have to post here, as this seems like a trivial effort.  But I am stuck.  I hate stuck.
      Thanks!
    • By Jemboy
      I use Vmware Workstation to run servers, client and other devices. The reason I use Workstation instead of Esxi is because I find it easier to copy the VM files when the host machine crashes (just unplug *raid 1* disk and add it to the new host). 
      Because I do not want to reinstall everytime, I often copy an existing VM to test something  or just use the copy a base to install other apps.

      In the past I used to use VmConverter, but the latest version won't copy/rename my latest VMs anymore and the older VmConverters do not support the last VM-format.
      So  I decided to make my own "VmConverter" and called it "VMRenamer".
      I post this as is (it's working for me) and I encourage others to enhance the program as needed.
      Pre-requisites:
      You have to have VMWorkstation installed (I asume on a X64 Windows) You may NOT have snapshotted the VMware in ANY way, even after deleting the snapshot the VM is changed to much for my program to reliabily rename/change the VM files. I have used some snippets from other in my source and I wish to thank them for posting those on the forum/internet.
      Some remarks or obsolute functions have not been removed, because I wanted to post this for you out there instead of delaying the post longer.
      Enjoy!
       
      VmRenamer_1Q_2.zip
    • By rudi
      Hi,
      When a non compiled AU3 script is run with #RequireAdmin, then if the UAC prompt can be authorized due to the fact, that the currently loggedon user has local admin rights, then the macro @UserProfileDir correctly reflects the profile dir of the user of the windows logon session.
       
      When the script with #RequireAdmin is started by a "normal user" without local admin rights, and I use a domain admin account to authorize the UAC prompt, then @UserProfileDir reflects the profile dir belonging to the AD-Admin account.
      As the script originally was started using the "regular user" I'm wondering, if there is a chance to "pass" the original user's @UserProfileDir to the UAC elevated script?
       
      As playing around with this feature I realize, that I basically don't know the exact mechanism of the UAC elevation authorization process:
      The script is started by right mouse click, execute script This is invoking e.g. "C:\Program Files (x86)\AutoIt3\AutoIt3.exe" "C:\Users\Rudi\Desktop\test.au3" as by this registy value: Windows Registry Editor Version 5.00 [HKEY_CLASSES_ROOT\AutoIt3Script\Shell\Run\Command] @="\"C:\\Program Files (x86)\\AutoIt3\\AutoIt3.exe\" \"%1\" %*" But what I honestly don't know is, how does the UAC propt interact in the program startup? I guess, that Autoit3.exe is parsing the AU3 source, is seeing the #RequireAdmin and then "relaunches itself with the AU3 as %1" requesting UAC elevated rights "from windows"??? With Process Explorer I can see, that The commandline then is this one with a "!" before "%1"
      "C:\Program Files (x86)\AutoIt3\AutoIt3.exe" !"C:\Users\Rudi\Desktop\test.au3"  It it should be something like this, then it might be possible to pass the original @UserProfileDir to the second, UAC elevated "Startup"??? <edit>
      I just noticed:
      When I use "WIN+R" and then directly use the command line, I see in Process Explorer, ...
      "C:\Program Files (x86)\AutoIt3\AutoIt3.exe" !"C:\Users\Rudi\Desktop\test.au3"
      ... then this script with #RequireAdmin is started *WITHOUT* UAC elevation.
      Guessing, that this ! is just reverting #RequireAdmin I tried the "opposite" one as well:
      AU3 script without #RequireAdmin Starting with "C:\Program Files (x86)\AutoIt3\AutoIt3.exe" !"C:\Users\Rudi\Desktop\test.au3" does not invoke UAC elevation prompt. So to me it looks like, this ! is a "status flag from Autoit3.exe to Autoit3.exe", that the elevation process was done already? amazing...
      the topic Autoit on Windows Vista is telling no details of  this UAC process...
      </edit>
       
      Regards, Rudi.
    • By JohnWIlling
      We are running on a VIrtual Machine via Remote Desktop.  We are having a situation where the loading of the chromedriver.exe is taking longer on the VM than on a physical machine.  It is causing the first call to  WD_Post in _WD_CreateSession to fail.
      To workaround the Issue, I used a simplified version of (_WinAPI_GetTcpTable) UDF to Loop waiting for the driver port to be in the listening state.  At that point, I know that the Post will work:
      Func _WaitForPort($ListenPort) Do Local $tMIB_TCPTABLE = DllStructCreate("dword[6]") Local $aRet = DllCall("Iphlpapi.dll", "DWORD", "GetTcpTable", "struct*", $tMIB_TCPTABLE, "DWORD*", 0, "BOOL", True) Local $dwSize = $aRet[2] $tMIB_TCPTABLE = DllStructCreate("DWORD[" & $dwSize / 4 & "]") $aRet = DllCall("Iphlpapi.dll", "DWORD", "GetTcpTable", "struct*", $tMIB_TCPTABLE, "DWORD*", $dwSize, "BOOL", True) If $aRet[0] <> 0 Then Return SetError(1) Local $iNumEntries = DllStructGetData($tMIB_TCPTABLE, 1, 1) Local $aRes[$iNumEntries][6] For $i = 0 To $iNumEntries - 1 Local $Type = DllStructGetData($tMIB_TCPTABLE, 1, 2 + $i * 5 + 0) $aRet = DllCall("ws2_32.dll", "ushort", "ntohs", "uint", DllStructGetData($tMIB_TCPTABLE, 1, 2 + $i * 5 + 2)) ; local port Local $Port = $aRet[0] ConsoleWrite("Checking Row:" & $i & " State:<" & $Type & "> Port:<" & $Port & ">" & @CRLF) ; Type 2 is LISTENING If $Type = 2 And $Port = $ListenPort Then Return 1; EndIF Next Sleep(500); Until 0; Return 0; EndFunc ;==>_WaitForPort  
    • By ur
      Which Powershell command in the PowerCLI module for VMware ESX used to interact with UI apps?
       
      When I launch any exe/any exeutable using powercli on guest VM using powercli command.
      Invoke-VMScript, I am able to run them in the background but not in the foreground.
       
      i.e., UI apps are not launching but showing the background as running in the task manager.
       
      We need our UI Automation scripts to execute in the VM, but it is not working.
       
      We are able to do in virtualbox and hyper-v but not in vmware esx using powercli.
       
      Please suggest.
×
×
  • Create New...