Leo1906

Differentiate between Mouseclick and Touch-Click

1 post in this topic

#1 ·  Posted (edited)

Hello there :D

I recently wanted to know how to differentiate between a mouse click and a touch click on controls in your GUI.
This can be interesting when using picture controls as buttons in combination with this UDF here.

No one could help me, so I tried my best to get it to work. And it works now! Unfortunately only for Windows 8 and above, as I had to use functions which are unavailable for older OS.
Additionally this was my first time writing DLLCalls myself. I hope (and think) that there are no mayor mistakes in it,

 

So here's the sample code:

#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <GDIPlus.au3>

OnAutoItExitRegister("_exit")

$play = 'iVBORw0KGgoAAAANSUhEUgAAABkAAAAZCAYAAAGz7rX1AAAACXBIWXMAAAsTAAALEwEAmpwYAAAKT2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AUkSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXXPues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgABeNMLCADATZvAMByH/w/qQplcAYCEAcB0kThLCIAUAEB6jkKmAEBGAYCdmCZTAKAEAGDLY2LjAFAtAGAnf+bTAICd+Jl7AQBblCEVAaCRACATZYhEAGg7AKzPVopFAFgwABRmS8Q5ANgtADBJV2ZIALC3AMDOEAuyAAgMADBRiIUpAAR7AGDIIyN4AISZABRG8lc88SuuEOcqAAB4mbI8uSQ5RYFbCC1xB1dXLh4ozkkXKxQ2YQJhmkAuwnmZGTKBNA/g88wAAKCRFRHgg/P9eM4Ors7ONo62Dl8t6r8G/yJiYuP+5c+rcEAAAOF0ftH+LC+zGoA7BoBt/qIl7gRoXgugdfeLZrIPQLUAoOnaV/Nw+H48PEWhkLnZ2eXk5NhKxEJbYcpXff5nwl/AV/1s+X48/Pf14L7iJIEyXYFHBPjgwsz0TKUcz5IJhGLc5o9H/LcL//wd0yLESWK5WCoU41EScY5EmozzMqUiiUKSKcUl0v9k4t8s+wM+3zUAsGo+AXuRLahdYwP2SycQWHTA4vcAAPK7b8HUKAgDgGiD4c93/+8//UegJQCAZkmScQAAXkQkLlTKsz/HCAAARKCBKrBBG/TBGCzABhzBBdzBC/xgNoRCJMTCQhBCCmSAHHJgKayCQiiGzbAdKmAv1EAdNMBRaIaTcA4uwlW4Dj1wD/phCJ7BKLyBCQRByAgTYSHaiAFiilgjjggXmYX4IcFIBBKLJCDJiBRRIkuRNUgxUopUIFVIHfI9cgI5h1xGupE7yAAygvyGvEcxlIGyUT3UDLVDuag3GoRGogvQZHQxmo8WoJvQcrQaPYw2oefQq2gP2o8+Q8cwwOgYBzPEbDAuxsNCsTgsCZNjy7EirAyrxhqwVqwDu4n1Y8+xdwQSgUXACTYEd0IgYR5BSFhMWE7YSKggHCQ0EdoJNwkDhFHCJyKTqEu0JroR+cQYYjIxh1hILCPWEo8TLxB7iEPENyQSiUMyJ7mQAkmxpFTSEtJG0m5SI+ksqZs0SBojk8naZGuyBzmULCAryIXkneTD5DPkG+Qh8lsKnWJAcaT4U+IoUspqShnlEOU05QZlmDJBVaOaUt2ooVQRNY9aQq2htlKvUYeoEzR1mjnNgxZJS6WtopXTGmgXaPdpr+h0uhHdlR5Ol9BX0svpR+iX6AP0dwwNhhWDx4hnKBmbGAcYZxl3GK+YTKYZ04sZx1QwNzHrmOeZD5lvVVgqtip8FZHKCpVKlSaVGyovVKmqpqreqgtV81XLVI+pXlN9rkZVM1PjqQnUlqtVqp1Q61MbU2epO6iHqmeob1Q/pH5Z/YkGWcNMw09DpFGgsV/jvMYgC2MZs3gsIWsNq4Z1gTXEJrHN2Xx2KruY/R27iz2qqaE5QzNKM1ezUvOUZj8H45hx+Jx0TgnnKKeX836K3hTvKeIpG6Y0TLkxZVxrqpaXllirSKtRq0frvTau7aedpr1Fu1n7gQ5Bx0onXCdHZ4/OBZ3nU9lT3acKpxZNPTr1ri6qa6UbobtEd79up+6Ynr5egJ5Mb6feeb3n+hx9L/1U/W36p/VHDFgGswwkBtsMzhg8xTVxbzwdL8fb8VFDXcNAQ6VhlWGX4YSRudE8o9VGjUYPjGnGXOMk423GbcajJgYmISZLTepN7ppSTbmmKaY7TDtMx83MzaLN1pk1mz0x1zLnm+eb15vft2BaeFos'
$play &= 'tqi2uGVJsuRaplnutrxuhVo5WaVYVVpds0atna0l1rutu6cRp7lOk06rntZnw7Dxtsm2qbcZsOXYBtuutm22fWFnYhdnt8Wuw+6TvZN9un2N/T0HDYfZDqsdWh1+c7RyFDpWOt6azpzuP33F9JbpL2dYzxDP2DPjthPLKcRpnVOb00dnF2e5c4PziIuJS4LLLpc+Lpsbxt3IveRKdPVxXeF60vWdm7Obwu2o26/uNu5p7ofcn8w0nymeWTNz0MPIQ+BR5dE/C5+VMGvfrH5PQ0+BZ7XnIy9jL5FXrdewt6V3qvdh7xc+9j5yn+M+4zw33jLeWV/MN8C3yLfLT8Nvnl+F30N/I/9k/3r/0QCngCUBZwOJgUGBWwL7+Hp8Ib+OPzrbZfay2e1BjKC5QRVBj4KtguXBrSFoyOyQrSH355jOkc5pDoVQfujW0Adh5mGLw34MJ4WHhVeGP45wiFga0TGXNXfR3ENz30T6RJZE3ptnMU85ry1KNSo+qi5qPNo3ujS6P8YuZlnM1VidWElsSxw5LiquNm5svt/87fOH4p3iC+N7F5gvyF1weaHOwvSFpxapLhIsOpZATIhOOJTwQRAqqBaMJfITdyWOCnnCHcJnIi/RNtGI2ENcKh5O8kgqTXqS7JG8NXkkxTOlLOW5hCepkLxMDUzdmzqeFpp2IG0yPTq9MYOSkZBxQqohTZO2Z+pn5mZ2y6xlhbL+xW6Lty8elQfJa7OQrAVZLQq2QqboVFoo1yoHsmdlV2a/zYnKOZarnivN7cyzytuQN5zvn//tEsIS4ZK2pYZLVy0dWOa9rGo5sjxxedsK4xUFK4ZWBqw8uIq2Km3VT6vtV5eufr0mek1rgV7ByoLBtQFr6wtVCuWFfevc1+1dT1gvWd+1YfqGnRs+FYmKrhTbF5cVf9go3HjlG4dvyr+Z3JS0qavEuWTPZtJm6ebeLZ5bDpaql+aXDm4N2dq0Dd9WtO319kXbL5fNKNu7g7ZDuaO/PLi8ZafJzs07P1SkVPRU+lQ27tLdtWHX+G7R7ht7vPY07NXbW7z3/T7JvttVAVVN1WbVZftJ+7P3P66Jqun4lvttXa1ObXHtxwPSA/0HIw6217nU1R3SPVRSj9Yr60cOxx++/p3vdy0NNg1VjZzG4iNwRHnk6fcJ3/ceDTradox7rOEH0x92HWcdL2pCmvKaRptTmvtbYlu6T8w+0dbq3nr8R9sfD5w0PFl5SvNUyWna6YLTk2fyz4ydlZ19fi753GDborZ752PO32oPb++6EHTh0kX/i+c7vDvOXPK4dPKy2+UTV7hXmq86X23qdOo8/pPTT8e7nLuarrlca7nuer21e2b36RueN87d9L158Rb/1tWeOT3dvfN6b/fF9/XfFt1+cif9zsu72Xcn7q28T7xf9EDtQdlD3YfVP1v+3Njv3H9qwHeg89HcR/cGhYPP/pH1jw9DBY+Zj8uGDYbrnjg+OTniP3L96fynQ89kzyaeF/6i/suuFxYvfvjV69fO0ZjRoZfyl5O/bXyl/erA6xmv28bCxh6+yXgzMV70VvvtwXfcdx3vo98PT+R8IH8o/2j5sfVT0Kf7kxmTk/8EA5jz/GMzLdsAAAAgY0hSTQAAeiUAAICDAAD5/wAAgOkAAHUwAADqYAAAOpgAABdvkl/FRgAAAaFJREFUeNpi/P//PwMKgAr8h7GZGNAAI7oWAAAAAP//whCYD9PPhFMZAAAA//+CWTOPgYEhEt16mDZGBgaGZVCz/jMwMExhYGBgYEFSnMrAwDAHWTcAAAD//8JwClZvYbMLJrkPmw44xhYgJ6AuRXEZI1GuAgAAAP//wu1cfOD///8wx81HctwPBgYGQVxqmXCYxc7AwPAOyZBUZEkmIh3EjcxhwaHoNwMDgywDA8NLqgUEAAAA//8iWRMTJUH9n4GB4QFBdWgaYHgCqRpg2JRUDf8ZGBieMDAwMMPUwUOJkZHxPwG/MhIbSm+gSQdrKKFjO2L9MIPY'
$play &= 'UHqOy5kYnqZZ0gAAAAD//yIv65Kb/pD8H8jAwHCBgYGhg4GBgYcqZmKxZD6WaNzFwMCgTa4lxIavKwMDwxWohY8YGBiiaJu/IUXSUqiFvxkYGCZiK0UptQS9fMxDKnUPMjAwmFPbEnTAxcDAwEFtS/5D629JaH1nCvUNAzHVAT7wmoGBoQZatf8jNkyJAUcZGBgKGBgYzlArM0ZCK4WphFINsWbSpVgBDADB5DrGESuD0wAAAABJRU5ErkJggg=='
$play = Binary(_WinAPI_Base64Decode($play))


$Form1 = GUICreate("Form1", 444, 372, -1, -1)
$pic1 = GUICtrlCreatePic("", 184, 112, 65, 57)
$pic1_dummy = GUICtrlCreateDummy()

_GDIPlus_Startup()
Global $play1 = _GDIPlus_BitmapCreateFromMemory($play, True)
_WinAPI_DeleteObject(GUICtrlSendMsg($pic1, $STM_SETIMAGE, $IMAGE_BITMAP, $play1))

DllCall("User32.dll", "bool", "RegisterTouchHitTestingWindow", "HWND", $Form1, "ULONG", 1)  ; register your GUI for recieving commands relatet to WM_TOUCHHITTESTING
GUIRegisterMsg(0x024D, "WM_TOUCHHITTESTING")

GUISetState(@SW_SHOW)

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $pic1_dummy
            MsgBox(0, 0, "touch-click")
        Case $Pic1
            MsgBox(0, 0, "Mouseclick")
    EndSwitch
WEnd

Func WM_TOUCHHITTESTING($hWnd, $iMsg, $wParam, $lParam)
    Local $send = 1
    Local $pic1_pos = _WinAPI_GetWindowRect(GUICtrlGetHandle($pic1))
    Local $struct = DllStructCreate("uint;long X;long Y")   ; first Parameter gives the accuracy of which this control is meant by a touch - second and third are the x/y points of the touch (not relative to GUI)

    DllCall("User32.dll", "bool", "EvaluateProximityToRect", "ptr", DllStructGetPtr($pic1_pos), "ptr", $lParam, "ptr", DllStructGetPtr($struct))
    If DllStructGetData($struct, 1) < 20 Then               ; if the proximity of the touch is near enough to the control (I chose 20, but you can vary yourself) then do something
        GUICtrlSendToDummy($pic1_dummy)                     ; you shouldn't do very intensive stuff inside the function itself. Outsource it to your main loop.
        $send = 0                                           ; no need to pass the command on to your GUI, as you take action yourself ..
    EndIf
    If $send Then _WinAPI_DefWindowProc($hWnd, $iMsg, $wParam, $lParam)
EndFunc



Func _WinAPI_Base64Decode($sB64String)
    Local $aCrypt = DllCall("Crypt32.dll", "bool", "CryptStringToBinaryA", "str", $sB64String, "dword", 0, "dword", 1, "ptr", 0, "dword*", 0, "ptr", 0, "ptr", 0)
    If @error Or Not $aCrypt[0] Then Return SetError(1, 0, "")
    Local $bBuffer = DllStructCreate("byte[" & $aCrypt[5] & "]")
    $aCrypt = DllCall("Crypt32.dll", "bool", "CryptStringToBinaryA", "str", $sB64String, "dword", 0, "dword", 1, "struct*", $bBuffer, "dword*", $aCrypt[5], "ptr", 0, "ptr", 0)
    If @error Or Not $aCrypt[0] Then Return SetError(2, 0, "")
    Return DllStructGetData($bBuffer, 1)
EndFunc

Func _exit()
    _WinAPI_DeleteObject($play1)
    _GDIPlus_Shutdown()
EndFunc

The magic happens in the WM_TOUCHHITTESTING function.
The _WinAPI_Base64Decode is written by @UEZ (as well as the way to use PNG as button (from binary) - thanks for that!)

On Windows 7 there is the WM_TOUCH function which should handle touch events, but I couldn't get it to work on controls ..??
If you will get the x/y coordinates of a touch (even when you touch on controls) you could make the "EvaluateProximityToRect" check yourself - as this function is not available in Windows 7. Shouldn't be that hard.

Btw.: this here is the MSDN articel related to this topic: https://msdn.microsoft.com/en-us/library/windows/desktop/hh454931(v=vs.85).aspx

So thats it. Any questions and comments welcome! :)

greetz,

Leo

Edited by Leo1906
added msdn link
1 person likes this

Share this post


Link to post
Share on other sites



Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • chacoya121
      By chacoya121
      can anyone help me plz
      how can you put mouseclick in if statement or is it possible
      example:
      if Mouseclick = "left" then Tooltip(" you click left mouse") thank you
    • xgjengen
      By xgjengen
      Hey guys!
      Im new when it comes to scripting and i follow some videos on how to make clicks and such. So i will say i know the basics alrady.
      However, im having a really big problem
      When i run the script is works fine with the clicks and all. But if i run it one or two more times it just moves to very inaccurate positions. YES, i have tried mulitple times to change positions and such but no luck. My script is working its just so inaccurate that im unable to use it. Are there any solution for this?
      I have also set the camera to a specific position. I have also tried other camera angles, no luck there either. Making more smoother macro movements does not seem to work aswell.
      I have also tried to have clicks away from the minimap, since i know some youtuber had problem with inaccuracy because of this.
      I came here because i was googling and did not see anyone else having this problem, or mentioning it. So if someone have a solution, i would be happy!! All i have seen are people trying to change the resolution. But it has nothing to do with that. I heard someone mention controlmouseclick. However are there any differences? I have also A window selection in my script so it changes to the window the macro will be running in. Resolution is not changing or anything. I have also tried other macros and same problem there.
       
      Also does anyone know or can teach me how pixel search works? I want the macro to be able to click at the correct character instead of waiting for the character to get in the position its searching for.
    • Gordi300
      By Gordi300
      Hey everyone!! 
      I'm trying to make a program in VB.NET that click inside a picturebox of an app without moving the mouse (using ControlClick).
      My code is this:
      1) ControlClick("Form1", "", "[CLASS:PictureBox; INSTANCE:12]", "left", 1, 5, 5) 2) ControlClick("Form1", "", "[CLASS:PictureBox; INSTANCE:1]", "left", 1, 5, 5) The code is correct  but the problem is that 1) works perfectly and 2) doesn't work. 
      '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
      Here I leave you the summary of each picturebox:
      >>>> Window <<<<
      Title:    Form1
      Class:    Form
      Position:    168, 0
      Size:    1030, 780
      Style:    0x16CA0000
      ExStyle:    0x00040100
      Handle:    0x00000000000B0092
       
      >>>> Control <<<<                PICTUREBOX 12
      Class:    PictureBox
      Instance:    12
      ClassnameNN:    PictureBox12
      Name:    
      Advanced (Class):    [CLASS:PictureBox; INSTANCE:12]
      ID:    28
      Text:    
      Position:    305, 721
      Size:    40, 40
      Style:    0x56010000
      ExStyle:    0x00000004
      Handle:    0x0000000000040520
       
      >>>> Control <<<<                PICTUREBOX 1
      Class:    PictureBox
      Instance:    1
      ClassnameNN:    PictureBox1
      Name:    
      Advanced (Class):    [CLASS:PictureBox; INSTANCE:1]
      ID:    4
      Text:    
      Position:    4, 176
      Size:    700, 533
      Style:    0x56010000
      ExStyle:    0x00000004
      Handle:    0x00000000000E02BE
       
      Thank you for you help!!!
    • Dequality
      By Dequality
      I'm currently wondering if there is somehow to actually make autoit click inside a minimized program? & sent keystrokes ? 
      2nd question, how can i make it do a mouse click at e.g. x200,y200 while the are at like x354,y313 (This is purely an example.) would love to be able to do other shizz on my pc while the script is running if possible.
      If this ISNT possible can u recommend a language / other scripting program which are able to do this ? willing to start over learning a new coding/scripting language.
       
      -Dequality.
    • DiveHigh
      By DiveHigh
      Hello There!  
      I've encountered a small issue i just cant seem to program my way out of...

      What im doing is:        ;or rather, what im trying to do is
      Pixelsearch for a color, within a part of my screen,
      If color is present, i will rightclick it, for a menu to appear,
      in which i want to Leftclick an option. Simplicity itself.   

      The issue is that because this menu appears in different parts of my screen, its complicated to select the option i want to click.
      There is no special color to it, and i cannot define the area in which to click, because the menu appears different places.
      The only thing that is in common for the menu is that the Leftclick needs to be (lets say) 30pixels below the rightclick
      .

      What can be done,  but seemingly not by me, is to get the coordinates of the rightclick, and use them as the "starting point" from where the coordinates for the leftclick will be.
      (Not sure if that makes sence, but maybe the code below will help you understand)

      You guys like example codes, so therefore i will give it my best shot, and hope it proves usefull

      $pos = Pixelsearch(200, 400, 200, 400, 0xFFFFFF)        ;This color is what i need to rightclick to bring up the menu
      If not @Error then                                                               ;If the color is present
      (Mouseclick("Right", $pos[0], $pos[1], 1, 1)                   ;Rightclick it, to bring up menu
      ;Here comes the part i cannot get to work
      $x = MousegetPos(X)                                                          ;To get the Coordinate "X", so we can calculated with it for the position of the leftclick
      $y = MousegetPos(Y)                                                           ; To get the Coordinate "Y", so that can be calculated with for the position of the leftclick

      MouseClick("Left", $x[0], $y-30[1], 1, 1)                         ;This was supposed to give me a leftclick that is 30pixels below the rightclick.
      (The MouseGetPos(X), may seem as a waste of space in this example, but in my script i will need both. 
              Im only using one in the example, mainly so that i wont confuse myself, but also to make it easier to digest for you guys)


      I have not been able to find any topic that covers this, but my apologies if such a topic already exist in the forums. 

      Maybe while im at it, can i ask,   what good does the [0] and [1] do, when using a "$variable" as coordinate?   


      Thanks in advance
      - DiveHigh