On Screen Keyboard strange behavior when running on a touchscreen

Qwerty212 · July 12, 2015

Hi.

I'm trying to create a custom On Screen Keyboard based on some of the examples in the forum, and the one I've used works perfectly when I'm using it with a mouse to click the buttons.

I'm coding a little script that will be running on a Windows 8.1 tablet and when I try to run a On Screen Keyboard example it repeates the keystrokes. If I want to type "3" and I click with my finger on the button "3", it types a "3". If then I click on another button the OSK types again the "3" and then the new number that I typed before :sweating:

This is the code I'm using as a base:

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include <ButtonConstants.au3>

;$WS_EX_NOACTIVATE = 0x08000000
$MA_NOACTIVATE = 3
$MA_NOACTIVATEANDEAT = 4

HotKeySet("{ESC}", "On_Exit")

Global $aKeys[48] = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "0", "-", "=", _
                     "Q", "W", "E", "R", "T", "Y", "U", "I", "O", "P", "[", "]", _
                     "A", "S", "D", "F", "G", "H", "J", "K", "L", ";", "'", '"', _
                     "Z", "X", "C", "V", "B", "N", "M", ",", ".", "/", "spc", "enter"]

; Create "keyboard" GUI
$hGUI = GUICreate("Test", 360, 120, 500, 100, $WS_POPUPWINDOW, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST, $WS_EX_NOACTIVATE))

$dummy1 = GUICtrlCreateDummy()
$iCount = 0
For $j = 0 To 3
    For $i = 0 To 11
        GUICtrlCreateButton("", $i * 30, $j * 30, 30, 30)
        GUICtrlSetData(-1, $aKeys[$iCount])
        $iCount += 1
;~         GUICtrlSetFont(-1, 10)
;~         GUICtrlSetBkColor(-1, 30000 + 2000 * ($i + 1) + 2000 * ($j + 1))
    Next
Next
$dummy2 = GUICtrlCreateDummy()
GUISetState()

GUIRegisterMsg($WM_MOUSEACTIVATE, 'WM_EVENTS')

Run("notepad.exe")

While 1
    $msg = GUIGetMsg()
    Switch $msg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $dummy1 To $dummy2
            Local $sText = ControlGetText($hGUI, "", $msg)
            ; Write key
            If $sText = "spc" Then
                Send("{SPACE}")
            ElseIf $sText = "enter" Then
                Send("{ENTER}")
            Else
                Send($sText)
            EndIf
    EndSwitch
WEnd


Func On_Exit()
    Exit
EndFunc

Func WM_EVENTS($hWndGUI, $MsgID, $WParam, $LParam)
    Switch $hWndGUI
        Case $hGUI
            Switch $MsgID
                Case $WM_MOUSEACTIVATE
                    ; Check mouse position
                    Local $aMouse_Pos = GUIGetCursorInfo($hGUI)
                    If $aMouse_Pos[4] <> 0 Then
                        Local $word = _WinAPI_MakeLong($aMouse_Pos[4], $BN_CLICKED)
                        _SendMessage($hGUI, $WM_COMMAND, $word, GUICtrlGetHandle($aMouse_Pos[4]))
                    EndIf
                    Return $MA_NOACTIVATEANDEAT
            EndSwitch
    EndSwitch
    Return $GUI_RUNDEFMSG
EndFunc

I don't have any other Windows' touch screen device to test, so if any of you can try the script I would trully apreciatte any feedback.

Maybe in touch screens the system handles in a different way the mouse click and the scripts fails... I don't know

Greets from Barcelona

Edited July 13, 2015 by Qwerty212

Qwerty212 · July 14, 2015

I have just tested it with a Windows XP machine with touch screen (not tablet) and it works like a charm.

Any clue about why it may doens't work correctly under a Windows 8.1 tablet system would be trully apreciatted.

Greets from Barcelona

Edited July 14, 2015 by Qwerty212

Kendlp · December 4, 2015

I am having this same problem with another onscreen keyboard script and the Windows PE 5 with a touchscreen. I tried it on a couple of different tablets with the same result as you described. If I plug in and use a mouse everything works as expected. Were you able to finds a resolution to this issue?

Qwerty212 · December 6, 2015

Hi Kendlp. I ended coding a on screen numpad keyboard

Kendlp · December 7, 2015

Sorry, I am not sure what you mean. Do you have an example you could share?

Qwerty212 · December 7, 2015

It's inside a huge code (more than 3000 lines)

I'll try to post an snippet tomorrow.

Greets from Barcelona

Fireworker · October 6, 2016

A long time is over...

Today i had the same problem in WinPE (Windows8.1) on a Tablet with only one usb port, without keyboard....

I have remove the "GUIRegisterMsg($WM_MOUSEACTIVATE, 'WM_EVENTS')" function

My Script below...

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=.\sourcen\keyboard.ico
#AutoIt3Wrapper_Outfile=.\bin\OnScreenKeyboard.exe
#AutoIt3Wrapper_UseUpx=y
#AutoIt3Wrapper_Compile_Both=y
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Res_Comment=On Screen Keyboard
#AutoIt3Wrapper_Res_Description=On Screen Keyboard
://////=__=
#AutoIt3Wrapper_Res_Fileversion=1.0.0.0
#AutoIt3Wrapper_Res_LegalCopyright=2016 IT
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <ButtonConstants.au3>

; --- Set options
AutoItSetOption("SendKeyDelay", 2)
AutoItSetOption("TrayMenuMode", 1)

; --- Picture to GUI
FileInstall(".\sourcen\win.ico", @TempDir & "\win.ico", 1)
FileInstall(".\sourcen\app.ico", @TempDir & "\app.ico", 1)
FileInstall(".\sourcen\back.ico", @TempDir & "\back.ico", 1)
FileInstall(".\sourcen\tab.ico", @TempDir & "\tab.ico", 1)
$WinIco  = @TempDir & "\win.ico"
$AppIco  = @TempDir & "\app.ico"
$BackIco = @TempDir & "\back.ico"
$TabIco  = @TempDir & "\tab.ico"

Global $Os = False, $Shift = False, $objWMIService, $colItems, $HResulution, $VResulution

$AppTitle       = "On Screen Keyboard"
$GuiWidth       = 671
$GuiHeight      = 231
$GuiBottomSpace = 70

If _WmiConnect() Then _GetResolution()
$GuiPostion = $VResulution - $GuiHeight - $GuiBottomSpace

If $VResulution = "" Then
    $VResulution = 768
    If _OsVersion() Then $GuiPostion = $VResulution
EndIf

$Gui           = GUICreate($AppTitle, $GuiWidth, $GuiHeight, -1, $GuiPostion, -1, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST, $WS_EX_NOACTIVATE))
                 GUISetFont(12, 500, 0, "Tahoma")

$NoFocusButton = GUICtrlCreateButton("a", 1, 1, 52, 5)
                 GUICTRLSetState($NoFocusButton, $GUI_HIDE)

$DummyOne      = GUICtrlCreateDummy()

; --- First row
$tilda         = GUICtrlCreateButton("`", 0, 0, 48, 47)
$1             = GUICtrlCreateButton("1", 47, 0, 48, 47)
$2             = GUICtrlCreateButton("2", 94, 0, 48, 47)
$3             = GUICtrlCreateButton("3", 141, 0, 48, 47)
$4             = GUICtrlCreateButton("4", 188, 0, 48, 47)
$5             = GUICtrlCreateButton("5", 235, 0, 48, 47)
$6             = GUICtrlCreateButton("6", 282, 0, 48, 47)
$7             = GUICtrlCreateButton("7", 329, 0, 48, 47)
$8             = GUICtrlCreateButton("8", 376, 0, 48, 47)
$9             = GUICtrlCreateButton("9", 423, 0, 48, 47)
$0             = GUICtrlCreateButton("0", 470, 0, 48, 47)
$Dash          = GUICtrlCreateButton("-", 517, 0, 48, 47)
$Eq            = GUICtrlCreateButton("=", 564, 0, 48, 47)
$Bs            = GUICtrlCreateButton("Back", 611, 0, 60, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $BackIco, -1)

; --- Second row
                 GUICtrlCreateButton("Tab", 0, 46, 60, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $TabIco, -1)
$q             = GUICtrlCreateButton("q",  59, 46, 48, 47)
$w             = GUICtrlCreateButton("w", 106, 46, 48, 47)
$e             = GUICtrlCreateButton("e", 153, 46, 48, 47)
$r             = GUICtrlCreateButton("r", 200, 46, 48, 47)
$t             = GUICtrlCreateButton("t", 247, 46, 48, 47)
$y             = GUICtrlCreateButton("y", 294, 46, 48, 47)
$u             = GUICtrlCreateButton("u", 341, 46, 48, 47)
$i             = GUICtrlCreateButton("i", 388, 46, 48, 47)
$o             = GUICtrlCreateButton("o", 435, 46, 48, 47)
$p             = GUICtrlCreateButton("p", 482, 46, 48, 47)
$OpenBracket   = GUICtrlCreateButton("[", 529, 46, 48, 47)
$CloseBracket  = GUICtrlCreateButton("]", 576, 46, 48, 47)
$BkSlash       = GUICtrlCreateButton("\", 623, 46, 48, 47)

; --- Third row
                 GUICtrlCreateButton("", 0, 92, 76, 47)
$a             = GUICtrlCreateButton("a", 75, 92, 48, 47)
$s             = GUICtrlCreateButton("s", 122, 92, 48, 47)
$d             = GUICtrlCreateButton("d", 169, 92, 48, 47)
$f             = GUICtrlCreateButton("f", 216, 92, 48, 47)
$g             = GUICtrlCreateButton("g", 263, 92, 48, 47)
$h             = GUICtrlCreateButton("h", 310, 92, 48, 47)
$j             = GUICtrlCreateButton("j", 357, 92, 48, 47)
$k             = GUICtrlCreateButton("k", 404, 92, 48, 47)
$l             = GUICtrlCreateButton("l", 451, 92, 48, 47)
$Semicolon     = GUICtrlCreateButton(";", 498, 92, 48, 47)
$Quotes        = GUICtrlCreateButton("'", 545, 92, 48, 47)
                 GUICtrlCreateButton("Enter", 592, 92, 79, 47)

; --- Fourth row
                 GUICtrlCreateButton("Shift", 0, 138, 100, 47)
$z             = GUICtrlCreateButton("z",  99, 138,  48, 47)
$x             = GUICtrlCreateButton("x", 146, 138,  48, 47)
$c             = GUICtrlCreateButton("c", 193, 138,  48, 47)
$v             = GUICtrlCreateButton("v", 240, 138,  48, 47)
$b             = GUICtrlCreateButton("b", 287, 138,  48, 47)
$n             = GUICtrlCreateButton("n", 334, 138,  48, 47)
$m             = GUICtrlCreateButton("m", 381, 138,  48, 47)
$Comma         = GUICtrlCreateButton(",", 428, 138,  48, 47)
$Period        = GUICtrlCreateButton(".", 475, 138,  48, 47)
$FrontSlash    = GUICtrlCreateButton("/", 522, 138,  48, 47)
                 GUICtrlCreateButton("", 569, 138, 102, 47)

; --- Fifth row
                 GUICtrlCreateButton("", 0, 184,  76, 47)
                 GUICtrlCreateButton("L-Win", 75, 184,  62, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $WinIco, -1)
                 GUICtrlCreateButton("Alt", 136, 184,  62, 47)
                 GUICtrlCreateButton("Space", 197, 184, 234, 47)
                 GUICtrlCreateButton("Alt", 430, 184,  62, 47)
                 GUICtrlCreateButton("R-Win", 491, 184,  62, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $WinIco, -1)
$Apps          = GUICtrlCreateButton("Cont", 552, 184,  58, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $AppIco, -1)
                 GUICtrlCreateButton("Close", 609, 184,  62, 47)

$DummyTwo = GUICtrlCreateDummy()

TraySetToolTip($AppTitle)
TraySetState()

GUICTRLSetState($NoFocusButton, $GUI_FOCUS)
GUISetState(@SW_SHOWNOACTIVATE)

While 1
    $msg = GUIGetMsg()
    Switch $msg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $DummyOne To $DummyTwo
            Local $sText = ControlGetText($Gui, "", $msg)

            If $sText = "Space" Then
                Send("{SPACE}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Enter" Then
                Send("{ENTER}")
                If $Os Then _NoFocus()
            ElseIf $sText = "L-Win" Then
                Send("{LWIN}")
                If $Os Then _NoFocus()
            ElseIf $sText = "R-Win" Then
                Send("{RWIN}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Tab" Then
                Send("{TAB}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Back" Then
                Send("{BS}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Cont" Then
                Send("{APPSKEY}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Shift" Then
                If Not $Shift Then
                    GUICtrlSetData($tilda, "~")
                    GUICtrlSetData($1, "!")
                    GUICtrlSetData($2, "@")
                    GUICtrlSetData($3, "#")
                    GUICtrlSetData($4, "$")
                    GUICtrlSetData($5, "%")
                    GUICtrlSetData($6, "^")
                    GUICtrlSetData($7, '&&')
                    GUICtrlSetData($8, "*")
                    GUICtrlSetData($9, "(")
                    GUICtrlSetData($0, ")")
                    GUICtrlSetData($Dash, "_")
                    GUICtrlSetData($Eq, "+")

                    GUICtrlSetData($q, "Q")
                    GUICtrlSetData($w, "W")
                    GUICtrlSetData($e, "E")
                    GUICtrlSetData($r, "R")
                    GUICtrlSetData($t, "T")
                    GUICtrlSetData($y, "Y")
                    GUICtrlSetData($u, "U")
                    GUICtrlSetData($i, "I")
                    GUICtrlSetData($o, "O")
                    GUICtrlSetData($p, "P")
                    GUICtrlSetData($OpenBracket, "{")
                    GUICtrlSetData($CloseBracket, "}")
                    GUICtrlSetData($BkSlash, "|")

                    GUICtrlSetData($a, "A")
                    GUICtrlSetData($s, "S")
                    GUICtrlSetData($d, "D")
                    GUICtrlSetData($f, "F")
                    GUICtrlSetData($g, "G")
                    GUICtrlSetData($h, "H")
                    GUICtrlSetData($j, "J")
                    GUICtrlSetData($k, "K")
                    GUICtrlSetData($l, "L")
                    GUICtrlSetData($Semicolon, ":")
                    GUICtrlSetData($Quotes, '"')

                    GUICtrlSetData($z, "Z")
                    GUICtrlSetData($x, "X")
                    GUICtrlSetData($c, "C")
                    GUICtrlSetData($v, "V")
                    GUICtrlSetData($b, "B")
                    GUICtrlSetData($n, "N")
                    GUICtrlSetData($m, "M")
                    GUICtrlSetData($Comma, "<")
                    GUICtrlSetData($Period, ">")
                    GUICtrlSetData($FrontSlash, "?")
                    $Shift = True
                    If $Os Then _NoFocus()
                Else
                    GUICtrlSetData($tilda, "`")
                    GUICtrlSetData($1, "1")
                    GUICtrlSetData($2, "2")
                    GUICtrlSetData($3, "3")
                    GUICtrlSetData($4, "4")
                    GUICtrlSetData($5, "5")
                    GUICtrlSetData($6, "6")
                    GUICtrlSetData($7, "7")
                    GUICtrlSetData($8, "8")
                    GUICtrlSetData($9, "9")
                    GUICtrlSetData($0, "0")
                    GUICtrlSetData($Dash, "-")
                    GUICtrlSetData($Eq, "=")

                    GUICtrlSetData($q, "q")
                    GUICtrlSetData($w, "w")
                    GUICtrlSetData($e, "e")
                    GUICtrlSetData($r, "r")
                    GUICtrlSetData($t, "t")
                    GUICtrlSetData($y, "y")
                    GUICtrlSetData($u, "u")
                    GUICtrlSetData($i, "i")
                    GUICtrlSetData($o, "o")
                    GUICtrlSetData($p, "p")
                    GUICtrlSetData($OpenBracket, "[")
                    GUICtrlSetData($CloseBracket, "]")
                    GUICtrlSetData($BkSlash, "\")

                    GUICtrlSetData($a, "a")
                    GUICtrlSetData($s, "s")
                    GUICtrlSetData($d, "d")
                    GUICtrlSetData($f, "f")
                    GUICtrlSetData($g, "g")
                    GUICtrlSetData($h, "h")
                    GUICtrlSetData($j, "j")
                    GUICtrlSetData($k, "k")
                    GUICtrlSetData($l, "l")
                    GUICtrlSetData($Semicolon, ";")
                    GUICtrlSetData($Quotes, "'")

                    GUICtrlSetData($z, "z")
                    GUICtrlSetData($x, "x")
                    GUICtrlSetData($c, "c")
                    GUICtrlSetData($v, "v")
                    GUICtrlSetData($b, "b")
                    GUICtrlSetData($n, "n")
                    GUICtrlSetData($m, "m")
                    GUICtrlSetData($Comma, ",")
                    GUICtrlSetData($Period, ".")
                    GUICtrlSetData($FrontSlash, "/")
                    $Shift = False
                    If $Os Then _NoFocus()
                EndIf
            ElseIf $sText = "Close" Then
                Exit
            Else
                If $Os Then _NoFocus()
                If $sText = '&&' Then
                    Send('&')
                Else
                    Send($sText, 1)
                EndIf
            EndIf
    EndSwitch
WEnd

; --- Set Focus To Help Button
Func _NoFocus()
    GUICTRLSetState($NoFocusButton, $GUI_FOCUS)
    ControlFocus($AppTitle, "", $Gui)
EndFunc

; --- Find OS Version
Func _OsVersion()
    If StringInStr(RegRead("HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion", "ProductName"), "Preinstallation") Then
        Return True
    Else
        Return False
    EndIf
EndFunc

; --- Connect to WMI
Func _WmiConnect()
    $objWMIService = ObjGet("winmgmts:\\" & @ComputerName & "\root\CIMV2")
    If Not @error Then
        Return True
    Else
        Return False
    EndIf
EndFunc

; --- Get Video Resolution
Func _GetResolution()
    $FoundAdapter = False
    $colItems = $objWMIService.ExecQuery("SELECT * FROM Win32_VideoController", "WQL", 0x10 + 0x20)
    If Not @error And IsObj($colItems) Then
        For $objItem In $colItems
            If Not ($objItem.CurrentHorizontalResolution = "") Or Not ($objItem.CurrentVerticalResolution = "") Then
                $FoundAdapter = True
            EndIf

            If $FoundAdapter = True Then
                $HResulution = $objItem.CurrentHorizontalResolution
                $VResulution = $objItem.CurrentVerticalResolution
                ExitLoop
            EndIf
            $FoundAdapter = False
        Next
        Return True
    Else
        Return False
    EndIf
EndFunc

Edited October 6, 2016 by Fireworker

Sign In

On Screen Keyboard strange behavior when running on a touchscreen

Recommended Posts

Qwerty212

Link to comment

Share on other sites

Qwerty212

Link to comment

Share on other sites

Kendlp

Link to comment

Share on other sites

Qwerty212

Link to comment

Share on other sites

Kendlp

Link to comment

Share on other sites

Qwerty212

Link to comment

Share on other sites

Fireworker

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta