Jump to content
Qwerty212

On Screen Keyboard strange behavior when running on a touchscreen

Recommended Posts

Qwerty212

Hi.

 

I'm trying to create a custom On Screen Keyboard based on some of the examples in the forum, and the one I've used works perfectly when I'm using it with a mouse to click the buttons.

I'm coding a little script that will be running on a Windows 8.1 tablet and when I try to run a On Screen Keyboard example it repeates the keystrokes. If I want to type "3" and I click with my finger on the button "3", it types a "3". If then I click on another button the OSK types again the "3" and then the new number that I typed before :sweating:

 

This is the code I'm using as a base:

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include <ButtonConstants.au3>

;$WS_EX_NOACTIVATE = 0x08000000
$MA_NOACTIVATE = 3
$MA_NOACTIVATEANDEAT = 4

HotKeySet("{ESC}", "On_Exit")

Global $aKeys[48] = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "0", "-", "=", _
                     "Q", "W", "E", "R", "T", "Y", "U", "I", "O", "P", "[", "]", _
                     "A", "S", "D", "F", "G", "H", "J", "K", "L", ";", "'", '"', _
                     "Z", "X", "C", "V", "B", "N", "M", ",", ".", "/", "spc", "enter"]

; Create "keyboard" GUI
$hGUI = GUICreate("Test", 360, 120, 500, 100, $WS_POPUPWINDOW, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST, $WS_EX_NOACTIVATE))

$dummy1 = GUICtrlCreateDummy()
$iCount = 0
For $j = 0 To 3
    For $i = 0 To 11
        GUICtrlCreateButton("", $i * 30, $j * 30, 30, 30)
        GUICtrlSetData(-1, $aKeys[$iCount])
        $iCount += 1
;~         GUICtrlSetFont(-1, 10)
;~         GUICtrlSetBkColor(-1, 30000 + 2000 * ($i + 1) + 2000 * ($j + 1))
    Next
Next
$dummy2 = GUICtrlCreateDummy()
GUISetState()

GUIRegisterMsg($WM_MOUSEACTIVATE, 'WM_EVENTS')

Run("notepad.exe")

While 1
    $msg = GUIGetMsg()
    Switch $msg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $dummy1 To $dummy2
            Local $sText = ControlGetText($hGUI, "", $msg)
            ; Write key
            If $sText = "spc" Then
                Send("{SPACE}")
            ElseIf $sText = "enter" Then
                Send("{ENTER}")
            Else
                Send($sText)
            EndIf
    EndSwitch
WEnd


Func On_Exit()
    Exit
EndFunc

Func WM_EVENTS($hWndGUI, $MsgID, $WParam, $LParam)
    Switch $hWndGUI
        Case $hGUI
            Switch $MsgID
                Case $WM_MOUSEACTIVATE
                    ; Check mouse position
                    Local $aMouse_Pos = GUIGetCursorInfo($hGUI)
                    If $aMouse_Pos[4] <> 0 Then
                        Local $word = _WinAPI_MakeLong($aMouse_Pos[4], $BN_CLICKED)
                        _SendMessage($hGUI, $WM_COMMAND, $word, GUICtrlGetHandle($aMouse_Pos[4]))
                    EndIf
                    Return $MA_NOACTIVATEANDEAT
            EndSwitch
    EndSwitch
    Return $GUI_RUNDEFMSG
EndFunc

I don't have any other Windows' touch screen device to test, so if any of you can try the script I would trully apreciatte any feedback.

Maybe in touch screens the system handles in a different way the mouse click and the scripts fails... I don't know :>

 

Greets from Barcelona

Edited by Qwerty212

Share this post


Link to post
Share on other sites
Qwerty212

I have just tested it with a Windows XP machine with touch screen (not tablet) and it works like a charm.

Any clue about why it may doens't work correctly under a Windows 8.1 tablet system would be trully apreciatted.

 

Greets from Barcelona

Edited by Qwerty212

Share this post


Link to post
Share on other sites
Kendlp

I am having this same problem with another onscreen keyboard script and the Windows PE 5 with a touchscreen. I tried it on a couple of different tablets with the same result as you described. If I plug in and use a mouse everything works as expected.  Were you able to finds a resolution to this issue?

Share this post


Link to post
Share on other sites
Qwerty212

Hi Kendlp. I ended coding a on screen numpad keyboard

Share this post


Link to post
Share on other sites
Kendlp

Sorry, I am not sure what you mean. Do you have an example you could share?

Share this post


Link to post
Share on other sites
Qwerty212

It's inside a huge code (more than 3000 lines)

I'll try to post an snippet tomorrow.

 

Greets from Barcelona

Share this post


Link to post
Share on other sites
Fireworker

A long time is over...

 

Today i had the same problem in WinPE (Windows8.1) on a Tablet with only one usb port, without keyboard....

I have remove the "GUIRegisterMsg($WM_MOUSEACTIVATE, 'WM_EVENTS')" function

 

My Script below...

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=.\sourcen\keyboard.ico
#AutoIt3Wrapper_Outfile=.\bin\OnScreenKeyboard.exe
#AutoIt3Wrapper_UseUpx=y
#AutoIt3Wrapper_Compile_Both=y
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Res_Comment=On Screen Keyboard
#AutoIt3Wrapper_Res_Description=On Screen Keyboard
://////=__=
#AutoIt3Wrapper_Res_Fileversion=1.0.0.0
#AutoIt3Wrapper_Res_LegalCopyright=2016 IT
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <ButtonConstants.au3>

; --- Set options
AutoItSetOption("SendKeyDelay", 2)
AutoItSetOption("TrayMenuMode", 1)

; --- Picture to GUI
FileInstall(".\sourcen\win.ico", @TempDir & "\win.ico", 1)
FileInstall(".\sourcen\app.ico", @TempDir & "\app.ico", 1)
FileInstall(".\sourcen\back.ico", @TempDir & "\back.ico", 1)
FileInstall(".\sourcen\tab.ico", @TempDir & "\tab.ico", 1)
$WinIco  = @TempDir & "\win.ico"
$AppIco  = @TempDir & "\app.ico"
$BackIco = @TempDir & "\back.ico"
$TabIco  = @TempDir & "\tab.ico"

Global $Os = False, $Shift = False, $objWMIService, $colItems, $HResulution, $VResulution

$AppTitle       = "On Screen Keyboard"
$GuiWidth       = 671
$GuiHeight      = 231
$GuiBottomSpace = 70

If _WmiConnect() Then _GetResolution()
$GuiPostion = $VResulution - $GuiHeight - $GuiBottomSpace

If $VResulution = "" Then
    $VResulution = 768
    If _OsVersion() Then $GuiPostion = $VResulution
EndIf

$Gui           = GUICreate($AppTitle, $GuiWidth, $GuiHeight, -1, $GuiPostion, -1, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST, $WS_EX_NOACTIVATE))
                 GUISetFont(12, 500, 0, "Tahoma")

$NoFocusButton = GUICtrlCreateButton("a", 1, 1, 52, 5)
                 GUICTRLSetState($NoFocusButton, $GUI_HIDE)

$DummyOne      = GUICtrlCreateDummy()

; --- First row
$tilda         = GUICtrlCreateButton("`", 0, 0, 48, 47)
$1             = GUICtrlCreateButton("1", 47, 0, 48, 47)
$2             = GUICtrlCreateButton("2", 94, 0, 48, 47)
$3             = GUICtrlCreateButton("3", 141, 0, 48, 47)
$4             = GUICtrlCreateButton("4", 188, 0, 48, 47)
$5             = GUICtrlCreateButton("5", 235, 0, 48, 47)
$6             = GUICtrlCreateButton("6", 282, 0, 48, 47)
$7             = GUICtrlCreateButton("7", 329, 0, 48, 47)
$8             = GUICtrlCreateButton("8", 376, 0, 48, 47)
$9             = GUICtrlCreateButton("9", 423, 0, 48, 47)
$0             = GUICtrlCreateButton("0", 470, 0, 48, 47)
$Dash          = GUICtrlCreateButton("-", 517, 0, 48, 47)
$Eq            = GUICtrlCreateButton("=", 564, 0, 48, 47)
$Bs            = GUICtrlCreateButton("Back", 611, 0, 60, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $BackIco, -1)

; --- Second row
                 GUICtrlCreateButton("Tab", 0, 46, 60, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $TabIco, -1)
$q             = GUICtrlCreateButton("q",  59, 46, 48, 47)
$w             = GUICtrlCreateButton("w", 106, 46, 48, 47)
$e             = GUICtrlCreateButton("e", 153, 46, 48, 47)
$r             = GUICtrlCreateButton("r", 200, 46, 48, 47)
$t             = GUICtrlCreateButton("t", 247, 46, 48, 47)
$y             = GUICtrlCreateButton("y", 294, 46, 48, 47)
$u             = GUICtrlCreateButton("u", 341, 46, 48, 47)
$i             = GUICtrlCreateButton("i", 388, 46, 48, 47)
$o             = GUICtrlCreateButton("o", 435, 46, 48, 47)
$p             = GUICtrlCreateButton("p", 482, 46, 48, 47)
$OpenBracket   = GUICtrlCreateButton("[", 529, 46, 48, 47)
$CloseBracket  = GUICtrlCreateButton("]", 576, 46, 48, 47)
$BkSlash       = GUICtrlCreateButton("\", 623, 46, 48, 47)

; --- Third row
                 GUICtrlCreateButton("", 0, 92, 76, 47)
$a             = GUICtrlCreateButton("a", 75, 92, 48, 47)
$s             = GUICtrlCreateButton("s", 122, 92, 48, 47)
$d             = GUICtrlCreateButton("d", 169, 92, 48, 47)
$f             = GUICtrlCreateButton("f", 216, 92, 48, 47)
$g             = GUICtrlCreateButton("g", 263, 92, 48, 47)
$h             = GUICtrlCreateButton("h", 310, 92, 48, 47)
$j             = GUICtrlCreateButton("j", 357, 92, 48, 47)
$k             = GUICtrlCreateButton("k", 404, 92, 48, 47)
$l             = GUICtrlCreateButton("l", 451, 92, 48, 47)
$Semicolon     = GUICtrlCreateButton(";", 498, 92, 48, 47)
$Quotes        = GUICtrlCreateButton("'", 545, 92, 48, 47)
                 GUICtrlCreateButton("Enter", 592, 92, 79, 47)

; --- Fourth row
                 GUICtrlCreateButton("Shift", 0, 138, 100, 47)
$z             = GUICtrlCreateButton("z",  99, 138,  48, 47)
$x             = GUICtrlCreateButton("x", 146, 138,  48, 47)
$c             = GUICtrlCreateButton("c", 193, 138,  48, 47)
$v             = GUICtrlCreateButton("v", 240, 138,  48, 47)
$b             = GUICtrlCreateButton("b", 287, 138,  48, 47)
$n             = GUICtrlCreateButton("n", 334, 138,  48, 47)
$m             = GUICtrlCreateButton("m", 381, 138,  48, 47)
$Comma         = GUICtrlCreateButton(",", 428, 138,  48, 47)
$Period        = GUICtrlCreateButton(".", 475, 138,  48, 47)
$FrontSlash    = GUICtrlCreateButton("/", 522, 138,  48, 47)
                 GUICtrlCreateButton("", 569, 138, 102, 47)

; --- Fifth row
                 GUICtrlCreateButton("", 0, 184,  76, 47)
                 GUICtrlCreateButton("L-Win", 75, 184,  62, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $WinIco, -1)
                 GUICtrlCreateButton("Alt", 136, 184,  62, 47)
                 GUICtrlCreateButton("Space", 197, 184, 234, 47)
                 GUICtrlCreateButton("Alt", 430, 184,  62, 47)
                 GUICtrlCreateButton("R-Win", 491, 184,  62, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $WinIco, -1)
$Apps          = GUICtrlCreateButton("Cont", 552, 184,  58, 47, $BS_ICON)
                 GUICtrlSetImage(-1, $AppIco, -1)
                 GUICtrlCreateButton("Close", 609, 184,  62, 47)

$DummyTwo = GUICtrlCreateDummy()

TraySetToolTip($AppTitle)
TraySetState()

GUICTRLSetState($NoFocusButton, $GUI_FOCUS)
GUISetState(@SW_SHOWNOACTIVATE)

While 1
    $msg = GUIGetMsg()
    Switch $msg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $DummyOne To $DummyTwo
            Local $sText = ControlGetText($Gui, "", $msg)

            If $sText = "Space" Then
                Send("{SPACE}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Enter" Then
                Send("{ENTER}")
                If $Os Then _NoFocus()
            ElseIf $sText = "L-Win" Then
                Send("{LWIN}")
                If $Os Then _NoFocus()
            ElseIf $sText = "R-Win" Then
                Send("{RWIN}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Tab" Then
                Send("{TAB}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Back" Then
                Send("{BS}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Cont" Then
                Send("{APPSKEY}")
                If $Os Then _NoFocus()
            ElseIf $sText = "Shift" Then
                If Not $Shift Then
                    GUICtrlSetData($tilda, "~")
                    GUICtrlSetData($1, "!")
                    GUICtrlSetData($2, "@")
                    GUICtrlSetData($3, "#")
                    GUICtrlSetData($4, "$")
                    GUICtrlSetData($5, "%")
                    GUICtrlSetData($6, "^")
                    GUICtrlSetData($7, '&&')
                    GUICtrlSetData($8, "*")
                    GUICtrlSetData($9, "(")
                    GUICtrlSetData($0, ")")
                    GUICtrlSetData($Dash, "_")
                    GUICtrlSetData($Eq, "+")

                    GUICtrlSetData($q, "Q")
                    GUICtrlSetData($w, "W")
                    GUICtrlSetData($e, "E")
                    GUICtrlSetData($r, "R")
                    GUICtrlSetData($t, "T")
                    GUICtrlSetData($y, "Y")
                    GUICtrlSetData($u, "U")
                    GUICtrlSetData($i, "I")
                    GUICtrlSetData($o, "O")
                    GUICtrlSetData($p, "P")
                    GUICtrlSetData($OpenBracket, "{")
                    GUICtrlSetData($CloseBracket, "}")
                    GUICtrlSetData($BkSlash, "|")

                    GUICtrlSetData($a, "A")
                    GUICtrlSetData($s, "S")
                    GUICtrlSetData($d, "D")
                    GUICtrlSetData($f, "F")
                    GUICtrlSetData($g, "G")
                    GUICtrlSetData($h, "H")
                    GUICtrlSetData($j, "J")
                    GUICtrlSetData($k, "K")
                    GUICtrlSetData($l, "L")
                    GUICtrlSetData($Semicolon, ":")
                    GUICtrlSetData($Quotes, '"')

                    GUICtrlSetData($z, "Z")
                    GUICtrlSetData($x, "X")
                    GUICtrlSetData($c, "C")
                    GUICtrlSetData($v, "V")
                    GUICtrlSetData($b, "B")
                    GUICtrlSetData($n, "N")
                    GUICtrlSetData($m, "M")
                    GUICtrlSetData($Comma, "<")
                    GUICtrlSetData($Period, ">")
                    GUICtrlSetData($FrontSlash, "?")
                    $Shift = True
                    If $Os Then _NoFocus()
                Else
                    GUICtrlSetData($tilda, "`")
                    GUICtrlSetData($1, "1")
                    GUICtrlSetData($2, "2")
                    GUICtrlSetData($3, "3")
                    GUICtrlSetData($4, "4")
                    GUICtrlSetData($5, "5")
                    GUICtrlSetData($6, "6")
                    GUICtrlSetData($7, "7")
                    GUICtrlSetData($8, "8")
                    GUICtrlSetData($9, "9")
                    GUICtrlSetData($0, "0")
                    GUICtrlSetData($Dash, "-")
                    GUICtrlSetData($Eq, "=")

                    GUICtrlSetData($q, "q")
                    GUICtrlSetData($w, "w")
                    GUICtrlSetData($e, "e")
                    GUICtrlSetData($r, "r")
                    GUICtrlSetData($t, "t")
                    GUICtrlSetData($y, "y")
                    GUICtrlSetData($u, "u")
                    GUICtrlSetData($i, "i")
                    GUICtrlSetData($o, "o")
                    GUICtrlSetData($p, "p")
                    GUICtrlSetData($OpenBracket, "[")
                    GUICtrlSetData($CloseBracket, "]")
                    GUICtrlSetData($BkSlash, "\")

                    GUICtrlSetData($a, "a")
                    GUICtrlSetData($s, "s")
                    GUICtrlSetData($d, "d")
                    GUICtrlSetData($f, "f")
                    GUICtrlSetData($g, "g")
                    GUICtrlSetData($h, "h")
                    GUICtrlSetData($j, "j")
                    GUICtrlSetData($k, "k")
                    GUICtrlSetData($l, "l")
                    GUICtrlSetData($Semicolon, ";")
                    GUICtrlSetData($Quotes, "'")

                    GUICtrlSetData($z, "z")
                    GUICtrlSetData($x, "x")
                    GUICtrlSetData($c, "c")
                    GUICtrlSetData($v, "v")
                    GUICtrlSetData($b, "b")
                    GUICtrlSetData($n, "n")
                    GUICtrlSetData($m, "m")
                    GUICtrlSetData($Comma, ",")
                    GUICtrlSetData($Period, ".")
                    GUICtrlSetData($FrontSlash, "/")
                    $Shift = False
                    If $Os Then _NoFocus()
                EndIf
            ElseIf $sText = "Close" Then
                Exit
            Else
                If $Os Then _NoFocus()
                If $sText = '&&' Then
                    Send('&')
                Else
                    Send($sText, 1)
                EndIf
            EndIf
    EndSwitch
WEnd

; --- Set Focus To Help Button
Func _NoFocus()
    GUICTRLSetState($NoFocusButton, $GUI_FOCUS)
    ControlFocus($AppTitle, "", $Gui)
EndFunc

; --- Find OS Version
Func _OsVersion()
    If StringInStr(RegRead("HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion", "ProductName"), "Preinstallation") Then
        Return True
    Else
        Return False
    EndIf
EndFunc

; --- Connect to WMI
Func _WmiConnect()
    $objWMIService = ObjGet("winmgmts:\\" & @ComputerName & "\root\CIMV2")
    If Not @error Then
        Return True
    Else
        Return False
    EndIf
EndFunc

; --- Get Video Resolution
Func _GetResolution()
    $FoundAdapter = False
    $colItems = $objWMIService.ExecQuery("SELECT * FROM Win32_VideoController", "WQL", 0x10 + 0x20)
    If Not @error And IsObj($colItems) Then
        For $objItem In $colItems
            If Not ($objItem.CurrentHorizontalResolution = "") Or Not ($objItem.CurrentVerticalResolution = "") Then
                $FoundAdapter = True
            EndIf

            If $FoundAdapter = True Then
                $HResulution = $objItem.CurrentHorizontalResolution
                $VResulution = $objItem.CurrentVerticalResolution
                ExitLoop
            EndIf
            $FoundAdapter = False
        Next
        Return True
    Else
        Return False
    EndIf
EndFunc

 

Edited by Fireworker

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×