Sign in to follow this  
Followers 0
dv8

I need help creating virtual keyboard. PLS

17 posts in this topic

Hello everybody. I recently discovered AutoIt, and I think it'll become my favourite automation tool. :lmao:

I'm a complete newbie (for now) and i need some help.

First some introduction to the problem:

I'm trying to create an information system for the patients of the medical center I'm working at. This system is actualy a website running on a PC with touchscreen (no keyboard or mouse). There is a page, which gives information for the health-insurance status of the patient, but first he needs to enter his PIN and a CAPCHA code. So I need virtual keyboard (only digits and letters) to appear on the screen whenever this page is loaded (and disappear when the page is closed). Pressing a button on the keyboard should fill the selected input field on the page.

Here is what i've done so far:

#include <GuiConstants.au3>

Opt("GUIOnEventMode", 1)

$mainwindow = GuiCreate("Keyboard", 400, 160)
WinSetOnTop ( "Keyboard", "", 1 )

GUISetOnEvent($GUI_EVENT_CLOSE, "CLOSEClicked")

; BUTTONS
$_1 = GuiCtrlCreateButton("1", 0, 0, 40, 40)
;GUICtrlSetOnEvent($_1, "ButtonPressed")
$_2 = GuiCtrlCreateButton("2", 40, 0, 40, 40)
$_3 = GuiCtrlCreateButton("3", 80, 0, 40, 40)
$_4 = GuiCtrlCreateButton("4", 120, 0, 40, 40)
$_5 = GuiCtrlCreateButton("5", 160, 0, 40, 40)
$_6 = GuiCtrlCreateButton("6", 200, 0, 40, 40)
$_7 = GuiCtrlCreateButton("7", 240, 0, 40, 40)
$_8 = GuiCtrlCreateButton("8", 280, 0, 40, 40)
$_9 = GuiCtrlCreateButton("9", 320, 0, 40, 40)
$_0 = GuiCtrlCreateButton("0", 360, 0, 40, 40)

$Q = GuiCtrlCreateButton("Q", 0, 40, 40, 40)
$W = GuiCtrlCreateButton("W", 40, 40, 40, 40)
$E = GuiCtrlCreateButton("E", 80, 40, 40, 40)
$R = GuiCtrlCreateButton("R", 120, 40, 40, 40)
$T = GuiCtrlCreateButton("T", 160, 40, 40, 40)
$Y = GuiCtrlCreateButton("Y", 200, 40, 40, 40)
$U = GuiCtrlCreateButton("U", 240, 40, 40, 40)
$I = GuiCtrlCreateButton("I", 280, 40, 40, 40)
$O = GuiCtrlCreateButton("O", 320, 40, 40, 40)
$P = GuiCtrlCreateButton("P", 360, 40, 40, 40)

$A = GuiCtrlCreateButton("A", 20, 80, 40, 40)
$S = GuiCtrlCreateButton("S", 60, 80, 40, 40)
$D = GuiCtrlCreateButton("D", 100, 80, 40, 40)
$F = GuiCtrlCreateButton("F", 140, 80, 40, 40)
$G = GuiCtrlCreateButton("G", 180, 80, 40, 40)
$H = GuiCtrlCreateButton("H", 220, 80, 40, 40)
$J = GuiCtrlCreateButton("J", 260, 80, 40, 40)
$K = GuiCtrlCreateButton("K", 300, 80, 40, 40)
$L = GuiCtrlCreateButton("L", 340, 80, 40, 40)

$Z = GuiCtrlCreateButton("Z", 40, 120, 40, 40)
$X = GuiCtrlCreateButton("X", 80, 120, 40, 40)
$C = GuiCtrlCreateButton("C", 120, 120, 40, 40)
$V = GuiCtrlCreateButton("V", 160, 120, 40, 40)
$B = GuiCtrlCreateButton("B", 200, 120, 40, 40)
$N = GuiCtrlCreateButton("N", 240, 120, 40, 40)
$M = GuiCtrlCreateButton("M", 280, 120, 40, 40)

WinWaitActive("http://healthinsurance.nssi.bg/report_healthhtm_1_15_OUT.asp - Microsoft Internet Explorer")
GUISetState(@SW_SHOW)

While 1
  Sleep(1000) ; Idle around
WEnd

;Func ButtonPressed()
;    ????????
;EndFunc

Func CLOSEClicked()
  If @GUI_WINHANDLE = $mainwindow Then
    Exit
  EndIf
EndFunc

As you can see I've managed to call the keyboard when the page is opened in Internet Explorer, but i'm having a problem with sending the numbers and letters to it.

Please help me with some ideas.

10x in advance.

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

maybe this

SEE MY LAST POST

8)

Edited by Valuater

NEWHeader1.png

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

10x very much. This works great!

Any idea how can I send the result of pressed buttons to an input text field in IE web page?

:lmao:

not working yet... out of time

see what you can do

SEE MY LAST POST

8)

Edited by Valuater

NEWHeader1.png

Share this post


Link to post
Share on other sites

Didn't LazyCat create such a thing one time ?

Maybe I'm becoming senile, but I'm pretty sure it was him how made it.

Anyway, I tried to find it, but it doesn't seem he created a topic for it..

Maybe he comment on this ? Lazy ?!

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Are you looking for something like this? I had originally intended to store row / column info in the multi-dimensioned array, but you get the idea.

#include<GUIConstants.au3>
$chars = '0123456789'
$chars = StringSplit($chars,'')

Opt("GUICoordMode", 2)

Dim $array[$chars[0] + 1][2]

GuiCreate('number keypad',400,100)
For $i = 1 To $chars[0] 
      $array[$i][1] = GuiCtrlCreateButton($chars[$i],10,-1,20,20)
Next


GuiSetState(@SW_SHOW)

While 1
    $msg = GUIGetMsg()
  Select
       Case $msg = $GUI_EVENT_CLOSE
    ExitLoop
 Case $msg < 1
 Case Else
    For $i = 1 to $chars[0]
       If  $msg = $array[$i][1] Then   Send($array[$i][0])
       ExitLoop
    Next
  EndSelect
Wend
Edited by flyingboz

Reading the help file before you post... Not only will it make you look smarter, it will make you smarter.

Share this post


Link to post
Share on other sites

For reliable IE automation, check out Dale's IE.au3 -- you'll need the beta version for COM capability. Otherwise it's status bar checking and sending tabs....


Reading the help file before you post... Not only will it make you look smarter, it will make you smarter.

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

got it ALL WORKING.... YEA!!!!!!!!!

SEE LAST POST

8)

Edited by Valuater

NEWHeader1.png

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

Or to use events. It makes the keyboard it bit more responsive.

I have modified Valuaters code for this a bit. :lmao:

Dick

#include <GuiConstants.au3>
#include <IE.au3>

Opt("GUIOnEventMode", 1)

Dim $BTN_[40]
Dim $x
Dim $ver = "1.0.1"
Dim $web_loc = "http://healthinsurance.nssi.bg/report_healthhtm_1_15_OUT.asp"; - Microsoft Internet Explorer"

$key = StringSplit("1,2,3,4,5,6,7,8,9,0,Q,W,E,R,T,Y,U,I,O,P,A,S,D,F,G,H,J,K,L,Z,X,C,V,B,N,M", ",")

$gui = GUICreate("Medical Center      v" & $ver, @DesktopWidth , @DesktopHeight - 60, -1, -1)
$B_oIE1 = ObjCreate("Shell.Explorer.2")
$internetgui1 = GUICtrlCreateObj($B_oIE1, 50, 5, @DesktopWidth - 100, @DesktopHeight - 200)
$B_oIE1.navigate ($web_loc)
$keybd = GUICtrlCreateButton("SHOW KEYBOARD", 100, @DesktopHeight - 150, 150, 40)
GUICtrlCreateLabel("Presented by, QTasc ", @DesktopWidth - 170, @DesktopHeight - 120, 120, 15, $SS_SUNKEN)
GUISetState()

$mainwindow = GuiCreate("Keyboard - Enter Your PIN Number", 400, 200, 20, @DesktopHeight - 300, -1, "", $gui)
; BUTTONS
$BTN_[1] = GuiCtrlCreateButton("1", 0, 0, 40, 40)
$BTN_[2] = GuiCtrlCreateButton("2", 40, 0, 40, 40)
$BTN_[3] = GuiCtrlCreateButton("3", 80, 0, 40, 40)
$BTN_[4] = GuiCtrlCreateButton("4", 120, 0, 40, 40)
$BTN_[5] = GuiCtrlCreateButton("5", 160, 0, 40, 40)
$BTN_[6] = GuiCtrlCreateButton("6", 200, 0, 40, 40)
$BTN_[7] = GuiCtrlCreateButton("7", 240, 0, 40, 40)
$BTN_[8] = GuiCtrlCreateButton("8", 280, 0, 40, 40)
$BTN_[9] = GuiCtrlCreateButton("9", 320, 0, 40, 40)
$BTN_[10] = GuiCtrlCreateButton("0", 360, 0, 40, 40)

$BTN_[11] = GuiCtrlCreateButton("Q", 0, 40, 40, 40)
$BTN_[12] = GuiCtrlCreateButton("W", 40, 40, 40, 40)
$BTN_[13] = GuiCtrlCreateButton("E", 80, 40, 40, 40)
$BTN_[14] = GuiCtrlCreateButton("R", 120, 40, 40, 40)
$BTN_[15] = GuiCtrlCreateButton("T", 160, 40, 40, 40)
$BTN_[16] = GuiCtrlCreateButton("Y", 200, 40, 40, 40)
$BTN_[17] = GuiCtrlCreateButton("U", 240, 40, 40, 40)
$BTN_[18] = GuiCtrlCreateButton("I", 280, 40, 40, 40)
$BTN_[19] = GuiCtrlCreateButton("O", 320, 40, 40, 40)
$BTN_[20] = GuiCtrlCreateButton("P", 360, 40, 40, 40)

$BTN_[21] = GuiCtrlCreateButton("A", 20, 80, 40, 40)
$BTN_[22] = GuiCtrlCreateButton("S", 60, 80, 40, 40)
$BTN_[23] = GuiCtrlCreateButton("D", 100, 80, 40, 40)
$BTN_[24] = GuiCtrlCreateButton("F", 140, 80, 40, 40)
$BTN_[25] = GuiCtrlCreateButton("G", 180, 80, 40, 40)
$BTN_[26] = GuiCtrlCreateButton("H", 220, 80, 40, 40)
$BTN_[27] = GuiCtrlCreateButton("J", 260, 80, 40, 40)
$BTN_[28] = GuiCtrlCreateButton("K", 300, 80, 40, 40)
$BTN_[29] = GuiCtrlCreateButton("L", 340, 80, 40, 40)

$BTN_[30] = GuiCtrlCreateButton("Z", 40, 120, 40, 40)
$BTN_[31] = GuiCtrlCreateButton("X", 80, 120, 40, 40)
$BTN_[32] = GuiCtrlCreateButton("C", 120, 120, 40, 40)
$BTN_[33] = GuiCtrlCreateButton("V", 160, 120, 40, 40)
$BTN_[34] = GuiCtrlCreateButton("B", 200, 120, 40, 40)
$BTN_[35] = GuiCtrlCreateButton("N", 240, 120, 40, 40)
$BTN_[36] = GuiCtrlCreateButton("M", 280, 120, 40, 40)
$BackSpace = GuiCtrlCreateButton("<-", 320, 120, 40, 40)
GUICtrlSetFont( -1, 10, 500)
$Input = GUICtrlCreateInput("", 40, 170, 150, 20)
GUICtrlSetFont( -1, 10)
$Send = GUICtrlCreateButton("&ENTER", 205, 165, 150, 30)

For $i = 1 To 37
    GUICtrlSetOnEvent($BTN_[$i], "ButtonPressed")
Next
GUICtrlSetOnEvent($BackSpace, "Back_Space")
GUICtrlSetOnEvent($Send, "Send_it")
GUISetOnEvent($GUI_EVENT_CLOSE, "CLOSEClicked")

GUISetState(@SW_SHOW)

WinSetOnTop ( "Keyboard", "", 1 )

While (1)
    Sleep(500)
WEnd

Func ButtonPressed()
    $i = @GUI_CtrlId - $BTN_[1] + 1; array starts at item 1, so add 1
    $info1 = GUICtrlRead($input)
    GUICtrlSetData($input, $info1 & $key[$i])
EndFunc

Func Back_Space()
    $info1 = GUICtrlRead($input)
    $info1 = StringTrimRight($info1, 1)
    GUICtrlSetData($input, $info1 )
EndFunc

Func CLOSEClicked()
  If @GUI_WINHANDLE = $mainwindow Then
    Exit
  EndIf
EndFunc

Func Send_it()
    $title = WinGetTitle("Keyboard")
    If $title = ("Keyboard - Enter Your PIN Number") Then
        
        $o_SearchForm = _IEFormGetObjByName ($B_oIE1, "FrmReportHealth")
        
        $o_Keyword_1 = _IEFormElementGetObjByName ($o_SearchForm, "egn")
        $info = GUICtrlRead($input)
        _IEFormElementSetValue ($o_Keyword_1, $Info)
        GUICtrlSetData($input, "" )
        WinSetTitle("Keyboard - Enter Your PIN Number", "", "Keyboard - Enter the CAPCHA Code")
    EndIf
    
    If $title = ("Keyboard - Enter the CAPCHA Code") Then
        
        $o_SearchForm = _IEFormGetObjByName ($B_oIE1, "FrmReportHealth")
        
        $o_Keyword_2 = _IEFormElementGetObjByName ($o_SearchForm, "key_ent")
        $info2 = GUICtrlRead($input)
        _IEFormElementSetValue ($o_Keyword_2, $info2)
        GUICtrlSetData($input, "" )
        WinSetTitle("Keyboard - Enter the CAPCHA Code", "","Keyboard - Enter Your PIN Number")
    EndIf
EndFunc

EDIT: corrected a typo.

Edited by Dickb

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

windows standardly comes with an on-screen keyboard.

%SystemRoot%\system32\osk.exe

@w0uter

You are right. Forgot about that. :lmao:

But making it in autoit is just more fun. ;)

Edited by Dickb

Share this post


Link to post
Share on other sites

windows standardly comes with an on-screen keyboard.

%SystemRoot%\system32\osk.exe

thats cool w0uter, i hadn't seen that before

1. the desired program is for a touch screen input, thus this very small for "fingers"

2. the key inputs by the requester is for numbers and letters only

@dickb

Nice addition, thanks

8)


NEWHeader1.png

Share this post


Link to post
Share on other sites

Or to use events. It makes the keyboard it bit more responsive.

I have modified Valuaters code for this a bit. :lmao:

Dick

You left a few things out, show keyboard, hide keyboard

.... while we are at it

#include <GuiConstants.au3>
#include <IE.au3>

Opt("GUIOnEventMode", 1)

Dim $BTN_[40]
Dim $x
Dim $ver = "1.0.1"
Dim $web_loc = "http://healthinsurance.nssi.bg/report_healthhtm_1_15_OUT.asp"; - Microsoft Internet Explorer"
Dim $left = 0
Dim $top = 0
$key = StringSplit("1,2,3,4,5,6,7,8,9,0,Q,W,E,R,T,Y,U,I,O,P,A,S,D,F,G,H,J,K,L,Z,X,C,V,B,N,M", ",")

$gui = GUICreate("Medical Center      v" & $ver, @DesktopWidth, @DesktopHeight - 60, -1, -1)
$B_oIE1 = ObjCreate("Shell.Explorer.2")
$internetgui1 = GUICtrlCreateObj($B_oIE1, 50, 5, @DesktopWidth - 100, @DesktopHeight - 200)
$B_oIE1.navigate ($web_loc)
$keybd = GUICtrlCreateButton("SHOW KEYBOARD", 100, @DesktopHeight - 150, 150, 40)
GUICtrlCreateLabel("Presented by, QTasc ", @DesktopWidth - 170, @DesktopHeight - 120, 120, 15, $SS_SUNKEN)
GUISetOnEvent($GUI_EVENT_CLOSE, "CLOSEClicked")
GUISetState()

$mainwindow = GUICreate("Keyboard - Enter Your PIN Number", 400, 200, 20, @DesktopHeight - 300, -1, "", $gui)

; BUTTONS
For $i = 1 To 36
    $BTN_[$i] = GUICtrlCreateButton($key[$i], $left, $top, 40, 40)
    GUICtrlSetOnEvent($BTN_[$i], "ButtonPressed")
    $left = $left + 40
    If $i = 10 Or $i = 20 Or $i = 29 Then $top = $top + 40
    If $i = 10 Then $left = 0
    If $i = 20 Then $left = 20
    If $i = 29 Then $left = 40
Next

$BackSpace = GUICtrlCreateButton("<-", 320, 120, 40, 40)
GUICtrlSetFont(-1, 10, 500)
$Input = GUICtrlCreateInput("", 40, 170, 150, 20)
GUICtrlSetFont(-1, 10)
$Send = GUICtrlCreateButton("&ENTER", 205, 165, 150, 30)

GUICtrlSetOnEvent($keybd, "Show_keys")
GUICtrlSetOnEvent($BackSpace, "Back_Space")
GUICtrlSetOnEvent($Send, "Send_it")
GUISetOnEvent($GUI_EVENT_CLOSE, "CLOSEClicked")
GUISetState(@SW_SHOW)

WinSetOnTop("Keyboard", "", 1)

While (1)
    Sleep(20)
WEnd

Func Show_keys()
    GUISetState(@SW_SHOW, $mainwindow)
EndFunc  ;==>Show_keys

Func ButtonPressed()
    $i = @GUI_CtrlId - $BTN_[1] + 1; array starts at item 1, so add 1
    $info1 = GUICtrlRead($Input)
    GUICtrlSetData($Input, $info1 & $key[$i])
EndFunc  ;==>ButtonPressed

Func Back_Space()
    $info1 = GUICtrlRead($Input)
    $info1 = StringTrimRight($info1, 1)
    GUICtrlSetData($Input, $info1)
EndFunc  ;==>Back_Space

Func CLOSEClicked()
    If @GUI_WinHandle = $mainwindow Then
        GUISetState(@SW_HIDE, $mainwindow)
    EndIf
    If @GUI_WinHandle = $gui Then
        Exit
    EndIf
EndFunc  ;==>CLOSEClicked

Func Send_it()
    $title = WinGetTitle("Keyboard")
    If $title = ("Keyboard - Enter Your PIN Number") Then
        $o_SearchForm = _IEFormGetObjByName ($B_oIE1, "FrmReportHealth")
        $o_Keyword_1 = _IEFormElementGetObjByName ($o_SearchForm, "egn")
        $info = GUICtrlRead($Input)
        _IEFormElementSetValue ($o_Keyword_1, $info)
        GUICtrlSetData($Input, "")
        WinSetTitle("Keyboard - Enter Your PIN Number", "", "Keyboard - Enter the CAPCHA Code")
    EndIf
    If $title = ("Keyboard - Enter the CAPCHA Code") Then
        $o_SearchForm = _IEFormGetObjByName ($B_oIE1, "FrmReportHealth")
        $o_Keyword_2 = _IEFormElementGetObjByName ($o_SearchForm, "key_ent")
        $info2 = GUICtrlRead($Input)
        _IEFormElementSetValue ($o_Keyword_2, $info2)
        GUICtrlSetData($Input, "")
        WinSetTitle("Keyboard - Enter the CAPCHA Code", "", "Keyboard - Enter Your PIN Number")
    EndIf
EndFunc  ;==>Send_it

8)


NEWHeader1.png

Share this post


Link to post
Share on other sites

You left a few things out, show keyboard, hide keyboard

.... while we are at it

Thanks and nice changes.

I have added some error detection to prevent the program to abort when the web page is not found.

This occured to me when I had no network connection. Hope this change resolves it.

Func Send_it()
    $title = WinGetTitle("Keyboard")
    If $B_oIE1 = 0 Then Return
    If _IEFormGetCount($B_oIE1) = 0 Then Return
    If $title = ("Keyboard - Enter Your PIN Number") Then
        $o_SearchForm = _IEFormGetObjByName ($B_oIE1, "FrmReportHealth")
        If $o_SearchForm = 0 Then Return
        $o_Keyword_1 = _IEFormElementGetObjByName ($o_SearchForm, "egn")
        $info = GUICtrlRead($Input)
        _IEFormElementSetValue ($o_Keyword_1, $info)
        GUICtrlSetData($Input, "")
        WinSetTitle("Keyboard - Enter Your PIN Number", "", "Keyboard - Enter the CAPCHA Code")
    EndIf
    If $title = ("Keyboard - Enter the CAPCHA Code") Then
        $o_SearchForm = _IEFormGetObjByName ($B_oIE1, "FrmReportHealth")
        If $o_SearchForm = 0 Then Return
        $o_Keyword_2 = _IEFormElementGetObjByName ($o_SearchForm, "key_ent")
        $info2 = GUICtrlRead($Input)
        _IEFormElementSetValue ($o_Keyword_2, $info2)
        GUICtrlSetData($Input, "")
        WinSetTitle("Keyboard - Enter the CAPCHA Code", "", "Keyboard - Enter Your PIN Number")
    EndIf
EndFunc ;==>Send_it

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

Everything works great. Some great ideas here. I got the "push" I needed. I'll have to modify some things to suit my needs, but 10x everyone! This is the most responsive forum I've came across!

By the way, is there something like IE.au3, but for Firefox? My intention was to use Firefox, because in fullscreen mode one can turn off all the control bars. I need this to prevent users getting access to desktop.

Edited by dv8

Share this post


Link to post
Share on other sites

Everything works great. Some great ideas here. I got the "push" I needed. I'll have to modify some things to suit my needs, but 10x everyone! This is the most responsive forum I've came across!

By the way, is there something like IE.au3, but for Firefox? My intention was to use Firefox, because in fullscreen mode one can turn off all the control bars. I need this to prevent users getting access to desktop.

as far as i know, there is nothing similar to IE.au3 for FireFox...

Glad you like it... also.. nice work Dickb

8)


NEWHeader1.png

Share this post


Link to post
Share on other sites

Everything works great. Some great ideas here. I got the "push" I needed. I'll have to modify some things to suit my needs, but 10x everyone! This is the most responsive forum I've came across!

By the way, is there something like IE.au3, but for Firefox? My intention was to use Firefox, because in fullscreen mode one can turn off all the control bars. I need this to prevent users getting access to desktop.

I have not tested this but take a look here:

http://www.autoitscript.com/forum/index.php?showtopic=20450

and

http://www.iol.ie/%7Elocka/mozilla/mozilla.htm

Maybe it can do what you want.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0