Jump to content

Examples for interactive coordinate selector or point picker?

Recommended Posts

Hello, it has been a long time since I have posted here :D

I am working on an AutoIt project where I need to enable to user to interactively choose any point or coordinate on the screen... something like a big overlay where the user can click anywhere on the screen to select that point. @UEZ's screenshot tool may have something similar to what I need:

I just checked the source code for the tool and I see that the Mark_Area function has a part in selecting the area to screenshot, I tried to figure out how it works but it is simply too complex and long...

So I was wondering if there were any examples of interactively selecting points on a screen? Maybe an UDF that I can just use in my script to make the whole thing a matter of adding few lines :lmao:o:)

Thank you for the replies in advance!

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

Thank you very much @UEZ :thumbsup:

I fiddled with your _GDIPlus example (which strangely doesn't seem to be using GDI+ much...) and in the end I created this:

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>

; #FUNCTION# ====================================================================================================================
; Name ..........: InteractiveCoordinateSelect
; Description ...: Lets the user interactively select a point on the screen
; Syntax ........: InteractiveCoordinateSelect()
; Parameters ....: None
; Return values .: Success: An array with the X and Y positions of the selected point
;                  Failure: False (This happens when the overlay window is closed)
; Author ........: Damon Harris (TheDcoder)
; Remarks .......: Inspired by UEZ's _GDIPlus_MarkScreenRegionAnimated
; Link ..........: https://git.io/fh9md
; Example .......: See Example.au3
; ===============================================================================================================================
Func InteractiveCoordinateSelect()
    Local $iOriginalGUIMode = Opt("GUIOnEventMode")
    Opt("GUIOnEventMode", 0)

    Local Const $hFullScreen = WinGetHandle("[TITLE:Program Manager;CLASS:Progman]")
    Local Const $aFullScreen = WinGetPos($hFullScreen)

    Local $hOverlay = GUICreate("", $aFullScreen[2], $aFullScreen[3], $aFullScreen[0], $aFullScreen[1], BitOR($WS_CLIPCHILDREN, $WS_POPUP), $WS_EX_TOPMOST)
    GUISetCursor(3 , $GUI_CURSOR_OVERRIDE, $hOverlay)
    WinSetTrans($hOverlay, "", 1)


    Local $aCursorInfo
    While True
        If GUIGetMsg() = $GUI_EVENT_CLOSE Then
            $aCursorInfo = False
        $aCursorInfo = GUIGetCursorInfo($hOverlay)
        If $aCursorInfo[2] Then
            ReDim $aCursorInfo[2]

    Opt("GUIOnEventMode", $iOriginalGUIMode)
    Return $aCursorInfo

A very simplified and barebones version of the UDF... I did however notice that I am not getting my full resolution (1920x1080) when I select the bottom and right most point on my screen, can someone else confirm that this is happening for them as well?

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

For those who want a script that they can paste together to just run the test, here you go:


Func Example()
    Local $aPosition = InteractiveCoordinateSelect()
    ConsoleWrite("X: " & $aPosition[0] & @CRLF & "Y: " & $aPosition[1] & @CRLF) 

; Don't forget to include the function code in your script... if that was not already obvious.

I appricate you testing.

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Dan_555
      Hi, here are few functions for the ListBox.
      I have searched the forum, but most of the functions are for listview, so i took one example code from melba23 (clear selection) and
      wrote few more functions. (Because my current project needs them).
      These functions work only on a Multi-selection ListBox . 
      Edit: Only 1 function does not work with single selection box.
      The functions do: Clear Selection, Delete Selected items, Invert Selection, Move selected items up and down.
       The example code has 2 Listboxes. The selected items on the left ListBox can be moved up and down. The right Listbox has buttons for the other functions.
      #include <GUIConstantsEx.au3> #include <GuiListBox.au3> #include <WindowsConstants.au3> #include <Array.au3> Local $singlesel = 0, $iMsgBoxAnswer = 0 ;MsgBox features: Title=Yes, Text=Yes, Buttons=Yes and No, Icon=Question, Modality=Task Modal $iMsgBoxAnswer = MsgBox(8228, "Choose Listbox selecton type", "Yes for single, No for multi selection box") If $iMsgBoxAnswer = 6 Then $singlesel = 1 ;Yes Local $BL_1,$BL_2,$BR_1,$BR_2,$BR_3,$BR_4,$BR_5,$BR_6 Global $hForm1 = GUICreate("Listbox test", 349, 287) $LB_1 = GUICtrlCreateList("", 6, 40, 157, 244, BitOR($LBS_NOTIFY, $LBS_MULTIPLESEL, $WS_HSCROLL, $WS_VSCROLL, $LBS_DISABLENOSCROLL)) If $singlesel = 1 Then $LB_2 = GUICtrlCreateList("", 179, 40, 157, 244, BitOR($LBS_NOTIFY, $WS_HSCROLL, $WS_VSCROLL, $LBS_DISABLENOSCROLL)) Else $LB_2 = GUICtrlCreateList("", 179, 40, 157, 244, BitOR($LBS_NOTIFY, $LBS_MULTIPLESEL, $WS_HSCROLL, $WS_VSCROLL, $LBS_DISABLENOSCROLL)) $BR_3 = GUICtrlCreateButton("Reverse Sel", 272, 22, 68, 17) EndIf $BL_1 = GUICtrlCreateButton("Up", 20, 3, 35, 18) $BL_2 = GUICtrlCreateButton("Down", 60, 3, 35, 18) $BR_1 = GUICtrlCreateButton("Up", 200, 3, 35, 18) $BR_2 = GUICtrlCreateButton("Down", 240, 3, 35, 18) $BR_4 = GUICtrlCreateButton("Clear Sel", 217, 22, 52, 17) $BR_5 = GUICtrlCreateButton("Delete", 175, 22, 40, 17) $BR_6 = GUICtrlCreateButton("Populate", 290, 3, 50, 18) GUISetState(@SW_SHOW) For $x = 0 To 50 If $x <= 10 Then GUICtrlSetData($LB_1, $x & " test", 0) GUICtrlSetData($LB_2, $x & " Test", 0) Next While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $BL_1 $a = Listbox_ItemMoveUD($LB_1, -1) If $a > -1 Then WinSetTitle($hForm1, "", "Moved items: " & $a) Case $BL_2 $a = Listbox_ItemMoveUD($LB_1, 1) If $a > -1 Then WinSetTitle($hForm1, "", "Moved items: " & $a) Case $BR_1 Listbox_ItemMoveUD($LB_2, -1) Case $BR_2 Listbox_ItemMoveUD($LB_2, 1) Case $BR_3 Listbox_ReverseSelection($LB_2) Case $BR_4 Listbox_ClearSelection($LB_2) Case $BR_5 Listbox_DeleteSelectedItems($LB_2) Case $BR_6 ;Populate GUICtrlSetData($LB_2, "") ; Clears the listbox For $x = 0 To 50 GUICtrlSetData($LB_2, $x & " Test", 0) Next EndSwitch WEnd ;note $hLB_ID - is the Listbox id Func Listbox_DeleteSelectedItems($hLB_ID) Local $aSel = _GUICtrlListBox_GetSelItems($hLB_ID) ;Get selected items Local $i, $slb = 0, $y, $x If $aSel[0] = 0 Then ;If the array is empty, there is no selection, or it is a single selection listbox For $x = 0 To _GUICtrlListBox_GetCount($hLB_ID) - 1 $y = _GUICtrlListBox_GetSel($hLB_ID, $x) If $y = True Then $slb = 1 _GUICtrlListBox_DeleteString($hLB_ID, $x) ;Perform a delete on single sel. LB ExitLoop EndIf Next EndIf If $slb = 0 Then _GUICtrlListBox_BeginUpdate($hLB_ID) For $i = $aSel[0] To 1 Step -1 ;Loop backwards and delete the selected items _GUICtrlListBox_DeleteString($hLB_ID, $aSel[$i]) Next _GUICtrlListBox_EndUpdate($hLB_ID) EndIf EndFunc ;==>Listbox_DeleteSelectedItems Func Listbox_ClearSelection($hLB_ID) ;Removes the selection from multi and single selection ListBox Local $aSel = _GUICtrlListBox_GetSelItems($hLB_ID) ;Code from Melba23 - Autoit Forum Local $slb, $x, $y If $aSel[0] = 0 Then _GUICtrlListBox_SetCurSel($hLB_ID, -1) $slb = 1 EndIf If $slb = 0 Then _GUICtrlListBox_BeginUpdate($hLB_ID) For $i = 1 To $aSel[0] _GUICtrlListBox_SetSel($hLB_ID, $aSel[$i], False) Next _GUICtrlListBox_EndUpdate($hLB_ID) EndIf EndFunc ;==>Listbox_ClearSelection Func Listbox_ReverseSelection($hLB_ID) ;Logically, this function works only on multi-selection listboxes Local $i Local $aCou = _GUICtrlListBox_GetCount($hLB_ID) Local $cSel = _GUICtrlListBox_GetCaretIndex($hLB_ID) ;Save the caret _GUICtrlListBox_BeginUpdate($hLB_ID) For $i = 0 To $aCou _GUICtrlListBox_SetSel($hLB_ID, $i, Not (_GUICtrlListBox_GetSel($hLB_ID, $i))) Next _GUICtrlListBox_SetCaretIndex($hLB_ID, $cSel) ;Restore the caret _GUICtrlListBox_EndUpdate($hLB_ID) EndFunc ;==>Listbox_ReverseSelection Func Listbox_ItemMoveUD($hLB_ID, $iDir = -1) ;Listbox_ItemMoveUD - Up/Down Move Multi/Single item in a ListBox ;$iDir: -1 up, 1 down ;Return values -1 nothing to do, 0 nothing moved, >0 performed moves Local $iCur, $iNxt, $aCou, $aSel, $i, $m = 0, $y, $slb = 0 ;Current, next, Count, Selection, loop , movecount $aSel = _GUICtrlListBox_GetSelItems($hLB_ID) ;Put selected items in an array $aCou = _GUICtrlListBox_GetCount($hLB_ID) ;Get total item count of the listbox If $aSel[0] = 0 Then $y = _GUICtrlListBox_GetCurSel($hLB_ID) If $y > -1 Then _ArrayAdd($aSel, $y) $aSel[0] = 1 $slb = 1 EndIf EndIf ;WinSetTitle($hGUI, "", $aSel[0]) ;Debugging info Select Case $iDir = -1 ;Move Up For $i = 1 To $aSel[0] If $aSel[$i] > 0 Then $iNxt = _GUICtrlListBox_GetText($hLB_ID, $aSel[$i] - 1) ;Save the selection index - 1 text _GUICtrlListBox_ReplaceString($hLB_ID, $aSel[$i] - 1, _GUICtrlListBox_GetText($hLB_ID, $aSel[$i])) ;Replace the index-1 text with the index text _GUICtrlListBox_ReplaceString($hLB_ID, $aSel[$i], $iNxt) ;Replace the selection with the saved var $m = $m + 1 EndIf Next For $i = 1 To $aSel[0] ;Restore the selections after moving If $aSel[$i] > 0 Then If $slb = 0 Then _GUICtrlListBox_SetSel($hLB_ID, $aSel[$i] - 1, 1) Else _GUICtrlListBox_SetCurSel($hLB_ID, $aSel[$i] - 1) EndIf EndIf Next Return $m Case $iDir = 1 ;Move Down If $aSel[0] > 0 Then For $i = $aSel[0] To 1 Step -1 If $aSel[$i] < $aCou - 1 Then $iNxt = _GUICtrlListBox_GetText($hLB_ID, $aSel[$i] + 1) _GUICtrlListBox_ReplaceString($hLB_ID, $aSel[$i] + 1, _GUICtrlListBox_GetText($hLB_ID, $aSel[$i])) _GUICtrlListBox_ReplaceString($hLB_ID, $aSel[$i], $iNxt) $m = $m + 1 EndIf Next EndIf For $i = $aSel[0] To 1 Step -1 ;Restore the selections after moving If $aSel[$i] < $aCou - 1 Then If $slb = 0 Then _GUICtrlListBox_SetSel($hLB_ID, $aSel[$i] + 1, 1) Else _GUICtrlListBox_SetCurSel($hLB_ID, $aSel[$i] + 1) EndIf EndIf Next Return $m EndSelect Return -1 EndFunc ;==>Listbox_ItemMoveUD  
    • By nacerbaaziz
      hi dears, i have a question please
      am now working for an audio player for the blind users
      and i added an option to open a file from the right click context menu.
      but here there is a small problem, i hope that you can help me to find a solution for it.
      the problem is as follow :
      when i added the problem into the context menu it work with successfuly if the user select just one file.
      but if the user selected more then one file, here the problem
      a multiple sections of the program will open.
      my question is :
      how i can detect the multiple selection from the context menu?
      i know that it so  hard, but i know also that you are harder than it
      for that dears please try to give me an solution for that problem.
      thanks in advance
    • By Eggie6
      Hi all,
      I have a script 
      #include <IE.au3> Local $oIE = _IEAttach("anythinghere") $tags = $oIE.document.GetElementsByTagName("div") $oCollection = $oIE.document.getElementsByClassName("x-panel-btn-td") MsgBox(0, "", $oCollection.length) which shows 8 results, how can i see what those results are, and i need also to get ID of 4th in a row of them

      Please help
    • By tentacole
      This is my first post, so I apologize if this is in the wrong place. 
      I've created a while loop to click in a certain area of an application, and have the y axis change at the end of each loop. My loop continues to click at x:27, x:10, even though the $y is adding 15 at the end of each loop. I did a Send($x) and Send($y) into a Notepad to see if the $y had changed after each loop, and the 15 was being added to $y each loop. (If this makes sense)
      I'm unsure where I'm going wrong, and would be extremely grateful if someone can point me in the correct direction to fix this. 
      Local $rDirectory = "H:\oDemandProject\fList.txt" Local $rLine = _FileCountLines($rDirectory) Local $x = 27 Local $y = 10 While $rline > 0 WinActivate("OnDemand", "-> 1") WinWaitActive("OnDemand", "-> 1") ControlClick("OnDemand", "", "[CLASS:AfxFrameOrView120u; INSTANCE:1]", "left", 4, $x, $y) $y += 15 $rLine -= 1 WEnd  
    • By ahha
      I'm using the excellent _Excel UDF to read a whole spreadsheet into an array I want to operate on.  I can read a specified range without a problem.  What I'd like to do is allow the user in Excel to select a range (contiguous is fine) and be able to read that range so that I know what range to operate on without having the user input the range via a keyboard.  I'm stuck and any hints greatly appreciated.  The code below uses the Helpfile .xls to illustrate where I crash.
      ; #AutoIt3Wrapper_run_debug_mode=Y ; use this to debug in console window <--- LOOK #include <Excel.au3> #include <MsgBoxConstants.au3> #include <Debug.au3> ;for _DebugArrayDisplay $sExcelFullFileName = "C:\Program Files (x86)\AutoIt3\Examples\Helpfile\Extras\_Excel1.xls" ;use Autoit test for example $oExcel = _Excel_Open() ;Create application object and open an Excel workbook $oWorkbook = _Excel_BookOpen($oExcel, $sExcelFullFileName) $aExcelArray = _Excel_RangeRead($oWorkbook, Default) _DebugArrayDisplay($aExcelArray, $aExcelArray) ;let's look at it $aI5K6 = _Excel_RangeRead($oWorkbook, Default,"I5:K6") ;read a rangeI5:K6 and return value _DebugArrayDisplay($aI5K6, $aI5K6) ;let's look at range it ;rather than defining the range explicity I'd like to read the selection in Excel and use it ;need a function like _Excel_SelectionRead ;try hacking Excel.au3 _Excel_RangeRead to see if can create _Excel_SelectionRead looks like .Selection .Value .Text and .Address possible MsgBox($MB_SYSTEMMODAL, "Info", "Select cells in Excel then click OK") ;ACTUALLY get range of the selection and this will suffice for us to use on the $aExcelArray $vRange = $oWorkbook.ActiveSheet.Selection.Address($vRange) ;tried all sorts of permutations and clearly I need help _DebugArrayDisplay($vRange, $vRange) ;let's look at range it ;$vResult = $oExcel.Transpose($vRange.Value) ;_DebugArrayDisplay($vResult, $vResult) ;let's look at range it _Excel_Close($oExcel, False, True) Exit  
  • Create New...