ferradavi

Finding position of a string in an active window

12 posts in this topic

Hi everyone!

How to find the starting position of a string (substring) in an active window?

Thanks 2 all 4 your attention

;)

Share this post


Link to post
Share on other sites



I think you will need to clarify your question a bit.  Do you mean finding the x,y coordinates of text that appears in a window?

Can you give an example of what you are trying to do?

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

I mean finding the x, y coordinates of text, i.e. in this window (or page if u prefer) I wanna move the mouse on "Started by ferradavi" (right under the topic's title), or other text I decide, when/if visible.

Edited by ferradavi

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Which browser?  The ie udf I believe can find a button or other type of control based on the text it displays.   I was looking into a problem like your of my own and really the easiest way is in Chrome is to send("^f)  then send whatever it is your trying to find.   This will highlight the text in a colored box.  Then pixelsearch the screen for the color of the box.... I really haven't found a better way yet.

I think it's possible to search the html source of the page but idk if there is a way to translate that into a specific location.   Now if what your describing is a link in the ie udf it's entirely possible to click on said link by the text but idk if you can just mousemove to it without clicking.   Probably would have to look at the code of the function that controls this.  See how it operates and modify it

Edited by markyrocks

Share this post


Link to post
Share on other sites

I would like to find a specific string, not necessarily in a browser page but in an active window (it could be in internet explorer or in notepad, word, excel...).

So, when the string is found the mouse moves on that string.

That's all

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

1 hour ago, markyrocks said:

Which browser?  The ie udf I believe can find a button or other type of control based on the text it displays.   I was looking into a problem like your of my own and really the easiest way is in Chrome is to send("^f)  then send whatever it is your trying to find.   This will highlight the text in a colored box.  Then pixelsearch the screen for the color of the box.... I really haven't found a better way yet.

I think it's possible to search the html source of the page but idk if there is a way to translate that into a specific location.   Now if what your describing is a link in the ie udf it's entirely possible to click on said link by the text but idk if you can just mousemove to it without clicking.   Probably would have to look at the code of the function that controls this.  See how it operates and modify it

markyrocks,

maybe in your case it could be useful something like this:

#include <IE.au3>
#include <Array.au3>

Local $obj_Internet = _IECreate("https://www.autoitscript.com/forum/topic/181577-finding-position-of-a-string-in-an-active-window/")

$ai_pos = _Client_XY_Word($obj_Internet, "autoit")
Switch @error
    Case 0
        _ArrayDisplay($ai_pos)
    Case 1
        MsgBox(16, "Error", "Not able to get the Tags Collection.")
    Case 2
        MsgBox(16, "Error", "String Not found")
EndSwitch
_IEQuit($obj_Internet)

Func _Client_XY_Word(ByRef $o_InternetExplorer, $s_word)

    $oATags = _IETagNameAllGetCollection($o_InternetExplorer)
    If IsObj($oATags) = 0 Then Return SetError(1, 0, 0)

    Local $as_NewID[1], $iPos

    For $oATag In $oATags
        $iPos = StringInStr($oATag.innerHTML, $s_word)
        If $iPos Then _Insert_ID($oATag, $as_NewID, $iPos, $s_word, $o_InternetExplorer)
    Next

    _ArrayDelete($as_NewID, 0)
    If IsArray($as_NewID) = 0 Then Return SetError(2, 0, 0)

    Local $1DArray[1], $oTemp
    For $i = 0 To UBound($as_NewID) - 1
        $oTemp = _IEGetObjById($o_InternetExplorer, $as_NewID[$i])
        If IsObj($oTemp) = 0 Then
            _ArrayAdd($1DArray, "")
            _ArrayAdd($1DArray, "")
        EndIf

        _ArrayAdd($1DArray, $oTemp.offsetLeft)
        _ArrayAdd($1DArray, $oTemp.offsetTop)
    Next

    Local $2DArray[UBound($as_NewID) / 2][2]
    For $i = 0 To (UBound($as_NewID) / 2) - 1

        $2DArray[$i][0] = $1DArray[(($i + 1) * 2) - 1]
        $2DArray[$i][1] = $1DArray[($i + 1) * 2]

    Next
    Return $2DArray
EndFunc   ;==>_Client_XY_Word


Func _Insert_ID(ByRef $o_Object, ByRef $sArray, $iWord_Pos, $s_word, ByRef $o_InternetExplorer)

    Static $iCount
    Switch $o_Object.tagname
        Case "HTML", "HEAD", "TITLE", "SCRIPT", "STYLE", "META"
            Return
    EndSwitch

    ConsoleWrite($o_Object.tagname & @CRLF)

    $iCount += 1
    _IEPropertySet($o_Object, "innerHTML", StringMid($o_Object.innerHTML, 1, $iWord_Pos - 1) & _    ;the HTML before the word
            "<span id = 'Autoitv.3 #" & $iCount & "'>" & $s_word & "</span>" & _ ;the word with a new tag
            StringMid($o_Object.innerHTML, $iWord_Pos + StringLen($s_word) + 1)) ;the rest of the HTML.
    If @error Then MsgBox(16, "Error", @error)  ;no error but still the span isn't found......???

    _ArrayAdd($sArray, "Autoitv.3 #" & $iCount)
EndFunc   ;==>_Insert_ID

 

Edited by ferradavi

Share this post


Link to post
Share on other sites

Ty for that btw.  I may have to use that.  Here's the issue,   autoit doesn't have a built in text reader.  The text has to exist in some form of data.  It is entirely possible to extract "visible text"  using wingettext  ().  This will tell you if the text exists in the window.   This doesn't work for every program,   the text has to be visible to the autoit info tool.  

From the results of wingettext  () there really is noway to pinpoint the exact position.   I'm sure many people have contemplated building an actual text reader that is pixel based.  It's just very long and time consuming process.  Trust me what your describing could be very useful.  Part of the issue though is different font styles and sizes would all need to be taken into consideration.   

As I described before the only way to do it that I know of is through some kind of work around and the problem with that is any work around would be program specific.   So if the text could appear on a variety of windows from different programs you'd basically have to build a work around for each specific potential scenario.  

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

Something i forgot that is possible would be if the text is predictable as far as size style ect.  It maybe possible to use this here this this.... I'm having an issue  just google autoit bmp in bmp.... I can't post a link for whatever reason 

Edited by markyrocks

Share this post


Link to post
Share on other sites

Something that just dawned on me is that it is definitely possible to  say have a random window and (one way or another )  selectall,  copy then search the clipboard for the text.   And through some trickery determine some kinda position 

Share this post


Link to post
Share on other sites

I don't believe it's solvable to do this job with only one method. But when you will try it with help of the ClipBoard-funcs read first all about Clipboard Management,

_ClipBoard_EnumFormats

is a good point to start.

Share this post


Link to post
Share on other sites

Thank you guys!

I will try asap.

Share this post


Link to post
Share on other sites

What exactly is the point of finding a string of text in some program somewhere on the screen and clicking on it, or at the very least moving the mouse pointer to it?

I can't think of a logical reason for wanting to do this in some random program, so I'm guessing that there's a specific reason and a specific program you're looking to do this in, correct?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now