Finding position of a string in an active window

[ferradavi]

Hi everyone!

How to find the starting position of a string (substring) in an active window?

Thanks 2 all 4 your attention

[willichan]

I think you will need to clarify your question a bit.  Do you mean finding the x,y coordinates of text that appears in a window?

Can you give an example of what you are trying to do?

[ferradavi]

I mean finding the x, y coordinates of text, i.e. in this window (or page if u prefer) I wanna move the mouse on "Started by ferradavi" (right under the topic's title), or other text I decide, when/if visible.

Edited March 31, 2016 by ferradavi

[markyrocks]

Which browser?  The ie udf I believe can find a button or other type of control based on the text it displays.   I was looking into a problem like your of my own and really the easiest way is in Chrome is to send("^f)  then send whatever it is your trying to find.   This will highlight the text in a colored box.  Then pixelsearch the screen for the color of the box.... I really haven't found a better way yet.

I think it's possible to search the html source of the page but idk if there is a way to translate that into a specific location.   Now if what your describing is a link in the ie udf it's entirely possible to click on said link by the text but idk if you can just mousemove to it without clicking.   Probably would have to look at the code of the function that controls this.  See how it operates and modify it

Edited March 31, 2016 by markyrocks

[ferradavi]

I would like to find a specific string, not necessarily in a browser page but in an active window (it could be in internet explorer or in notepad, word, excel...).

So, when the string is found the mouse moves on that string.

That's all

[ferradavi]

1 hour ago, markyrocks said:

Which browser?  The ie udf I believe can find a button or other type of control based on the text it displays.   I was looking into a problem like your of my own and really the easiest way is in Chrome is to send("^f)  then send whatever it is your trying to find.   This will highlight the text in a colored box.  Then pixelsearch the screen for the color of the box.... I really haven't found a better way yet.

I think it's possible to search the html source of the page but idk if there is a way to translate that into a specific location.   Now if what your describing is a link in the ie udf it's entirely possible to click on said link by the text but idk if you can just mousemove to it without clicking.   Probably would have to look at the code of the function that controls this.  See how it operates and modify it

markyrocks,

maybe in your case it could be useful something like this:

expandcollapse popup

#include <IE.au3>
#include <Array.au3>

Local $obj_Internet = _IECreate("https://www.autoitscript.com/forum/topic/181577-finding-position-of-a-string-in-an-active-window/")

$ai_pos = _Client_XY_Word($obj_Internet, "autoit")
Switch @error
    Case 0
        _ArrayDisplay($ai_pos)
    Case 1
        MsgBox(16, "Error", "Not able to get the Tags Collection.")
    Case 2
        MsgBox(16, "Error", "String Not found")
EndSwitch
_IEQuit($obj_Internet)

Func _Client_XY_Word(ByRef $o_InternetExplorer, $s_word)

    $oATags = _IETagNameAllGetCollection($o_InternetExplorer)
    If IsObj($oATags) = 0 Then Return SetError(1, 0, 0)

    Local $as_NewID[1], $iPos

    For $oATag In $oATags
        $iPos = StringInStr($oATag.innerHTML, $s_word)
        If $iPos Then _Insert_ID($oATag, $as_NewID, $iPos, $s_word, $o_InternetExplorer)
    Next

    _ArrayDelete($as_NewID, 0)
    If IsArray($as_NewID) = 0 Then Return SetError(2, 0, 0)

    Local $1DArray[1], $oTemp
    For $i = 0 To UBound($as_NewID) - 1
        $oTemp = _IEGetObjById($o_InternetExplorer, $as_NewID[$i])
        If IsObj($oTemp) = 0 Then
            _ArrayAdd($1DArray, "")
            _ArrayAdd($1DArray, "")
        EndIf

        _ArrayAdd($1DArray, $oTemp.offsetLeft)
        _ArrayAdd($1DArray, $oTemp.offsetTop)
    Next

    Local $2DArray[UBound($as_NewID) / 2][2]
    For $i = 0 To (UBound($as_NewID) / 2) - 1

        $2DArray[$i][0] = $1DArray[(($i + 1) * 2) - 1]
        $2DArray[$i][1] = $1DArray[($i + 1) * 2]

    Next
    Return $2DArray
EndFunc   ;==>_Client_XY_Word


Func _Insert_ID(ByRef $o_Object, ByRef $sArray, $iWord_Pos, $s_word, ByRef $o_InternetExplorer)

    Static $iCount
    Switch $o_Object.tagname
        Case "HTML", "HEAD", "TITLE", "SCRIPT", "STYLE", "META"
            Return
    EndSwitch

    ConsoleWrite($o_Object.tagname & @CRLF)

    $iCount += 1
    _IEPropertySet($o_Object, "innerHTML", StringMid($o_Object.innerHTML, 1, $iWord_Pos - 1) & _    ;the HTML before the word
            "<span id = 'Autoitv.3 #" & $iCount & "'>" & $s_word & "</span>" & _ ;the word with a new tag
            StringMid($o_Object.innerHTML, $iWord_Pos + StringLen($s_word) + 1)) ;the rest of the HTML.
    If @error Then MsgBox(16, "Error", @error)  ;no error but still the span isn't found......???

    _ArrayAdd($sArray, "Autoitv.3 #" & $iCount)
EndFunc   ;==>_Insert_ID

 

Edited March 31, 2016 by ferradavi

[markyrocks]

Ty for that btw.  I may have to use that.  Here's the issue,   autoit doesn't have a built in text reader.  The text has to exist in some form of data.  It is entirely possible to extract "visible text"  using wingettext  ().  This will tell you if the text exists in the window.   This doesn't work for every program,   the text has to be visible to the autoit info tool.  

From the results of wingettext  () there really is noway to pinpoint the exact position.   I'm sure many people have contemplated building an actual text reader that is pixel based.  It's just very long and time consuming process.  Trust me what your describing could be very useful.  Part of the issue though is different font styles and sizes would all need to be taken into consideration.   

As I described before the only way to do it that I know of is through some kind of work around and the problem with that is any work around would be program specific.   So if the text could appear on a variety of windows from different programs you'd basically have to build a work around for each specific potential scenario.  

[markyrocks]

Something i forgot that is possible would be if the text is predictable as far as size style ect.  It maybe possible to use this here this this.... I'm having an issue  just google autoit bmp in bmp.... I can't post a link for whatever reason 

Edited March 31, 2016 by markyrocks

[markyrocks]

Something that just dawned on me is that it is definitely possible to  say have a random window and (one way or another )  selectall,  copy then search the clipboard for the text.   And through some trickery determine some kinda position 

[AutoBert]

I don't believe it's solvable to do this job with only one method. But when you will try it with help of the ClipBoard-funcs read first all about Clipboard Management,

_ClipBoard_EnumFormats

is a good point to start.

[ferradavi]

Thank you guys!

I will try asap.

[BrewManNH]

What exactly is the point of finding a string of text in some program somewhere on the screen and clicking on it, or at the very least moving the mouse pointer to it?

I can't think of a logical reason for wanting to do this in some random program, so I'm guessing that there's a specific reason and a specific program you're looking to do this in, correct?