All visible text from all available controls?

[Maggotus]

My first post - please forgive me as I couldn't find anything as I dug for answers.

I've written simple scripts in the past to place visible or hidden text to clipboard, but this one has me stumped. I know that certain applications just can't be read from, but I'm curious whether the text is in a control that is behind another control, and not accessible from the Window Info tool.

Is anyone able to help me write a script that can loop through all available controls within a window, and output any visible or hidden text to a text file? It seems this is the only way to confirm that the text is truly unavailable to AutoIt.

OCR is unfortunately not an option for me in this particular implementation.

Thanks in advance.

[Tvern]

You can use_WinAPI_EnumWindows() and then WinGetText() on the returned handles.

Here is something I never got around to finishing that will give you an example. (Right click any window to use)

expandcollapse popup

#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include <Constants.au3>
#include <Misc.au3>
#include <Array.au3>
#Include <GuiListView.au3>
#Include <GuiTreeView.au3>

Opt("GUIOnEventMode",1)
Global $tPoint = DllStructCreate($tagPoint)
Global $ahTreeViewItems[1] = [0]
Global $ahWindows[1]
Global $hWndItemClicked
Global $hGUI = GUICreate("Extended Window Info",800,500)
GUISetOnEvent(-3,"_Exit")
Global $hTreeView = _GUICtrlTreeView_Create($hGUI,10,10,200,400)
Global $hListView = _GUICtrlListView_Create($hGUI,"",220,10,410,400)
_GUICtrlListView_InsertColumn($hListView, 1, "Property", 100)
_GUICtrlListView_InsertColumn($hListView, 1, "Value", 400)
GUISetState()
GUIRegisterMsg($WM_NOTIFY, "WM_NOTIFY")

While 1
    If _IsPressed("02") Then _WinGetInfo()
    If $hWndItemClicked Then _ControlInfo()
    Sleep(10)
WEnd

Func _WinGetInfo()
    _DestroyTreeItems()
    _GUICtrlTreeView_BeginUpdate($hTreeView)
    Local $hWnd
    _Pos()
    $hWnd = _WinAPI_WindowFromPoint($tPoint)
    $hWnd = _WinAPI_GetAncestor($hWnd,2)
    _CreateNodeRec($hWnd, $hTreeView)
    _GUICtrlTreeView_EndUpdate($hTreeView)
    While _IsPressed("0D")
        Sleep(10)
    WEnd
EndFunc

Func _CreateNodeRec($hWnd, $hParentTreeItem)
    $hParentTreeItem = _AddTreeItem($hWnd, $hParentTreeItem)
    Local $aWins = _WinAPI_EnumWindows(False, $hWnd)
    For $i = 1 To $aWins[0][0]
        If _WinAPI_GetParent($aWins[$i][0]) = $hWnd Then
            _CreateNodeRec($aWins[$i][0], $hParentTreeItem)
        EndIf
    Next
EndFunc

Func _AddTreeItem($hWnd, $hParentTreeItem)
    ConsoleWrite($hWnd & @CRLF)
    _ArrayAddValue($ahWindows,$hWnd)
    If $hParentTreeItem = $hTreeView Then
        _ArrayAddValue($ahTreeViewItems,_GUICtrlTreeView_Add($hTreeView,0,_WinAPI_GetClassName($hWnd)))
    Else
        _ArrayAddValue($ahTreeViewItems,_GUICtrlTreeView_AddChild($hTreeView,$hParentTreeItem,_WinAPI_GetClassName($hWnd)))
    EndIf
    Return $ahTreeViewItems[$ahTreeViewItems[0]]
EndFunc

Func _ArrayAddValue(ByRef $avArray, $vValue)
    $avArray[0] += 1
    ReDim $avArray[$avArray[0]+1]
    $avArray[$avArray[0]] = $vValue
EndFunc

Func _ControlInfo()
    _GUICtrlListView_DeleteAllItems($hListView)
    _GUICtrlListView_BeginUpdate($hListView)
    For $i = 1 To $ahTreeViewItems[0]
        If $hWndItemClicked = $ahTreeViewItems[$i] Then
            If Not WinExists($ahWindows[$i]) Then
                _DestroyTreeItems()
                ExitLoop
            EndIf
            Local $aPos = WinGetPos($ahWindows[$i])
            _AddListItem("Title", WinGetTitle($ahWindows[$i]))
            _AddListItem("Class", _WinAPI_GetClassName($ahWindows[$i]))
            _AddListItem("Text", WinGetText($ahWindows[$i]))
            _AddListItem("Position", $aPos[0] & "x" & $aPos[1])
            _AddListItem("Size", $aPos[2] & "x" & $aPos[3])
            _AddListItem("Style", Hex(_WinAPI_GetWindowLong($ahWindows[$i], $GWL_STYLE),8))
            _AddListItem("ExStyle", Hex(_WinAPI_GetWindowLong($ahWindows[$i], $GWL_EXSTYLE),8))
            _AddListItem("Handle", $ahWindows[$i])
            ExitLoop
        EndIf
    Next
    _GUICtrlListView_EndUpdate($hListView)
    $hWndItemClicked = False
    Return
EndFunc

Func _DestroyTreeItems()
    _GUICtrlTreeView_BeginUpdate($hTreeView)
    _GUICtrlTreeView_DeleteAll($hTreeView)
    _GUICtrlTreeView_EndUpdate($hTreeView)
    Global $ahTreeViewItems[1] = [0]
    Global $ahWindows[1] = [0]
    _GUICtrlListView_AddSubItem($hListView,_GUICtrlListView_AddItem($hListView, "Error!"),"Window is no longer valid",1)
EndFunc

Func _AddListItem($sProperty, $sValue)
    _GUICtrlListView_AddSubItem($hListView,_GUICtrlListView_AddItem($hListView, $sProperty),$sValue,1)
EndFunc

Func WM_NOTIFY($hWnd, $Msg, $wParam, $lParam)
    #forceref $hWnd, $Msg, $wParam
    Local $hWndFrom, $iIDFrom, $iCode, $tNMHDR, $tInfo
    $tNMHDR = DllStructCreate($tagNMHDR, $lParam)
    $hWndFrom = HWnd(DllStructGetData($tNMHDR, "hWndFrom"))
    $iIDFrom = DllStructGetData($tNMHDR, "IDFrom")
    $iCode = DllStructGetData($tNMHDR, "Code")
    Switch $hWndFrom
        Case $hTreeView
            Switch $iCode
                Case $NM_CLICK
                    $hWndItemClicked = _GUICtrlTreeView_HitTestItem($hTreeView, _WinAPI_GetMousePosX(True, $hTreeView), _WinAPI_GetMousePosY(True, $hTreeView))
            EndSwitch
    EndSwitch
    Return 0
EndFunc

Func _Pos()
    Local $aPos = MouseGetPos()
    DllStructSetData($tPoint, "x", $aPos[0])
    DllStructSetData($tPoint, "y", $aPos[1])
    Return
EndFunc

Func _Exit()
    Exit
EndFunc

Once you have the list, you could also filter out any control that does not overlap the mousepos.

[Maggotus]

Excellent - Let me give this a shot and see what I can do with it. Thank you.