Get text under mouse in Firefox

[lod3n]

This is from one of my responses to a support posting here:

http://www.autoitscript.com/forum/index.php?showtopic=32546

I'm posting it in scripts and scraps too because I think it's really cool, and has a lot of potential. I think using MSA might provide us with a way of automating Firefox similar to what we can do with IE. I know some people here would really be interested in that possibility. This page:

http://www.mozilla.org/access/windows/msaa-server

seems to indicate that quite a bit of the DOM would be accessible to us. However, I am clueless as to how to proceed, or whether we even can just using COM.

I stumbled upon MSAA or MSA, meaning Microsoft Active Accessibility, which sounds like it might provide what you're looking for. The trouble is that obtaining an IAccessible object with the commands AccessibleObjectFromWindow or AccessibleObjectFromPoint from oleacc.dll is not something that I know how to do. Perhaps someone on this forum whose kung fu is superior to mine could take a crack at it.

In the mean time, someone named Jose Roca has provided some PowerBASIC samples that include a nifty EXE which is easily scrapable in AutoIt. Take a look here:

http://com.it-berater.org/other_wrappers.htm

Scroll to the bottom and download TB_IACCS.zip, extract it, and create Iaccessible.au3:

run("EX_TestIAcc_01.exe")
winwait("IAccessible Test")
$hwnd = WinGetHandle("IAccessible Test","")
$cHwnd = ControlGetHandle($hwnd,"","Edit1")
winsetstate($hwnd,"",@SW_MINIMIZE)
$oldtext = ""
while 1
    sleep(100)
    $text = ControlGetText ($hwnd, "text", $cHwnd)
    if $text <> $oldtext Then
        $oldtext = $text
        $pos = MouseGetPos()
        ToolTip ($text , $pos[0]+10 , $pos[1]+10)
    EndIf
    if not winexists($hwnd) then exitloop
WEnd

This works pretty well on established software. I can even hover my mouse over text in Firefox and AutoIt can retreive the text, even from an Input tag. This could be the basis for your translation script, but the way I have implemented it is extremely unsatisfying.

Edited September 11, 2006 by lod3n