Danp2 Posted July 1, 2009 Share Posted July 1, 2009 I am attempting to use the elementFromPoint method to retrieve the text below the mouse cursor. In testing, it seems like the Y coordinates are off by approx the width of the menu, toolbars, and tabs. Is this normal? If so, how should I adjust for this? expandcollapse popup#include <IE.au3> #Include <GUIConstantsEx.au3> Opt("MouseCoordMode", 2) HotKeySet("^+a", "Test") While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit EndSwitch WEnd Func Test() Local $oIE, $pos, $element, $t $pos = MouseGetPos() ; get topmost browser tab $oIE = _IEAttach ("", "instance", 1) If @error = $_IEStatus_Success Then ; get window title $hwnd = _IEPropertyGet($oIE, "hwnd") $title = WinGetTitle($hwnd) $oIE = _IEAttach($title, "windowtitle") EndIf ;MsgBox(0,'yoffset', _IEPropertyGet($oIE, "browsery")) $element = $oIE.document.elementFromPoint($pos[0], $pos[1]) If IsObj($element) Then MsgBox(0,$pos[0] & ' ' & $pos[1], _IEPropertyGet($element, 'innertext')) EndIf EndFunc Also, I think I found an issue with the _IEPropertyGet function. To see, uncomment the line MsgBox(0,'yoffset', _IEPropertyGet($oIE, "browsery")) Dan Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
Authenticity Posted July 1, 2009 Share Posted July 1, 2009 #include <IE.au3> #include <WinAPI.au3> $_IEErrorNotify = False Dim $oIE = _IECreate('http://www.google.com') _IEErrorHandlerRegister() Dim $hWnd = ControlGetHandle(_IEPropertyGet($oIE, 'hwnd'), '', 'Internet Explorer_Server1') Dim $tPt ;_IEHeadInsertEventScript($oIE, 'document', 'onmousedown', 'alert("clientX: " + event.clientX + " clientY: " + event.clientY + "\noffsetX: " + event.offsetX + " offsetY: " + event.offsetY + "\nsrcElement: " + event.srcElement + " srcElementId: " + event.srcElement.id + " srcElementName: " + event.srcElement.name, 0, "Info");') While _IEPropertyGet($oIE, 'visible') Sleep(800) $tPt = _WinAPI_GetMousePos(True, $hWnd) $oElem = _elementFromPoint(DllStructGetData($tPt, 1), DllStructGetData($tPt, 2)) ConsoleWrite($oElem.tagName & ': ' & $oElem.id & ', ' & $oElem.name & @LF) WEnd Func _elementFromPoint($iX, $iY) Return $oIE.document.elementFromPoint($iX, $iY) EndFunc Maybe you can use something like _IEHeadInsertEventScript() with document event 'onmousemove' and use IEEval() (search the forum). Link to comment Share on other sites More sharing options...
Danp2 Posted July 1, 2009 Author Share Posted July 1, 2009 That's some interesting code, but I don't think I can use it to address my particular need. In the example I posted, I'm using a hotkey to allow the user to execute a routine after they've positioned the mouse over the desired element. I need to be able to determine the element the mouse is positioned over. Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
Danp2 Posted July 10, 2009 Author Share Posted July 10, 2009 FYI, I finally got this working by subtracting the value of $oIE.document.parentwindow.screenTop from the vertical position returned by MouseGetPos(). Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
DaleHohm Posted July 10, 2009 Share Posted July 10, 2009 I was actually reluctant to implement the coordinate routines in _IEPropertyGet because they have a history of being calculated incorrectly and changing from IE version to IE version. These did work when they were first released. What browser version are you testing with? Dale Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model Automate input type=file (Related) Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded Better Better? IE.au3 issues with Vista - Workarounds SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead? Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble Link to comment Share on other sites More sharing options...
Danp2 Posted July 11, 2009 Author Share Posted July 11, 2009 I was actually reluctant to implement the coordinate routines in _IEPropertyGet because they have a history of being calculated incorrectly and changing from IE version to IE version. These did work when they were first released. What browser version are you testing with?DaleIE8 Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
DaleHohm Posted July 12, 2009 Share Posted July 12, 2009 I see that I didn't read your post carefully enough. Thought you were using the _IEPropertyGet coordinate functions, be you were not. The error you report with _IEPropertyGet($oIE, "browsery") is because I am neglecting to trap the error and send back bad object type (what you are asking for is invalid). The browserX and browserY can only be used on a document element, not on the browser itself as you attempted to do here. Dale Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model Automate input type=file (Related) Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded Better Better? IE.au3 issues with Vista - Workarounds SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead? Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble Link to comment Share on other sites More sharing options...
Danp2 Posted July 12, 2009 Author Share Posted July 12, 2009 Based on your experience, is there a better way to identify the element below the mouse cursor? Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
DaleHohm Posted July 12, 2009 Share Posted July 12, 2009 Based on your experience, is there a better way to identify the element below the mouse cursor?No, this would be the most straight forward method. Once you figure out the geometry it should work reliably.Dale Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model Automate input type=file (Related) Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded Better Better? IE.au3 issues with Vista - Workarounds SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead? Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble Link to comment Share on other sites More sharing options...
Danp2 Posted January 9, 2010 Author Share Posted January 9, 2010 I've been struggling with this for a while now, and I think that I finally found a reliable way to get the mouse coordinates within the client region by using the createEventObject() function: $evt = $oIE.document.createEventObject() $element = $oIE.document.elementFromPoint($evt.clientX, $evt.clientY) Using the above code, the event object's clientX & clientY properties are used to retrieve the element below the mouse cursor. In my limited testing, I have yet to find a scenario where this code doesn't work. Thoughts? Dan Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
Danp2 Posted October 8, 2012 Author Share Posted October 8, 2012 I've been struggling with this for a while now, and I think that I finally found a reliable way to get the mouse coordinates within the client region by using the createEventObject() function: $evt = $oIE.document.createEventObject() $element = $oIE.document.elementFromPoint($evt.clientX, $evt.clientY) I've been successfully using this method with IE for several years. Now I want to do the same thing with Firefox. Can this be done without using the onmousemove event handler? Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
AdmiralAlkex Posted October 8, 2012 Share Posted October 8, 2012 I've been successfully using this method with IE for several years. Now I want to do the same thing with Firefox. Can this be done without using the onmousemove event handler?I don't know. Perhaps the FF UDF. Did you look if it has something like that? .Some of my scripts: ShiftER, Codec-Control, Resolution switcher for HTC ShiftSome of my UDFs: SDL UDF, SetDefaultDllDirectories, Converting GDI+ Bitmap/Image to SDL Surface Link to comment Share on other sites More sharing options...
Danp2 Posted October 9, 2012 Author Share Posted October 9, 2012 Yes, I'm using the FF UDF to interact with the website. Since my earlier post, I've been playing around with using the _FFCmd function to temporarily adding a listener for the mousemove event. Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
Danp2 Posted October 9, 2012 Author Share Posted October 9, 2012 OK... here's what I've come up with thus far using FF UDF: Local $pos = MouseGetPos(), $element _FFConnect(Default, Default, 3000) _FFObjNew("mouseevent", "null") _FFCmd("FFau3.mouseevent=null;") _FFCmd("var listener = function (event) {FFau3.mouseevent = event};") _FFCmd("content.document.addEventListener('mousemove',listener, false);") ; Force minimal mouse movement MouseMove($pos[0]+1, $pos[1]) MouseMove($pos[0]-1, $pos[1]) _FFCmd("content.document.removeEventListener('mousemove',listener, false);") $element = _FFCmd("content.document.elementFromPoint(FFau3.mouseevent.clientX, FFau3.mouseevent.clientY)") Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
Danp2 Posted October 17, 2012 Author Share Posted October 17, 2012 Here is my revised version, which doesn't require the MouseMove trick: Func _FFGetMouseElement($aPoint) Local $aFFPos $aFFPos = _ScreenToClient(_FFWindowGetHandle(), $aPoint[0], $aPoint[1]) ; Adjust Y value for height of Tabs, Toolbars, etc $aFFPos[1] -= _FFCmd("window.content.mozInnerScreenY - window.mozInnerScreenY") _FFCmd("FFau3.xpath=FFau3.WCD.elementFromPoint(" & $aFFPos[0] & ", " & $aFFPos[1] & ");") EndFunc Func _ScreenToClient($h_wnd, $i_x, $i_y); Returns an array, [0] = x, [1] = y If IsString($h_wnd) Then $h_wnd = WinGetHandle($h_wnd) Local $t_xy_coords = DllStructCreate("int x; int y") DllStructSetData($t_xy_coords, "x", $i_x) DllStructSetData($t_xy_coords, "y", $i_y) DllCall("user32.dll", "int", "ScreenToClient", "hwnd", $h_wnd, "ptr", DllStructGetPtr($t_xy_coords)) Local $a_ret_coords[2] = [DllStructGetData($t_xy_coords, "x"), DllStructGetData($t_xy_coords, "y")] Return $a_ret_coords EndFunc Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
chachew Posted May 27, 2014 Share Posted May 27, 2014 (edited) So this works very well for me except if i move my cursor out of the IE browser window. I get the error that the Variable must be of type "Object".: ConsoleWrite($oElem.tagName & ': ' & $oElem.id & $oElem.name & @LF) ConsoleWrite($oElem^ ERROR Is there a way to ignore the error while i dont have the cursor in the IE window? Edited May 27, 2014 by chachew Link to comment Share on other sites More sharing options...
JohnOne Posted May 27, 2014 Share Posted May 27, 2014 Yes. Use IsObj() function before trying to use an object. AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now