IE elementFromPoint fails intermittently with Bad variable type error

[NRK]

Hi All, 

The below mentioned code to highlight HTML element under mouse pointer works fine. But it fails intermittently in the  $oDoc.elementFromPoint call and exits the script. The error number is "0x80020008" and the error description is "Bad variable type". I am using 64 bit windows 7 Enterprise edition Service pack1 with IE 11. Please help me in solving this issue. I tried all possible combinations to make it work consistently, but still it fails. 

If it is a known open issue, please let me know any alternate approach without using elementFromPoint call. 

Thanks for your support. 

expandcollapse popup

#include <IE.au3>
#include <WinAPI.au3>
#include <GUIConstantsEx.au3>
; #include <WinAPIEx.au3>
#include <WindowsConstants.au3>
Local $oIE = _IECreate("https://www.autoitscript.com/forum/")
Local $oDoc = _IEDocGetObj($oIE)
Local $oIE_hwnd = _IEPropertyGet($oIE, "hwnd")
Local $oElement, $oElement_save, $aMousePos
Local $aBrowserPos[6] ; x , y , width , height , x right , y bottom
Local $aOverlayPos[4] ; coordinates of the overlay frame
; Create a Phantom window with a visible frame
Local $aParentPos, $AlphaKey = 0xABABAB
Local $iBorderColor = 0x00FF00, $iBorderThickness = 5
$hOverlay = GUICreate("", -50, -50, 10, 10, $WS_POPUPWINDOW, $WS_EX_LAYERED + $WS_EX_TOPMOST) ; transparent
$Panel = GUICreate("", 48, 48, 10, 10, $WS_CHILD + $WS_VISIBLE, -1, $hOverlay) ; this new window becomes a child of $hOverlay window
GUISetBkColor($iBorderColor, $hOverlay)
GUISetBkColor($AlphaKey, $Panel)
_WinAPI_SetLayeredWindowAttributes($hOverlay, $AlphaKey, 0, $LWA_COLORKEY)
GUISetState(@SW_SHOW, $hOverlay)


While WinExists($oIE_hwnd)
    $aMousePos = MouseGetPos()
    ; Now we get reference to the DOM object behind the mouse pointer
    $oElement = $oDoc.elementFromPoint($aMousePos[0] - $oIE.document.parentwindow.screenLeft, $aMousePos[1] - $oIE.document.parentwindow.screenTop)
    If IsObj($oElement) And $oElement <> $oElement_save Then
        $oElement_save = $oElement
        $aBrowserPos[0] = Int($oIE.document.parentwindow.screenLeft) ;      X position on the desktop of the left edge of the browser
        $aBrowserPos[1] = Int($oIE.document.parentwindow.screenTop) ;       Y position on the desktop of the top border of the browser
        $aBrowserPos[2] = Int($oIE.document.documentElement.clientWidth) ;  Width of browser's client area
        $aBrowserPos[3] = Int($oIE.document.documentElement.clientHeight) ; Height of browser's client area
        $aBrowserPos[4] = $aBrowserPos[0] + $aBrowserPos[2] ;               X position on the desktop of the right border of the browser
        $aBrowserPos[5] = $aBrowserPos[1] + $aBrowserPos[3] ;               Y position on the desktop of the bottom border of the browser
        ;
        $oRect = $oElement.getBoundingClientRect() ; coordinates of the DOM object within the client area of the browser
        ;
        $aOverlayPos[0] = Int($oRect.left) + $aBrowserPos[0] ; X position of the left overlay border (absolute desktop coordinates)
        If $aOverlayPos[0] < $aBrowserPos[0] Then $aOverlayPos[0] = $aBrowserPos[0] ; if left border of the DOM obj is out of browser, overlay border stays anyway within the screen
        $aOverlayPos[1] = Int($oRect.Top) + $aBrowserPos[1] ; Y  position of the top overlay border
        If $aOverlayPos[1] < $aBrowserPos[1] Then $aOverlayPos[1] = $aBrowserPos[1] ; if top border of obj is out of browser, overlay border stays anyway within the screen
        $aOverlayPos[2] = Int($oRect.right - $oRect.left) ; width of the obj
        If $aOverlayPos[0] + $aOverlayPos[2] > $aBrowserPos[4] Then $aOverlayPos[2] = $aBrowserPos[4] - $aOverlayPos[0] ; if obj's width is out of bounds, force the edge within the browser
        $aOverlayPos[3] = Int($oRect.bottom - $oRect.top) ; Height of the obj
        If $aOverlayPos[1] + $aOverlayPos[3] > $aBrowserPos[5] Then $aOverlayPos[3] = $aBrowserPos[5] - $aOverlayPos[1] ; stay within browser's bounds
        ;
        WinMove($hOverlay, "", $aOverlayPos[0], $aOverlayPos[1], $aOverlayPos[2], $aOverlayPos[3])
        $aParentPos = WinGetPos($hOverlay) ; x, y, width, height
        WinMove($Panel, "", $iBorderThickness - 1, $iBorderThickness - 1, Int($aParentPos[2] - $iBorderThickness * 2), Int($aParentPos[3] - $iBorderThickness * 2))
    EndIf
 WEnd

 

Edited April 26, 2018 by Melba23
Added code tags

[Melba23]

NRK,

Welcome to the AutoIt forums.

But please stick to just the one thread in future - I have deleted the other one.

M23

[NRK]

Sure, Posted in the wrong place, hence posted it here again.

[Danp2]

Adding a COM error handler will solve this issue. One way to accomplish this would be to call _IEErrorHandlerRegister() at the beginning of your script.

[NRK]

Danp2, 

Thanks for the response. I will test this and confirm.

[mLipok]

Could you also say something about AutoIt and InternetExplorer version which you use ?

 

[NRK]

By adding _IEErrorHandlerRegister(), it intercepts the COM error and continues the loop, however after handling the error for around 30 times, it comes out the program with exit code 255. Here is the updated code which I used to test it with _IEErrorHandlerRegister() added. I guess it comes out of the script because of the way in which it is coded inside elementFromPoint. Please share your thoughts and possible solutions.

expandcollapse popup

#include <IE.au3>
#include <WinAPI.au3>
#include <GUIConstantsEx.au3>
; #include <WinAPIEx.au3>
#include <WindowsConstants.au3>
_IEErrorHandlerRegister(_ErrFunc)
Local $oIE = _IECreate("https://www.autoitscript.com/forum/")
Local $oDoc = _IEDocGetObj($oIE)
Local $oIE_hwnd = _IEPropertyGet($oIE, "hwnd")
Local $oElement, $oElement_save, $aMousePos
Local $aBrowserPos[6] ; x , y , width , height , x right , y bottom
Local $aOverlayPos[4] ; coordinates of the overlay frame
; Create a Phantom window with a visible frame
Local $aParentPos, $AlphaKey = 0xABABAB
Local $iBorderColor = 0x00FF00, $iBorderThickness = 5
$hOverlay = GUICreate("", -50, -50, 10, 10, $WS_POPUPWINDOW, $WS_EX_LAYERED + $WS_EX_TOPMOST) ; transparent
$Panel = GUICreate("", 48, 48, 10, 10, $WS_CHILD + $WS_VISIBLE, -1, $hOverlay) ; this new window becomes a child of $hOverlay window
GUISetBkColor($iBorderColor, $hOverlay)
GUISetBkColor($AlphaKey, $Panel)
_WinAPI_SetLayeredWindowAttributes($hOverlay, $AlphaKey, 0, $LWA_COLORKEY)
GUISetState(@SW_SHOW, $hOverlay)

While WinExists($oIE_hwnd)
    $aMousePos = MouseGetPos()
    ; Now we get reference to the DOM object behind the mouse pointer
    $oElement = $oDoc.elementFromPoint($aMousePos[0] - $oIE.document.parentwindow.screenLeft, $aMousePos[1] - $oIE.document.parentwindow.screenTop)
    ConsoleWrite("################ Continuing the execution after exception ####################" & @CRLF)
    If IsObj($oElement) And $oElement <> $oElement_save Then
        $oElement_save = $oElement
        $aBrowserPos[0] = Int($oIE.document.parentwindow.screenLeft) ;      X position on the desktop of the left edge of the browser
        $aBrowserPos[1] = Int($oIE.document.parentwindow.screenTop) ;       Y position on the desktop of the top border of the browser
        $aBrowserPos[2] = Int($oIE.document.documentElement.clientWidth) ;  Width of browser's client area
        $aBrowserPos[3] = Int($oIE.document.documentElement.clientHeight) ; Height of browser's client area
        $aBrowserPos[4] = $aBrowserPos[0] + $aBrowserPos[2] ;               X position on the desktop of the right border of the browser
        $aBrowserPos[5] = $aBrowserPos[1] + $aBrowserPos[3] ;               Y position on the desktop of the bottom border of the browser
        ;
        $oRect = $oElement.getBoundingClientRect() ; coordinates of the DOM object within the client area of the browser
        ;
        $aOverlayPos[0] = Int($oRect.left) + $aBrowserPos[0] ; X position of the left overlay border (absolute desktop coordinates)
        If $aOverlayPos[0] < $aBrowserPos[0] Then $aOverlayPos[0] = $aBrowserPos[0] ; if left border of the DOM obj is out of browser, overlay border stays anyway within the screen
        $aOverlayPos[1] = Int($oRect.Top) + $aBrowserPos[1] ; Y  position of the top overlay border
        If $aOverlayPos[1] < $aBrowserPos[1] Then $aOverlayPos[1] = $aBrowserPos[1] ; if top border of obj is out of browser, overlay border stays anyway within the screen
        $aOverlayPos[2] = Int($oRect.right - $oRect.left) ; width of the obj
        If $aOverlayPos[0] + $aOverlayPos[2] > $aBrowserPos[4] Then $aOverlayPos[2] = $aBrowserPos[4] - $aOverlayPos[0] ; if obj's width is out of bounds, force the edge within the browser
        $aOverlayPos[3] = Int($oRect.bottom - $oRect.top) ; Height of the obj
        If $aOverlayPos[1] + $aOverlayPos[3] > $aBrowserPos[5] Then $aOverlayPos[3] = $aBrowserPos[5] - $aOverlayPos[1] ; stay within browser's bounds
        ;
        WinMove($hOverlay, "", $aOverlayPos[0], $aOverlayPos[1], $aOverlayPos[2], $aOverlayPos[3])
        $aParentPos = WinGetPos($hOverlay) ; x, y, width, height
        WinMove($Panel, "", $iBorderThickness - 1, $iBorderThickness - 1, Int($aParentPos[2] - $iBorderThickness * 2), Int($aParentPos[3] - $iBorderThickness * 2))
    EndIf
 WEnd

 ; User's COM error function. Will be called if COM error occurs
Func _ErrFunc($oError)
    ; Do anything here.
    ;MsgBox($MB_SYSTEMMODAL, "_IEErrorNotify Status", "Notification now is " & _IEErrorNotify())

    ConsoleWrite(@ScriptName & " (" & $oError.scriptline & ") : ==> COM Error intercepted !" & @CRLF & _
            @TAB & "err.number is: " & @TAB & @TAB & "0x" & Hex($oError.number) & @CRLF & _
            @TAB & "err.windescription:" & @TAB & $oError.windescription & @CRLF & _
            @TAB & "err.description is: " & @TAB & $oError.description & @CRLF & _
            @TAB & "err.source is: " & @TAB & @TAB & $oError.source & @CRLF & _
            @TAB & "err.helpfile is: " & @TAB & $oError.helpfile & @CRLF & _
            @TAB & "err.helpcontext is: " & @TAB & $oError.helpcontext & @CRLF & _
            @TAB & "err.lastdllerror is: " & @TAB & $oError.lastdllerror & @CRLF & _
            @TAB & "err.scriptline is: " & @TAB & $oError.scriptline & @CRLF & _
            @TAB & "err.retcode is: " & @TAB & "0x" & Hex($oError.retcode) & @CRLF & @CRLF)
    $oError = 0
EndFunc   ;==>_ErrFunc

 

[NRK]

I am using 64 bit windows 7 Enterprise edition Service pack1 with IE 11.0.9600.18977 and AutoIT version v3.3.14.3

[Juvigy]

Put this in the beginning : #AutoIt3Wrapper_UseX64=Y

[Danp2]

@Juvigy Can you explain how that will solve the OP's issue?

[NRK]

It did not solve the issue. Below is the code snippet after adding #AutoIt3Wrapper_UseX64=Y directive. 

 

#include <IE.au3>
#include <WinAPI.au3>
#include <GUIConstantsEx.au3>
; #include <WinAPIEx.au3>
#include <WindowsConstants.au3>
#AutoIt3Wrapper_UseX64=Y
_IEErrorHandlerRegister(_ErrFunc)
Local $oIE = _IECreate("https://www.autoitscript.com/forum/")
Local $oDoc = _IEDocGetObj($oIE)
Local $oIE_hwnd = _IEPropertyGet($oIE, "hwnd")
Local $oElement, $oElement_save, $aMousePos
Local $aBrowserPos[6] ; x , y , width , height , x right , y bottom
Local $aOverlayPos[4] ; coordinates of the overlay frame
; Create a Phantom window with a visible frame
Local $aParentPos, $AlphaKey = 0xABABAB

 

[Danp2]

12 hours ago, NRK said:

By adding _IEErrorHandlerRegister(), it intercepts the COM error and continues the loop, however after handling the error for around 30 times, it comes out the program with exit code 255.

The only time the error occurs is when the you close the IE window, so I believe your analysis here is faulty. The ConsoleWrite you added only indicates that the code has continued running after the prior command, not that an error has occurred.

I suspect that you are running out of memory / resources. When I ran your revised code, it looped over 1500 times without encountering an error. I did observed that memory usage was increasing as the script ran, but haven't tried to figure out why this is occurring.

[Juvigy]

1 hour ago, Danp2 said:

@Juvigy Can you explain how that will solve the OP's issue?

I had similar issues with objects that was caused by a bug in AutoitxX86 version. Putting the line forces usage of X64bit and resolved my issues.

[junkew]

Try to reproduce with objcreate internetexplorer.application. Then you exclude at least if its caused by ie.udf itself.

Try in excel vba with the object if you get same behavior.

[NRK]

On 4/27/2018 at 4:49 PM, Danp2 said:

The only time the error occurs is when the you close the IE window, so I believe your analysis here is faulty. The ConsoleWrite you added only indicates that the code has continued running after the prior command, not that an error has occurred.

I suspect that you are running out of memory / resources. When I ran your revised code, it looped over 1500 times without encountering an error. I did observed that memory usage was increasing as the script ran, but haven't tried to figure out why this is occurring.

The COM error "Bad Variable Type" came much earlier before I closed IE window. Idea behind the ConsoleWrite is to prove that after error is handled, the loop continues.  In my machine, when I tested, the memory consumption did not increase beyond 2%. I don't think memory is an issue as there is no local variable defined inside the loop.  

[NRK]

3 hours ago, junkew said:

Try to reproduce with objcreate internetexplorer.application. Then you exclude at least if its caused by ie.udf itself.

Try in excel vba with the object if you get same behavior.

I do not have much experience in vba excel or objecreate, can you please share a sample code or outline so that I can elaborate it and test it. 

[NRK]

On 4/27/2018 at 5:24 PM, Juvigy said:

I had similar issues with objects that was caused by a bug in AutoitxX86 version. Putting the line forces usage of X64bit and resolved my issues.

I am using IE 32 bit. Will that be the reason behind this issue? Does #AutoIt3Wrapper_UseX64=Y work fine with 32 bit version of IE? 

[junkew]

No then you should put value to n