Jump to content
Sign in to follow this  
elFic

IE Embedded - capturing events

Recommended Posts

Hello Everyone,

My name is Łukasz (I am from Poland) and this is my first question on this (and any other) forum.
So far I was able to find answers / solutions for all my problems and doubts related to autoit on this forum (or with Google). Now I decided to ask You for help.

I am trying to made GUI of my program using Embedded IE. It works well but my problem is to capture events like button click.
I've browsed a lot for solution - without success.

Please find below very minimal example of what I am doing.

#Include <IE.au3>

HotKeySet("{ESC}", "_Close")

_IEErrorHandlerRegister(_ErrFunc)

$oIE = _IECreateEmbedded()
$hGui = GUICreate("Test", 700, 500, -1, -1)
GUICtrlCreateObj($oIE, 0, 0, 700, 500)

_IENavigate($oIE, "about:blank")

$sHTML = ""
$sHTML &= "<html>"
$sHTML &= "<head>"
$sHTML &= "<title>Test</title>"
$sHTML &= "</head>"
$sHTML &= "<body>"
$sHTML &= '<button type="button" id="btnMenu">=</button>'
$sHTML &= '<button type="button" id="btnClose">X</button>'
$sHTML &= "</body>"
$sHTML &= "</html>"
_IEDocWriteHTML($oIE, $sHTML)
_IEAction($oIE, "refresh")

GUISetState()

$oButtonMenu = _IEGetObjById($oIE, "btnMenu")
$oButtonClose = _IEGetObjById($oIE, "btnClose")

$oEvent = ObjEvent($oButtonMenu, "_MY_EVENT_HANDLER_","HTMLButtonElementEvents2")
$oEvent2 = ObjEvent($oButtonClose, "_MY_EVENT_HANDLER_","HTMLButtonElementEvents2")


While (1)
    Sleep(20)
WEnd

Func _MY_EVENT_HANDLER_onclick($oEventObj)

    MsgBox(4096, "Yay!", "You clicked the button. But which one ?")

EndFunc   ;==>_MY_EVENT_HANDLER_onclick

Func _Close()
    Exit
EndFunc

Func _ErrFunc($oError)
    ; Do anything here.
    ConsoleWrite(@ScriptName & " (" & $oError.scriptline & ") : ==> COM Error intercepted !" & @CRLF & _
            @TAB & "err.number is: " & @TAB & @TAB & "0x" & Hex($oError.number) & @CRLF & _
            @TAB & "err.windescription:" & @TAB & $oError.windescription & @CRLF & _
            @TAB & "err.description is: " & @TAB & $oError.description & @CRLF & _
            @TAB & "err.source is: " & @TAB & @TAB & $oError.source & @CRLF & _
            @TAB & "err.helpfile is: " & @TAB & $oError.helpfile & @CRLF & _
            @TAB & "err.helpcontext is: " & @TAB & $oError.helpcontext & @CRLF & _
            @TAB & "err.lastdllerror is: " & @TAB & $oError.lastdllerror & @CRLF & _
            @TAB & "err.scriptline is: " & @TAB & $oError.scriptline & @CRLF & _
            @TAB & "err.retcode is: " & @TAB & "0x" & Hex($oError.retcode) & @CRLF & @CRLF)
EndFunc   ;==>_ErrFunc

and the error I have

test.au3 (31) : ==> COM Error intercepted !
    err.number is:      0x80028017
    err.windescription: Nazwa pola nie jest zdefiniowana w rekordzie.

    err.description is:     
    err.source is:      
    err.helpfile is:    
    err.helpcontext is:     
    err.lastdllerror is:    0
    err.scriptline is:  31
    err.retcode is:     0x00000000

test.au3 (32) : ==> COM Error intercepted !
    err.number is:      0x80028017
    err.windescription: Nazwa pola nie jest zdefiniowana w rekordzie.

    err.description is:     
    err.source is:      
    err.helpfile is:    
    err.helpcontext is:     
    err.lastdllerror is:    0
    err.scriptline is:  32
    err.retcode is:     0x00000000
 

Nazwa pola nie jest zdefiniowana w rekordzie = Field name not defined in the record.

These two errors are referring to

$oEvent = ObjEvent($oButtonMenu, "_MY_EVENT_HANDLER_","HTMLButtonElementEvents2")
$oEvent2 = ObjEvent($oButtonClose, "_MY_EVENT_HANDLER_","HTMLButtonElementEvents2")

I made a lot of different tries, I have experimented with ObjEvent interface name, but still have same issue.

One more detail that can be important - I am using FEATURE_BROWSER_EMULATION to set up embedded IE to IE11. I have checked IE10 with same result.
Can You please point me what I am doing wrong?

Thank You.

Edited by elFic
copy-paste mistake in the code

Share this post


Link to post
Share on other sites

Hi Subz,

Thank You for reply.
Yes, I have seen this topic before. But I have not found the answer for my problem.

The interesting thing is the code I posted works well when IE is not embedded.
I can assume the embedded IE requires different interface for ObjEvent. But I am not sure if it is right assumption.

Share this post


Link to post
Share on other sites

try this way...

#include <IE.au3>

HotKeySet("{ESC}", "_Close")

_IEErrorHandlerRegister(_ErrFunc)

$oIE = _IECreateEmbedded()
$hGui = GUICreate("Test", 700, 500, -1, -1)
GUICtrlCreateObj($oIE, 0, 0, 700, 500)

_IENavigate($oIE, "about:blank")

$sHTML = ""
$sHTML &= "<html>"
$sHTML &= "<head>"
$sHTML &= "<title>Test</title>"
$sHTML &= "</head>"
$sHTML &= "<body>"
$sHTML &= '<button type="button" id="btnMenu">=</button>'
$sHTML &= '<button type="button" id="btnClose">X</button>'
$sHTML &= "</body>"
$sHTML &= "</html>"
_IEDocWriteHTML($oIE, $sHTML)
_IEAction($oIE, "refresh")

GUISetState()
; -------------------------------------------------------------------------------
; inject a reference to the javascript global object into the WebBrowser document
; this works for the BrowserControl, but it doesn't on the IE browser
;
; http://perfectionkills.com/unnecessarily-comprehensive-look-into-a-rather-insignificant-issue-of-global-objects-creation/#ecmascript_5_strict_mode
$oIE.document.parentwindow.setTimeout("JSglobal = (1,eval)(""this"");", 0)
Local $ohJS
Do
    Sleep(1000)
    $ohJS = Execute('$oIE.document.parentwindow.JSglobal')
Until IsObj($ohJS)

; getting references to javascript objects in this way will then work on ObjEvent
$oButtonMenu = $ohJS.btnMenu ;_IEGetObjById($oIE, "btnMenu")
$oButtonClose = $ohJS.btnClose ; _IEGetObjById($oIE, "btnClose")
; -------------------------------------------------------------------------------

$oEvent = ObjEvent($oButtonMenu, "_MY_EVENT_HANDLER_", "HTMLButtonElementEvents2")
$oEvent2 = ObjEvent($oButtonClose, "_MY_EVENT_HANDLER_", "HTMLButtonElementEvents2")


While (1)
    Sleep(20)
WEnd

Volatile Func _MY_EVENT_HANDLER_onclick($oEventObj)
    ; https://msdn.microsoft.com/en-us/library/aa703876(v=vs.85).aspx
    MsgBox(4096, "Yay!", "You clicked the button. But which one ? this ID --> " & $oEventObj.srcElement.id)

EndFunc   ;==>_MY_EVENT_HANDLER_onclick

Func _Close()
    ; the end
    $oEvent.stop
    $oEvent2.stop
    $oEvent = 0
    $oEvent2 = 0
    $oIE = 0 ; Remove IE from memory
    GUIDelete($hGui) ; Remove GUI
    Exit
EndFunc   ;==>_Close

Func _ErrFunc($oError)
    ; Do anything here.
    ConsoleWrite(@ScriptName & " (" & $oError.scriptline & ") : ==> COM Error intercepted !" & @CRLF & _
            @TAB & "err.number is: " & @TAB & @TAB & "0x" & Hex($oError.number) & @CRLF & _
            @TAB & "err.windescription:" & @TAB & $oError.windescription & @CRLF & _
            @TAB & "err.description is: " & @TAB & $oError.description & @CRLF & _
            @TAB & "err.source is: " & @TAB & @TAB & $oError.source & @CRLF & _
            @TAB & "err.helpfile is: " & @TAB & $oError.helpfile & @CRLF & _
            @TAB & "err.helpcontext is: " & @TAB & $oError.helpcontext & @CRLF & _
            @TAB & "err.lastdllerror is: " & @TAB & $oError.lastdllerror & @CRLF & _
            @TAB & "err.scriptline is: " & @TAB & $oError.scriptline & @CRLF & _
            @TAB & "err.retcode is: " & @TAB & "0x" & Hex($oError.retcode) & @CRLF & @CRLF)
EndFunc   ;==>_ErrFunc

p.s. For another example of use for the WebControl, have a look to this script also... (https://www.autoitscript.com/forum/topic/186422-who-is-who-a-little-dragdrop-game/)

 


small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

Hello Chimp, 

Thank You very much for this example. It works perfect. 
And thank You for the links in comments. 
It made me clear what is (1,eval) used for.

I am really appreciate Your support.
Regards 
Łukasz 

Share this post


Link to post
Share on other sites

FYI, I was battling with this a few days ago as well - ie11 embedded, and for whatver reason a td with the Js attached to it of onclick(oie.navigate()) caused the object to crash. Unsure of there is a workaround. 

Share this post


Link to post
Share on other sites
On 3/1/2019 at 8:50 PM, Chimp said:

try this way...

Dear Chimp,

Below is code from your previous comment here

only small changes trying to capture document events from an html form in the embedded webcontrol

i'm finding this approach simpler to understand, and seems to work ok.

For sure, i'm not understanding some concepts?

Thanks for your comments

#include <GUIConstantsEx.au3>
#include <MsgBoxConstants.au3>
Global $g_idGUIEdit

Example()
Exit ; End of our Demo.

Func Example()
    Local $hGUIMain = GUICreate("Event Test", 1000, 600)
    $g_idGUIEdit = GUICtrlCreateEdit("", 5, 405, 990, 175)
    ; GUICtrlSetBkColor(-1, 0x000000)
    ; GUICtrlSetColor(-1, 0x00FF00)
    GUICtrlSetFont(-1, 9, 400, -1, 'Courier New')
    GUICtrlCreateLabel("Below are some Browser events 'captured' from the above web page by AutoIt", 5, 385, 990, 20)
    Local $idGUIExit = GUICtrlCreateButton(" Close and exit", 5, 580, 990, 15)
    GUISetState() ;Show GUI

    ; We prepare the Internet Explorer as our test subject
    Global $oIE = ObjCreate("Shell.Explorer.2")
    $hIE = GUICtrlCreateObj($oIE, 5, 5, 990, 380) ; <- insert $oIE in the AutoIt GUI

    ; Here we load an example Web page just to have something viewed in the browser
    ;$oIE.navigate('http://samples.msdn.microsoft.com/workshop/samples/author/dhtml/refs/onmousemoveEX.htm')

   ;>>>>
   ;These lines from: Chimp, Jan 2017 (edited)
   ;Who is Who ? (a little drag&drop game)
   ;https://www.autoitscript.com/forum/topic/186422-who-is-who-a-little-dragdrop-game/
   ;Global $oIE = ObjCreate("Shell.Explorer.2") ; Create a BrowserControl
   ;Local $hGUI = GUICreate("", 660, 600, 30, 30)
   ;GUICtrlCreateObj($oIE, 0, 0, 660, 600) ; Place BrowserControl on the GUI
   ;GUISetState() ;Show GUI
   $oIE.navigate('about:blank') ; file:///' & @ScriptDir & '\WhoIsWho.html')
   While Not String($oIE.readyState) = 'complete' ; wait for about:blank
     Sleep(100)
   WEnd

   ; this waits till the document is ready to be used (portion of code from IE.au3)
   While Not (String($oIE.readyState) = "complete" Or $oIE.readyState = 4)
     Sleep(100)
   WEnd
   While Not (String($oIE.document.readyState) = "complete" Or $oIE.document.readyState = 4)
     Sleep(100)
   WEnd

   $oIE.document.Write(_GetHTML()) ; inject lising directly to the HTML document:
   $oIE.document.close() ; close the write stream
   $oIE.document.execCommand("Refresh")
   While Not String($oIE.readyState) = 'complete' ; wait for readyState after a refresh
     Sleep(100)
   WEnd

   $oDocument = $oIE.document
   ;>>>>

;~     Sleep(1000) ; Give it some time to load the web page
;~     Do ; wait for document
;~         Sleep(250)
;~         $oDocument = $oIE.document
;~     Until IsObj($oDocument)

    ;   +  Scripting Object Interfaces
    ;   |  ---------------------------
    ;   |  https://msdn.microsoft.com/en-us/library/hh801967(v=vs.85).aspx
    ;   |
    ;   +-->   HTMLDocumentEvents2 interface (catch OnClick, OnMouseOver, .... etc
    ;          -----------------------------
    ;          https://msdn.microsoft.com/en-us/library/aa769764(v=vs.85).aspx
    ;
    Global $oEventObject = ObjEvent($oDocument, "IEEvent2_", "HTMLDocumentEvents2")

    If @error Then
        MsgBox($MB_OK, "AutoIt COM Test", _
                "ObjEvent: Can't use event interface 'HTMLDocumentEvents2'. Error code: " & Hex(@error, 8))
        Exit
    EndIf

    ; GUISwitch($hGUIMain) ; Switch back to our GUI in case IE stole the focus

    ; Waiting for user to close the GUI.
    Local $iMsg
    While 1
        $iMsg = GUIGetMsg()
        If $iMsg = $GUI_EVENT_CLOSE Or $iMsg = $idGUIExit Then ExitLoop
    WEnd

    $oEventObject.Stop ; Tell IE we don't want to receive events.
    $oEventObject = 0 ; Kill the Event Object
    $oIE = 0 ; Remove IE from memory (not really necessary).

    GUIDelete() ; Remove GUI
EndFunc   ;==>Example

; A few Internet Explorer Event Functions
; ( reference to the Event Obj interface: )
; ( https://msdn.microsoft.com/en-us/library/aa703876(v=vs.85).aspx )
;
Volatile Func IEEvent2_onClick($oEvent)
   ConsolePrint("mouse click: " & $oEvent.clientX & ',' & $oEvent.clientY & '  on ' & $oEvent.srcElement.NodeName & '  id: ' & $oEvent.srcElement.id)
   If $oEvent.srcElement.NodeName = "INPUT" Then
      If $oEvent.srcElement.type = "radio" Or $oEvent.srcElement.type = "checkbox" Then
         If $oEvent.srcElement.checked=true Then
            ConsolePrint("name: " & $oEvent.srcElement.name & " id " & $oEvent.srcElement.id  & " True")
         Else
            ConsolePrint("name: " & $oEvent.srcElement.name & " id " & $oEvent.srcElement.id & " False")
         EndIf
      EndIf
   EndIf
EndFunc   ;==>IEEvent2_onClick

Volatile Func IEEvent2_onDblClick($oEvent)
    ConsolePrint("mouse DoubleClick: @" & $oEvent.clientX & ',' & $oEvent.clientY)
EndFunc   ;==>IEEvent2_onDblClick

;~ Volatile Func IEEvent2_onMouseMove($oEvent)
;~     ConsolePrint("mouse moved to:" & @TAB & "Xpos = " & $oEvent.clientX & @TAB & "Ypos = " & $oEvent.clientY)
;~ EndFunc   ;==>IEEvent2_onMouseMove

Func ConsolePrint($sMsg)
    Local Const $iMaxLines = 9 ; keep last 9 log lines only
    $sMsg = @HOUR & ':' & @MIN & ':' & @SEC & ':' & @MSEC & @TAB & $sMsg & @CRLF
    $sMsg = StringReplace(GUICtrlRead($g_idGUIEdit) & $sMsg, @CR, @CR)
    If @extended > $iMaxLines Then ; more than $iMaxLines
        $sMsg = StringMid($sMsg, StringInStr($sMsg, @CR, 0, -1 * $iMaxLines) + 2)
    EndIf
    GUICtrlSetData($g_idGUIEdit, $sMsg)
EndFunc   ;==>ConsolePrint

Func _GetHTML()
   ;form html code from: Marc Clifton, 16 Feb 2013, https://www.codeproject.com/Articles/547451/WebBrowser-Element-Events-and-Values
   Local $sHTML = _
         "<!DOCTYPE HTML>" & @CRLF & _
         "<html>" & @CRLF & _
         "<head>" & @CRLF & _
         "<meta http-equiv=""X-UA-Compatible"" content=""IE=edge"" />" & @CRLF & _
         "<title>Test</title>" & @CRLF & _
         "</head>" & @CRLF & _
         "<body>" & @CRLF & _
         '<button type="button" id="btnMenu">=</button>' & @CRLF & _
         '<button type="button" id="btnClose">X</button>' & @CRLF & _
         '<form>' & @CRLF & _
         'First name: <input id="firstname" type="text" name="firstname"/><br/>' & @CRLF & _
         'Last name: <input id="lastname" type="text" name="lastname"/><br/>' & @CRLF & _
         'Password: <input id="password" type="password" name="pwd"/><br><br/>' & @CRLF & _
         '<input type="radio" id="male" name="sex" value="male"/>Male<br/>' & @CRLF & _
         '<input type="radio" id="female" name="sex" value="female"/>Female<br/><br/>' & @CRLF & _
         '<input type="checkbox" id="bike" name="vehicle" value="Bike"/>I have a bike<br/>' & @CRLF & _
         '<input type="checkbox" id="car" name="vehicle" value="Car"/>I have a car <br/><br/>' & @CRLF & _
         '<input type="button" id="ok" value="OK"/><br/>' & @CRLF & _
         '<input type="button" id="cancel" value="Cancel"/><br/><br/>' & @CRLF & _
         '</from>' & @CRLF & _
         "</body>" & @CRLF & _
         "</html>"
   Return $sHTML
 EndFunc   ;==>_GetHTML

 

Edited by robertocm

Share this post


Link to post
Share on other sites

May I ask, how are you able to declare the same variable with different functions/strings. I was sure that it would only overwrite an existing string or function. I'm looking at the $sHTML variable that is just under the #includes in Chimps' script.

 

Regards,

Supra

Share this post


Link to post
Share on other sites

Hi @robertocmthanks for testing and posting your results...
For what I understand, in the way that you have used, you trap events coming from the html "document" object, and then you have to check further, in the event function of the document, from which specific element the event was fired using a series of if ... then statements. This is also a way to go, but if you want to trap events, directly from a specific DOM object, then you have to pass a reference of that specific object to the AutoIt  ObjEvent() function. Also, not all and not always the events fired by a specific child dom object are "bubbled" up to the top DOM object, that is the "document" object. In that case you are not able to catch the event through the document object because it doesn't receive it from the child elements. Have a look to the "Background" session of this link (https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/aa752045(v%3dvs.85)) for a better explanation of how events are forwarded up in the DOM hierarchy.

Hi @supraaxdd, the variable $sHTML is not "reassigned" each time, but since is used the concatenation operator (that is the &= sign) and not the assignment operator (that is the = sign) the value is "appended" to the current content of the variable and is not overwritten.

Edited by Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites
5 hours ago, Chimp said:

Hi @robertocmthanks for testing and posting your results...
For what I understand, in the way that you have used, you trap events coming from the html "document" object, and then you have to check further, in the event function of the document, from which specific element the event was fired using a series of if ... then statements. This is also a way to go, but if you want to trap events, directly from a specific DOM object, then you have to pass a reference of that specific object to the AutoIt  ObjEvent() function. Also, not all and not always the events fired by a specific child dom object are "bubbled" up to the top DOM object, that is the "document" object. In that case you are not able to catch the event through the document object because it doesn't receive it from the child elements. Have a look to the "Background" session of this link (https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/aa752045(v%3dvs.85)) for a better explanation of how events are forwarded up in the DOM hierarchy.

Hi @supraaxdd, the variable $sHTML is not "reassigned" each time, but since is used the concatenation operator (that is the &= sign) and not the assignment operator (that is the = sign) the value is "appended" to the current content of the variable and is not overwritten.

That explains, I completely forgot about the other operators. Thanks for explaining :)

 

Regards,

Supra

Share this post


Link to post
Share on other sites

@supraaxdd When replying, can you please stop quoting the entire previous post? Everyone can read what the other person posted, so there's no reason to quote the thing in its entirety. If you must quote something, quote only the information the pertains to your reply and discard the rest. Also, there's no need to quote someone's code unless you've modified it in some way. You're just making it harder for everyone else to follow the flow of the thread by unnecessary over-quoting.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

I'm sorry @BrewManNH, It's a horrible habit that I need to work on. I got multiple complaints and I always foget about them. I'm sorry about it, I will try my best to limit my quoting to a bare minimum.

Regards,

Supra

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...