Microsoft Edge - WebView2, embed web code in your native application

[mLipok]

Anybody could help with using QueryInterface in AutoIt to embed Microsoft Edge using WebView2 Interface ?

Here are links to information/documentation:

Image comes from: https://docs.microsoft.com/en-us/microsoft-edge/webview2/media/webview2/whatwebview.png

Introduction to Microsoft Edge WebView2
https://docs.microsoft.com/en-us/microsoft-edge/webview2/

Understand WebView2 SDK versions
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/versioning

Quote

• Win32 C/C++. When using QueryInterface to obtain a new interface, check for a return value of E_NOINTERFACE. This value may indicate that the WebView2 Runtime is a previous version, and doesn't support that interface.

Download:
https://developer.microsoft.com/en-US/microsoft-edge/webview2/
https://developer.microsoft.com/en-US/microsoft-edge/webview2/#download-section
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution

API Reference:
https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/?view=webview2-1.0.622.22

HowTo/Concepts:
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/versioning
https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/win32
https://docs.microsoft.com/en-us/microsoft-edge/webview2/howto/debug?tabs=devtools
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/security
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/userdatafolder

https://mybuild.microsoft.com/sessions/9198aeac-0c8e-4d32-96d1-cbb99a390aa6

Example:
https://github.com/MicrosoftEdge/WebView2Samples

Related:

https://www.essentialobjects.com/Products/WebBrowser/Default.aspx?gclid=CjwKCAiA17P9BRB2EiwAMvwNyMe6UI9VHej-JYDBUoyGDFz40KaaJ_RidY-75Fhr-PHz65sSVC0XlxoC5loQAvD_BwE

https://docs.microsoft.com/en-us/microsoft-edge/webview2/howto/webdriver

 

btw.
I'm not WindowsApi expert (QueryInterface, C++, DllCall).
What I trying to say is:     It means that I do not done anything in this regards .... as so far.

 

 

TIMELINE/ChangeLog:
 

1. 2020-11-12 - Project propsed by @mLipok
2. 2021-02-03 - First version provided by @LarsJ

Edited February 3, 2021 by mLipok

[Gianni]

Is there a way to use this stuff by making use of the following other stuff? or am I just raving?
sorry if I'm just delirious ...

 

[ptrex]

@Chimp, 

Yes in theory it should be possible...

 

 

Edited November 25, 2020 by ptrex

[Gianni]

hmmm, thanks @ptrex for your hopeful response
I'll try to see if I can understand something  .... sure that if there was even just a simple draft, an "hello world" to start experimenting things would be easier for the rest of us ....

Thanks again

[ptrex]

If I see the time available I will do some tests and if successful I will update here.

[LarsJ]

The easiest way to get started is probably to translate the code in this AutoHotkey thread into AutoIt or at least take the AutoHotkey code as a starting point. We have tried this before a few years ago with relatively good success.

[ptrex]

hi Lars,

I tried yesterday do get it running in plain .NET environment. 

I was able to get the WebView2 control on the Gui but that is as far as got... no webpage is loading so far. 

So I am missing probably some dependencies or it needs a specific .NET environment. As long as it doens't run in .NET I can't move it to au3.

 

[ptrex]

If you read this here, I guess it is too early to start playing with it... 

https://stackoverflow.com/questions/61291674/how-can-i-add-webview2-control-in-visual-studio-toolbar

https://docs.microsoft.com/en-us/microsoft-edge/webview2/releasenotes

There are quite a lot of dependencies needed to make it all work well...

If you have not got all the versions aligned it is a no go at the moment ?

[LarsJ]

AutoHotkey example
Even if the .NET version of the WebView2 code does not work, it may well be that the Windows API version works. The AutoHotkey code actually works.

If you want to try the AutoHotkey code, use this procedure. I've only tested on Windows 7. Not on Windows 10. I think the procedure is the same. 

Install AutoHotkey and SciTE for AutoHotkey:

• Download and install AutoHotkey_1.1.33.02_setup.exe.
  By default, AutoHotkey is installed under C:\Program Files\AutoHotkey.
• Download and install SciTE4AutoHotkey. Click "Installer".
• Download AutoHotkey v2 by clicking Current Alpha Release and extract the zip-file.
• Rename C:\Program Files\AutoHotkey\AutoHotkey.exe to AutoHotkey-a.exe or similar.
• Rename (or copy) AutoHotkeyU64.exe from the AutoHotkey v2 zip-file to AutoHotkey.exe.
• Copy the new AutoHotkey.exe to C:\Program Files\AutoHotkey

Download and install the WebView2 Runtime: Evergreen Bootstrapper.

Download AutoHotkey code and data to test WebView2:

• Download webview2.zip at the bottom of this post and extract the zip-file.
• Open webview2.ahk in SciTE4AutoHotkey and run the script by pressing F5.

The AutoHotkey post describes that you need to install a beta or development version of Microsoft Edge to make the script work. But it has not been necessary.

Now the AutoHotkey code must be translated to AutoIt. We've done this before and we can probably do it again. It's not necessarily completely trivial. But probably not completely impossible either.

This is the AutoHotkey code to translate, webview2.ahk:

expandcollapse popup

#Requires AutoHotkey v2.0-a121
#SingleInstance Force

; DllCall("User32.dll\SetProcessDpiAwarenessContext", "Ptr", 0xFFFFFFFC) ; not sure if this is needed.  Works fine without it.

main := gui.new("+Resize")
main.OnEvent("Close", "ExitApp")

main.Show(Format("w{} h{}", A_ScreenWidth * 0.4, A_ScreenHeight * 0.6))

; need to have the size also otherwise when you un-maximize it the window has zero height and width
wv := WebView.new(main)
wv.create()

return

mainSize(controller, win, minMax, *) {
    ; for performance reasons, you should make the webview2 not visible when the parent window is minimized/hidden.
    ComCall(3, controller, "Int*", isVisible := 0) ; IWebView2WebViewController::get_IsVisible
    if (minMax == -1) {
        if (isvisible) {
            ComCall(4, controller, "Int", false) ; IWebView2WebViewController::put_IsVisible
        }
    }
    else {
        if (!isVisible) {
            ComCall(4, controller, "Int", true) ; IWebView2WebViewController::put_IsVisible
        }
        else {
            DllCall("User32.dll\GetClientRect", "Ptr", win.hWnd, "Ptr", RECT := BufferAlloc(16))
            ComCall(6, controller, "Ptr", RECT) ; IWebView2WebViewController::put_Bounds
        }
        
    }
}

class WebView {    
    __new(parent, dllPath := "WebView2Loader.dll") {
        if (Type(parent) != "Gui") {
            throw Exception("Parent must be of type: Gui")
        }
        
        this.parent := parent
        this.dllPath := dllPath
        this.controllerCompletedHandler := ICoreWebView2CreateCoreWebView2ControllerCompletedHandler.new(this)
        this.envCompletedHandler := IWebView2CreateWebView2EnvironmentCompletedHandler.new(this.parent, this.controllerCompletedHandler)
    }
    
    OnControllerCompleted(cb) {
        this.controllerCreatedCallbacks.push(cb)
    }
    
    create() {
        if (R := DllCall(this.dllPath . "\CreateCoreWebView2Environment", "Ptr", this.envCompletedHandler, "UInt")) {
            MsgBox("ERROR " . Format("{:08X}", R))
        }
    }
}

class IWebView2CreateWebView2EnvironmentCompletedHandler extends IUnknown {
    __New(parent, controllerCompleted) {
        super.__new()
        
        this.parent := parent
        this.controllerCompleted := controllerCompleted
    }
    
    Invoke(thisPtr, hresult, ICoreWebView2Env) {
        ; ICoreWebView2Environment::CreateCoreWebView2Controller Method.
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2environment#createcorewebview2controller
        ComCall(3, ICoreWebView2Env, "Ptr", this.parent.hwnd, "Ptr", this.controllerCompleted)

        return 0  ; S_OK.
    }
}

class ICoreWebView2CreateCoreWebView2ControllerCompletedHandler extends IUnknown {
    __new(wv) {
        super.__new()
        
        this.wv := wv
    }
    
    Invoke(thisPtr, HRESULT, IWebView2WebViewController) {    
        ObjAddRef(IWebView2WebViewController) ; This was key to retain a reference to the Controller
        
        this.wv.parent.OnEvent("Size", Func("mainSize").bind(IWebView2WebViewController))
        
        ; Resize WebView to fit the bounds of the parent window.
        ; IWebView2WebViewController::put_Bounds Method.
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2controller#put_bounds
        DllCall("User32.dll\GetClientRect", "Ptr", this.wv.parent.hWnd, "Ptr", RECT := BufferAlloc(16))
        ComCall(6, IWebView2WebViewController, "Ptr", RECT)

        ; IWebView2WebViewController::get_CoreWebView2
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2controller#get_corewebview2
        ComCall(25, IWebView2WebViewController, "Ptr*", coreWebView := 0)
        this.wv.coreWebView := coreWebView
        
        ; ICoreWebView2::add_navigationstarting
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2#add_navigationstarting
        ; not really sure what the last parameter (EventRegistrationToken) is for but passing an empty buffer makes it work
        ComCall(7, this.wv.coreWebView, "Ptr", ICoreWebView2NavigationStartingEventHandler.new(), "Ptr", token := BufferAlloc(8))
        
        ; ICoreWebView2::Navigate
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2#navigate
        ; ComCall(5, this.wv.coreWebView, "Str", "https://getbootstrap.com/")
        ComCall(5, this.wv.coreWebView, "Str", "file:///" A_ScriptDir "/index.html")

        return 0  ; S_OK.
    }
}

class ICoreWebView2NavigationStartingEventHandler extends IUnknown {
    Invoke(thisPtr, IWebView2WebView, IWebView2NavigationStartingEventArgs) {
        ComCall(4, IWebView2NavigationStartingEventArgs, "Int*", isUserInitiated := 0)
        ComCall(5, IWebView2NavigationStartingEventArgs, "Int*", isRedirected := 0)
        
        if (isUserInitiated && !isRedirected) {
            ComCall(3, IWebView2NavigationStartingEventArgs, "Str*", uri := "")
            
            if (MsgBox("You are about to navigate to: " . uri . "`n`nDo you want to continue?", "Navigation warning", "YN Icon!") = "No") {
                
                ; IWebView2NavigationStartingEventArgs::put_Cancel
                ComCall(8, IWebView2NavigationStartingEventArgs, "Int", true)
                
                ; ICoreWebView2::NavigateToString
                ComCall(6, IWebView2WebView, "Str", "<h1>Navigation Canceled</h1><p>You chose to cancel navigation to the following URL: " . uri . "</p>")
            }
        }
        
        return 0 ; S_OK
    }
}

class IUnknown {
    methods := ["QueryInterface", "AddRef", "Release", "Invoke"]
    vtbl := BufferAlloc(4 * A_PtrSize)
    
    __New() {
        for (name in this.methods) {
            method := this.GetMethod(name)
            NumPut("UPtr", CallbackCreate(method.bind(this), "", method.MinParams - 1), this.vtbl, (A_Index - 1) * A_PtrSize)
        }
        
        this.ptr := DllCall("Kernel32.dll\GlobalAlloc", "UInt", 0, "Ptr", A_PtrSize + 4, "UPtr")
        NumPut("UPtr", this.vtbl.ptr, this.ptr)
        NumPut("UInt", 1, this.ptr, A_PtrSize)
    }
    
    __Delete() {
        DllCall("Kernel32.dll\GlobalFree", "Ptr", this)
    }
    
    QueryInterface(riid, ppvObject) {
        ; No idea why this isn't called at all...
    }
    
    AddRef(interface) {
        ObjAddRef(interface)
    }
    
    Release(interface) {
        ObjRelease(interface)
    }
}

The example is implemented in AutoIt code in WebView2-3.au3 in the 7z-file at bottom of this post.

The AutoHotkey code is included in WebView2.ahk in the 7z-file.

Edited February 7, 2021 by LarsJ
The text after the code box

[ptrex]

Hi Lars,

Thanks for the update on this.

The Windows API  sure do work I don't doubt that. But I am not a Windows API guy 😞

So if someone feels he or she can do the AHK to AU3 translation that would be great.

In the meantime I will try to get the .Net version to work and report back if I have something going.

 

[Gianni]

Thanks all for the tips and the considerations on this matter,
....also, just to know,
since this other project (https://www.autoitscript.com/forum/topic/193846-chromium-embedded-framework-for-autoit3/?do=findComment&comment=1469397) already seems to work on AutoIt, I am wondering if the "Chromium Embedded Framework" used by that project is the same "environment" used by Edge too, that is, if the underlying engine involved is ultimately the same?? ( ie if both share the same DLLs or whatever?)
Thanks again

p.s.
I'm wondering also if Chrome and Edge has the same engine (that is if both uses the same phisical files on the disk??)

 

Edited November 27, 2020 by Chimp

[LarsJ]

This weekend I've tested the AutoHotkey code on Windows 10. There are links to two different scripts in the posts above. None of these scripts works on Windows 10. So there is no code to translate to AutoIt. Of course, I've installed the Microsoft Edge Chromium version on Windows 10 and the required runtime code. The Edge version is the same on Windows 10 and Windows 7.

To get ahead, it's probably easiest to look at the getting started example. The required WebView2 COM interfaces used in this example are not included in the Windows SDK and must be installed through the WebView2 NuGet package.

And since these COM interfaces are also needed for tag descriptions for ObjCreateInterface() and ObjectFromTag() (for callback interfaces as in UIA Events), it makes good sense to continue with this example.

The problem with this approach is that it'll be time consuming. It'll require more time than I'm interested in spending right now. Therefore, I'll not go ahead with this project. Sorry.

Instead, I'll continue with the projects on rectangular selections in listviews. I'll be working on these projects through the rest of the winter.

Whether I'll continue with the WebView2 project afterwards, time will tell.

[ptrex]

Hi Lars,

Same thing for me. 

Over the weekend I spend quite a lot of time trying to get the simple .NET example going. But also no success 😞

It seems that the libraries are not yet stable enough to start building upon.

If you read the simple getting started example, which promises to get everyone going in a few lines.

https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/winforms

But unfortunately it did not...

I am stuck at the moment to get the "WebView2Loader.dll" loaded.

Anyhow if there is news I will release an update.

 

 

[LarsJ]

We're back in business.

Tonight I've installed the latest version of Visual Studio, the WebView2 GitHub sample files and the WebView2 NuGet package as described in the documentation for the WebView2 getting started example. Step 1 - 3 in the documentation. After a single small error, the build process succeeded in the second attempt.

The executable WebView2 file started automatically (the Bing search shown in the image in the documentation), I searched for "autoit forums" and then browsed through exactly this thread.

So now we have code that works on Windows 10, we have code that can be translated into AutoIt, and we have the WebView2.h header file from the NuGet package that seems to contain all the information needed to create tag description files for ObjCreateInterface() and ObjectFromTag().

I've once again changed the priority of my projects. They are all exciting. But this is the most exciting right now. Already this weekend, I hope to be able to take the first steps in the translation task.

HelloWebView.cpp and WebView2.h are contained in the 7z-file.

WebView2.7z

Edited December 3, 2020 by LarsJ
WebView2.7z

[ptrex]

Hi Lars,

Great news !

Anyhow I have been playing around with the .NET version and assemblies that have been release.

But I and many others can't get that running at the moment because there too many dependencies that are not aligned right now.

And versions are changing rapidly too. So I can't move forward on this one unfortunately until the releases are getting more stable.

 

[mLipok]

as a curiosity

[TheDcoder]

I was afraid that after M$ drops support for IE and removes it completely from Windows, we won't have any default installed browser which can be scripted, but good to see that they are making Edge usable over WinAPI

And since Edge is based on Chromium, it would be awesome if the changes are somehow backported, then we can use the same API to interact with Chrome!

Edit: Forgot that Chromium is not a GPL project, so Edge isn't open-source and thus the changes can't be backported... without the help of reverse engineering anyway. Oh well, I guess Edge will do better than IE atleast.

Edited December 3, 2020 by TheDcoder

[LarsJ]

Getting started example step 3: WebView window
First AutoIt implementation. Tested on Windows 10/7, 32/64 bit. Execute WebView2.au3:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w- 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

#AutoIt3Wrapper_UseX64=y

Opt( "MustDeclareVars", 1 )

#include <WindowsConstants.au3>
#include <GUIConstantsEx.au3>
#include <WinAPICom.au3>
#include <WinAPI.au3>

Global $hGui, $oCoreWebView2Environment, $oCoreWebView2Controller

#include "..\Includes\WV2Interfaces.au3"

WebView2()

Func WebView2()
  $hGui = GUICreate( "WebView2 Sample", 1200, 900, -1, -1, $WS_OVERLAPPEDWINDOW )

  _WinAPI_CoInitialize( $COINIT_APARTMENTTHREADED )

  CoreWebView2CreateCoreWebView2EnvironmentCompletedHandlerCreate()
  ConsoleWrite( "$pCoreWebView2CreateCoreWebView2EnvironmentCompletedHandler = " & $pCoreWebView2CreateCoreWebView2EnvironmentCompletedHandler & @CRLF & @CRLF )

  CoreWebView2CreateCoreWebView2ControllerCompletedHandlerCreate()
  ConsoleWrite( "$pCoreWebView2CreateCoreWebView2ControllerCompletedHandler = " & $pCoreWebView2CreateCoreWebView2ControllerCompletedHandler & @CRLF & @CRLF )

  Local $hWebView2Loader = DllOpen( @AutoItX64 ? "WebView2Loader-x64.dll" : "WebView2Loader-x86.dll" )
  Local $aRet = DllCall( $hWebView2Loader, "long", "CreateCoreWebView2EnvironmentWithOptions", "wstr", "", "wstr", "", _
                         "ptr", NULL, "ptr", $pCoreWebView2CreateCoreWebView2EnvironmentCompletedHandler )
  If @error Or $aRet[0] Then Return ConsoleWrite( "CreateCoreWebView2EnvironmentWithOptions ERR" & @CRLF )
  ConsoleWrite( "CreateCoreWebView2EnvironmentWithOptions OK" & @CRLF & @CRLF )

  GUISetState( @SW_SHOW )

  While 1
    Switch GUIGetMsg()
      Case $GUI_EVENT_MAXIMIZE, $GUI_EVENT_RESIZED
        Local $tRect = _WinAPI_GetClientRect( $hGui )
        $oCoreWebView2Controller.put_Bounds( $tRect )

      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  CoreWebView2CreateCoreWebView2ControllerCompletedHandlerDelete()
  CoreWebView2CreateCoreWebView2EnvironmentCompletedHandlerDelete()
  DllClose( $hWebView2Loader )
EndFunc

Func CoreWebView2CreateCoreWebView2EnvironmentCompletedHandler_Invoke( $pSelf, $long, $ptr ) ; Ret: long  Par: long;ptr*
  ConsoleWrite( "CoreWebView2CreateCoreWebView2EnvironmentCompletedHandler_Invoke" & @CRLF )
  $oCoreWebView2Environment = ObjCreateInterface( $ptr, $sIID_ICoreWebView2Environment, $dtag_ICoreWebView2Environment )
  ConsoleWrite( "IsObj( $oCoreWebView2Environment ) = " & IsObj( $oCoreWebView2Environment ) & @CRLF & @CRLF )
  $oCoreWebView2Environment.CreateCoreWebView2Controller( $hGui, $pCoreWebView2CreateCoreWebView2ControllerCompletedHandler )
  Return 0
  #forceref $pSelf, $long
EndFunc

Func CoreWebView2CreateCoreWebView2ControllerCompletedHandler_Invoke( $pSelf, $long, $ptr ) ; Ret: long  Par: long;ptr*
  ConsoleWrite( "CoreWebView2CreateCoreWebView2ControllerCompletedHandler_Invoke" & @CRLF )
  $oCoreWebView2Controller = ObjCreateInterface( $ptr, $sIID_ICoreWebView2Controller, $dtag_ICoreWebView2Controller )
  ConsoleWrite( "IsObj( $oCoreWebView2Controller ) = " & IsObj( $oCoreWebView2Controller ) & @CRLF )
  $oCoreWebView2Controller.AddRef()
  $oCoreWebView2Controller.get_CoreWebView2( $pCoreWebView2 )
  $oCoreWebView2 = ObjCreateInterface( $pCoreWebView2, $sIID_ICoreWebView2, $dtag_ICoreWebView2 )
  ConsoleWrite( "IsObj( $oCoreWebView2 ) = " & IsObj( $oCoreWebView2 ) & @CRLF & @CRLF )
  Local $tRect = _WinAPI_GetClientRect( $hGui )
  $oCoreWebView2Controller.put_Bounds( $tRect )
  $oCoreWebView2.Navigate( "https://www.bing.com/" )
  Return 0
  #forceref $pSelf, $long
EndFunc

The code is translated to AutoIt from the C++ code in the Getting started example step 3 and from the AutoHotkey code in WebView2.ahk.

A slightly updated example is included in WebView2-1.au3 in the 7z-file at bottom of this post.

Edited February 7, 2021 by LarsJ
The text after the code box

[mLipok]

Thanks.
I can't test today, as I will have cinema night with my wife
But this is great news.

Regards,
mLipok

 

[mLipok]

1 hour ago, LarsJ said:

First AutoIt implementation.

If somebody never use this before... How to start testing ?

Is this following requirment still valid ?

On 11/27/2020 at 12:55 AM, LarsJ said:

Download and install the WebView2 Runtime: Evergreen Bootstrapper.

Only x64 ?