Jump to content
mLipok

Microsoft Edge - WebView2, embed web code in your native application

Recommended Posts

Anybody could help with using QueryInterface in AutoIt to embed Microsoft Edge using WebView2 Interface ?

Here are links to information/documentation:

whatwebview.png

Image comes from: https://docs.microsoft.com/en-us/microsoft-edge/webview2/media/webview2/whatwebview.png

Introduction to Microsoft Edge WebView2
https://docs.microsoft.com/en-us/microsoft-edge/webview2/

Understand WebView2 SDK versions
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/versioning

Quote
  • Win32 C/C++. When using QueryInterface to obtain a new interface, check for a return value of E_NOINTERFACE. This value may indicate that the WebView2 Runtime is a previous version, and doesn't support that interface.

Download:
https://developer.microsoft.com/en-US/microsoft-edge/webview2/
https://developer.microsoft.com/en-US/microsoft-edge/webview2/#download-section
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution

API Reference:
https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/?view=webview2-1.0.622.22

HowTo/Concepts:
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/versioning
https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/win32
https://docs.microsoft.com/en-us/microsoft-edge/webview2/howto/debug?tabs=devtools
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/security
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/userdatafolder

https://mybuild.microsoft.com/sessions/9198aeac-0c8e-4d32-96d1-cbb99a390aa6

Example:
https://github.com/MicrosoftEdge/WebView2Samples

Related:

https://www.essentialobjects.com/Products/WebBrowser/Default.aspx?gclid=CjwKCAiA17P9BRB2EiwAMvwNyMe6UI9VHej-JYDBUoyGDFz40KaaJ_RidY-75Fhr-PHz65sSVC0XlxoC5loQAvD_BwE

https://docs.microsoft.com/en-us/microsoft-edge/webview2/howto/webdriver

 

btw.
I'm not WindowsApi expert (QueryInterface, C++, DllCall).
What I trying to say is:     It means that I do not done anything in this regards .... as so far.

 

Edited by mLipok

Signature beginning:   Wondering who uses AutoIt and what it can be used for ?
* GHAPI UDF - modest beginning - communication with GitHub REST API Forum Rules *
Include Dependency Tree (Tool for analyzing script relations)
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskSchedulerIE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) *

PDF Related:How to get reference to PDF object embeded in IE *

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

I also encourage you to check awesome @trancexx code:  * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuffOnHungApp handlerAvoid "AutoIt Error" message box in unknown errors  * HTML editor

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2021-01-03

Share this post


Link to post
Share on other sites

Is there a way to use this stuff by making use of the following other stuff? or am I just raving?
sorry if I'm just delirious ...

 


small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

hmmm, thanks @ptrex for your hopeful response :lmao:
I'll try to see if I can understand something :unsure: .... sure that if there was even just a simple draft, an "hello world" to start experimenting things would be easier for the rest of us .... :whistle:

Thanks again :)


small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

The easiest way to get started is probably to translate the code in this AutoHotkey thread into AutoIt or at least take the AutoHotkey code as a starting point. We have tried this before a few years ago with relatively good success.

Share this post


Link to post
Share on other sites

hi Lars,

I tried yesterday do get it running in plain .NET environment. 

I was able to get the WebView2 control on the Gui but that is as far as got... no webpage is loading so far. 

So I am missing probably some dependencies or it needs a specific .NET environment. As long as it doens't run in .NET I can't move it to au3.

 


Share this post


Link to post
Share on other sites

If you read this here, I guess it is too early to start playing with it... 

https://stackoverflow.com/questions/61291674/how-can-i-add-webview2-control-in-visual-studio-toolbar

https://docs.microsoft.com/en-us/microsoft-edge/webview2/releasenotes

There are quite a lot of dependencies needed to make it all work well...

If you have not got all the versions aligned it is a no go at the moment ?


Share this post


Link to post
Share on other sites

Even if the .NET version of the WebView2 code does not work, it may well be that the Windows API version works. The AutoHotkey code actually works.

If you want to try the AutoHotkey code, use this procedure. I've only tested on Windows 7. Not on Windows 10. I think the procedure is the same. 

Install AutoHotkey and SciTE for AutoHotkey:

  • Download and install AutoHotkey_1.1.33.02_setup.exe.
    By default, AutoHotkey is installed under C:\Program Files\AutoHotkey.
  • Download and install SciTE4AutoHotkey. Click "Installer".
  • Download AutoHotkey v2 by clicking Current Alpha Release and extract the zip-file.
  • Rename C:\Program Files\AutoHotkey\AutoHotkey.exe to AutoHotkey-a.exe or similar.
  • Rename (or copy) AutoHotkeyU64.exe from the AutoHotkey v2 zip-file to AutoHotkey.exe.
  • Copy the new AutoHotkey.exe to C:\Program Files\AutoHotkey

Download and install the WebView2 Runtime: Evergreen Bootstrapper.

Download AutoHotkey code and data to test WebView2:

  • Download webview2.zip at the bottom of this post and extract the zip-file.
  • Open webview2.ahk in SciTE4AutoHotkey and run the script by pressing F5.

HqicNVD.png

The AutoHotkey post describes that you need to install a beta or development version of Microsoft Edge to make the script work. But it has not been necessary.

Now the AutoHotkey code must be translated to AutoIt. We've done this before and we can probably do it again. It's not necessarily completely trivial. But probably not completely impossible either.

This is the AutoHotkey code to translate, webview2.ahk:

#Requires AutoHotkey v2.0-a121
#SingleInstance Force

; DllCall("User32.dll\SetProcessDpiAwarenessContext", "Ptr", 0xFFFFFFFC) ; not sure if this is needed.  Works fine without it.

main := gui.new("+Resize")
main.OnEvent("Close", "ExitApp")

main.Show(Format("w{} h{}", A_ScreenWidth * 0.4, A_ScreenHeight * 0.6))

; need to have the size also otherwise when you un-maximize it the window has zero height and width
wv := WebView.new(main)
wv.create()

return

mainSize(controller, win, minMax, *) {
    ; for performance reasons, you should make the webview2 not visible when the parent window is minimized/hidden.
    ComCall(3, controller, "Int*", isVisible := 0) ; IWebView2WebViewController::get_IsVisible
    if (minMax == -1) {
        if (isvisible) {
            ComCall(4, controller, "Int", false) ; IWebView2WebViewController::put_IsVisible
        }
    }
    else {
        if (!isVisible) {
            ComCall(4, controller, "Int", true) ; IWebView2WebViewController::put_IsVisible
        }
        else {
            DllCall("User32.dll\GetClientRect", "Ptr", win.hWnd, "Ptr", RECT := BufferAlloc(16))
            ComCall(6, controller, "Ptr", RECT) ; IWebView2WebViewController::put_Bounds
        }
        
    }
}

class WebView {    
    __new(parent, dllPath := "WebView2Loader.dll") {
        if (Type(parent) != "Gui") {
            throw Exception("Parent must be of type: Gui")
        }
        
        this.parent := parent
        this.dllPath := dllPath
        this.controllerCompletedHandler := ICoreWebView2CreateCoreWebView2ControllerCompletedHandler.new(this)
        this.envCompletedHandler := IWebView2CreateWebView2EnvironmentCompletedHandler.new(this.parent, this.controllerCompletedHandler)
    }
    
    OnControllerCompleted(cb) {
        this.controllerCreatedCallbacks.push(cb)
    }
    
    create() {
        if (R := DllCall(this.dllPath . "\CreateCoreWebView2Environment", "Ptr", this.envCompletedHandler, "UInt")) {
            MsgBox("ERROR " . Format("{:08X}", R))
        }
    }
}

class IWebView2CreateWebView2EnvironmentCompletedHandler extends IUnknown {
    __New(parent, controllerCompleted) {
        super.__new()
        
        this.parent := parent
        this.controllerCompleted := controllerCompleted
    }
    
    Invoke(thisPtr, hresult, ICoreWebView2Env) {
        ; ICoreWebView2Environment::CreateCoreWebView2Controller Method.
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2environment#createcorewebview2controller
        ComCall(3, ICoreWebView2Env, "Ptr", this.parent.hwnd, "Ptr", this.controllerCompleted)

        return 0  ; S_OK.
    }
}

class ICoreWebView2CreateCoreWebView2ControllerCompletedHandler extends IUnknown {
    __new(wv) {
        super.__new()
        
        this.wv := wv
    }
    
    Invoke(thisPtr, HRESULT, IWebView2WebViewController) {    
        ObjAddRef(IWebView2WebViewController) ; This was key to retain a reference to the Controller
        
        this.wv.parent.OnEvent("Size", Func("mainSize").bind(IWebView2WebViewController))
        
        ; Resize WebView to fit the bounds of the parent window.
        ; IWebView2WebViewController::put_Bounds Method.
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2controller#put_bounds
        DllCall("User32.dll\GetClientRect", "Ptr", this.wv.parent.hWnd, "Ptr", RECT := BufferAlloc(16))
        ComCall(6, IWebView2WebViewController, "Ptr", RECT)

        ; IWebView2WebViewController::get_CoreWebView2
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2controller#get_corewebview2
        ComCall(25, IWebView2WebViewController, "Ptr*", coreWebView := 0)
        this.wv.coreWebView := coreWebView
        
        ; ICoreWebView2::add_navigationstarting
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2#add_navigationstarting
        ; not really sure what the last parameter (EventRegistrationToken) is for but passing an empty buffer makes it work
        ComCall(7, this.wv.coreWebView, "Ptr", ICoreWebView2NavigationStartingEventHandler.new(), "Ptr", token := BufferAlloc(8))
        
        ; ICoreWebView2::Navigate
        ; https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/0-9-538/icorewebview2#navigate
        ; ComCall(5, this.wv.coreWebView, "Str", "https://getbootstrap.com/")
        ComCall(5, this.wv.coreWebView, "Str", "file:///" A_ScriptDir "/index.html")

        return 0  ; S_OK.
    }
}

class ICoreWebView2NavigationStartingEventHandler extends IUnknown {
    Invoke(thisPtr, IWebView2WebView, IWebView2NavigationStartingEventArgs) {
        ComCall(4, IWebView2NavigationStartingEventArgs, "Int*", isUserInitiated := 0)
        ComCall(5, IWebView2NavigationStartingEventArgs, "Int*", isRedirected := 0)
        
        if (isUserInitiated && !isRedirected) {
            ComCall(3, IWebView2NavigationStartingEventArgs, "Str*", uri := "")
            
            if (MsgBox("You are about to navigate to: " . uri . "`n`nDo you want to continue?", "Navigation warning", "YN Icon!") = "No") {
                
                ; IWebView2NavigationStartingEventArgs::put_Cancel
                ComCall(8, IWebView2NavigationStartingEventArgs, "Int", true)
                
                ; ICoreWebView2::NavigateToString
                ComCall(6, IWebView2WebView, "Str", "<h1>Navigation Canceled</h1><p>You chose to cancel navigation to the following URL: " . uri . "</p>")
            }
        }
        
        return 0 ; S_OK
    }
}

class IUnknown {
    methods := ["QueryInterface", "AddRef", "Release", "Invoke"]
    vtbl := BufferAlloc(4 * A_PtrSize)
    
    __New() {
        for (name in this.methods) {
            method := this.GetMethod(name)
            NumPut("UPtr", CallbackCreate(method.bind(this), "", method.MinParams - 1), this.vtbl, (A_Index - 1) * A_PtrSize)
        }
        
        this.ptr := DllCall("Kernel32.dll\GlobalAlloc", "UInt", 0, "Ptr", A_PtrSize + 4, "UPtr")
        NumPut("UPtr", this.vtbl.ptr, this.ptr)
        NumPut("UInt", 1, this.ptr, A_PtrSize)
    }
    
    __Delete() {
        DllCall("Kernel32.dll\GlobalFree", "Ptr", this)
    }
    
    QueryInterface(riid, ppvObject) {
        ; No idea why this isn't called at all...
    }
    
    AddRef(interface) {
        ObjAddRef(interface)
    }
    
    Release(interface) {
        ObjRelease(interface)
    }
}

 

Edited by LarsJ
AutoHotkey code

Share this post


Link to post
Share on other sites

Hi Lars,

Thanks for the update on this.

The Windows API  sure do work I don't doubt that. But I am not a Windows API guy 😞

So if someone feels he or she can do the AHK to AU3 translation that would be great.

In the meantime I will try to get the .Net version to work and report back if I have something going.

 


Share this post


Link to post
Share on other sites

Thanks all for the tips and the considerations on this matter,
....also, just to know,
since this other project (https://www.autoitscript.com/forum/topic/193846-chromium-embedded-framework-for-autoit3/?do=findComment&comment=1469397) already seems to work on AutoIt, I am wondering if the "Chromium Embedded Framework" used by that project is the same "environment" used by Edge too, that is, if the underlying engine involved is ultimately the same?? ( ie if both share the same DLLs or whatever?)
Thanks again

p.s.
I'm wondering also if Chrome and Edge has the same engine (that is if both uses the same phisical files on the disk??)

 

Edited by Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

This weekend I've tested the AutoHotkey code on Windows 10. There are links to two different scripts in the posts above. None of these scripts works on Windows 10. So there is no code to translate to AutoIt. Of course, I've installed the Microsoft Edge Chromium version on Windows 10 and the required runtime code. The Edge version is the same on Windows 10 and Windows 7.

To get ahead, it's probably easiest to look at the getting started example. The required WebView2 COM interfaces used in this example are not included in the Windows SDK and must be installed through the WebView2 NuGet package.

And since these COM interfaces are also needed for tag descriptions for ObjCreateInterface() and ObjectFromTag() (for callback interfaces as in UIA Events), it makes good sense to continue with this example.

The problem with this approach is that it'll be time consuming. It'll require more time than I'm interested in spending right now. Therefore, I'll not go ahead with this project. Sorry.

Instead, I'll continue with the projects on rectangular selections in listviews. I'll be working on these projects through the rest of the winter.

Whether I'll continue with the WebView2 project afterwards, time will tell.

Share this post


Link to post
Share on other sites

Hi Lars,

Same thing for me. 

Over the weekend I spend quite a lot of time trying to get the simple .NET example going. But also no success 😞

It seems that the libraries are not yet stable enough to start building upon.

If you read the simple getting started example, which promises to get everyone going in a few lines.

https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/winforms

But unfortunately it did not...

I am stuck at the moment to get the "WebView2Loader.dll" loaded.

Anyhow if there is news I will release an update.

 

 


Share this post


Link to post
Share on other sites

We're back in business.

Tonight I've installed the latest version of Visual Studio, the WebView2 GitHub sample files and the WebView2 NuGet package as described in the documentation for the WebView2 getting started example. Step 1 - 3 in the documentation. After a single small error, the build process succeeded in the second attempt.

The executable WebView2 file started automatically (the Bing search shown in the image in the documentation), I searched for "autoit forums" and then browsed through exactly this thread.

So now we have code that works on Windows 10, we have code that can be translated into AutoIt, and we have the WebView2.h header file from the NuGet package that seems to contain all the information needed to create tag description files for ObjCreateInterface() and ObjectFromTag().

I've once again changed the priority of my projects. They are all exciting. But this is the most exciting right now. Already this weekend, I hope to be able to take the first steps in the translation task.

HelloWebView.cpp and WebView2.h are contained in the 7z-file.

WebView2.7z

Edited by LarsJ
WebView2.7z

Share this post


Link to post
Share on other sites

Hi Lars,

Great news !

Anyhow I have been playing around with the .NET version and assemblies that have been release.

But I and many others can't get that running at the moment because there too many dependencies that are not aligned right now.

And versions are changing rapidly too. So I can't move forward on this one unfortunately until the releases are getting more stable.

 


Share this post


Link to post
Share on other sites

as a curiosity

image.png.5de062dae53a948db1d1d216228197b9.png


Signature beginning:   Wondering who uses AutoIt and what it can be used for ?
* GHAPI UDF - modest beginning - communication with GitHub REST API Forum Rules *
Include Dependency Tree (Tool for analyzing script relations)
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskSchedulerIE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) *

PDF Related:How to get reference to PDF object embeded in IE *

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

I also encourage you to check awesome @trancexx code:  * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuffOnHungApp handlerAvoid "AutoIt Error" message box in unknown errors  * HTML editor

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2021-01-03

Share this post


Link to post
Share on other sites

I was afraid that after M$ drops support for IE and removes it completely from Windows, we won't have any default installed browser which can be scripted, but good to see that they are making Edge usable over WinAPI :)

And since Edge is based on Chromium, it would be awesome if the changes are somehow backported, then we can use the same API to interact with Chrome!

Edit: Forgot that Chromium is not a GPL project, so Edge isn't open-source and thus the changes can't be backported... without the help of reverse engineering anyway. Oh well, I guess Edge will do better than IE atleast.

Edited by TheDcoder

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...