Jump to content

Microsoft Edge - WebView2, embed web code in your native application


mLipok
 Share

Recommended Posts

Hi @UEZ
your _WinAPI_SHCreateStreamOnFileEx function is awesome!
I just put that function in my draft script and changed the parameters of the $dtag_ICoreWebView2 variable a bit and the CapturePreview method started working! Thanks so much for that 🙂
However it seems that the generated screenshot has some problems.
It seems that the generated screenshot doesn't capture the whole page but only a part;
also maybe you need somehow to close the stream after taking the screenshot? it appears that the file is only "flushed to disk" when the script is closed.
However this seems to be the right way even if still to be improved....

I've updated the two draft scripts (which are now starting to work) in my previous post

See You UEZ and thanks again
PS
If anyone has any improvements or new features to add to this material, they are welcome

Edited by Gianni

 

image.jpeg.9f1a974c98e9f77d824b358729b089b0.jpeg Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Link to comment
Share on other sites

When I add _WinAPI_ReleaseStream($hStream) then it seems to work properly:

...
            Case $idWebScreenShot ; CapturePreview
                ; create a stream on a file [By the great UEZ! :) ]
                FileDelete($sScrrenShot_Path)
                $hStream = _WinAPI_SHCreateStreamOnFileEx($sScrrenShot_Path, BitOR($STGM_READWRITE, $STGM_CREATE), $FILE_ATTRIBUTE_NORMAL, True)

                ConsoleWrite("-Debug stream --: " & @error & @TAB & VarGetType($hStream) & @TAB & $hStream & @TAB & VarGetType($oCoreWebView2) & @CRLF)
                ; load a web page
                ; $oCoreWebView2.Navigate("https://www.autoitscript.com")
                ; MsgBox(0, '', 'Pause to give time to load page')
                ConsoleWrite("------------------" & @CRLF)

                $oCoreWebView2.CapturePreview($COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT_PNG, $hStream, $pICoreWebView2CapturePreviewCompletedHandler)
                ; above call Forces in turn CoreWebView2CapturePreviewCompletedHandler_Invoke() callback function below to be executed after screenshot is taken
                ; MsgBox(0, '', 'Pause post screenshot')
                _WinAPI_ReleaseStream($hStream)

            Case $GUI_EVENT_CLOSE
...

Any idea how to get the image as a handle in the memory instead of writing it to a file?

Here the ChatGPT suggestion:

// Define a global or class-level buffer to hold the screenshot data
std::vector<BYTE> screenshotBuffer;

// Show the user a dialog and capture a screenshot of the WebView to memory
void FileComponent::CaptureScreenshotToMemory()
{
    // Clear the previous screenshot buffer
    screenshotBuffer.clear();

    CHECK_FAILURE(m_webView->CapturePreview(
        COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT_PNG,
        nullptr,
        Callback<ICoreWebView2CapturePreviewCompletedHandler>(
            [this](HRESULT error_code, IStream* preview_stream) -> HRESULT {
                CHECK_FAILURE(error_code);

                // Read the screenshot data from the stream into the memory buffer
                STATSTG stat;
                ULONG bytesRead = 0;
                preview_stream->Stat(&stat, STATFLAG_NONAME);
                screenshotBuffer.resize(stat.cbSize.LowPart);
                preview_stream->Read(&screenshotBuffer[0], stat.cbSize.LowPart, &bytesRead);

                // Handle the captured screenshot in memory (e.g., display or process it)
                // ...

                return S_OK;
            })
            .Get()));
}

I don't know how to init preview_stream as IStream. Maybe solution can be found here:

 

Btw, I made some small changes to the  _WinAPI_SHCreateStreamOnFileEx function.

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to comment
Share on other sites

9 hours ago, UEZ said:

Any idea how to get the image as a handle in the memory instead of writing it to a file?

_WinAPI_CreateStreamOnHGlobal() seems to work.
However, it is important to wait for the screenshot to be completed before reading the stream (I inserted a loop that waits for $GlobalFlag to be True. It is set to True by the callback function when the screenshot is completed). Below snippet shows the captured screenshot in a temporary GUI.....

Case $idWebScreenShot ; CapturePreview
    Local $hStream = _WinAPI_CreateStreamOnHGlobal()
    $GlobalFlag = False
    $oCoreWebView2.CapturePreview($COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT_PNG, $hStream, $pICoreWebView2CapturePreviewCompletedHandler)
    ; above call Forces in turn CoreWebView2CapturePreviewCompletedHandler_Invoke() callback function below to be executed after screenshot is taken
                
    Do ; wait until screenshot is completw
        Sleep(200)
    Until $GlobalFlag ; is set to True in the CoreWebView2CapturePreviewCompletedHandler_Invoke() below callback function
                
    Local $hBitmapFromStream = _GDIPlus_BitmapCreateFromStream($hStream) ;create bitmap from a stream
    WinAPI_ReleaseStream($hStream)
    Local $iX = _GDIPlus_ImageGetWidth($hBitmapFromStream)
    Local $iY = _GDIPlus_ImageGetHeight($hBitmapFromStream)
    Local $hTempGui = GUICreate('', $iX, $iY, 5, 5) ; to show the screenshot
    GUISetState(@SW_SHOW,$hTempGui)
    Local $hGraphics = _GDIPlus_GraphicsCreateFromHWND($hTempGui) ;create a graphics object from a window handle
    _GDIPlus_GraphicsDrawImage($hGraphics, $hBitmapFromStream, 0, 0) ;display streamed image
    MsgBox(0, '', 'pause')
    ;cleanup resources
    _GDIPlus_GraphicsDispose($hGraphics)
    _GDIPlus_BitmapDispose($hBitmapFromStream)
    GUIDelete($hTempGui)
Case $GUI_EVENT_CLOSE

 or something like that as suggested in this post ... :P

Local $hStream = _WinAPI_CreateStreamOnHGlobal()
    Local $hMemory = _WinAPI_GetHGlobalFromStream($hStream)
    Local $iMemSize = _MemGlobalSize($hMemory)
    Local $pMem = _MemGlobalLock($hMemory)
    Local $tData = DllStructCreate("byte[" & $iMemSize & "]", $pMem)
    Local $bData = DllStructGetData($tData, 1)
    _WinAPI_ReleaseStream($hStream)
    _MemGlobalFree($hMemory)
    Return $bData

 

Edited by Gianni

 

image.jpeg.9f1a974c98e9f77d824b358729b089b0.jpeg Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Link to comment
Share on other sites

@Gianni I already tested all that stuff but it didn't work because _GDIPlus_BitmapCreateFromStream always returned 0 / error. 😉

If it is working for you then probably I made some mistakes. 

 

Edit:

"However, it is important to wait for the screenshot to be completed before reading the stream" - that was the problem why it didn't work - thx.

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to comment
Share on other sites

10 hours ago, Gianni said:

(I inserted a loop that waits for $GlobalFlag to be True. It is set to True by the callback function when the screenshot is completed).

$GlobalFlag is always false and I couldn't find a definition of this variable in any of the files. I used Sleep(1000) for this example.

Where is it?

 

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to comment
Share on other sites

Hi @UEZ

sorry for the mess of my snippets

I declared the Global variable $GlobalFlag at the beginning of one of my test drafts but I didn't post it, I only posted the core piece of code...

Global $GlobalFlag = False

Also, since the CapturePreview function seems to be asynchronous, I've set the $GlobalFlag setting inside the callback function for the CapturePreview so that it's only set to True when the capture is complete.

This functio is executed automatically only when CapturePreview has ended

Func CoreWebView2CapturePreviewCompletedHandler_Invoke($pSelf, $iErrorCode)   ; Ret: dword
    ConsoleWrite("CoreWebView2CapturePreviewCompletedHandler_Invoke()" & @CRLF)
    ConsoleWrite("CapturePreview returned Error code: " & $iErrorCode & @CRLF)
    $GlobalFlag = True
    Return 0     ; For AddRef/Release
    #forceref $pSelf, $iErrorCode
EndFunc   ;==>CoreWebView2CapturePreviewCompletedHandler_Invoke

in this way the "wait" loop is exceeded only when the $GlobalFlag becomes True in the CallBack function, i.e. when the screenshot capture is finished.

Sorry again for the mess... as soon as I can I try to group all the positive "discoveries" into a single script.

 

image.jpeg.9f1a974c98e9f77d824b358729b089b0.jpeg Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Link to comment
Share on other sites

How did you do it? My _GDIPlus_BitmapCreateFromStream is still 0.

Case $idWebScreenShot ; CapturePreview

                $GlobalFlag = False

                Local $hStream = _WinAPI_CreateStreamOnHGlobal()

                $oCoreWebView2.CapturePreview($COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT_PNG, $hStream, $pICoreWebView2CapturePreviewCompletedHandler)
                ; above call Forces in turn CoreWebView2CapturePreviewCompletedHandler_Invoke() callback function below to be executed after screenshot is taken

                Do ; wait until screenshot is completw
                    Sleep(10)
                Until $GlobalFlag ; is set to True in the CoreWebView2CapturePreviewCompletedHandler_Invoke() below callback function

                Local $hBitmapFromStream = _GDIPlus_BitmapCreateFromStream($hStream) ;create bitmap from a stream
                MsgBox(0, "", $hBitmapFromStream)

                _WinAPI_ReleaseStream($hStream)
                
                Local $iX = _GDIPlus_ImageGetWidth($hBitmapFromStream)
                Local $iY = _GDIPlus_ImageGetHeight($hBitmapFromStream)

                Local $hTempGui = GUICreate('', $iX, $iY, 5, 5) ; to show the screenshot
                GUISetState(@SW_SHOW, $hTempGui)

                Local $hGraphics = _GDIPlus_GraphicsCreateFromHWND($hTempGui) ;create a graphics object from a window handle
                _GDIPlus_GraphicsDrawImage($hGraphics, $hBitmapFromStream, 0, 0) ;display streamed image

                MsgBox(0, '', 'pause')
                ;cleanup resources
                _GDIPlus_GraphicsDispose($hGraphics)
                _GDIPlus_BitmapDispose($hBitmapFromStream)
                GUIDelete($hTempGui)

 

Func CoreWebView2CapturePreviewCompletedHandler_Invoke($pSelf, $iErrorCode)   ; Ret: dword
    #forceref $pSelf, $iErrorCode
    ConsoleWrite("CoreWebView2CapturePreviewCompletedHandler_Invoke()" & @CRLF)
    ConsoleWrite("CapturePreview returned Error code: " & $iErrorCode & @CRLF)
    $GlobalFlag = True
    Return 0     ; For AddRef/Release
EndFunc   ;==>CoreWebView2CapturePreviewCompletedHandler_Invoke

 

Link to comment
Share on other sites

I'm just struggling with this to take a screenshot of a given website without showing the site in a gui. Is it somehow possible? Any suggestions?

 

Thx.

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to comment
Share on other sites

16 minutes ago, KaFu said:

Not directly, but I guess it might work if you create the process on a different desktop (_WinAPI_CreateDesktop) without switching to it.

I'm wondering if the second DESKTOP needs to be active to transmit signals, or it will be the same as with a minimized RDP session when AutoIt GUI based programs don't work.

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) * Show_Macro_Values.au3 *

 

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskSchedulerIE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related:How to get reference to PDF object embeded in IE * IE on Windows 11

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

I also encourage you to check awesome @trancexx code:  * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuffOnHungApp handlerAvoid "AutoIt Error" message box in unknown errors  * HTML editor

winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2023-04-24

Link to comment
Share on other sites

8 hours ago, KaFu said:

I use this technique in SSD to switch the Windows Sound dialog on the invisible secondary desktop with controlsend().

Here a dirty PoC, works fine for me.

_WinAPI_CreateDesktop_Wrapper.zip 793.5 kB · 11 downloads

Thanks KaFu, but I would like to take a screenshot of the entire site, not just the GUI size. Further, your code returns a hStream handle but hStream doesn't contain any bitmap data because it is not filled up by $oCoreWebView2.CapturePreview. Reason is that the website is not rendered. You must add some Sleep() to give some time for rendering.

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to comment
Share on other sites

11 hours ago, UEZ said:

Thanks KaFu, but I would like to take a screenshot of the entire site, not just the GUI size. Further, your code returns a hStream handle but hStream doesn't contain any bitmap data because it is not filled up by $oCoreWebView2.CapturePreview. Reason is that the website is not rendered. You must add some Sleep() to give some time for rendering.

That was not your question 😉. I've play around with the function last week and never could render more than the "visible" client size (even if the window is larger), seems the function is lazy.

There's a sleep(1000) in the script, maybe that's not enough on your system, for me it worked fine.

For a full-size preview of a website maybe go back to this "Shell.Explorer.2" solution?

 

Link to comment
Share on other sites

Your are right, my question was not precise enough but the web screenshot from Ward or any other IE solution doesn't work anymore when IE is not active anymore, for example on my Win11 notebook.

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to comment
Share on other sites

When I remove the GUI style and use a GUI like this:

$hGui = GUICreate("WebView2 Sample", 100,100)

I can resize it to any arbitrary size after the GUISetState() like this

WinMove($hGui,"",0,0,1080,8000)

Performing this on a different desktop allows for saving the full website invisible.

Now a challenge is to determine the full size of the webpage in px. Maybe a function to check when a single color starts to repeat to fill all following lines, and then cut it off?

Edit:

A solution also might be to cycle through increasing screenshots until the scrollbar on the right disappears.

Sleep() time depends on the responsiveness of the webpage (obviously), for how long it takes to fully render.

Edit2:

Do you assume that "Shell.Explorer.2" will not work on Win11, or did you try it? I'm still on Win10, but search results indicate it still should work,

Edit3:

The sleep() call should be replace with an event listener for "NavigationCompleted"

Edited by KaFu
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...