Jump to content
argumentum

[Solved] non-block download to memory ?

Recommended Posts

argumentum
Posted (edited)
#include <InetConstants.au3>
#include <MsgBoxConstants.au3>
#include <WinAPIFiles.au3>

; Download a file in the background.
; Wait for the download to complete.

Example()

Func Example()
    ; Save the downloaded file to the temporary folder.
    Local $sFilePath = _WinAPI_GetTempFileName(@TempDir)

    ; Download the file in the background with the selected option of 'force a reload from the remote site.'
    Local $hDownload = InetGet("http://www.autoitscript.com/autoit3/files/beta/update.dat", $sFilePath, $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)

    ; Wait for the download to complete by monitoring when the 2nd index value of InetGetInfo returns True.
    Do
        Sleep(1)
        
        ; so this is non-blocking
        
        
        ; I can do other stuff
        ; that I need to get done
        ; without waiting for
        ; the download to get done
        
        ; but if I use  InetRead("URL", 9) I don't write to disk   =D
                
        ; can I have a $sFilePath in memory or a way not to write to file ???
        
        
    Until InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)

    ; Retrieve the number of total bytes received and the filesize.
    Local $iBytesSize = InetGetInfo($hDownload, $INET_DOWNLOADREAD)
    Local $iFileSize = FileGetSize($sFilePath)

    ; Close the handle returned by InetGet.
    InetClose($hDownload)

    ; Display details about the total number of bytes read and the filesize.
    MsgBox($MB_SYSTEMMODAL, "", "The total download size: " & $iBytesSize & @CRLF & _
            "The total filesize: " & $iFileSize)

    ; Delete the file.
    FileDelete($sFilePath)
EndFunc   ;==>Example

 

Edited by argumentum
[Solved]

Share this post


Link to post
Share on other sites
Deye


Didn't really try this but you could try WinHttp.au3 functions to download the data
On the option: find a way to Bin the data to file ..

Share this post


Link to post
Share on other sites
AutoBert

It works for me, the file has a size from 456 bytes. But deleting the file works also.

Share this post


Link to post
Share on other sites
argumentum
Posted (edited)

@AutoBert, I was looking for an answer along the way of using-file-attribute-temporary-and-file-flag-delete-on-close .

@Deye, I was also looking for the above mentioned approach, but if nothing else, I could use the WinHttp.au3 functions.

Edit: I'm looking for an example using _WinAPI_CreateFileEx for this =)

from https://blogs.msdn.microsoft.com/larryosterman/2004/04/19/its-only-temporary/

NT has a whole lot of really cool features that aren’t always obvious without REALLY looking closely at the documentation.

One of my favorite is what I call “temporary” temporary files.

A “temporary” temporary file is one whose storage is never written to disk (in the absence of memory pressure).  It behaves just like a file does (because it’s a file) but the cache manager disables the lazy writer on the file, and the filesystem knows to never commit the pages containing the file’s metadata to disk.

To create a “temporary” file, you call CreateFile specifying FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE in the dwFlagsAndAttributes attribute.  This combination of bits acts as a hint to the filesystem that the file data should never be flushed to disk.  In other words, such a file can be created, written to, and read from without the system ever touching the disk.

 

Edited by argumentum
found _WinAPI_CreateFileEx

Share this post


Link to post
Share on other sites
argumentum

ok, better example of what I'd like to have:

#cs ----------------------------------------------------------------------------

 AutoIt Version: 3.3.14.5
 Author:         myName

 Script Function:
    Template AutoIt script.

#ce ----------------------------------------------------------------------------

; Script Start - Add your code below here



#include <WinAPIHObj.au3>
#include <WinAPIFiles.au3>

#include <InetConstants.au3>
#include <MsgBoxConstants.au3>
#include <WinAPIFiles.au3>

; Download a file in the background.
; Wait for the download to complete.

Example()

Func Example()
    ; Save the downloaded file to the temporary folder.
    Local $sFilePath = _WinAPI_GetTempFileName(@TempDir)

    ConsoleWrite('$sFilePath = "' & $sFilePath & '"' & @CRLF)


    ; using the $FILE_ATTRIBUTE_TEMPORARY & $FILE_FLAG_DELETE_ON_CLOSE
    Local $hFile = _WinAPI_CreateFileEx($sFilePath, $CREATE_ALWAYS, BitOR($GENERIC_READ, $GENERIC_WRITE), BitOR($FILE_SHARE_DELETE,$FILE_SHARE_READ,$FILE_SHARE_WRITE), BitOR($FILE_ATTRIBUTE_TEMPORARY, $FILE_FLAG_DELETE_ON_CLOSE))

    ; Download the file in the background with the selected option of 'force a reload from the remote site.'
    Local $hDownload = InetGet("http://www.autoitscript.com/autoit3/files/beta/update.dat", $hFile, $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)

    ; Wait for the download to complete by monitoring when the 2nd index value of InetGetInfo returns True.
    Do
        Sleep(250)
    Until InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)

    ; Retrieve the number of total bytes received and the filesize.
    Local $iBytesSize = InetGetInfo($hDownload, $INET_DOWNLOADREAD)

    ; Close the handle returned by InetGet.
    InetClose($hDownload)

    Local $iSize = ByteSuffix( _WinAPI_GetFileSizeEx($hFile))
    Local $iFileSize = ByteSuffix(FileGetSize($hFile))

    ; Display details about the total number of bytes read and the filesize.
    ConsoleWrite( "The total download size: " & $iBytesSize & ' - ( if more than zero, the file downloaded )' & @CRLF )
    ConsoleWrite( "filesize as per FileGetSize: " & $iFileSize & ' - ( if more than zero, the file got written )' & @CRLF)
    ConsoleWrite( "filesize as per _WinAPI_GetFileSizeEx: " & $iSize & ' - ( if more than zero, the file got written )' & @CRLF)

    ConsoleWrite('--- file start ---' & @CRLF)
    ConsoleWrite(FileRead($hFile)) ; <----------- this is not working  =/
    ConsoleWrite('--- file end ---' & @CRLF)


    ;;; from _WinAPI_CreateFile example
    Local $sText, $nBytes
    ConsoleWrite('--- file start ---' & @CRLF)
    ; 2) read 6 bytes from position 3
    $tBuffer = DllStructCreate("byte[6]")
    $iSizeEx = _WinAPI_GetFileSizeEx($hFile)
    _WinAPI_SetFilePointer($hFile, 3)
    _WinAPI_ReadFile($hFile, $tBuffer, 6, $nBytes) ; <----------- this is not working  =/
    $sText = BinaryToString(DllStructGetData($tBuffer, 1))
    ConsoleWrite('2):' & $iSizeEx & ' ' & $sText & @CRLF)
    ConsoleWrite('--- file end ---' & @CRLF)
    ;;; from _WinAPI_CreateFile example


    _WinAPI_CloseHandle($hFile)
;~  ; Delete the file.
;~  FileDelete($sFilePath)
EndFunc   ;==>Example

Func ByteSuffix($iBytes)
    Local $iIndex = 0, $aArray = [' bytes', ' KB', ' MB', ' GB', ' TB', ' PB', ' EB', ' ZB', ' YB']
    While $iBytes > 1023
        $iIndex += 1
        $iBytes /= 1024
    WEnd
    Return Round($iBytes) & $aArray[$iIndex]
EndFunc   ;==>ByteSuffix

but it does not work :'(

Share this post


Link to post
Share on other sites
Danyfirex

Hello InetGet does not support pass a file handle.  

Between check this link interesting infomation about FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE 

https://stackoverflow.com/questions/5242676/using-file-attribute-temporary-and-file-flag-delete-on-close

 

Saludos

Share this post


Link to post
Share on other sites
argumentum
1 hour ago, Danyfirex said:

Same link I posted =/

 

1 hour ago, Danyfirex said:

Hello InetGet does not support pass a file handle

Well, If you run the script, you'll see in the console: 

$sFilePath = "C:\Users\PACSAD~1\AppData\Local\Temp\C447.tmp"
The total download size: 456 - ( if more than zero, the file downloaded )
filesize as per FileGetSize: 456 bytes - ( if more than zero, the file got written )
filesize as per _WinAPI_GetFileSizeEx: 0 bytes - ( if more than zero, the file got written )
--- file start ---
--- file end ---
--- file start ---
2):0 --- file end ---

So, "filesize as per FileGetSize: 456 bytes" means that , something got written, somewhere. Maybe the access permissions are wrong ?. No clue.
 

Share this post


Link to post
Share on other sites
Danyfirex

Of course basically you write a file with name format like 0xXXXXXXX in script directory because InetGet does not support pass a handle so you basically past string "0x000000" and the download work correctly but you are no able to get what you expect. 

 

Also check that InetGet internally do not use FILE_READ_ATTRIBUTES and you will be not able to get  the file size
 CreateFileW ( "0x00000224", GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, 0x04caf7ac, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL )  

You dont get any result int _WinAPI_GetFileSizeEx because the handle you pass to inetget is never used InetGet never write to that file it write to the internal created one named like 0x000000.

 

Saludos

  • Like 1

Share this post


Link to post
Share on other sites
argumentum
10 hours ago, Danyfirex said:

You dont get any result int _WinAPI_GetFileSizeEx because the handle you pass to inetget is never used InetGet never write to that file it write to the internal created one named like 0x000000.

Thanks @Danyfirex.

So as a solution I'll fork the download to another process ( that would download once a second ) and get the data via IPC, to free the main script from posible timeouts waiting for the data.

[Solved]

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×