Binary Download with Basic Authentication

[josheli]

Hi. I've been a user/lurker for awhile and have been searching/waiting for a way, without using the COM functions, to download binary files through HTTP using basic authentication. Anyway, with the help and inspiration of the InetGetSource from wOuter, the _BinaryWrite functions from Larry, and MSDN, i've created the following snippet to log into a website using basic authentication, and download a binary file to the local box without creating a temp file.

Some notes:

- Contains some Progress window code, which is obviously not necessary.

- Comments include the wininet dll function prototypes from MSDN, just for info

- Some of my DLLCall function parameter types may be wrong, wasn't sure on some types to use

- My code quality and error handling leave something to be desired

- It uses a buffer of 4096 bytes (not sure if this is too much/too little/just right), but it works

- I've used it to download a 375mb file which took about 15 minutes, so maybe it will be useful.

- sorry about the formatting

expandcollapse popup

;contains _APIFileOpen(), _BinaryWrite(), etc...
#include <BinaryFile.au3>

; Download a binary file through HTTP Basic Authentication
;
; $domain - domain only, without "http://" or "/path", e.g. "www.example.com"
; $user  - basic auth username
; $pass  - basic auth password
; $dl_file   - Request URI you would find in a HTTP GET header, e.g.; "/somedir/some_file_to_download.zip"
; $sv_file  - Local file path and name to save to, e.g. "C:\downloads\example-file.zip"
;
Func download_file($domain, $user, $pass, $dl_file, $sv_file)

  Local $h_Wininet = DllOpen("wininet.dll")
  Local $ai_InternetOpen
  Local $ai_InternetConn
  Local $ai_InternetCloseHandle
  Local $av_InternetReadFile[5] = [0, 0, '', 0, 1]
  Local $sentReq = 0
  Local $http_port = 80

  ProgressOn("Download File", "Connecting...")

#cs
HINTERNET InternetOpen(
  LPCTSTR lpszAgent,
  DWORD dwAccessType,
  LPCTSTR lpszProxyName,
  LPCTSTR lpszProxyBypass,
  DWORD dwFlags
);
#ce
  $ai_InternetOpen = DllCall($h_Wininet, 'long', 'InternetOpen', 'str', "User Agent!", 'long', 0, 'str', '', 'str', '', 'long', 0)
  If @error Or $ai_InternetOpen[0] = 0 Then
        DllClose($h_Wininet)
        SetError(1)
    ErrorBox("InternetOpen error.")
        Return 0
  EndIf

#cs
InternetConnect(
  HINTERNET hInternet,
  LPCTSTR lpszServerName,
  INTERNET_PORT nServerPort,
  LPCTSTR lpszUsername,
  LPCTSTR lpszPassword,
  DWORD dwService,
  DWORD dwFlags,
  DWORD_PTR dwContext
);
#ce

  $ai_InternetConn = DllCall($h_Wininet, 'long', 'InternetConnect', 'long', $ai_InternetOpen[0], 'str', $domain, 'short', $http_port, 'str', $user, 'str', $pass, 'int', 3, 'int', 0, 'int', 0)
   If @error Or $ai_InternetConn[0] = 0 Then
        DllCall($h_Wininet, 'int', 'InternetCloseHandle', 'long', $ai_InternetOpen[0])
        DllClose($h_Wininet)
        SetError(1)
    ErrorBox("InternetConnect Error.")
        Return 0
    EndIf

#cs
BOOL InternetSetOption(
  HINTERNET hInternet,
  DWORD dwOption,
  LPVOID lpBuffer,
  DWORD dwBufferLength
);
#ce
  
 ;INTERNET_OPTION_USERNAME = 28
  Local $userlen = StringLen($user) + 1
  Local $userset = DllCall($h_Wininet, 'long', 'InternetSetOption', 'int', $ai_InternetConn[0], 'long', 28, 'str', $user, 'long', $userlen)

 ;INTERNET_OPTION_PASSWORD = 29
  Local $passlen = StringLen($pass) + 1
  Local $passset = DllCall($h_Wininet, 'long', 'InternetSetOption', 'int', $ai_InternetConn[0], 'long', 29, 'str', $pass, 'long', $passlen)

#cs
HINTERNET HttpOpenRequest(
  HINTERNET hConnect,
  LPCTSTR lpszVerb,
  LPCTSTR lpszObjectName,
  LPCTSTR lpszVersion,
  LPCTSTR lpszReferer,
  LPCTSTR* lpszAcceptTypes,
  DWORD dwFlags,
  DWORD_PTR dwContext
);
#ce

  $ai_HttpOpenReq = DllCall($h_Wininet, 'long', 'HttpOpenRequest', 'long', $ai_InternetConn[0], 'str', "GET", 'str', $dl_file, 'str', "HTTP/1.1", 'str', "", 'str', "*/*", 'int', 3, 'int', 0x80000000, 'int', 0)
   If @error Or $ai_HttpOpenReq[0] = 0 Then
        DllCall($h_Wininet, 'int', 'InternetCloseHandle', 'long', $ai_InternetConn[0])
        DllClose($h_Wininet)
        SetError(1)
    ErrorBox("HttpOpenRequest Error.")
        Return 0
    EndIf

#cs
BOOL HttpSendRequest(
  HINTERNET hRequest,
  LPCTSTR lpszHeaders,
  DWORD dwHeadersLength,
  LPVOID lpOptional,
  DWORD dwOptionalLength
);
#ce
  $sentReq = DllCall($h_Wininet, 'short', 'HttpSendRequest', 'long', $ai_HttpOpenReq[0], 'str', '', 'long', '', 'ptr', '', 'int', 0)
   If @error Or $sentReq = 0 Then
        DllCall($h_Wininet, 'int', 'InternetCloseHandle', 'long', $ai_InternetConn[0])
        DllClose($h_Wininet)
        SetError(1)
    ErrorBox("HttpSendRequest Error.")
        Return 0
    EndIf

   Local $prog = 1
   ProgressSet($prog)

    #region InternetReadFile
    Local $v_Struct = DllStructCreate ('udword')
    Local $bytes_read = 0, $buf_writ = 0
    Local $fh = _APIFileOpen($sv_file)
    Local $buflen = 4096
    Local $buf = DLLStructCreate("byte[" & $buflen & "]")
    
    While $av_InternetReadFile[4] <> 0
        
        $av_InternetReadFile = DllCall($h_Wininet, 'int', 'InternetReadFile', 'long', $ai_HttpOpenReq[0], 'ptr', DllStructGetPtr($buf), 'long', $buflen, 'ptr', DllStructGetPtr ($v_Struct))
        
;# of bytes read
    $av_InternetReadFile[4] = DllStructGetData($v_Struct, 1)
    
    If @Error Then
      ProgressOff()
      ErrorBox("Error: " & @Error)
      ExitLoop
    ElseIf $av_InternetReadFile[4] < 1 Then
      ExitLoop
    EndIf

    $bytes_read += $av_InternetReadFile[4]
    
    $buf_writ = _BinaryFileWrite($fh, $buf)

    If $buf_writ < 1 Then
      ProgressOff()
      ErrorBox("Error: " & @Error)
      ExitLoop
    EndIf
    
    If Mod($bytes_read/$buflen, 15) = 0 Then
      If $prog >= 100 Then
        $prog = 0
      EndIf
      $prog += 1
      ProgressSet($prog, "Read " & $bytes_read & " Bytes", "Downloading...")
    EndIf
            
    WEnd
    #endregion
    
    #region Finish
    DllStructDelete ($v_Struct)
    DllCall($h_Wininet, 'int', 'InternetCloseHandle', 'long', $ai_HttpOpenReq[0])
    DllCall($h_Wininet, 'int', 'InternetCloseHandle', 'long', $ai_InternetOpen[0])
    DllClose($h_Wininet)

    _APIFileClose($fh)
    DLLStructDelete($buf)
    #endregion

    ProgressOff()

EndFunc

[jay]

I cant get this to work can you explain it a little better?

Thanks

Jay

[vAnA]

good simple

[drlava]

Could this pointer buffer technique be used to sllow _InetGetSource to work with binary files?

Currently it only returns up to the first null in the file.