Jump to content
Sign in to follow this  

Multi segment file downloading

Recommended Posts


The reason i wrote this was because i didnt found any download manager in the whole web! where you can customize the starting position of the segments.

Most programs will just start one segment at 0%, the second at 50%, third at %25 ,75%, 12,5% and so on.

There is very little information about multi part/segement downloading in the forums so i gathered some informations and knocked this script together.

No messing with dlls or IE functions, just plain TCPSend and TCPRecv (i like it pure :). You can adjust the header to your needs.

The initial segment size is 2 MB and the segments will start downloading from the beginning of the file.

This is ideal for viewing movies or avi files while downloading.

You can adjust the segment size while downloading.

If you want more then 4 segements you'll have to figure out yourself how to do it ;)

The GUI is still basic. I was too lazy to implement the Progress bar.

You can see the completion percent in the consolewrites as soon as a segment finishes.

#include <Array.au3>
#include <GUIConstantsEx.au3>
#include <EditConstants.au3>
#include <ProgressConstants.au3>
#include <WindowsConstants.au3>
#include <WinAPI.au3>

;Create a GUI with a progress indicator for download status
; #### Unfinished ####
Opt("GUIOnEventMode", 1)

$Form1 = GUICreate("Multi Segment downloader", 294, 56, 192, 124)
$Progress1 = GUICtrlCreateProgress(8, 8, 275, 20)
$Label1 = GUICtrlCreateLabel("Chunksize (MB):", 8, 32)
$edit = GUICtrlCreateInput("2",90,32,50,20)
$Updown1 = GUICtrlCreateUpdown($edit)


$sFile = @ScriptDir & '\file.avi' ; File to wrtie the download data to

$download= "http://speedtest.qsc.de/500MB.qsc" ; Internetfile to download

$refer = $download ; $refer = complete http path

$download = StringReplace($download,"http://","",1)

$string = StringSplit($download, "/",2)

;$string[0] = speedtest.qsc.de ; needed for 'Host:' in http referr

$PFileURL = ""
For $i = 1 To UBound($string)-1
    $PFileURL &= "/"&$string[$i]

;$PFileURL = /folder/misc/500MB.qsc ; needed for file path in http referr


$ip = TCPNameToIP($string[0])

ConsoleWrite($string[0]&" -> IP: "&$ip & @CRLF)

$size = 10000000 ; preset the size to 10MB, just in case we have an problem getting it from http responses

;~ $size = InetGetSize("http://"&$download) ; <-- some servers dont allow this

ConsoleWrite("Size of file: "&$size&" bytes"& @CRLF)

dim $chunkarr[4][7] ; [4] = maximum nr of simultanious chunks
;[0] = socket
;[1] = chunkstart
;[2] = chunkend
;[3] = chunksize
;[4] = data
;[5] = downloding
;[6] found the 0d0a0d0a after header?
ConsoleWrite(@CRLF& @CRLF & @CRLF)

$chunksize = GUICtrlRead($edit)*1024*1024 ; initial size of each segment

ConsoleWrite("Chunksize: "&$chunksize&" bytes"& @CRLF)

$nextchunkPos = 0 ; Starting at Byte

FileDelete ( $sFile)

$hFile = _WinAPI_CreateFile($sFile, 3,4,2)

While 1
    $chunksize = GUICtrlRead($edit)*1024*1024
    $chunksactive = 0
    For $i = 0 To UBound($chunkarr)-1 ; go through each chunk
        if $chunkarr[$i][5] = 0 and $nextchunkPos < $size then ; Unused ChunkSlot --> Request new chunk
            $chunkarr[$i][1] = $nextchunkPos    ; start
            $chunkarr[$i][2] = $nextchunkPos+$chunksize-1   ; end
            $chunkarr[$i][4] = Binary("")       ; data
            $chunkarr[$i][5] = 1        ; downloading
            $chunkarr[$i][6] = 0        ; found the 0d0a0d0a after header
            if $chunkarr[$i][2]+$chunksize >= $size then $chunkarr[$i][2] = $size-1; if last chunk set end to last byte

            $chunkarr[$i][3] = $chunkarr[$i][2]-$chunkarr[$i][1]+1  ; size

            $nextchunkPos = $chunkarr[$i][2]+1

        if $chunkarr[$i][5] = 1 then ; current chunk is marked downloading
            $chunksactive += 1
    if $chunksactive = 0 then ExitLoop

ConsoleWrite( @CRLF&" #####    DOWNLOAD FINISHED    #####" & @CRLF)


func RequestChunk($chunknr)
    ; Prepare Header
    $h  = "GET "&$PFileURL&" HTTP/1.1" & @CRLF
    $h &= "Referer: "&$refer & @CRLF
    $h &= "Range: bytes=" & $chunkarr[$chunknr][1] & "-" & $chunkarr[$chunknr][2] & @CRLF ; <----  ### important part for multi segment downloading ###
    $h &= "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv: Gecko/20100722 Ant.com Toolbar 2.0.1 Firefox/3.6.8" & @CRLF
    $h &= "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" & @CRLF
    $h &= "Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3" & @CRLF
    $h &= "Accept-Encoding: gzip,deflate" & @CRLF
    $h &= "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7" & @CRLF
    $h &= "Keep-Alive: 115" & @CRLF
    $h &= "Connection: keep-alive" & @CRLF
    $h &= "Host: " & $string[0] & @CRLF
    $h &= "Pragma: no-cache" & @CRLF
    $h &= @CRLF
;~  ConsoleWrite("+############### HEADER ################"& @CRLF)
;~  ConsoleWrite($h & @CRLF)

;~  ConsoleWrite("+-------------------- HEADER END ------------------"& @CRLF)
    $chunkarr[$chunknr][0] = TCPConnect($ip, 80)
    $bs = TCPSend( $chunkarr[$chunknr][0], $h)
    ConsoleWrite( "-Chunk "&$chunknr&" request: Range: bytes= " & $chunkarr[$chunknr][1] & "-" & $chunkarr[$chunknr][2] &" / "&$chunkarr[$i][3]&" ("&$bs&" bytes sent)"&@CRLF)
    If  $chunkarr[$chunknr][0] = -1 Or $err Then
        ConsoleWrite("  TCPConnect error" & @CRLF)


func RecvChunk($chunknr)

    $data = TCPRecv( $chunkarr[$chunknr][0], 800,1)
    If @error <> 0 Then ; this chunk closed its connection
        ConsoleWrite(@CRLF&"Closed Connection (Chunk "&$chunknr&" finished)" & @CRLF)

        if BinaryLen($chunkarr[$chunknr][4]) < $chunkarr[$chunknr][3] then ; downloaded bytes are not equal requested size
            $cs = TCPCloseSocket ( $chunkarr[$chunknr][0] )
            ConsoleWrite("!Chunk mismatch: "&$chunknr&" finished ("&$cs&") with "&BinaryLen($chunkarr[$chunknr][4])&" bytes ("&floor(($nextchunkPos/$size)*100)&"%)"&@CRLF)
            ConsoleWrite(BinaryToString($chunkarr[$chunknr][4]) & @CRLF)
            ConsoleWrite("!resetting chunk("&$chunknr&") data and request again... " & @CRLF)
            $chunkarr[$chunknr][4] = Binary("")     ; reset data

        else; Chunk finished
            $chunkarr[$chunknr][5] = 0 
            $tBuffer = DLLStructCreate("byte[" & BinaryLen ($chunkarr[$chunknr][4]) & "]")

            DLLStructSetData($tBuffer, 1, $chunkarr[$chunknr][4])
            local $nBytes

            $fp = _WinAPI_SetFilePointer($hFile, $chunkarr[$chunknr][1])
    ;~      ConsoleWrite("File Pointer to "&$chunkarr[$chunknr][1]&" --> Result: "&$fp & @CRLF)

            $wf = _WinAPI_WriteFile($hFile,DLLStructGetPtr($tBuffer), $chunkarr[$i][3], $nBytes)
    ;~      ConsoleWrite("Write File "&$chunkarr[$i][3]&" bytes: "&$wf & @CRLF)

            $tBuffer = 0
            ;~ ConsoleWrite('3) ' & FileRead($sFile) & @LF)
            $cs = TCPCloseSocket ( $chunkarr[$chunknr][0] )
            ConsoleWrite("+Chunk "&$chunknr&" finished ("&$cs&") with "&BinaryLen($chunkarr[$chunknr][4])&" bytes ("&floor(($nextchunkPos/$size)*100)&"%)"&@CRLF)
        return 0

    ; This chunk is still receiving data..
    if not ($data = "") then
        $timer = TimerInit()
            if $chunkarr[$chunknr][6] = 0 then ; if we didnt found the 0d0a0d0a position yet
                $data2 = binary("")
                For $i = 0 To BinaryLen($data)
                    ; search the first bytes of the incoming packet for the 0d0a0d0a string
                    ; $data2 will be used as fill up variable and usually contains the whole header
                    ; cant use stringinstr or Regex functions since any 0x00 character in the file will mess both up seriously!
                    $bit = BinaryMid($data,$i,1)
                    $data2 &= $bit

                    if StringRight($data2,8) = binary("0D0A0D0A") then ; found header, stored in $data2
                        $marr = StringRegExp(BinaryToString($data2),"Content-Range: bytes \d+-\d+/(\d+)",3) ; get exact size of file from header
                        $size = $marr[0]

                        $data2 = binary("")
                        $chunkarr[$chunknr][6] = 1 ; found the 0d0a0d0a position
                        ConsoleWrite("Content Range Size: [" &$marr[0]&"], 0D0A0D0A pos: ["&$i&"]"&@CRLF)
                $data = $data2; reset $data, from now on the file starts

            $chunkarr[$chunknr][4] &= $data ; save the data of this chunk to global array
            $data = TCPRecv( $chunkarr[$chunknr][0], 4048,1)
        until $data == "" or TimerDiff($timer)  > 10000 ; 10 second timeout
    return 1

Teamspeak 3 User Viewer - Quick and functional TS3 Query script, which shows online users.Cached Screenshot Deleter - Deletes older Fraps Screenshots if they exceed a specified limit.Unresolved Topics:Intercept and modify dragdrop text behaviour in scite

Share this post

Link to post
Share on other sites

qsek, nice script

I've been tweaking on it a little - hope you don't mind.

Would like to see it come to full fruition - hope others will join in.

Among other things - I've added 1 progress bar.

I tried to add 4 of them, but it was too cpu intensive.

So, the one progress bar only samples the 1st segment in real time.

(it is not an exact percentage - but a representation of overall progress of all 4 segments combined)

- Edit: ( old script deleted - See new post below )

Edited by ripdad

"The mediocre teacher tells. The Good teacher explains. The superior teacher demonstrates. The great teacher inspires." -William Arthur Ward

Share this post

Link to post
Share on other sites

Good work both ! ;)

This look promising !

AutoIt X86 - SciTE 3.6.0WIN 8.1 X64 - Other Example Scripts

Share this post

Link to post
Share on other sites


- Rearranged and cleaned code

- Changed algo

- Added InputBox

- Added Time

Edit - ( old script deleted )

- Some improvements & added a few goodies

Off to a new project .. hopefully someone else will pick up!


Edited by ripdad

"The mediocre teacher tells. The Good teacher explains. The superior teacher demonstrates. The great teacher inspires." -William Arthur Ward

Share this post

Link to post
Share on other sites

Last Update

"The mediocre teacher tells. The Good teacher explains. The superior teacher demonstrates. The great teacher inspires." -William Arthur Ward

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this