Jump to content
FaridAgl

UrlDownloadEx() - Download with maximum speed

Recommended Posts

Of course I ran it once and was like "Where is the AutoIt exe?" but then I commented out line 8 and voila it worked. I somehow get the feeling I gave you inspiration for the delegate function BytesReceived() concept. -_0

5* from me at least.

Edited by guinness

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Could you explain the usage of $vParam to me please. I get it's used in the callback function, but I can't see why it is needed.


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

I somehow get the feeling I gave you inspiration for the delegate function BytesReceived() concept. -_0

Well, I was going to admit it ;)

 

Could you explain the usage of $vParam to me please. I get it's used in the callback function, but I can't see why it is needed.

I believe this is somehow the usual way of defining a callback (delegate), an additional parameter that can deliver whatever you may need into your callback function.

Share this post


Link to post
Share on other sites

Oh, I forgot to say.

it's not possible to abort (cancel) the download by returning False from the callback function, and retrieve the downloaded data, that makes no sense because the file is getting downloaded as multiple segments, and there isn't any reliable data to retrieve as long as the download isn't completed. This is why 1st example doesn't create any file.

Share this post


Link to post
Share on other sites

Well, I was going to admit it ;)

 

I believe this is somehow the usual way of defining a callback (delegate), an additional parameter that can deliver whatever you may need into your callback function.

Haha. I understand, it's some sort of user parameter.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Haha. I understand, it's some sort of user parameter.

That's so true.

 

Thank you for sharing this! Very nice job!

Thank you for your support.

Also I would like to say thank you to trancexx, because of her great WinHTTP UDF.

Share this post


Link to post
Share on other sites

Hey,

Nice work! The speed difference surely depends on the speed of the server, and how much speed it is able to give per connection. I made a little Example where you can see the speed. The file is hosted on a very fast server, so the difference shouldn't be that huge. But you can try it with other files from slower servers or may servers which are on the other side of the world (from your position ;) )

Thanks for sharing Farid!

#include "UrlDownloadEx.au3"


HotKeySet("{ESC}","_exit")
Global $t1 = TimerInit()
ProgressOn("Exit with {ESC}", "Download 50mb","Start download...")
AdlibRegister("_update",250)



Global $vResult = UrlDownloadEx("http://download.thinkbroadband.com/50MB.zip", @ScriptDir & "\[temp]50MB.zip", 1, BytesReceived);Quite the same speed over the complete download
ConsoleWrite(StringFormat("Result:\t%s\nSize:\t%u\nError:\t%u\nTimer:\t%u\n", $vResult, @extended, @error, TimerDiff($t1)))
AdlibRegister("_update",1000)

If Not BytesReceived(0,0,0,0) Then Exit;if abborted, exit

Global $vResult = UrlDownloadEx("http://download.thinkbroadband.com/50MB.zip", @ScriptDir & "\[temp]50MB2.zip", Default, BytesReceived);speed differ from time to time. you have to but an higher amout of MS in the AdlibRegister (like 1000 instead of 250)
ConsoleWrite(StringFormat("Result:\t%s\nSize:\t%u\nError:\t%u\nTimer:\t%u\n", $vResult, @extended, @error, TimerDiff($t1)))

Func BytesReceived($iReceivedBytes, $iTotalReceivedBytes, $iDownloadSize, $vParam)
    Local Static $hTimer = TimerInit(), $iBytes, $iKbS, $sKbS, $iDownloadSize2, $iTotalReceivedBytes2, $fExit = False
    If $vParam = 1 Then
        ProgressSet($iTotalReceivedBytes2 / $iDownloadSize2 * 100, StringFormat("%.1f%%\r\n%.2f %s",$iTotalReceivedBytes2 / $iDownloadSize2 * 100,$iKbS,$sKbS))
        Local $iTime = 0
        $iKbS = 1000 / TimerDiff($hTimer) * $iBytes
        While $iKbS > 1000
            $iTime += 1
            $iKbS /= 1024
        WEnd
        Switch $iTime
            Case 0
                $sKbS = "Bytes/s"
            Case 1
                $sKbS = "KB/s"
            Case 2
                $sKbS = "MB/s"
            Case 3
                $sKbS = "GB/s"
        EndSwitch

        $iKbS = $iKbS
        $hTimer = TimerInit()
        $iBytes = 0
    ElseIf $vParam = 2 Then
        $fExit = True
    Else
        $iBytes += $iReceivedBytes
        $iDownloadSize2 = $iDownloadSize
        $iTotalReceivedBytes2 = $iTotalReceivedBytes
    EndIf
    Return Not $fExit
EndFunc   ;==>BytesReceived


Func _update()
    BytesReceived(0, 0, 0, 1)
EndFunc   ;==>speed

Func _exit()
    BytesReceived(0, 0, 0, 2)
EndFunc

Greetz,

Spider 

Example 3.au3

Edited by GtaSpider

www.AutoIt.de - Moderator of the German AutoIt Forum

 

Share this post


Link to post
Share on other sites

Thank you, a very nice example.

You are absolutely true, it depends on the servers and how the server is configured. Usually servers have a download speed limit per each connection, so downloading a file with multiple connection can boost the download speed.

However, if your Internet connection's download bandwidth is low, or the server has no limit on the download speed, the difference shouldn't be that much.

Also some servers won't allow multiple connections for downloading a single file, or they don't even support the "Range" http header field.

But, most of the times, you would enjoy using this method of downloading.

Share this post


Link to post
Share on other sites

>"C:\Users\hppc\Desktop\autoit-v3.3.11.6\install\SciTE4AutoIt3(8)\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:\Users\hppc\Desktop\UrlDownloadEx + Examples(1)\Example 1.au3" /UserParams    
+>15:37:05 Starting AutoIt3Wrapper v.2.2.0.3 SciTE v.3.4.1.0   Keyboard:0001041F  OS:WIN_8/  CPU:X64 OS:X64    Environment(Language:041F)
+>         SciTEDir => C:\Users\hppc\Desktop\autoit-v3.3.11.6\install\SciTE4AutoIt3(8)   UserDir => C:\Users\hppc\Desktop\autoit-v3.3.11.6\install\SciTE4AutoIt3(8)\AutoIt3Wrapper
>Running AU3Check (3.3.11.6)  from:C:\Users\hppc\Desktop\autoit-v3.3.11.6\install  input:C:\Users\hppc\Desktop\UrlDownloadEx + Examples(1)\Example 1.au3
+>15:37:05 AU3Check ended.rc:0
>Running:(3.3.11.6):C:\Users\hppc\Desktop\autoit-v3.3.11.6\install\autoit3.exe "C:\Users\hppc\Desktop\UrlDownloadEx + Examples(1)\Example 1.au3"
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

this code worked

but not 50mb

try it

#include "UrlDownloadEx.au3"


HotKeySet("{ESC}","_exit")
Global $t1 = TimerInit()
ProgressOn("Exit with {ESC}", "Download 50mb","Start download...")
AdlibRegister("_update",250)



Global $vResult = UrlDownloadEx("http://www.autoitscript.com/files/autoit3/autoit-v3-setup.exe", @ScriptDir & "\AutoIt 3.3.12.0.exe", 1, BytesReceived);Quite the same speed over the complete download
ConsoleWrite(StringFormat("Result:\t%s\nSize:\t%u\nError:\t%u\nTimer:\t%u\n", $vResult, @extended, @error, TimerDiff($t1)))
AdlibRegister("_update",1000)

If Not BytesReceived(0,0,0,0) Then Exit;if abborted, exit

;Global $vResult = UrlDownloadEx("http://download.thinkbroadband.com/50MB.zip", @ScriptDir & "\[temp]50MB2.zip", Default, BytesReceived);speed differ from time to time. you have to but an higher amout of MS in the AdlibRegister (like 1000 instead of 250)
;ConsoleWrite(StringFormat("Result:\t%s\nSize:\t%u\nError:\t%u\nTimer:\t%u\n", $vResult, @extended, @error, TimerDiff($t1)))

Func BytesReceived($iReceivedBytes, $iTotalReceivedBytes, $iDownloadSize, $vParam)
    Local Static $hTimer = TimerInit(), $iBytes, $iKbS, $sKbS, $iDownloadSize2, $iTotalReceivedBytes2, $fExit = False
    If $vParam = 1 Then
        ProgressSet($iTotalReceivedBytes2 / $iDownloadSize2 * 100, StringFormat("%.1f%%\r\n%.2f %s",$iTotalReceivedBytes2 / $iDownloadSize2 * 100,$iKbS,$sKbS))
        Local $iTime = 0
        $iKbS = 1000 / TimerDiff($hTimer) * $iBytes
        While $iKbS > 1000
            $iTime += 1
            $iKbS /= 1024
        WEnd
        Switch $iTime
            Case 0
                $sKbS = "Bytes/s"
            Case 1
                $sKbS = "KB/s"
            Case 2
                $sKbS = "MB/s"
            Case 3
                $sKbS = "GB/s"
        EndSwitch

        $iKbS = $iKbS
        $hTimer = TimerInit()
        $iBytes = 0
    ElseIf $vParam = 2 Then
        $fExit = True
    Else
        $iBytes += $iReceivedBytes
        $iDownloadSize2 = $iDownloadSize
        $iTotalReceivedBytes2 = $iTotalReceivedBytes
    EndIf
    Return Not $fExit
EndFunc   ;==>BytesReceived


Func _update()
    BytesReceived(0, 0, 0, 1)
EndFunc   ;==>speed

Func _exit()
    BytesReceived(0, 0, 0, 2)
EndFunc
 added udf but dont work ?

but example 3 worked

dowload

Edited by mesale0077

Share this post


Link to post
Share on other sites

Error:  14

It means the callback function returned False, so the download got aborted.

Check your code and make sure you are returning True from the callback function all the times.

Remember, it's necessary for the callback function to return True, if you don't return any value from the callback function, it will be considered as False and the download progress will be aborted.

Share this post


Link to post
Share on other sites

Fantastic work!

I tested It in a dirty download benchmark with temporary links-> .

But Combofix.exe don't get downloaded...

Share this post


Link to post
Share on other sites

The static link is only 

http://www.bleepingcomputer.com/download/combofix/dl/12/

my example search the temporary link (using the static link and the filename.exe as parameters) and it tries to download.

the temporary link is like

http://download.bleepingcomputer.com/dl/3a314486dc4cdc746d35b26ea1bb0837/53b10c72/windows/security/anti-virus/c/combofix/ComboFix.exe

Your UrlDownloadEx() is OK with the other two downloads, I think it's a property of the combofix hosting server, maybe it doesn't permit downloading in pieces ?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...