Sign in to follow this  
Followers 0
t0nZ

How to download files from temporary links

6 posts in this topic

#1 ·  Posted (edited)

So my problem was to download files from temporary links.

If you consider Combofix, an util to clean Windows from malware, it's a program updated daily and I want always the last version.

But the download link is temporary and always changing, and I don't trust other sites static links.

I developed a system to download a list of links, the main idea is to have the list of links in the page and after I go parsing the list to found the correct temporary links.

It's dirty but it's working.

This is a "lite" version for the forum.:

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=Icone\Full ico\down.ico
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

;NSC 2014
; download temporary links demo


#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <ProgressConstants.au3>
#include <WindowsConstants.au3>
#include <Array.au3>
#include <GuiEdit.au3>
#include <IE.au3>
#include <string.au3>

Global $uProgress1, $UPDATE, $updateArray, $updateDownload, $downlist, $adown, $conta
Global $downpath = @DesktopDir & "\GetDown"

;_________________

Gui()
Prepare()
downlist()
downloop()
end()

Exit


Func Gui()

    #Region ### START Koda GUI section ### Form=C:\Program Files (x86)\AutoIt3\Examples\NSC_test\formS\update.kxf
    $UForm1 = GUICreate("GET DOWN lite V.1.5", 300, 200, 200, 150)
    GUISetBkColor(0xC0C0C0)
    $uProgress1 = GUICtrlCreateProgress(265, 5, 30, 190, BitOR($PBS_SMOOTH, $PBS_VERTICAL, $WS_BORDER))
    GUICtrlSetColor(-1, 0x0033FF)
    GUICtrlSetBkColor(-1, 0x000066)
    $UPDATE = GUICtrlCreateEdit("", 5, 5, 250, 190, BitOR($ES_AUTOVSCROLL, $ES_AUTOHSCROLL, $ES_WANTRETURN, $WS_BORDER))
    GUICtrlSetData(-1, "Gettin'down temporary links" & @CRLF)
    GUICtrlSetFont(-1, 8, 800, 0, "Verdana")
    GUICtrlSetColor(-1, 0x99FFFF)
    GUICtrlSetBkColor(-1, 0x330000)
    GUICtrlSetCursor(-1, 3)
    GUISetState(@SW_SHOW)
    #EndRegion ### END Koda GUI section ###

EndFunc   ;==>Gui

Func Prepare()
    If Not FileExists($downpath) Then DirCreate($downpath)
EndFunc   ;==>Prepare

Func downlist()
    Dim $adown[3] ; in other scripts I download a txt file from internet.
    $adown[0] = "http://www.bleepingcomputer.com/download/adwcleaner/dl/125/|adwcleaner.exe"
    $adown[1] = "http://www.bleepingcomputer.com/download/combofix/dl/12/|combofix.exe"
    $adown[2] = "http://www.bleepingcomputer.com/download/rkill/dl/10/|rkill.exe"
    _GUICtrlEdit_AppendText($UPDATE, "download list assigned. " & @CRLF)
EndFunc   ;==>downlist



Func downloop()

    While 1
        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"

        Local $oIE = _IECreate($arraydown[0], 0, 0, 1, 0) ;hidden IE
        Local $oLinks = _IELinkGetCollection($oIE) ; list of links!
        Local $iNumLinks = @extended
        _GUICtrlEdit_AppendText($UPDATE, "Searching in " & $iNumLinks & " links for " & $arraydown[0] & @CRLF)
        Local $sTxt = $iNumLinks & " links found" & @CRLF & @CRLF ; uncomment for list of links (1/3)
        For $oLink In $oLinks
            $sTxt &= $oLink.href & @CRLF ;uncomment for list of links (2/3)
            If StringInStr($oLink.href, $arraydown[1]) Then
                Inetdown($oLink.href, $arraydown[1])
            EndIf
        Next
        MsgBox($MB_SYSTEMMODAL, "Link Info", $sTxt) ;uncomment for list of links (3/3)
    WEnd

EndFunc   ;==>downloop


Func Inetdown($filedown, $nomefiledown)
    $updateDownload = InetGet($filedown, $downpath & "\" & $nomefiledown, 1, 1)
    _GUICtrlEdit_AppendText($UPDATE, "downloading " & $nomefiledown & @CRLF)

    Do
        Sleep(500)
        _GUICtrlEdit_AppendText($UPDATE, "bytes downloaded :" & InetGetInfo($updateDownload, 0) & @CRLF)
        GUICtrlSetData($uProgress1, (100 * InetGetInfo($updateDownload, 0) / InetGetInfo($updateDownload, 1)))
    Until InetGetInfo($updateDownload, 2) ; Check if the download is complete.

    InetClose($updateDownload) ; Close the handle to release resources.
    _GUICtrlEdit_AppendText($UPDATE, "downloaded in " & $downpath & @CRLF)
EndFunc   ;==>Inetdown

Func end()
    Run("explorer.exe " & $downpath)
    MsgBox(64, "NSC GET DOWN", "Files downloaded!")
EndFunc   ;==>end

the Function downloop() contains the trick.

The complete version downloads from a public server the list of the files to download, and for every file there is a command to choose various download methods and  to exucute commands (run the files etc) after the download, and at the end I have a mail with a report.

Edited by t0nZ

Share this post


Link to post
Share on other sites



Why use IE ?

You have just to extract links from source and find the one you want (temporary or not)

#include <Array.au3>

$sSrc = BinaryToString ( InetRead ( 'http://www.bleepingcomputer.com/download/combofix/' ) )
$aLinks = StringRegExp ( $sSrc, '(?s)(?i)<a href="(.*?)"', 3 )
$iIndex = _ArraySearch ( $aLinks, 'download/combofix', 0, 0, 0, 1 )
ConsoleWrite ( '+ Link Download Page : ' & $aLinks[$iIndex] & @Crlf )

$sSrc = StringReplace ( BinaryToString ( InetRead ( $aLinks[$iIndex] ) ), "'", '"' )
$aLink = StringRegExp ( $sSrc, '(?s)(?i)<a href="(.*?)"', 3 )
$iIndex = _ArraySearch ( $aLink, 'ComboFix.exe', 0, 0, 0, 1 )
ConsoleWrite ( '! Link Download ComboFix.exe : ' & $aLink[$iIndex] & @Crlf )
1 person likes this

AutoIt 3.3.14.2 X86 - SciTE 3.6.0WIN 8.1 X64 - Other Example Scripts

Share this post


Link to post
Share on other sites

Thank you for the snippet !

I Like this method, it's faster and more efficient (like a lot of things without IE....)

So I have modified my example to try three methods, with the evidence of timings.

Third method ? Your snippet + >this.

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=Icone\Full ico\down.ico
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

;NSC 2014
; download temporary links demo


#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <ProgressConstants.au3>
#include <WindowsConstants.au3>
#include <Array.au3>
#include <GuiEdit.au3>
#include <IE.au3>
#include <string.au3>
#include "UrlDownloadEx.au3"

Global $uProgress1, $UPDATE, $updateArray, $updateDownload, $downlist, $adown, $conta
Global $downpath1 = @DesktopDir & "\GetDown1"
Global $downpath2 = @DesktopDir & "\GetDown2"
Global $downpath3 = @DesktopDir & "\GetDown3"

;_________________

Gui()
Prepare()
downlist()
downloop1()
downlist()
downloop2()
downlist()
downloop3()
end()

Exit


Func Gui()

    #Region ### START Koda GUI section ### Form=C:\Program Files (x86)\AutoIt3\Examples\NSC_test\formS\update.kxf
    $UForm1 = GUICreate("GET DOWN lite V.1.7", 300, 200, 200, 150)
    GUISetBkColor(0xC0C0C0)
    $uProgress1 = GUICtrlCreateProgress(265, 5, 30, 190, BitOR($PBS_SMOOTH, $PBS_VERTICAL, $WS_BORDER))
    GUICtrlSetColor(-1, 0x0033FF)
    GUICtrlSetBkColor(-1, 0x000066)
    $UPDATE = GUICtrlCreateEdit("", 5, 5, 250, 190, BitOR($ES_AUTOVSCROLL, $ES_AUTOHSCROLL, $ES_WANTRETURN, $WS_BORDER))
    GUICtrlSetData(-1, "Gettin'down temporary links" & @CRLF)
    GUICtrlSetFont(-1, 8, 800, 0, "Verdana")
    GUICtrlSetColor(-1, 0x99FFFF)
    GUICtrlSetBkColor(-1, 0x330000)
    GUICtrlSetCursor(-1, 3)
    GUISetState(@SW_SHOW)
    #EndRegion ### END Koda GUI section ###

EndFunc   ;==>Gui

Func Prepare()
    If Not FileExists($downpath1) Then DirCreate($downpath1)
    If Not FileExists($downpath2) Then DirCreate($downpath2)
    If Not FileExists($downpath3) Then DirCreate($downpath3)
EndFunc   ;==>Prepare

Func downlist()
    Dim $adown[3] ; in other scripts I download a txt file from internet.
    $adown[0] = "http://www.bleepingcomputer.com/download/adwcleaner/dl/125/|adwcleaner.exe"
    $adown[1] = "http://www.bleepingcomputer.com/download/combofix/dl/12/|combofix.exe"
    $adown[2] = "http://www.bleepingcomputer.com/download/rkill/dl/10/|rkill.exe"
    _GUICtrlEdit_AppendText($UPDATE, "download list assigned. " & @CRLF)
EndFunc   ;==>downlist

Func downloop1()
    MsgBox(64, "getdown1", "start downloading IE mode")
    $iBegin = TimerInit()
    While 1

        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"

        ;;;___________________
        Local $oIE = _IECreate($arraydown[0], 0, 0, 1, 0) ;hidden IE
        Local $oLinks = _IELinkGetCollection($oIE) ; list of links!
        Local $iNumLinks = @extended
        _GUICtrlEdit_AppendText($UPDATE, "Searching in " & $iNumLinks & " links for " & $arraydown[0] & @CRLF)
        ;Local $sTxt = $iNumLinks & " links found" & @CRLF & @CRLF ; uncomment for list of links (1/3)
        For $oLink In $oLinks
            ;   $sTxt &= $oLink.href & @CRLF ;uncomment for list of links (2/3)
            If StringInStr($oLink.href, $arraydown[1]) Then
                Inetdown($oLink.href, $downpath1 & "\" & $arraydown[1])
            EndIf
        Next
        ;MsgBox($MB_SYSTEMMODAL, "Link Info", $sTxt) ;uncomment for list of links (3/3)
    WEnd
    $iDiff = Int(TimerDiff($iBegin) / 1000)
    $iMin = Int($iDiff / 60)
    $iSec = $iDiff - ($iMin * 60)
    MsgBox(64, "Downloaded in ", $iMin & ":" & StringFormat("%02i", $iSec))

EndFunc   ;==>downloop1


Func downloop2()
    MsgBox(64, "getdown2", "start downloading INETGET mode")
    $iBegin = TimerInit()
    While 1
        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"


        $sSrc = StringReplace(BinaryToString(InetRead($arraydown[0])), "'", '"')
        $aLink = StringRegExp($sSrc, '(?s)(?i)<a href="(.*?)"', 3)
        $iIndex = _ArraySearch($aLink, $arraydown[1], 0, 0, 0, 1)
        ConsoleWrite('! Link Download  ' & $aLink[$iIndex] & @CRLF)
        Inetdown($aLink[$iIndex], $downpath2 & "\" & $arraydown[1])
    WEnd
    $iDiff = Int(TimerDiff($iBegin) / 1000)
    $iMin = Int($iDiff / 60)
    $iSec = $iDiff - ($iMin * 60)

    MsgBox(64, "Downloaded in ", $iMin & ":" & StringFormat("%02i", $iSec))
EndFunc   ;==>downloop2


Func downloop3()
    MsgBox(64, "getdown3", "start downloading INETGET mode + UrlDownloadEX")
    $iBegin = TimerInit()
    While 1
        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"


        $sSrc = StringReplace(BinaryToString(InetRead($arraydown[0])), "'", '"')
        $aLink = StringRegExp($sSrc, '(?s)(?i)<a href="(.*?)"', 3)
        $iIndex = _ArraySearch($aLink, $arraydown[1], 0, 0, 0, 1)
        ConsoleWrite('! Link Download ' & $aLink[$iIndex] & @CRLF)
        UrlDownloadEx($aLink[$iIndex], $downpath3 & "\" & $arraydown[1])
    WEnd
    $iDiff = Int(TimerDiff($iBegin) / 1000)
    $iMin = Int($iDiff / 60)
    $iSec = $iDiff - ($iMin * 60)

    MsgBox(64, "Downloaded in ", $iMin & ":" & StringFormat("%02i", $iSec))
EndFunc   ;==>downloop3

Func Inetdown($filedown, $nomefiledown)
    $updateDownload = InetGet($filedown, $nomefiledown, 1, 1)
    _GUICtrlEdit_AppendText($UPDATE, "downloading " & $nomefiledown & @CRLF)

    Do
        Sleep(500)
        _GUICtrlEdit_AppendText($UPDATE, "bytes downloaded :" & InetGetInfo($updateDownload, 0) & @CRLF)
        GUICtrlSetData($uProgress1, (100 * InetGetInfo($updateDownload, 0) / InetGetInfo($updateDownload, 1)))
    Until InetGetInfo($updateDownload, 2) ; Check if the download is complete.

    InetClose($updateDownload) ; Close the handle to release resources.
    _GUICtrlEdit_AppendText($UPDATE, "downloaded " & $nomefiledown & @CRLF)
EndFunc   ;==>Inetdown

Func end()
    Run("explorer.exe " & $downpath1)
    Run("explorer.exe " & $downpath2)
    Run("explorer.exe " & $downpath3)
    MsgBox(64, "NSC GET DOWN", "Files downloaded!")
EndFunc   ;==>end

Share this post


Link to post
Share on other sites

 

Thank you for the snippet !

I Like this method, it's faster and more efficient (like a lot of things without IE....)

So I have modified my example to try three methods, with the evidence of timings.

Third method ? Your snippet + >this.

 

When it's possible to avoid using IE, so do it !

Glad to help you !  :)


AutoIt 3.3.14.2 X86 - SciTE 3.6.0WIN 8.1 X64 - Other Example Scripts

Share this post


Link to post
Share on other sites

Surely, but the snippet for downloading from temporary links it would prove useful... soon or later...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • Tippex
      By Tippex
      I'm having a problem with _IEBodyReadText() which is a function from the IE.au3 UDF include.
      My problem is that randomly it will crash my script with:
      Return SetError($_IESTATUS_Success, 0, $oObject.document.body.innerText) Return SetError($_IESTATUS_Success, 0, $oObject.document.body^ ERROR I don't know why the error isn't caught by __IEErrorHandlerRegister but looking at this function, I can see that there is a check done by IsObj($oObject) to ensure that the object "$oObject" exists but no check that it contains the property "innerText"
      ; #FUNCTION# ==================================================================================================================== ; Author ........: Dale Hohm ; =============================================================================================================================== Func _IEBodyReadText(ByRef $oObject) If Not IsObj($oObject) Then __IEConsoleWriteError("Error", "_IEBodyReadText", "$_IESTATUS_InvalidDataType") Return SetError($_IESTATUS_InvalidDataType, 1, 0) EndIf If Not __IEIsObjType($oObject, "browserdom") Then __IEConsoleWriteError("Error", "_IEBodyReadText", "$_IESTATUS_InvalidObjectType", "Expected document element") Return SetError($_IESTATUS_InvalidObjectType, 1, 0) EndIf ; Return SetError($_IESTATUS_Success, 0, $oObject.document.body.innerText) EndFunc ;==>_IEBodyReadText I'm struggling to find out how to test if "$oObject.document.body.innerText" exists before allowing the following Return statement to reference it with the assumption that it does exist (crashing my script when it doesn't).
      Any ideas please?
    • Nareshm
      By Nareshm
      How can i add sleep command using internet speed.
      If my internet speed is 40-50Kbps then sleep for 5 seconds,
      If my internet speed is 30-40Kbps then sleep for 4 seconds,
      If my internet speed is 20-30Kbps then sleep for 3 seconds.

      *Sleep command used bitween my costom script, Example,
      RunWait("rasdial /disconnect", "", @SW_HIDE) ; Stop connect from start
      Sleep(5000)
      RunWait("rasdial Internet", "", @SW_HIDE) ; Connect again
       
    • Nareshm
      By Nareshm
      Please Somebody Help me i am trying to create new script but, i don't know how to create it.

      I am Creating script like this :
      ; My script here [First Script]
      Check the internet connection, if internet connected then
      ; My Script here [Second Script]
      If internet not connected, wait for connect, after connected run my second script
      If Control id/windows/title matched do nothing and end/exit autoit script
      If Control id/windows/title not matched repeat my first and second script again until control id/windows/tittle not matched

      match with :
      >>>> Window <<<<
      Title:    PS :: Version - 2.8.0.0  - [PS Bill]
      Class:    WindowsForm734a
      Position:    -8, -8
      Size:    1382, 744
      Style:    0x17CD0000
      ExStyle:    0x00050100
      Handle:    0x0000000000130572
      >>>> Control <<<<
      Class:    WindowsForm734a
      Instance:    15
      ClassnameNN:    WindowsForm34a15
      Name:    BillPrint
      Advanced (Class):    [NAME:BillPrint]
      ID:    7875
      Text:    
      Position:    39, 310
      Size:    1221, 306
      ControlClick Coords:    745, 20
      Style:    0x56010000
      ExStyle:    0x00000000
      Handle:    0x00000000000C0468
       
    • Nareshm
      By Nareshm
      How can i stop/pause my internet connection for only 7 seconds, after 7 seconds automaticaly resume my internet.
      Please give me scripte
    • Ubermensch
      By Ubermensch
      Hello guys,
      I am trying to create autoit codes in order to determine if IE button was clicked or pressed.
      I am not sure if it is possible for IE button.
      Here is my initial code.
      #include <IE.au3> #include <MsgBoxConstantsau #include <WinAPI.au3> AutoIt $searchTab = "Untitled 1 - Internet Explorer" WinActivate("Untitled 1 - Internet Explorer") Local $oIE = _IEAttach("", "instance", 1) For $i = 0 To 100 Send("^" & $i) Sleep(250) If Not(StringInStr(WinGetTitle("[ACTIVATE]", $searchTab) = 0) Then While 1 If $oGetItem = 1 Then MsgBox($MB_SYSTEMMODAL, "Attention!", "It's working!") EndIf WEnd ExitLoop EndIf Next