Jump to content
Sign in to follow this  
t0nZ

How to download files from temporary links

Recommended Posts

t0nZ

So my problem was to download files from temporary links.

If you consider Combofix, an util to clean Windows from malware, it's a program updated daily and I want always the last version.

But the download link is temporary and always changing, and I don't trust other sites static links.

I developed a system to download a list of links, the main idea is to have the list of links in the page and after I go parsing the list to found the correct temporary links.

It's dirty but it's working.

This is a "lite" version for the forum.:

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=Icone\Full ico\down.ico
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

;NSC 2014
; download temporary links demo


#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <ProgressConstants.au3>
#include <WindowsConstants.au3>
#include <Array.au3>
#include <GuiEdit.au3>
#include <IE.au3>
#include <string.au3>

Global $uProgress1, $UPDATE, $updateArray, $updateDownload, $downlist, $adown, $conta
Global $downpath = @DesktopDir & "\GetDown"

;_________________

Gui()
Prepare()
downlist()
downloop()
end()

Exit


Func Gui()

    #Region ### START Koda GUI section ### Form=C:\Program Files (x86)\AutoIt3\Examples\NSC_test\formS\update.kxf
    $UForm1 = GUICreate("GET DOWN lite V.1.5", 300, 200, 200, 150)
    GUISetBkColor(0xC0C0C0)
    $uProgress1 = GUICtrlCreateProgress(265, 5, 30, 190, BitOR($PBS_SMOOTH, $PBS_VERTICAL, $WS_BORDER))
    GUICtrlSetColor(-1, 0x0033FF)
    GUICtrlSetBkColor(-1, 0x000066)
    $UPDATE = GUICtrlCreateEdit("", 5, 5, 250, 190, BitOR($ES_AUTOVSCROLL, $ES_AUTOHSCROLL, $ES_WANTRETURN, $WS_BORDER))
    GUICtrlSetData(-1, "Gettin'down temporary links" & @CRLF)
    GUICtrlSetFont(-1, 8, 800, 0, "Verdana")
    GUICtrlSetColor(-1, 0x99FFFF)
    GUICtrlSetBkColor(-1, 0x330000)
    GUICtrlSetCursor(-1, 3)
    GUISetState(@SW_SHOW)
    #EndRegion ### END Koda GUI section ###

EndFunc   ;==>Gui

Func Prepare()
    If Not FileExists($downpath) Then DirCreate($downpath)
EndFunc   ;==>Prepare

Func downlist()
    Dim $adown[3] ; in other scripts I download a txt file from internet.
    $adown[0] = "http://www.bleepingcomputer.com/download/adwcleaner/dl/125/|adwcleaner.exe"
    $adown[1] = "http://www.bleepingcomputer.com/download/combofix/dl/12/|combofix.exe"
    $adown[2] = "http://www.bleepingcomputer.com/download/rkill/dl/10/|rkill.exe"
    _GUICtrlEdit_AppendText($UPDATE, "download list assigned. " & @CRLF)
EndFunc   ;==>downlist



Func downloop()

    While 1
        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"

        Local $oIE = _IECreate($arraydown[0], 0, 0, 1, 0) ;hidden IE
        Local $oLinks = _IELinkGetCollection($oIE) ; list of links!
        Local $iNumLinks = @extended
        _GUICtrlEdit_AppendText($UPDATE, "Searching in " & $iNumLinks & " links for " & $arraydown[0] & @CRLF)
        Local $sTxt = $iNumLinks & " links found" & @CRLF & @CRLF ; uncomment for list of links (1/3)
        For $oLink In $oLinks
            $sTxt &= $oLink.href & @CRLF ;uncomment for list of links (2/3)
            If StringInStr($oLink.href, $arraydown[1]) Then
                Inetdown($oLink.href, $arraydown[1])
            EndIf
        Next
        MsgBox($MB_SYSTEMMODAL, "Link Info", $sTxt) ;uncomment for list of links (3/3)
    WEnd

EndFunc   ;==>downloop


Func Inetdown($filedown, $nomefiledown)
    $updateDownload = InetGet($filedown, $downpath & "\" & $nomefiledown, 1, 1)
    _GUICtrlEdit_AppendText($UPDATE, "downloading " & $nomefiledown & @CRLF)

    Do
        Sleep(500)
        _GUICtrlEdit_AppendText($UPDATE, "bytes downloaded :" & InetGetInfo($updateDownload, 0) & @CRLF)
        GUICtrlSetData($uProgress1, (100 * InetGetInfo($updateDownload, 0) / InetGetInfo($updateDownload, 1)))
    Until InetGetInfo($updateDownload, 2) ; Check if the download is complete.

    InetClose($updateDownload) ; Close the handle to release resources.
    _GUICtrlEdit_AppendText($UPDATE, "downloaded in " & $downpath & @CRLF)
EndFunc   ;==>Inetdown

Func end()
    Run("explorer.exe " & $downpath)
    MsgBox(64, "NSC GET DOWN", "Files downloaded!")
EndFunc   ;==>end

the Function downloop() contains the trick.

The complete version downloads from a public server the list of the files to download, and for every file there is a command to choose various download methods and  to exucute commands (run the files etc) after the download, and at the end I have a mail with a report.

Edited by t0nZ

Share this post


Link to post
Share on other sites
wakillon

Why use IE ?

You have just to extract links from source and find the one you want (temporary or not)

#include <Array.au3>

$sSrc = BinaryToString ( InetRead ( 'http://www.bleepingcomputer.com/download/combofix/' ) )
$aLinks = StringRegExp ( $sSrc, '(?s)(?i)<a href="(.*?)"', 3 )
$iIndex = _ArraySearch ( $aLinks, 'download/combofix', 0, 0, 0, 1 )
ConsoleWrite ( '+ Link Download Page : ' & $aLinks[$iIndex] & @Crlf )

$sSrc = StringReplace ( BinaryToString ( InetRead ( $aLinks[$iIndex] ) ), "'", '"' )
$aLink = StringRegExp ( $sSrc, '(?s)(?i)<a href="(.*?)"', 3 )
$iIndex = _ArraySearch ( $aLink, 'ComboFix.exe', 0, 0, 0, 1 )
ConsoleWrite ( '! Link Download ComboFix.exe : ' & $aLink[$iIndex] & @Crlf )
  • Like 1

AutoIt 3.3.14.2 X86 - SciTE 3.6.0WIN 8.1 X64 - Other Example Scripts

Share this post


Link to post
Share on other sites
t0nZ

Thank you for the snippet !

I Like this method, it's faster and more efficient (like a lot of things without IE....)

So I have modified my example to try three methods, with the evidence of timings.

Third method ? Your snippet + >this.

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=Icone\Full ico\down.ico
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

;NSC 2014
; download temporary links demo


#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <ProgressConstants.au3>
#include <WindowsConstants.au3>
#include <Array.au3>
#include <GuiEdit.au3>
#include <IE.au3>
#include <string.au3>
#include "UrlDownloadEx.au3"

Global $uProgress1, $UPDATE, $updateArray, $updateDownload, $downlist, $adown, $conta
Global $downpath1 = @DesktopDir & "\GetDown1"
Global $downpath2 = @DesktopDir & "\GetDown2"
Global $downpath3 = @DesktopDir & "\GetDown3"

;_________________

Gui()
Prepare()
downlist()
downloop1()
downlist()
downloop2()
downlist()
downloop3()
end()

Exit


Func Gui()

    #Region ### START Koda GUI section ### Form=C:\Program Files (x86)\AutoIt3\Examples\NSC_test\formS\update.kxf
    $UForm1 = GUICreate("GET DOWN lite V.1.7", 300, 200, 200, 150)
    GUISetBkColor(0xC0C0C0)
    $uProgress1 = GUICtrlCreateProgress(265, 5, 30, 190, BitOR($PBS_SMOOTH, $PBS_VERTICAL, $WS_BORDER))
    GUICtrlSetColor(-1, 0x0033FF)
    GUICtrlSetBkColor(-1, 0x000066)
    $UPDATE = GUICtrlCreateEdit("", 5, 5, 250, 190, BitOR($ES_AUTOVSCROLL, $ES_AUTOHSCROLL, $ES_WANTRETURN, $WS_BORDER))
    GUICtrlSetData(-1, "Gettin'down temporary links" & @CRLF)
    GUICtrlSetFont(-1, 8, 800, 0, "Verdana")
    GUICtrlSetColor(-1, 0x99FFFF)
    GUICtrlSetBkColor(-1, 0x330000)
    GUICtrlSetCursor(-1, 3)
    GUISetState(@SW_SHOW)
    #EndRegion ### END Koda GUI section ###

EndFunc   ;==>Gui

Func Prepare()
    If Not FileExists($downpath1) Then DirCreate($downpath1)
    If Not FileExists($downpath2) Then DirCreate($downpath2)
    If Not FileExists($downpath3) Then DirCreate($downpath3)
EndFunc   ;==>Prepare

Func downlist()
    Dim $adown[3] ; in other scripts I download a txt file from internet.
    $adown[0] = "http://www.bleepingcomputer.com/download/adwcleaner/dl/125/|adwcleaner.exe"
    $adown[1] = "http://www.bleepingcomputer.com/download/combofix/dl/12/|combofix.exe"
    $adown[2] = "http://www.bleepingcomputer.com/download/rkill/dl/10/|rkill.exe"
    _GUICtrlEdit_AppendText($UPDATE, "download list assigned. " & @CRLF)
EndFunc   ;==>downlist

Func downloop1()
    MsgBox(64, "getdown1", "start downloading IE mode")
    $iBegin = TimerInit()
    While 1

        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"

        ;;;___________________
        Local $oIE = _IECreate($arraydown[0], 0, 0, 1, 0) ;hidden IE
        Local $oLinks = _IELinkGetCollection($oIE) ; list of links!
        Local $iNumLinks = @extended
        _GUICtrlEdit_AppendText($UPDATE, "Searching in " & $iNumLinks & " links for " & $arraydown[0] & @CRLF)
        ;Local $sTxt = $iNumLinks & " links found" & @CRLF & @CRLF ; uncomment for list of links (1/3)
        For $oLink In $oLinks
            ;   $sTxt &= $oLink.href & @CRLF ;uncomment for list of links (2/3)
            If StringInStr($oLink.href, $arraydown[1]) Then
                Inetdown($oLink.href, $downpath1 & "\" & $arraydown[1])
            EndIf
        Next
        ;MsgBox($MB_SYSTEMMODAL, "Link Info", $sTxt) ;uncomment for list of links (3/3)
    WEnd
    $iDiff = Int(TimerDiff($iBegin) / 1000)
    $iMin = Int($iDiff / 60)
    $iSec = $iDiff - ($iMin * 60)
    MsgBox(64, "Downloaded in ", $iMin & ":" & StringFormat("%02i", $iSec))

EndFunc   ;==>downloop1


Func downloop2()
    MsgBox(64, "getdown2", "start downloading INETGET mode")
    $iBegin = TimerInit()
    While 1
        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"


        $sSrc = StringReplace(BinaryToString(InetRead($arraydown[0])), "'", '"')
        $aLink = StringRegExp($sSrc, '(?s)(?i)<a href="(.*?)"', 3)
        $iIndex = _ArraySearch($aLink, $arraydown[1], 0, 0, 0, 1)
        ConsoleWrite('! Link Download  ' & $aLink[$iIndex] & @CRLF)
        Inetdown($aLink[$iIndex], $downpath2 & "\" & $arraydown[1])
    WEnd
    $iDiff = Int(TimerDiff($iBegin) / 1000)
    $iMin = Int($iDiff / 60)
    $iSec = $iDiff - ($iMin * 60)

    MsgBox(64, "Downloaded in ", $iMin & ":" & StringFormat("%02i", $iSec))
EndFunc   ;==>downloop2


Func downloop3()
    MsgBox(64, "getdown3", "start downloading INETGET mode + UrlDownloadEX")
    $iBegin = TimerInit()
    While 1
        $filedown = _ArrayPop($adown)
        If @error <> 0 Then ExitLoop

        $arraydown = _StringExplode($filedown, "|") ; example of line processed "www.yourlinkhere.com|yourfile.exe"


        $sSrc = StringReplace(BinaryToString(InetRead($arraydown[0])), "'", '"')
        $aLink = StringRegExp($sSrc, '(?s)(?i)<a href="(.*?)"', 3)
        $iIndex = _ArraySearch($aLink, $arraydown[1], 0, 0, 0, 1)
        ConsoleWrite('! Link Download ' & $aLink[$iIndex] & @CRLF)
        UrlDownloadEx($aLink[$iIndex], $downpath3 & "\" & $arraydown[1])
    WEnd
    $iDiff = Int(TimerDiff($iBegin) / 1000)
    $iMin = Int($iDiff / 60)
    $iSec = $iDiff - ($iMin * 60)

    MsgBox(64, "Downloaded in ", $iMin & ":" & StringFormat("%02i", $iSec))
EndFunc   ;==>downloop3

Func Inetdown($filedown, $nomefiledown)
    $updateDownload = InetGet($filedown, $nomefiledown, 1, 1)
    _GUICtrlEdit_AppendText($UPDATE, "downloading " & $nomefiledown & @CRLF)

    Do
        Sleep(500)
        _GUICtrlEdit_AppendText($UPDATE, "bytes downloaded :" & InetGetInfo($updateDownload, 0) & @CRLF)
        GUICtrlSetData($uProgress1, (100 * InetGetInfo($updateDownload, 0) / InetGetInfo($updateDownload, 1)))
    Until InetGetInfo($updateDownload, 2) ; Check if the download is complete.

    InetClose($updateDownload) ; Close the handle to release resources.
    _GUICtrlEdit_AppendText($UPDATE, "downloaded " & $nomefiledown & @CRLF)
EndFunc   ;==>Inetdown

Func end()
    Run("explorer.exe " & $downpath1)
    Run("explorer.exe " & $downpath2)
    Run("explorer.exe " & $downpath3)
    MsgBox(64, "NSC GET DOWN", "Files downloaded!")
EndFunc   ;==>end

Share this post


Link to post
Share on other sites
wakillon

 

Thank you for the snippet !

I Like this method, it's faster and more efficient (like a lot of things without IE....)

So I have modified my example to try three methods, with the evidence of timings.

Third method ? Your snippet + >this.

 

When it's possible to avoid using IE, so do it !

Glad to help you !  :)


AutoIt 3.3.14.2 X86 - SciTE 3.6.0WIN 8.1 X64 - Other Example Scripts

Share this post


Link to post
Share on other sites
t0nZ

Surely, but the snippet for downloading from temporary links it would prove useful... soon or later...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • MakzNovice
      By MakzNovice
      Hello Experts,
      I am working on custom application details of which I am unable to share.
      But here is statement of my problem, I have two IE windows one window with Title "Parent " and other with "TableEdit", but for some reason when I get WinList they are shown as 2 processes each. ( see attached image)
      Since my most important motive is to modify table data in 'TableEdit' I try to access the table in it
      My code snippet is 
      ***********************************************************************************************
      $searchString = "TableEdit"
      Local $aList = WinList("[REGEXPTITLE:(?i)(.*Internet Explorer.*)]")
      _ArrayDisplay($aList)
      For $i = 1 To $aList[0][0]
          If $aList[$i][0] <> "" And BitAND(WinGetState($aList[$i][1]), 2) Then
              ;MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1])
              If 0 <> StringInStr($aList[$i][0],"TableEdit") Then
                  $objEditIE = _IEAttach ($aList[$i][1])
                  ExitLoop
              EndIf
          EndIf
      Next
      WinActivate($objEditIE)
      $oPageContentListFrame = _IEFrameGetObjByName($objEditIE, "iframe_1")
      $iNumFrames = @extended
      MsgBox(0,"Got pagecontent",@error) ; returns 3, indicating invalid data type
      MsgBox(0,"Number of Frame",$iNumFrames) ; returns 1, expected value is 1 
      ***********************************************************************************************
      and If I do code as below the return value is different
      $searchString = "TableEdit"
       $objEditIE = _IEAttach ($searchString, "title")
      WinActivate($objEditIE)
      $oPageContentListFrame = _IEFrameGetObjByName($objEditIE, "iframe_1")
      $iNumFrames = @extended
      MsgBox(0,"Got pagecontent",@error) ; returns 0
      MsgBox(0,"Number of Frame",$iNumFrames) ; returns 0. expected value is 1
      ***********************************************************************************************
      What is wrong with my code? 
      Why do I see two process each for IE for each window?
       
      WinListImage.bmp
    • tintin92
      By tintin92
      Hello,

      How can I automatically download a list of zip files, like the ones in the quote, using a AutoIt script?
      Is it possible with AutoIt script?
       
    • ur
      By ur
      I have an aws cloud machine where I kept all the build process to create some set of zip files and uploading them to OneDrive.
      Once it is completed, I have another local machine in our environment which is a acting as a local share server also.
      Is there anyway to notify this machine(without public IP) to start downloading these files.
      I have written an AutoIT exe to download these files and copy them to respective locations.But don't know how to notify this from the remote machine.
       
      Any suggestion??
       
    • Nareshm
      By Nareshm
      Its Possible To auto Login with autoit script ?
    • Tippex
      By Tippex
      I have a problem trying to use _IEAttach to recover control after a Windows Explorer reset (e.g. on Windows 7), since all the 
      InternetExplorer.Application objects appear to be destroyed by this event. An example of this is when a PC has some kind of fault causing a pop-up error message "Windows Explorer has stopped working". I can simulate this event by using Windows Task Manager to end the Explorer.exe process then to file, new task (run...) Explorer.exe
      After this event, _IEAttach gives an @error of: 
      7 ($_IEStatus_NoMatch) - No Match
      However, using WinList() followed by WinGetText() I can recover the URL for an IE session that has lost its 
      InternetExplorer.Application object which could be used to _IECreate a new session (the $iTryAttach fails) and WinKill() the original session but the result is a if a refresh had been done (any form entries are lost) . Unfortunately form entries do not appear as text or hidden text to Windows so I'm looking for the best way to read the contents of an Internet Explorer screen (even if just text without structure). 
      When there is no InternetExplorer.Application object (because of some Windows fault) but still with an Internet Explorer session running, I tried to read its contents by a "Select All" & "Copy to Clipboard" but its form entries are blanked and combo boxes choices are all displayed with no way of finding out which ones were selected.
      Local $ClipBoardSave = _ClipBoard_GetData() ;not ClipGet() which is text only ClipPut("") ;Clear User's Clipboard Local $sText = "" Local $hWnd = WinActivate("Window Title", "") If IsHWnd($hWnd) Then Local $bStatusCtrlSend = ControlSend("Window Title", "", "", "^a^c") If $bStatusCtrlSend Then Local $bStatusCtrlClick = ControlClick("Window Title", "", "") If $bStatusCtrlClick Then $sText = ClipGet() _ClipBoard_SetData($ClipBoardSave) ;not ClipPut($ClipBoardSave) which is text only  
      Is  there a solution to this please?
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.