Finding Download Links and Using Them

[Chimaera]

Ok im adding a little updater to give me the ability to update the 5 tools in the program.

There is no switches etc i just need it to check a webpage probably Filehippo or the main website and retrieve the latest file

eg would be AVG Free

I used Yashied's code to give me the link to the download

expandcollapse popup

ConsoleWrite(_HTTPGetResponse('http://free.avg.com/gb-en/download-file-dm-afh') & @CR)

Func _HTTPGetResponse($sUrl)

    Local $Error, $Socket, $Recv = ''
    Local $Host = StringRegExpReplace($sUrl, '.*://(.*?)/.*', '\1')
    Local $Page = StringRegExpReplace($sUrl, '.*://.*?(/.*)', '\1')
    Local $Send = 'HEAD ' & $Page & ' HTTP/1.1' & @CRLF & _
                  'Host: ' & $Host & @CRLF & _
                  'User-Agent: AutoIt/' & @AutoItVersion & ' (Windows; U; Windows NT 5.1; en-US; rv:1.8.1)' & @CRLF & _
                  'Referer: ' & $Host & @CRLF & _
                  'Connection: close' & @CRLF & @CRLF

    TCPStartup()
    If @error Then
        Return SetError(@error, 0, '')
    EndIf
    $IP = TCPNameToIP($Host)
    If @error Then
        Return SetError(@error, TCPShutdown(), '')
    EndIf
    $Socket = TCPConnect($IP, 80)
    If @error Then
        Return SetError(@error, TCPShutdown(), '')
    EndIf
    If Not TCPSend($Socket, $Send) Then
        Return SetError(@error, TCPCloseSocket($Socket) + TCPShutdown(), '')
    EndIf
    Do
        $Recv &= TCPRecv($Socket, 16)
    Until @error
    If Not $Recv Then
        $Error = @error
    Else
        $Error = 0
    EndIf
    TCPCloseSocket($Socket)
    TCPShutdown()
    Return SetError($Error, 2, $Recv)
EndFunc   ;==>_HTTPGetResponse

Which gives me this

HTTP/1.1 302 Moved Temporarily

Server: Apache

Location: http://download.avgfree.com/filedir/inst/avg_free_stb_en_2011_1392_free.exe

X-AVG: harpo13

Content-Type: text/html

Date: Wed, 10 Aug 2011 09:47:45 GMT

Connection: close

Set-Cookie: cmc=222; expires=Wed, 10-Aug-2011 09:47:48 GMT; path=/; domain=avg.com

Set-Cookie: avg_tracking_redirect=download-file-dm-afh; path=/

But unfortunatly they keep changing the links again as they are trying to protect them which is an ass, and on some like Ccleaner slim build i cant get them at all

Im calling the links at the min from an ini file like this

InetGet( IniRead( @ScriptDir & "\file_includes\tools_links.ini","AVG_Free","Link",  "NotFound"), @ScriptDir & "\file_includes\install_avg_free.exe", 1)

Which basically contains the Location link from above

The question really is, is there a simple ish way to do this which gets the right link? bear in mind ive not done Internet stuff before.

One other option i have would be to include another file update program with the script like Ketarin with the list of programs within and run from CMD line silent, but this to me adds complexity to the job.

So any suggestions i should look at or udf's that might be relevant?

Thanks

EDIT typos

Edited August 10, 2011 by Chimaera

[wakillon]

Your free.avg example works well.

Is it only for ccleaner that doesn't work ?

[Chimaera]

lol i knew that was going to trip me up sorry it failed this morning and thats the renewed link i did just after

They only seem to work for a day or less sometimes

I was using filehippo before copying from the if your download dosent start link

This is a failed one

http://www.filehippo.com/download/file/4dd125ede1b7855bc03997219f0fdc5171298aaadc9730bd9a0bee3f5eb6d07a/

Edited August 10, 2011 by Chimaera

[wakillon]

I suggest you a simply way for get it from the official CCleaner WebSite

#include <String.au3>

Global $sURL = "http://www.piriform.com/ccleaner/builds" 
$_Version = _StringBetween ( BinaryToString ( InetRead ( $sURL ) ), 'ccsetup', '.exe' )
If Not @error Then
    $_DownloadLink = 'http://download.piriform.com/ccsetup' & $_Version[0] & '.exe'
    ConsoleWrite ( "-->-- $_DownloadLink : " & $_DownloadLink & @Crlf )
EndIf

Edited August 10, 2011 by wakillon

[Chimaera]

Ok thanks for that but i dont understand it

Im currently trying to get it to download the version i need by modding it

Questions

Global $sURL = "http://www.piriform.com/ccleaner/builds"

do you always set the page with the downloads on it as this one?, as i have more than one product to find

How did you arrive at this address?

$_DownloadLink = 'http://download.piriform.com/ccsetup'

I just tried this to pick up the slim one

Global $sURL = "http://www.piriform.com/ccleaner/builds"
$_Version = _StringBetween ( BinaryToString ( InetRead ( $sURL ) ), 'ccsetup', '_slim.exe' )
If Not @error Then
    $_DownloadLink = 'http://download.piriform.com/ccsetup' & $_Version[0] & '_slim.exe'
    ConsoleWrite ( "-->-- $_DownloadLink : " & $_DownloadLink & @Crlf )
EndIf

And it returned this

-->-- $_DownloadLink : http://download.piriform.com/ccsetup309.exe<br /><br /><a href="download/standard"><img src="http://static.piriform.com/pf/download.png" /></a></div><blockquote><p><b>

Builds below are for system admins and advanced users.

<br />

They are not for distribution or mirroring.

</b></p></blockquote><div class="itembox"><img src="http://static.piriform.com/pf/install_48_grey.gif" align="left" /><b>CCleaner - Portable</b><br />

Zip file, no installer

<br /><br /><font color="green">3,090 kb</font>

- ccsetup309.zip<br /><br /><a href="download/portable"><img src="http://static.piriform.com/pf/download.png" /></a></div><div class="itembox"><img src="http://static.piriform.com/pf/install_48_grey.gif" align="left" /><b>CCleaner - PortableU3</b><br />

For U3 compatible USB thumbdrives

<br /><br /><font color="green">3,175 kb</font>

- ccsetup309.u3p<br /><br /><a href="download/portableU3"><img src="http://static.piriform.com/pf/download.png" /></a></div><div class="itembox"><img src="http://static.piriform.com/pf/install_48_grey.gif" align="left" /><b>CCleaner - Slim</b><br />

Installer, no toolbar

<br /><br /><font color="green">2,470 kb</font>

- ccsetup309_slim.exe

Another example i tried

#include <String.au3>

Global $sURL = "http://free.avg.com/gb-en/download-file-dm-afh"
$_Version = _StringBetween ( BinaryToString ( InetRead ( $sURL ) ), 'avg_free_x86_all_2011_', '.exe' )
If Not @error Then
    $_DownloadLink = 'http://af-download.avg.com/avg_free_x86_all_2011_' & $_Version[0] & '.exe'
    ConsoleWrite ( "-->-- $_DownloadLink : " & $_DownloadLink & @Crlf )
EndIf

Sorry i just cant get it working except for your original version

Edited August 10, 2011 by Chimaera

[wakillon]

It doesn't work for slim version.

For standart edition :open this page http://www.piriform.com/ccleaner/download/standard

put your mouse over "please click on this link" and right click, copy adress of this link

What's difference between this 2 versions?

[Chimaera]

I use slim cos i need silent install without crap like Chrome

There doesn't seem to be a uniform way to give a link and find "easily" the download link

even Yashieds code doesn't always work

[wakillon]

An alternative solution for slim version !

#include <Array.au3>

Global $_URL = "http://www.freewarehunter.com/files/"
Global $sSource = BinaryToString ( InetRead ( $_URL ) )
Global $_ArrayLinks = StringRegExp ( $sSource, '(?s)(?i)<a href="(.*?)">', 3 )
$_ArrayLinks = _DeleteArrayElementWithoutString ( $_ArrayLinks, 'ccsetup' )
_ArrayDisplay ( $_ArrayLinks )
Exit

Func _DeleteArrayElementWithoutString ( $_Array, $_String )
    Local $_Item
    For $_Element In $_Array
        If StringInStr ( $_Element, $_String ) = 0 Then
            _ArrayDelete ( $_Array, $_Item )
        Else
            $_Item+=1
        EndIf
    Next
    Return ( $_Array )
EndFunc ;==> _DeleteArrayElementWithoutString ( )

i let you choose.

Edited August 10, 2011 by wakillon

[Chimaera]

Thanks wakillon for the help

This just dosent seem to be working out how i thought it would be, it seems a whole lot more complicated.

Anyway i shall have to keep plodding on and see what happens

Thanks for your time.

[storme]

G'day M8

Just a word.

The slim version disappears from CCleaners website at every update.

Some time later (approximately a week) they put up the latest slim version.

The assumption was that the authors want to make some money off the full version (adds) and this forces people to get the full version for a week or so.

BTW I have a function to download files from filehippo. If you want it drop me an IM.

C YA

John Morrison

aka

Storm-E

[wakillon]

G'day M8

Just a word.

The slim version disappears from CCleaners website at every update.

Some time later (approximately a week) they put up the latest slim version.

The assumption was that the authors want to make some money off the full version (adds) and this forces people to get the full version for a week or so.

BTW I have a function to download files from filehippo. If you want it drop me an IM.

C YA

John Morrison

aka

Storm-E

Your function doesn't work for CCleaner Slim, but for standard version.

[storme]

Your function doesn't work for CCleaner Slim, but for standard version.

Too true FileHippo doesn't do the Slim verison of Ccleaner at all.

However, the filehippodownload function would be useful for the other programs Chimaera wants to download.

I think a better solution would be to work out how to stop google Crome from installing as teh slim isn't always available.

Then ccleaner could be downlaoded from anywhere.

Your freewarehunter is a good alternative but Chimaera wouldn't always have the latest version of course. However, that maybe of no importance for the script.

Anyway gota keep going

John Morrison

[ripdad]

This works for me ...

InetGet('http://www.piriform.com/ccleaner/download/slim/downloadfile', @ScriptDir & '\ccsetup-slim.exe', 1, 1)

An SRE would be able to get the version number from this page: http://www.piriform.com/ccleaner/builds

[Chimaera]

BTW I have a function to download files from filehippo. If you want it drop me an IM.

I might have a chat about it m8

InetGet('http://www.piriform.com/ccleaner/download/slim/downloadfile', @ScriptDir & '\ccsetup-slim.exe', 1, 1)

An SRE would be able to get the version number from this page: http://www.piriform.com/ccleaner/builds

Not trying to be dumb but whats an SRE?

I use this at present to establish version numbers

$CC_key = _GetUpdates("Ccleaner")
ConsoleWrite($CC_key & @CRLF)
IniWrite( @ScriptDir & "\latest_version.ini", "Ccleaner", "Version ",$CC_key)



Func _GetUpdates($sID) ; by GEOSoft (If I remember)
    Local $aReturn, $bRead, $sRead
    $bRead = InetRead("http://www.filehippo.com/search?q=" & $sID)
    $sRead = BinaryToString($bRead)

    $aReturn = StringRegExp($sRead, "(?i)" & $sID & "\s?((?:\d{1,5})(?:\.\d{1,5}){1,4})", 1)
    If @error Then
        Return SetError(1, 0, "None - Error")
    EndIf
    Return $aReturn[0]
EndFunc   ;==>_GetUpdates

@ripdad does your inetget take any notice of numbers or must it have the exact file name?

[storme]

This works for me ...

InetGet('http://www.piriform.com/ccleaner/download/slim/downloadfile', @ScriptDir & '\ccsetup-slim.exe', 1, 1)

An SRE would be able to get the version number from this page: http://www.piriform.com/ccleaner/builds

The trouble is 'http://www.piriform.com/ccleaner/download/slim/downloadfile' will only work until the next download then it disappears for a week or 2 then starts working again when the author puts up the new slim version.

It had me real confused the first time as I lost the old slim version and couldn't download the new one.

I found a forum post explaining what the situation is.

I'd think that "freewarehunter" that wakillon suggested maybe a good fall back.

eg

1. Try to download from piriform (as above)

2. If it Fails get latest from freewarehunter

That way you'd always have the latest "slim" version.

The only other choice is to automate around the "google chrome" install or uninstall it after the ccleaner install.

My 2C

John Morrison

BTW what does SRE stand for? (be kind it's 2:30am here and my brain is fried)

[Chimaera]

The fetching down of slim version isnt critical, i dont mind if its 14 days between updates.

I agree a second download choice might be a sensible idea as long as both have the right version.

[ripdad]

SRE = StringRegExp

Chimaera, once you get the version number from the SRE, then you can use that in your filename

instead of the static one I put for InetGet.

storme, ahh - I was unaware that the file gets removed - thats weird.

I suppose if InetGet returns an error, then that would be the case.

[Chimaera]

SRE = StringRegExp

Thank god you didnt mean this lol

http://www.channel4.com/learning/microsites/L/lifestuff/content/up_close/letstalksex/sre.html

[BrewManNH]

There's a program that can do this called Ketarin and will monitor the software that you have entered into the database. You might take a look at how it works, because it uses a scripting type set up for the programs that will give you an idea of how they do it.

[Chimaera]

I already use it m8 but for this script im doing i dont want to add it into the script to handle the updates, im just trying to find a simple way to grab five files and with ripdads suggestion and the file hippo one from storme it might just work

Thanks

Chimaera