Interpretating IE and Download box

[masCh]

Hello everyone! I'm new..

I've spent some hours on this forum trying to write my own script. From the searches I have managed to compile some working codes available in this forum into a semi-working script.

What I'm trying to achieve is basically serial downloading from RapidShare.

The method is, create an empty text file called rs.txt

Then copy the addresses of the files you wish to download into the file rs.txt for eg:

http://rapidshare.com/files/230057387/adsenseisalive.pdf

http://rapidshare.com/files/230057387/adsenseisalive2.pdf

http://rapidshare.com/files/230057387/adsenseisalive3.pdf

Then execute the script, it will search for each file to download, and choose Free User

The code is below:

expandcollapse popup

;
; AutoIt Version: 3.0
; Language:    English
; Platform:    Win9x/NT
; Author:        Nusantara Technologies (azrudi@nusantech.com)
;
; Script Function:
;   Plays with RapidShare.
;



#include <IE.au3>
#include <winAPI.au3>

$file = FileOpen("rs.txt",0)
$counter = 1

;Check if file opened for reading OK
If $file = -1 Then
MsgBox(0, "Error", "Unable to open file.")
Exit
EndIf


;Read in lines of text until the EOF is reached
While 1
$alamat = FileReadLine($file)
If @error = -1 Then ExitLoop


;Obtain address
msgbox(0,"Hmm",$alamat,5)
$oIE = _IECreate($alamat)

; This may need more drilling down into the DOM...
$colInputs = _IETagNameGetCollection($oIE, "input")
$sMsg = ""
For $oInput In $colInputs
    If String($oInput.value) = "Free user" Then 
        $oIE = _IEAction($oInput, "click")

Global $winHandle = WinGetHandle("RapidShare") 
_WinAPI_SetWindowPos($winhandle, "hwnd_top", 0, 0, 1024, 400, "SWP_SHOWWINDOW")
Sleep(3000)
Send("{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}{DOWN}")
Sleep(50000)
MouseClick("left",510,250)
Sleep(2000)
Send("{TAB}{TAB}{TAB}{ENTER}")
Sleep(1000)
Send("{ENTER}")
;********************
;**
;** somehow must detect once download completes
;** then close IE
;** then resume script
;*********************

ExitLoop
    EndIf
Next


$i = 0
while $i < $counter
$i = $i + 1
Wend

$counter = $counter + 1

Wend

FileClose($file)

I have two problems:

1. I have no idea how to detect if the download has been complete or not? I want it so that once download of current file is complete, then can move on to the next file from rs.txt

2. The way I handled clicking on the "Download!" image after the 45 sec wait is a bit crude. Basically I just resized the browser, scrolled using up/down keys and then click somewhere in the middle of the page. So it might not work for all your PCs. I wonder if there is a better way to find the "Download!" image and click on it properly?

Thanks!

p/s attaching the sample data file

rs.txt

[Authenticity]

I guess you'll understand, if no first read the help file. The concept is the matter =]

expandcollapse popup

#include <HTTP.au3>

Dim $sHost = 'rs69.rapidshare.com', $sPage = '/files/9720545/MoreFloyd.rar'
Dim $sRet, $aMatch, $iTmp, $sTmp
Dim $sHTTP = _HTTPConnect($sHost)
If @error Then Exit

_HTTPPost($sHost, $sPage, $sHTTP, 'dl.start=Free')
$sRet = _HTTPRead($sHTTP)
If $sRet = '' Then OhBoy()

If StringRegExp($sRet, '(?i)<h1>error</h1>') Then OhBoy('You have reached the download limit for free-users')

$aMatch = StringRegExp($sRet, '(?i)var c\s*=\s*(\d+);', 1)
If Not IsArray($aMatch) Then OhBoy()

$iTmp = Int($aMatch[0])
$aMatch = StringRegExp($sRet, "(?i)document\.dlf\.action=\\'([^']*)\\", 1)
If Not IsArray($aMatch) Then OhBoy()

$sTmp = $aMatch[0]
$aMatch = StringRegExp($sRet, '(?i)(\d+)\s+kb', 1)
If Not IsArray($aMatch) Then OhBoy()

For $i = $iTmp To 1 Step -1
    ToolTip(@TAB & $i, @DesktopWidth-100, 0, 'Waiting...', 1)
    Sleep(1000)
Next
ToolTip('')

$iTmp = Int($aMatch[0])
ProgressOn('Downloading "' & $sTmp & '"', 'Please wait while downloading the file')

InetGet($sTmp, @ScriptDir & '\' &  StringTrimLeft($sTmp, StringInStr($sTmp, '/', 0, -1)), 1, 1)

While @InetGetActive
    Local $iKBRead = @InetGetBytesRead/0x400
    Local $iPrcnt = Round(100*$iKBRead/$iTmp)
    ProgressSet($iPrcnt, $iKBRead & '/' & $iTmp & ' KB')
    Sleep(200)
WEnd

ProgressOff()
_HTTPClose($sHTTP)


Func OhBoy($sStr = 'Unknown')
    MsgBox(0x10, 'Error!', $sStr)
    _HTTPClose($sHTTP)
    Exit
EndFunc

[masCh]

I guess you'll understand, if no first read the help file. The concept is the matter =]

expandcollapse popup

#include <HTTP.au3>

Dim $sHost = 'rs69.rapidshare.com', $sPage = '/files/9720545/MoreFloyd.rar'
Dim $sRet, $aMatch, $iTmp, $sTmp
Dim $sHTTP = _HTTPConnect($sHost)
If @error Then Exit

_HTTPPost($sHost, $sPage, $sHTTP, 'dl.start=Free')
$sRet = _HTTPRead($sHTTP)
If $sRet = '' Then OhBoy()

If StringRegExp($sRet, '(?i)<h1>error</h1>') Then OhBoy('You have reached the download limit for free-users')

$aMatch = StringRegExp($sRet, '(?i)var c\s*=\s*(\d+);', 1)
If Not IsArray($aMatch) Then OhBoy()

$iTmp = Int($aMatch[0])
$aMatch = StringRegExp($sRet, "(?i)document\.dlf\.action=\\'([^']*)\\", 1)
If Not IsArray($aMatch) Then OhBoy()

$sTmp = $aMatch[0]
$aMatch = StringRegExp($sRet, '(?i)(\d+)\s+kb', 1)
If Not IsArray($aMatch) Then OhBoy()

For $i = $iTmp To 1 Step -1
    ToolTip(@TAB & $i, @DesktopWidth-100, 0, 'Waiting...', 1)
    Sleep(1000)
Next
ToolTip('')

$iTmp = Int($aMatch[0])
ProgressOn('Downloading "' & $sTmp & '"', 'Please wait while downloading the file')

InetGet($sTmp, @ScriptDir & '\' &  StringTrimLeft($sTmp, StringInStr($sTmp, '/', 0, -1)), 1, 1)

While @InetGetActive
    Local $iKBRead = @InetGetBytesRead/0x400
    Local $iPrcnt = Round(100*$iKBRead/$iTmp)
    ProgressSet($iPrcnt, $iKBRead & '/' & $iTmp & ' KB')
    Sleep(200)
WEnd

ProgressOff()
_HTTPClose($sHTTP)


Func OhBoy($sStr = 'Unknown')
    MsgBox(0x10, 'Error!', $sStr)
    _HTTPClose($sHTTP)
    Exit
EndFunc

Thank you! Did you write this script yourself?

It looks far more advanced than my {TAB}{TAB}{TAB} solution!

And because of that, I have no idea how the script works! (I am trying to learn at the same time as having a working script)

What do these mean?

'(?i)var c\s*=\s*(\d+);'

(?i)document\.dlf\.action=\\'([^']*)\\"

'(?i)(\d+)\s+kb'

And I guess, to make it work with a premium account, I just change dl.Start=Free to dl.Start=Premium?

One last thing, at the very top the server is rs69.rapidshare.com and specified specifically.. what if one day rapidshare changes the host to rs70.rapidshare.com or rs69.rapidshare.de? Is there a way to read this server address from the page itself?

Thanks, you've been a great help at making a functional script! All I want to do now is to learn more!

[Authenticity]

Hmm I think you should read the help file for things that are not clear because there might be a lot of things you don't know ;]

Basically it's as followed:

If the host is maintained in the poor way I showed you'll be lucky if it'll redirect you to the new address though it's unlikely that they'll change their host because it's a site of millions of users and so it's even more unlikely, maybe with redirection... Anyway, you should maintain the input data like the host or the files as input from file or something that a download patch can fix.

The regular expressions can be explained but you'll need to read the help file anyway to understand what all the symbols mean:

- First see if you have an error in the page like file could not be found or the user need to wait for 40 or so minutes

before downloading the next file because of file size download delays restriction of their side.

- Then get the c variable which determines how much seconds you need to wait before it's possible to download the file.

- Get the file download url.

- Get the file name of the url path to save it in the script directory.

- Get it's size so the user can see the progression.

...

You need to have a little tiny miny understanding in HTTP request the understand the other _HTTP* functions but it's quite self explanatory and by reading a few examples you'll get it pretty fast.

[masCh]

I noticed the code includes a file called HTTP.au3 but I cant find it on my AutoIt install.

So I searched the forum and found this:

http://www.autoitscript.com/forum/index.ph...mp;#entry319354

Is this the right file?

So I added this file into the AutoIt3\Include directory and tried to run the script.

It gave me an error with no line or no information, like this (see attachment).

I dont know what's wrong, so I am posting the whole code here:

expandcollapse popup

;
; AutoIt Version: 3.0
; Language:    English
; Platform:    Win9x/NT
; Author:        Nusantara Technologies (azrudi@nusantech.com)
;
; Script Function:
;   Plays with RapidShare.
;



#include <HTTP.au3>

$file = FileOpen("rs.txt",0)
$counter = 1

;Check if file opened for reading OK
If $file = -1 Then
MsgBox(0, "Error", "Unable to open file.")
Exit
EndIf


;Read in lines of text until the EOF is reached
Dim $sRet, $aMatch, $iTmp, $sTmp

While 1
$alamat = FileReadLine($file)
If @error = -1 Then ExitLoop

$mula=StringInStr($alamat,'rapidshare.com')+14
if $mula=14 Then 
        $mula=StringInStr($alamat,'rapidshare.de')+13
        If $mula=13 Then OhBoy("Server Not Found!")
    EndIf
    
$bilang=StringLen($alamat)-$mula+1
$sPage=StringMid($alamat,$mula,$bilang)

$sHost='rs69.rapidshare.com'
$sHTTP= _HTTPConnect($sHost)
if @error Then Exit

_HTTPPost($sHost, $sPage, $sHTTP, 'dl.start=Free')
$sRet = _HTTPRead($sHTTP)
If $sRet = '' Then OhBoy()

If StringRegExp($sRet, '(?i)<h1>error</h1>') Then OhBoy('You have reached the download limit for free-users')

$aMatch = StringRegExp($sRet, '(?i)var c\s*=\s*(\d+);', 1)
If Not IsArray($aMatch) Then OhBoy()

$iTmp = Int($aMatch[0])
$aMatch = StringRegExp($sRet, "(?i)document\.dlf\.action=\\'([^']*)\\", 1)
If Not IsArray($aMatch) Then OhBoy()

$sTmp = $aMatch[0]
$aMatch = StringRegExp($sRet, '(?i)(\d+)\s+kb', 1)
If Not IsArray($aMatch) Then OhBoy()

For $i = $iTmp To 1 Step -1
    ToolTip(@TAB & $i, @DesktopWidth-100, 0, 'Waiting...', 1)
    Sleep(1000)
Next
ToolTip('')

$iTmp = Int($aMatch[0])
ProgressOn('Downloading "' & $sTmp & '"', 'Please wait while downloading the file')

InetGet($sTmp, @ScriptDir & '\' &  StringTrimLeft($sTmp, StringInStr($sTmp, '/', 0, -1)), 1, 1)

While @InetGetActive
    Local $iKBRead = @InetGetBytesRead/0x400
    Local $iPrcnt = Round(100*$iKBRead/$iTmp)
    ProgressSet($iPrcnt, $iKBRead & '/' & $iTmp & ' KB')
    Sleep(200)
WEnd

ProgressOff()
_HTTPClose($sHTTP)




$i = 0
while $i < $counter
$i = $i + 1
Wend

$counter = $counter + 1

Wend

FileClose($file)

Func OhBoy($sStr = 'Unknown')
    MsgBox(0x10, 'Error!', $sStr)
    _HTTPClose($sHTTP)
    Exit
EndFunc

Does anyone know how to deal with that error?

Edited May 7, 2009 by masCh

[Authenticity]

Yes, it's the correct library, I've changed mine so if you want you can change yours to a sane value of TCPRecv heh.

Find and change the line Dim $recv = TCPRecv($socket,16) with Dim $recv = TCPRecv($socket,0x2000)

About the error message, it's hard to identify where it's called from, call OhBoy (lol) with like OhBoy(1) for the first unknown error and OhBoy(2) for the second unknown error, et cetera.

[masCh]

Yes, it's the correct library, I've changed mine so if you want you can change yours to a sane value of TCPRecv heh.

Find and change the line Dim $recv = TCPRecv($socket,16) with Dim $recv = TCPRecv($socket,0x2000)

About the error message, it's hard to identify where it's called from, call OhBoy (lol) with like OhBoy(1) for the first unknown error and OhBoy(2) for the second unknown error, et cetera.

Thanks, I added the numbers to differentiate the error locations..

Its from here:

$aMatch = StringRegExp($sRet, '(?i)var c\s*=\s*(\d+);', 1)
If Not IsArray($aMatch) Then OhBoy(2)

[masCh]

Thanks, I added the numbers to differentiate the error locations..

Its from here:

$aMatch = StringRegExp($sRet, '(?i)var c\s*=\s*(\d+);', 1)
If Not IsArray($aMatch) Then OhBoy(2)

Anyone know about this? I have no idea what is meant by $aMatch = StringRegExp($sRet, '(?i)var c\s*=\s*(\d+);', 1) so I dont know what the problem is..

[Authenticity]

Can you post one of the download links please?

[masCh]

Can you post one of the download links please?

For the code? Or a download from rapidshare?

The links are at the first post of this thread, in the attached rs.txt

If you want the AutoIt code download, its attached here..

nusantech_rs.au3

[Authenticity]

Yes, I see what is the problem. I've innocently use rs69.rapidshare.com because of the previous particular case. Few minutes to fix it please. ;]

[Authenticity]

expandcollapse popup

#include <HTTP.au3>
#include <INet.au3>
#include <ProgressConstants.au3>

Dim $sURL = 'http://rapidshare.com/files/230057387/adsenseisalive.pdf'

Dim $sHTTP
Dim $sHost, $sPage
Dim $sRet, $sDownloadLoc, $sFile
Dim $iDelay, $iSize, $iKBRead, $iPrcnt

$sRet = _INetGetSource($sURL)
$sRet = _SingleMatch($sRet, '(?i)<form id="ff" action="([^"]*)')
If $sRet = '' Then OhBoy('Bad URL')

$sHost = _SingleMatch($sRet, '(?i)http://([^/]+)')
$sPage = _SingleMatch($sURL, '(?i)rapidshare.com(.*)')
If $sHost = '' Or $sPage = '' Then OhBoy('Bad URL')

$sHTTP = _HTTPConnect($sHost)
If @error Then Exit MsgBox(0x10, 'Error!', 'Could not connect to: ' & $sHost)

_HTTPPost($sHost, $sPage, $sHTTP, 'dl.start=Free')
$sRet = _HTTPRead($sHTTP)
If $sRet = '' Then OhBoy()

If StringRegExp($sRet, '(?i)<h1>error</h1>') Then OhBoy('You have reached the download limit for free-users')

$iDelay = _SingleMatch($sRet, '(?i)var c\s*=\s*(\d+);')

If $iDelay <> '' Then
    $iDelay = Int($iDelay)
    For $i = $iDelay To 1 Step -1
        ToolTip(@TAB & $i, @DesktopWidth-100, 0, 'Waiting...', 1)
        Sleep(1000)
    Next
    ToolTip('')
EndIf

$sDownloadLoc = _SingleMatch($sRet, "(?i)document\.dlf\.action=\\'([^']*)\\")
If $sDownloadLoc = '' Then OhBoy('Could not find download location')

$iSize = StringRegExp($sRet, '(?i)(\d+)\s+kb', 1)
If $iSize = '' Then
    $iSize = 0
Else
    $iSize = Int($iSize)
EndIf

$sFile = StringTrimLeft($sURL, StringInStr($sURL, '/', 0, -1))

InetGet($sDownloadLoc, @ScriptDir & '\' &  $sFile, 0, 1)

If $iSize > 0 Then
    ProgressOn('Downloading "' & $sFile & '"', 'Please wait while downloading the file')
    
    While @InetGetActive
        $iKBRead = @InetGetBytesRead/0x400
        $iPrcnt = Round(100*$iKBRead/$iSize)
        ProgressSet($iPrcnt, $iKBRead & '/' & $iSize & ' KB')
        Sleep(200)
    WEnd
    ProgressOff()
Else
    Local $hGUI, $Progress, $LabelUpper, $LabelBottom
    
    $hGUI = GUICreate('Downloading "' & $sFile & '"', 250, 120)
    $LabelUpper = GUICtrlCreateLabel('Please wait while downloading the file', 20, 20)
    $Progress = GUICtrlCreateProgress(20, 50, 210, 20, $PBS_MARQUEE)
    $LabelBottom = GUICtrlCreateLabel('', 30, 85, 100, 23)
    GUISetState()
    GUICtrlSendMsg($Progress, $PBM_SETMARQUEE, True, 100)
    
    While @InetGetActive
        GUICtrlSetData($LabelBottom, @InetGetBytesRead/1024 & ' KB')
        Sleep(100)
    WEnd
    
    GUICtrlSendMsg($Progress, $PBM_SETMARQUEE, False, 0)
    GUIDelete()
EndIf

_HTTPClose($sHTTP)
Exit

Func OhBoy($sStr = 'Unknown')
    MsgBox(0x10, 'Error!', $sStr)
    _HTTPClose($sHTTP)
    Exit
EndFunc

Func _SingleMatch($sString, $sPattern)
    Local $aMatch = StringRegExp($sString, $sPattern, 1)
    If IsArray($aMatch) Then Return $aMatch[0]
    Return ''
EndFunc

[masCh]

That worked, but how did you figure out dl.Start = Free?

Where on the Rapidshare page does it say that?

[Authenticity]

Usually if it's on a form you'll see something like <form action="something"...> so you can see what it post/get to the server. If you want it more explicitly you can get DebugBar and examine the HTML tab and see what is the header data of the request and the additional data. For example, in this case it sends additional data by specifying "Content-Length: 13" at the header and the additional data is "dl.Start=Free". It should be prepended by a blank line.

[masCh]

Thank's Authenticity

You've been a great help!