hankjrfan Posted June 13, 2005 Share Posted June 13, 2005 I have wrote a script that downloads a webpage and extracts just the info I need from it. But the problem is it is way too slow. The downloaded file ranges between 80kb and 200kb. It takes between 12 and 30 minutes to go through the file. I would really like to try to get this time down. About 5 minutes would be ideal. All comments and criticisms are welcome. expandcollapse popupDim $world="Nova" Dim $line Dim $rows=20000 Dim $cols=5 Dim $name Dim $level Dim $vocation Dim $array[$rows][$cols] For $n=0 To $rows-1 $array[$n][0]="0" Next While 1 $begin = TimerInit() InetGet("http://www.tibia.com/statistics/?subtopic=whoisonline&world=" & $world,@ScriptDir & "\" & $world & ".txt") $file=FileOpen($world & ".txt", 0) While 1 $line = $line & FileRead($file, 1) If @error = -1 Then ExitLoop Wend $string = "<TR><TD>Currently " $temp = StringInStr($line,$string) $line = StringTrimLeft($line, $temp-1) $string = '<FORM ACTION="http://www.tibia.com/community/?subtopic=character" METHOD=post>' $temp = StringInStr($line,$string) $line = StringTrimRight($line,StringLen($line)-$temp+1) $line = StringReplace($line,"+"," ") $line = StringReplace($line,'<A NAME="'," ") $line = StringReplace($line,"%27"," ") ;Finds Number of Players Online $string = "<TR><TD>Currently " $temp = StringInStr($line,$string) $line = StringTrimLeft($line, $temp+17) $string = " players are online.<BR>" $temp = StringInStr($line,$string) $online=StringLeft($line,$temp-1) $string = '<a target=_top HREF="http://www.tibia.com/community/?subtopic=character&name' $temp = StringInStr($line,$string) $line = StringTrimLeft($line, $temp-1) For $n=1 To $online $found=0 ;Finds Name $string = "name=" $temp = StringInStr($line,$string) $line = StringTrimLeft($line, $temp+4) $string = '">' $temp = StringInStr($line,$string) $name=StringLeft($line,$temp-1) ;Finds Level $string = "10%>" $temp = StringInStr($line,$string) $line = StringTrimLeft($line, $temp+3) $string = "</TD>" $temp = StringInStr($line,$string) $level=Int(StringLeft($line,$temp-1)) If $level<8 Then ContinueLoop ;Finds Vocation $string = "20%>" $temp = StringInStr($line,$string) $line = StringTrimLeft($line, $temp+3) $string = "</TD>" $temp = StringInStr($line,$string) $vocation=StringLeft($line,$temp-1) If $vocation="None" Then ContinueLoop For $i=0 To $rows-1 If $array[$i][0]=$name Then $found=1 $array[$i][3]=$array[$i][3]+1 $array[$i][4]=@HOUR & ":" & @MIN $array[$i][1]=$level $array[$i][2]=$vocation EndIf Next If $found<>1 Then For $i=0 To $rows-1 If $array[$i][0]="0" Then $array[$i][0]=$name $array[$i][1]=$level $array[$i][2]=$vocation $array[$i][3]=1 $array[$i][4]=@HOUR & ":" & @MIN ExitLoop EndIf Next EndIf Next FileClose($file) FileMove($world & ".txt",$world & "bak.txt",1) FileMove($world & "bak.csv",$world & "bak2.csv",1) FileMove($world & ".csv",$world & "bak.csv",1) $file = FileOpen($world & ".csv",1) For $n=0 To $rows-1 If $array[$n][0]="0" Then ExitLoop $name=$array[$n][0] $level=$array[$n][1] $vocation=$array[$n][2] $times=$array[$n][3] $time=$array[$n][4] $line=$name & "," & $level & "," & $vocation & "," & $times & "," & $time FileWriteLine($file,$line) Next FileClose($file) $dif = TimerDiff($begin) FileWriteLine("worldwatch.log","Minutes to run: " & $dif/1000/60) WEnd Link to comment Share on other sites More sharing options...
herewasplato Posted June 13, 2005 Share Posted June 13, 2005 try replacing: While 1 $line = $line & FileRead($file, 1) If @error = -1 Then ExitLoop Wend with $line = FileRead($file,FileGetSize($file)) let me know...... [size="1"][font="Arial"].[u].[/u][/font][/size] Link to comment Share on other sites More sharing options...
hankjrfan Posted June 13, 2005 Author Share Posted June 13, 2005 try replacing:While 1$line = $line & FileRead($file, 1)If @error = -1 Then ExitLoopWendwith$line = FileRead($file,FileGetSize($file))let me know......<{POST_SNAPBACK}>It does not seem to work. I am not sure why it seems like it should. When I use your method the output file is empty. Link to comment Share on other sites More sharing options...
herewasplato Posted June 13, 2005 Share Posted June 13, 2005 It does not seem to work. I am not sure why it seems like it should. When I use your method the output file is empty.<{POST_SNAPBACK}>I just seemed to remember a one line solution for reading the entire contents of a file into a variable - I searched on FileRead and grabbed the first example that I noticed.http://www.autoitscript.com/forum/index.ph...indpost&p=84379It really should not save you that much time, but is better than reading the file one character at a time.I'm not sure what else to suggest - good luck [size="1"][font="Arial"].[u].[/u][/font][/size] Link to comment Share on other sites More sharing options...
hankjrfan Posted June 13, 2005 Author Share Posted June 13, 2005 I just seemed to remember a one line solution for reading the entire contents of a file into a variable - I searched on FileRead and grabbed the first example that I noticed.http://www.autoitscript.com/forum/index.ph...indpost&p=84379It really should not save you that much time, but is better than reading the file one character at a time.I'm not sure what else to suggest - good luck<{POST_SNAPBACK}>I got it!! The problem at first was that I was useing the file handle instead of the file name. Well anyway this method is much faster than the one I was useing.Many Thanks Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now