Jump to content

code optamization help


sam2332
 Share

Recommended Posts

Ok im having some lag issues.

I am splitting a proxy list that is immensely huge (like 2.5 gb)and, I need some optimizations if you could share some :)

here is my code

#include <array.au3>
$file = FileOpen(@ScriptDir & "\proxys.txt", 0)
$list = FileRead($file)
FileClose($file)
$splita = StringSplit($list, @CRLF)
$file = FileOpen(@ScriptDir &"\fixed proxys.txt",1)
For $i = 1 To $splita[0] - 1
    $dataa = StringSplit($splita[$i], "      ")
    $dataa = StringSplit($dataa[1],":")
    for $b = 0 to ubound($dataa)-1
    Next
    if $dataa[0] = 2 then
        ConsoleWrite($dataa[1] & ":" & $dataa[2] & @CRLF)
    FileWrite($file,$dataa[1] & ":" & $dataa[2] & @crlf)
    EndIf
Next
FileClose($file)

the proxys are in this format

85.119.217.110:1080 server support: 4 Belgium (Meise)

85.119.217.111:1080 server support: 4 Belgium (Meise)

85.119.217.112:1080 server support: 4 Belgium (Meise)

85.119.217.113:1080 server support: 4 Belgium (Meise)

194.7.83.94:1080 server support: 4 Belgium (Tessenderlo)

200.87.196.131:1080 server support: 4 Bolivia (La Paz)

any help would be greatly appreciated

edit: hahahha i didnt even notice the empty for next loop lol it was for debug purposes xD

Edited by sam2332
Link to comment
Share on other sites

Try this.

But I'm not RegExp expert so check if RegExp format string is correct first!

#include <array.au3>

$file = FileOpen(@ScriptDir & "\proxys.txt", 0)
$list = FileRead($file)
FileClose($file)

$splita = StringRegExp($list, '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d{1,4}', 3) ; 85.119.217.110:1080
$file = FileOpen(@ScriptDir &"\fixed proxys.txt",1)
For $i = 1 To UBound($splita) - 1
 ConsoleWrite($splita[$i] & @CRLF)
    FileWrite($file,$splita[$i] & @crlf)
Next
FileClose($file)

EDIT: In your code you have empty inner FOR/NEXT loop!!

Edited by Zedna
Link to comment
Share on other sites

lol, so you read 2.5 GB at once... only this could take a fare amount of time and then working on that giant data is also fare amount of time. Do a test, read first like 32 MB chunks and go upward 64, 128, .. until you find something better for data reading and manipulation. If you'll read at the middle of the last line of the chunk, leave it at a buffer for the next read. I did so and it was pretty fast and perfectly correct.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...