Jump to content
Sign in to follow this  
rudi

Howto speedup Array cleaning up?

Recommended Posts

rudi

Hi.

Perhaps it's best to describe fist, what I want to do:

I have some files of GPS data representing LPG pumps in Europe.

File1: 24h open, EC-Cards accepted

File2: 24h open, cash (includes entries from file1)

File3: 7 days open, EC-Cards accepted (includes entries from File1 and File2)

File4: 7 days open, cash (includes entries from File1, 2, 3)

File5: Mo-Fr, Cards accepted (includes 1-4)

File6: Mo-Fr, cash (and so on... up to file8)

I wrote a working Autoit script to cleaning up file2 to file8, but it's ugly slow. What I do is this:

_FileReadToArray("File1.txt",$UpArray)

_FileReadToArray("File2.txt",$DownArray)

Then I compare the GPS coordinates for each entry ($x) of $UpArray with also each entry ($y) of $DownArray. When they equal, I use _ArrayDelete($DownArray,$y) for cleaning up.

Finally I write back the cleaned up array to File2.txt, then I continue with the next pair of files. (1 against 2 to 8, 2 against 3 to 8, ...)

The program is ugly slow. Is there a faster way to cleanup from a file2 all those lines, that start with the same substring found in any line of file1?

Thanks for any suggestions, Rudi.

PS: More background information:

www.Gas-Tankstellen.de offers the very kind service to download all known LPG petrol pumps from a always up-to-date database. These data can be loaded into several Car GPS systems, and I want to modify the offered data in a way that I will have different icons for "24h, EC-Card accepted". So the files will have to be uniquiified (each LPG station has to show up in only *ONE* file), and therefore I wrote this extremly slow program.


Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Share this post


Link to post
Share on other sites
weaponx

I don't recommend using _ArrayDelete(). It is a very expensive function because it redimensions your array. Also people tend to misuse it in a loop and then wonder why they see:

"the array subscript or dimension has been exceeded"

Instead of _ArrayDelete maybe just set that element to NULL with "", then when you write the array back to a file just have it skip NULL elements, this will be much faster.

Share this post


Link to post
Share on other sites
andybiochem

I wanted to know the answer to this ages ago, this is what I found...

Instead of using this:

(note the backward steps through the array)

for $i = (UBound($array) - 1) to 1 step -1
    
    if $array[$i] = "remove" Then
        _ArrayDelete($array,$i)
    EndIf

Next

Use this:

(note the forward steps through the array - much nicer!)

$string = ""

for $i = 1 to (UBound($array) - 1)
    
    if $array[$i] <> "remove" Then
        $string &= $array[$i] & "|"
    EndIf

Next

$array = StringSplit($string,"|")
_ArrayPop($array)

I didn't come up with this idea, I read it in another post but can't remember whose. Credit to them anyway; it's an incredibly fast way of removing unwanted elements....or rather, just keeping wanted ones.


- Table UDF - create simple data tables - Line Graph UDF GDI+ - quickly create simple line graphs with x and y axes (uses GDI+ with double buffer) - Line Graph UDF - quickly create simple line graphs with x and y axes (uses AI native graphic control) - Barcode Generator Code 128 B C - Create the 1/0 code for barcodes. - WebCam as BarCode Reader - use your webcam to read barcodes - Stereograms!!! - make your own stereograms in AutoIT - Ziggurat Gaussian Distribution RNG - generate random numbers based on normal/gaussian distribution - Box-Muller Gaussian Distribution RNG - generate random numbers based on normal/gaussian distribution - Elastic Radio Buttons - faux-gravity effects in AutoIT (from javascript)- Morse Code Generator - Generate morse code by tapping your spacebar!

Share this post


Link to post
Share on other sites
rudi

I don't recommend using _ArrayDelete(). It is a very expensive function because it redimensions your array. Also people tend to misuse it in a loop and then wonder why they see:

"the array subscript or dimension has been exceeded"

That didn't give me trouble, as after each _ArrayDelete() I did a "MyArr[0]=ubound(MyArr) - 1", it's just about speed...

Instead of _ArrayDelete maybe just set that element to NULL with "", then when you write the array back to a file just have it skip NULL elements, this will be much faster.

That's a very good approach. What do you think ("Speed" point of view) of setting the value of entries to be cleaned up to "False", like this:

Dim $TempString

$MyArray = StringSplit("one,two,tree,four,five,six,seven,eight,nine,ten", ",")
$MyArray[3]=False
$MyArray[5]=False
$MyArray[8]=False
for $i = 1 to $MyArray[0]
    if $MyArray[$i] Then
        $TempString &= $MyArray[$i] & @CRLF ; each line, also the last one, "closed" with CRLF
    EndIf
Next
$hanlde=FileOpen("CleanedUp.txt",2)
FileWrite($hanlde,$TempString)
FileClose($hanlde)

Run("notepad CleanedUp.txt")

Thanks, Rudi.

[edit] How to set an array element to NULL? Array[$x]="" ?

Edited by rudi

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×