Optimize my code...file2array

[sshrum]

I'm writing a function that will allow me to take a file and dump it to an array (keep reading...) and additionally, if a separator is passed it will stringsplit each line into a 2-dimension array (ah!).

Test file was a 5.4 meg file with 30302 lines

Func _File2Array($sFilename, $sSeperator="")
    Consolewrite("(_File2Array) ")
    $iTimer = TimerInit()
    Dim $aTemp[1]
    $aTemp[0] = 0
    if fileexists($sFilename) then
        $iSize = round(FileGetSize($sFilename)/1024/1024,2)
        consolewrite("Reading '" & $sFilename & "' (" & $iSize & " megs); ")
        $aTemp = StringSplit(FileRead($sFilename), @CR) 
        if $sSeperator <> "" Then
            $iCols = ubound(StringSplit($aTemp[1], $sSeperator))-1
            Dim $aTemp2[$aTemp[0]+1][$iCols]
            $aTemp2[0][0]=$aTemp[0]
            For $i = 1 to $aTemp[0]
                $aRecord = StringSplit($aTemp[$i], $sSeperator)
                For $j = 1 to $aRecord[0]
                    $aTemp2[$i][$j-1] = $aRecord[$j]
                Next
            Next
            $aTemp = $aTemp2
        EndIf
        ConsoleWrite("complete [" & Round(TimerDiff($iTimer)/1000,2) & " secs]" & @CRLF)
    Else
        ConsoleWrite("could not locate file: '" & $sFilename & "'" & @CRLF)     
    EndIf
    return $aTemp
EndFunc

The 1-dim run took .26 secs.

The 2-dim run (14 columns) took over 3.8 secs.

Does anyone know what I can do to make the 2-dim process faster in AutoIT or is this the fastest I can get it (no C++ remarks)?

Seems fast to me. Just looking for input.

TIA

EDIT: modified the code a schooch

Edited July 19, 2007 by sshrum

[randallc]

Hi,

Seems fast.

"In autoIt" is open to interpretation!?

Here it is, ? 2-3x faster using my "array2d.au3" UDF from signature link; but that uses vbscript too?..

#include<array2d.au3>
local $time1 = TimerInit(),$ar_FileReadArray,$sFilename=@ScriptDir&"\DELL4700FastSearchNew_S_.txt";,$a=_FileReadToArray($sFilename,$ar_FileReadArray)
$ar_FileReadArray=StringSplit(FileRead($sFilename), @CR)  
local $ar_FileReadArray2D=_Array2DCreateFromArraySt($ar_FileReadArray, 1,  0, "|"),$a=ConsoleWrite("FileToArray2D.au3" & "=" & Round(TimerDiff($time1), 2) & " MSEC." & @LF)
local $a=ConsoleWrite("UBound($ar_FileReadArray2D)" & "=" & UBound($ar_FileReadArray2D) & @LF)
local $a=ConsoleWrite("$ar_FileReadArray2D[0][0]" & "=" & $ar_FileReadArray2D[0][0] & @LF),$time1 = TimerInit()
local $a=ConsoleWrite("$ar_FileReadArray2D[UBound($ar_FileReadArray2D)-1][UBound($ar_FileReadArray2D,2)-1]" & "=" & $ar_FileReadArray2D[UBound($ar_FileReadArray2D)-1][UBound($ar_FileReadArray2D,2)-1] & @LF)

local $ar_FileReadArray2D=_File2Array($sFilename, "|"),$a=ConsoleWrite("FileToArray2D.au3" & "=" & Round(TimerDiff($time1), 2) & " MSEC." & @LF)
local $a=ConsoleWrite("UBound($ar_FileReadArray2D)" & "=" & UBound($ar_FileReadArray2D) & @LF)
local $a=ConsoleWrite("$ar_FileReadArray2D[0][0]" & "=" & $ar_FileReadArray2D[0][0] & @LF)
local $a=ConsoleWrite("$ar_FileReadArray2D[UBound($ar_FileReadArray2D)-1][UBound($ar_FileReadArray2D,2)-1]" & "=" & $ar_FileReadArray2D[UBound($ar_FileReadArray2D)-1][UBound($ar_FileReadArray2D,2)-1] & @LF)oÝ÷ Ù«­¢+ØÐíIÕ¹¹¥¹è ̸ȸԸÀ¤éèÀäÈíAɽɴ¥±ÌÀäÈíÕѽ%ÐÌÀäÈíÑÀäÈíÕѽ¥Ð̹áÅÕ½ÐíèÀäÈíÁɽɵÌÀäÈíMÉ¡¹¥¹ÀäÈí¥±Q½ÉÉäɹÔÌÅÕ½Ðì)¥±Q½ÉÉäɹÔÌôÈÈÐ̸àÈ5M¸)U ½Õ¹ ÀÌØíÉ}¥±IÉÉäɤôÌÔàÀÔ(ÀÌØíÉ}¥±IÉÉäÉlÁulÁtõAI=IML¹AP(ÀÌØíÉ}¥±IÉÉäÉmU   ½Õ¹ ÀÌØíÉ}¥±IÉÉäɤ´ÅumU    ½Õ¹ ÀÌØíÉ}¥±IÉÉäɰȤ´ÅtõE%(¡}¥±ÉÉÉä¤I¥¹ÌäíèÀäÈíÁɽɵÌÀäÈíMÉ¡¹¥¹ÀäÈí10ÐÜÀÁÍÑMÉ¡9Ý}M|¹ÑáÐÌäì ̸ääµÌ¤ì½µÁ±ÑlظäÐÍÍt)¥±Q½ÉÉäɹÔÌôÜÈÀظÈÌ5M¸)U    ½Õ¹ ÀÌØíÉ}¥±IÉÉäɤôÌÔàÀØ(ÀÌØíÉ}¥±IÉÉäÉlÁulÁtôÌÔàÀÔ(ÀÌØíÉ}¥±IÉÉäÉmU   ½Õ¹ ÀÌØíÉ}¥±IÉÉäɤ´ÅumU    ½Õ¹ ÀÌØíÉ}¥±IÉÉäɰȤ´ÅtõE%

best, randall

[sshrum]

Sweet...

Nah...my comment about C++ is to ward off all those people that answer requests for faster with "use C++". I know that but that's not what I was looking for.

I'm going to play around with your functions in a bit...need to vent and sleep.

I just spent 2 freakin hours fixing a freakin 1-line input error I made AND kept overlooking...ARGH!

There are some of my own homebrewed that I still need: _ArrayShuffle (simple random row replacement), and _ArrayMerge (combine like-dim'ed arrays). I'd love to see these in a UDF.