Jump to content

File to 2D Array

CosmosTunes
 Share

Recommended Posts

CosmosTunes Seeker

CosmosTunes

Posted
Posted

Hi,

i need to read a file with some entries into a 2d array.

the file is like:

5555;Test;0

5556;Test2;1

5557;Test100;50

But both of my solutions are very slow (tested with 6277 entries):

~ Timecheck ~

_ReadFileTo2DArray(): 70.09

_ReadFileTo2DArray2(): 69.84

Any help would be apreciated.

CosmosTunes

Local $sFilename = "\data\list.txt", $iTimeF1, $iTimeF2
$iTimeF1 = _ReadFileTo2DArray($sFilename)
$iTimeF2 = _ReadFileTo2DArray2($sFilename)
MsgBox(0, "Info", "~ Timecheck ~" & @CRLF & "_ReadFileTo2DArray(): " & $iTimeF1 & @CRLF & "_ReadFileTo2DArray2(): " & $iTimeF2)


Func _ReadFileTo2DArray($sFilename)
    Local $hFile, $aFinalArray[1][3], $iStarttime = TimerInit()
    $hFile = FileOpen(@ScriptDir & $sFilename, 0)
    If $hFile = -1 Then
        SetError(1)
        Return 0
    EndIf
    Local $sTemp = FileRead($hFile)
; remove last line separator if any at the end of the file
    If StringRight($sTemp, 1) = @LF Then $sTemp = StringTrimRight($sTemp, 1)
    If StringRight($sTemp, 1) = @CR Then $sTemp = StringTrimRight($sTemp, 1)
    $aArray = StringSplit($sTemp, @CRLF, 1); Try Windows @CRLF first
    If @error Then $aArray = StringSplit($sTemp, @LF); Unix @LF is next most common
    If @error Then $aArray = StringSplit($sTemp, @CR); Finally try Mac @CR
    FileClose($hFile)
    For $i = 1 To $aArray[0]
        $aPart = StringSplit($aArray[$i], ";")
        If $aPart[0] = 3 Then
            ReDim $aFinalArray[$i+1][3]
            $aFinalArray[$i][0] = Number($aPart[1])
            $aFinalArray[$i][1] = String($aPart[2])
            If Number($aPart[3]) > 0 Then
                $aFinalArray[$i][2] = Number($aPart[3])
            Else
                $aFinalArray[$i][2] = 0
            EndIf               
        EndIf
    Next
    $aFinalArray[0][0] = UBound($aFinalArray) - 1
    Return Round(TimerDiff($iStarttime) / 1000, 2)
EndFunc ;==>_ReadFileTo2DArray


Func _ReadFileTo2DArray2($sFilename)
    Local $hFile, $sLine, $aList[1][3], $iCounter = 1, $iStarttime = TimerInit()
    If FileExists(@ScriptDir & $sFilename) Then
        $hFile = FileOpen(@ScriptDir & $sFilename, 0)
        If $hFile = -1 Then
            SetError(1)
            Return 0
        EndIf
        While 1
            $sLine = FileReadLine($hFile)
            If @error = -1 Then ExitLoop
            $aPart = StringSplit($sLine, ";")
            If $aPart[0] = 3 Then
                ReDim $aList[$iCounter+1][3]
                $aList[$iCounter][0] = Number($aPart[1])
                $aList[$iCounter][1] = String($aPart[2])
                If Number($aPart[3]) > 0 Then
                    $aList[$iCounter][2] = Number($aPart[3])
                Else
                    $aList[$iCounter][2] = 0
                EndIf
                $iCounter = $iCounter + 1
            EndIf
        WEnd
        $aList[0][0] = UBound($aList) - 1
        FileClose($hFile)
    Else
        SetError(2)
        Return 0
    EndIf
    Return Round(TimerDiff($iStarttime) / 1000, 2)
EndFunc ;==>_ReadFileTo2DArray2
Link to comment
Share on other sites
martin Universalist

martin

Posted
Posted

Hi,

i need to read a file with some entries into a 2d array.

the file is like:

5555;Test;0

5556;Test2;1

5557;Test100;50

But both of my solutions are very slow (tested with 6277 entries):

~ Timecheck ~

_ReadFileTo2DArray(): 70.09

_ReadFileTo2DArray2(): 69.84

Any help would be apreciated.

CosmosTunes

Local $sFilename = "\data\list.txt", $iTimeF1, $iTimeF2
$iTimeF1 = _ReadFileTo2DArray($sFilename)
$iTimeF2 = _ReadFileTo2DArray2($sFilename)
MsgBox(0, "Info", "~ Timecheck ~" & @CRLF & "_ReadFileTo2DArray(): " & $iTimeF1 & @CRLF & "_ReadFileTo2DArray2(): " & $iTimeF2)


Func _ReadFileTo2DArray($sFilename)
    Local $hFile, $aFinalArray[1][3], $iStarttime = TimerInit()
    $hFile = FileOpen(@ScriptDir & $sFilename, 0)
    If $hFile = -1 Then
        SetError(1)
        Return 0
    EndIf
    Local $sTemp = FileRead($hFile)
; remove last line separator if any at the end of the file
    If StringRight($sTemp, 1) = @LF Then $sTemp = StringTrimRight($sTemp, 1)
    If StringRight($sTemp, 1) = @CR Then $sTemp = StringTrimRight($sTemp, 1)
    $aArray = StringSplit($sTemp, @CRLF, 1); Try Windows @CRLF first
    If @error Then $aArray = StringSplit($sTemp, @LF); Unix @LF is next most common
    If @error Then $aArray = StringSplit($sTemp, @CR); Finally try Mac @CR
    FileClose($hFile)
    For $i = 1 To $aArray[0]
        $aPart = StringSplit($aArray[$i], ";")
        If $aPart[0] = 3 Then
            ReDim $aFinalArray[$i+1][3]
            $aFinalArray[$i][0] = Number($aPart[1])
            $aFinalArray[$i][1] = String($aPart[2])
            If Number($aPart[3]) > 0 Then
                $aFinalArray[$i][2] = Number($aPart[3])
            Else
                $aFinalArray[$i][2] = 0
            EndIf               
        EndIf
    Next
    $aFinalArray[0][0] = UBound($aFinalArray) - 1
    Return Round(TimerDiff($iStarttime) / 1000, 2)
EndFunc;==>_ReadFileTo2DArray


Func _ReadFileTo2DArray2($sFilename)
    Local $hFile, $sLine, $aList[1][3], $iCounter = 1, $iStarttime = TimerInit()
    If FileExists(@ScriptDir & $sFilename) Then
        $hFile = FileOpen(@ScriptDir & $sFilename, 0)
        If $hFile = -1 Then
            SetError(1)
            Return 0
        EndIf
        While 1
            $sLine = FileReadLine($hFile)
            If @error = -1 Then ExitLoop
            $aPart = StringSplit($sLine, ";")
            If $aPart[0] = 3 Then
                ReDim $aList[$iCounter+1][3]
                $aList[$iCounter][0] = Number($aPart[1])
                $aList[$iCounter][1] = String($aPart[2])
                If Number($aPart[3]) > 0 Then
                    $aList[$iCounter][2] = Number($aPart[3])
                Else
                    $aList[$iCounter][2] = 0
                EndIf
                $iCounter = $iCounter + 1
            EndIf
        WEnd
        $aList[0][0] = UBound($aList) - 1
        FileClose($hFile)
    Else
        SetError(2)
        Return 0
    EndIf
    Return Round(TimerDiff($iStarttime) / 1000, 2)
EndFunc;==>_ReadFileTo2DArray2
In _ReadFileTo2DArray it looks like you could be wasting a lot of time with ReDim. I think it would be better to ReDim only once. So create the array

Dim $aFinalArray[$aArray[0]][3]

Have a variable starting at zero which you increment every time you add to the array, and when you have finished ReDim to the number of elements filled.

Serial port communications UDF Includes functions for binary transmission and reception.printing UDF Useful for graphs, forms, labels, reports etc.Add User Call Tips to SciTE for functions in UDFs not included with AutoIt and for your own scripts.Functions with parameters in OnEvent mode and for Hot Keys One function replaces GuiSetOnEvent, GuiCtrlSetOnEvent and HotKeySet.UDF IsConnected2 for notification of status of connected state of many urls or IPs, without slowing the script.
Link to comment
Share on other sites
  • Moderators
SmOke_N Universalist

SmOke_N

Posted
  • Moderators
Posted (edited)

Try this:

Func _my_filereadto2darray($s_filename)
    If FileExists($s_filename) = 0 Then Return SetError(1, 0, 0)
    Local $s_temp = FileRead($s_filename)
    ;Strip any cr's and lf's off the end until we are at the last line of code:
    $s_temp = StringRegExpReplace($s_temp, "[\r|\n]+\z", "")
    ;Get only the lines with 3 semi colons on it, if you don't want just digits to be checked...
    ;for the first string, then change (\d+) to (.+?)
    Local $a_sre = StringRegExp($s_temp, "(?:\A|\n)(\d+);(.+?);(.+?)(?:\r|\z)", 3)
    If @error Then Return SetError(2, 0, 0)
    Local $a_ret[UBound($a_sre)][3], $i_count = 0
    For $i = 0 To UBound($a_sre) - 1 Step 3
        $i_count += 1
        $a_ret[$i_count][0] = $a_sre[$i]
        $a_ret[$i_count][1] = $a_sre[$i + 1]
        $a_ret[$i_count][2] = $a_sre[$i + 2]
    Next
    If $i_count = 0 Then Return SetError(3, 0, 0)
    ReDim $a_ret[$i_count + 1][3]
    $a_ret[0][0] = $i_count
    Return $a_ret
EndFunc
Edit: Was missing a semi colon for a comment

Edited by SmOke_N

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Link to comment
Share on other sites
CosmosTunes Seeker

CosmosTunes

Posted
  • Author
Posted

Try this:

Func _my_filereadto2darray($s_filename)
    If FileExists($s_filename) = 0 Then Return SetError(1, 0, 0)
    Local $s_temp = FileRead($s_filename)
    ;Strip any cr's and lf's off the end until we are at the last line of code:
    $s_temp = StringRegExpReplace($s_temp, "[\r|\n]+\z", "")
    ;Get only the lines with 3 semi colons on it, if you don't want just digits to be checked...
    ;for the first string, then change (\d+) to (.+?)
    Local $a_sre = StringRegExp($s_temp, "(?:\A|\n)(\d+);(.+?);(.+?)(?:\r|\z)", 3)
    If @error Then Return SetError(2, 0, 0)
    Local $a_ret[UBound($a_sre)][3], $i_count = 0
    For $i = 0 To UBound($a_sre) - 1 Step 3
        $i_count += 1
        $a_ret[$i_count][0] = $a_sre[$i]
        $a_ret[$i_count][1] = $a_sre[$i + 1]
        $a_ret[$i_count][2] = $a_sre[$i + 2]
    Next
    If $i_count = 0 Then Return SetError(3, 0, 0)
    ReDim $a_ret[$i_count + 1][3]
    $a_ret[0][0] = $i_count
    Return $a_ret
EndFunc
Edit: Was missing a semi colon for a comment
Omg? Perfect 0.21 seconds and gives the same array!

thanks man :]

Link to comment
Share on other sites
  • Moderators
SmOke_N Universalist

SmOke_N

Posted
  • Moderators
Posted

Omg? Perfect 0.21 seconds and gives the same array!

thanks man :]

You could make it a bit faster and take out the:

;Strip any cr's and lf's off the end until we are at the last line of code:

$s_temp = StringRegExpReplace($s_temp, "[\r|\n]+\z", "")

completely, the other regex won't allow blank lines anyway.

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Link to comment
Share on other sites
CosmosTunes Seeker

CosmosTunes

Posted
  • Author
Posted (edited)

You could make it a bit faster and take out the:

;Strip any cr's and lf's off the end until we are at the last line of code:

$s_temp = StringRegExpReplace($s_temp, "[\r|\n]+\z", "")

completely, the other regex won't allow blank lines anyway.

true give me same array.

time 0.20 seconds ;]

need to dig deeper into regex i think :P

Edited by CosmosTunes
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...