Seeking clarification about _FileReadToArray $sDelimiter

[timmy2]

I figure there must be a reason for _FileReadToArray to exist when FileReadToArray can already read an entire text file into an array, but I don't understand the _FileReadToArray examples in the Help File.  The threads about _FileReadToArray here in the forum all seem to end up using StringSplit to work with delimiters, yet _FileReadToArray has some sort of built-in delimiter capability. 

Here's where I was hoping _FileReadToArray might be applicable.  Let's assume the text file I wish to read into my script is like this:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Cras hendrerit tristique purus quis eleifend.
@@
Maecenas interdum dui in magna tincidunt, 
eget imperdiet tellus ornare.
  * Fusc
  * blandit
  * tellu
  * metus
Nulla nec blandit ligula. 
@@
Fusce nec rhoncus enim. Cras facilisis velit velit,
in fringilla felis porttitor sed. Cras vel nisi
lobortis, vestibulum justo quis, lacinia est.
Maecenas eu dolor ac est posuere semper a a enim.

I would like to import the text file into a 1D array, with the @@'s used as delimiters.  So in this example the resulting 1D array would have 3 elements in it. Note that I want to retain the formatting and line endings (CRLF's).

Can _FileReadToArray be used to accomplish this on its own?

Thank you.

[Anoop]

Hi,

Please check the syntax and description.

_FileReadToArray ( $sFilePath, ByRef $vReturn [, $iFlags = $FRTA_COUNT [, $sDelimiter = ""]] )

$sDelimiter -

Used to further split each line of the file - e.g. reading CSV files into a 2D array.

 

So, it splits each line with the delimiter. The delimiter is not for the entire line.

[pluto41]

 

#include <Array.au3>
#include <File.au3>

Local $j = 1, $sLine = ""
Local $aRetArray, $aNewArray[2], $sFile = "c:\Temp\delimiter.txt"   ; <== lorum ipsum test file

; Read file with Count
_FileReadToArray ( $sFile,$aRetArray, $FRTA_COUNT, "" )

For $i = 1 to $aRetArray[0]
    If StringInStr ( $aRetArray[$i], "@@" ) Then
        AddLineToNewArray ( )
    Else
        $sLine &= $aRetArray[$i]
    EndIf
Next

; Add The Last part of the File to the NewArray
AddLineToNewArray ( )

; Display NewArray Result
_ArrayDisplay ( $aNewArray )

Func AddLineToNewArray ( )
        ReDim $aNewArray [$aNewArray [0] + 2]
        $aNewArray[$j] = $sLine
        $aNewArray[0] = $j
        $sLine = ""
        $j += 1
EndFunc

a Workaround.

[AutoBert]

Another workaround:

#include <Array.au3>
#include <File.au3>

Global $aArray[1]
$sFile='lorem.txt'  ;note: if origin file is needed work with a copy 
_ReplaceStringInFile($sFile,@CRLF,'|')
_ReplaceStringInFile($sFile,'@@|',@CRLF)
_FileReadToArray($sFile,$aArray)
_ArrayDisplay($aArray,'"|" = @CRLF')

For $i = 1 To UBound($aArray)-1
    $aArray[$i]=StringReplace($aArray[$i],'|',@CRLF)
Next
_ArrayDisplay($aArray)

But you can use StringSplit for getting same result:

#include <Array.au3>
#include <File.au3>

Global $aArray[1]
$sFile='lorem.txt'
$sData=FileRead($sFile)
$aSplit=StringSplit($sData,'@@'&@CRLF, $STR_ENTIRESPLIT)
_ArrayDisplay($aSplit)

 

[timmy2]

20 hours ago, Anoop said:

Hi,

Please check the syntax and description.

_FileReadToArray ( $sFilePath, ByRef $vReturn [, $iFlags = $FRTA_COUNT [, $sDelimiter = ""]] )

$sDelimiter -

Used to further split each line of the file - e.g. reading CSV files into a 2D array.

 

So, it splits each line with the delimiter. The delimiter is not for the entire line.

@Anoop, what exactly is "each line of the file"? Does this mean up to the first ASCII line ending, like a CR? (I kinda doubt that because the delimiter can be defined.) So, in my example text file, why can't "@" or "@@" be delimiters? 

I see the other much-appreciated replies continue to use stringsplit, so back to my original question: what's the point of the delimiter option? I could must as easily use FileReadToArray and then use stringsplit.

Sorry (truly), but I am not getting it. 

[Anoop]

Hello,

As you have rightly said, _FileReadToArray is with FileReadToArray and StringSplit.

I have copied below the _FileReadToArray function.

expandcollapse popup

Func _FileReadToArray($sFilePath, ByRef $vReturn, $iFlags = $FRTA_COUNT, $sDelimiter = "")
    ; Clear the previous contents
    $vReturn = 0

    If $iFlags = Default Then $iFlags = $FRTA_COUNT
    If $sDelimiter = Default Then $sDelimiter = ""

    ; Set "array of arrays" flag
    Local $bExpand = True
    If BitAND($iFlags, $FRTA_INTARRAYS) Then
        $bExpand = False
        $iFlags -= $FRTA_INTARRAYS
    EndIf
    ; Set delimiter flag
    Local $iEntire = $STR_CHRSPLIT
    If BitAND($iFlags, $FRTA_ENTIRESPLIT) Then
        $iEntire = $STR_ENTIRESPLIT
        $iFlags -= $FRTA_ENTIRESPLIT
    EndIf
    ; Set row count and split count flags
    Local $iNoCount = 0
    If $iFlags <> $FRTA_COUNT Then
        $iFlags = $FRTA_NOCOUNT
        $iNoCount = $STR_NOCOUNT
    EndIf

    ; Check delimiter
    If $sDelimiter Then
        ; Read file into an array
        Local $aLines = FileReadToArray($sFilePath)
        If @error Then Return SetError(@error, 0, 0)

        ; Get first dimension and add count if required
        Local $iDim_1 = UBound($aLines) + $iFlags
        ; Check type of return array
        If $bExpand Then ; All lines have same number of fields
            ; Count fields in first line
            Local $iDim_2 = UBound(StringSplit($aLines[0], $sDelimiter, $iEntire + $STR_NOCOUNT))
            ; Size array
            Local $aTemp_Array[$iDim_1][$iDim_2]
            ; Declare the variables
            Local $iFields, _
                    $aSplit
            ; Loop through the lines
            For $i = 0 To $iDim_1 - $iFlags - 1
                ; Split each line as required
                $aSplit = StringSplit($aLines[$i], $sDelimiter, $iEntire + $STR_NOCOUNT)
                ; Count the items
                $iFields = UBound($aSplit)
                If $iFields <> $iDim_2 Then
                    ; Return error
                    Return SetError(3, 0, 0)
                EndIf
                ; Fill this line of the array
                For $j = 0 To $iFields - 1
                    $aTemp_Array[$i + $iFlags][$j] = $aSplit[$j]
                Next
            Next
            ; Check at least 2 columns
            If $iDim_2 < 2 Then Return SetError(4, 0, 0)
            ; Set dimension count
            If $iFlags Then
                $aTemp_Array[0][0] = $iDim_1 - $iFlags
                $aTemp_Array[0][1] = $iDim_2
            EndIf
        Else ; Create "array of arrays"
            ; Size array
            Local $aTemp_Array[$iDim_1]
            ; Loop through the lines
            For $i = 0 To $iDim_1 - $iFlags - 1
                ; Split each line as required
                $aTemp_Array[$i + $iFlags] = StringSplit($aLines[$i], $sDelimiter, $iEntire + $iNoCount)
            Next
            ; Set dimension count
            If $iFlags Then
                $aTemp_Array[0] = $iDim_1 - $iFlags
            EndIf
        EndIf
        ; Return the array
        $vReturn = $aTemp_Array
    Else ; 1D
        If $iFlags Then
            Local $hFileOpen = FileOpen($sFilePath, $FO_READ)
            If $hFileOpen = -1 Then Return SetError(1, 0, 0)
            Local $sFileRead = FileRead($hFileOpen)
            FileClose($hFileOpen)

            If StringLen($sFileRead) Then
                $vReturn = StringRegExp(@LF & $sFileRead, "(?|(\N+)\z|(\N*)(?:\R))", 3)
                $vReturn[0] = UBound($vReturn) - 1
            Else
                Return SetError(2, 0, 0)
            EndIf
        Else
            $vReturn = FileReadToArray($sFilePath)
            If @error Then
                $vReturn = 0
                Return SetError(@error, 0, 0)
            EndIf
        EndIf

    EndIf
    Return 1
EndFunc   ;==>_FileReadToArray

 

[Melba23]

timmy2,

FileReadToArray is an internal (and hence very fast) AutoIt function which reads a file into an 1D array with each line (determined by the use of @CR, @LF or @CRLF EOL markers) in a separate element.

_FileReadToArray is a function within the standard File.au3 library which will also allow you to read a file into an array, but if used with the optional $sDelimiter parameter it also allows each line to be further split, resulting in a 2D array - as explained in the Help file, this is designed for use with CSV or similarly formatted files. This added functionality is written in AutoIt code and so the function is (as would be expected) slower that the internal function if all you need is to split the file at a standard EOL marker.

If you wish to split a file with no EOL markers but on a delimiter such as "@" or "@@" then you need to use StringSplit.

Clearer now?

M23

[timmy2]

Thank you, @Melba23.  Yes, clearer now.  I think what left me in doubt is the statement, "If a delimiter is not specified... (EOL will be used)". I assumed that meant if I DO supply a delimiter it will be the sole delimiter, and therefore the function might split up the text in my example file.

I believe I recall seeing an old post by you asking for comments about this function's Help file.  I was tempted to add a comment (with all due respect - and some bowing and scraping because you've been extremely helpful in the past). It would be wonderful if the help file could include real-world-for-the-layman examples, in addition to the examples that, while craftily coded, sometimes require study and reverse-engineering.

I frequently refer to the help file to see if a function is applicable to my goal.  I'm shopping for the right "tool" and just want to know what a function does. The descriptions are sometimes a bit cryptic so I jump to the examples in hopes of understanding what the function does. If the example requires a time and mental investment, it is akin to an online shopper being confronted with riddles instead of quick answers. Truly, no offense intended. This is just feedback from someone who eventually "gets it", but sometimes needs a little elaboration.

[timmy2]

On 9/17/2016 at 3:28 AM, AutoBert said:

But you can use StringSplit for getting same result:

#include <Array.au3>
#include <File.au3>

Global $aArray[1]
$sFile='lorem.txt'
$sData=FileRead($sFile)
$aSplit=StringSplit($sData,'@@'&@CRLF, $STR_ENTIRESPLIT)
_ArrayDisplay($aSplit)

It can't get any easier than your suggested code.  Thank you @AutoBert !  Although not perfectly apt, the aphorism, "When you hear hoofbeats, think of horses not zebras" comes to mind.  I clearly over-thought this.

[Melba23]

timmy2,

Quote

I think what left me in doubt is the statement, "If a delimiter is not specified... (EOL will be used)".

I do not see any such comment in the Help file. What I see is:

$sDelimiter [optional] : Used to further split each line of the file - e.g. reading CSV files into a 2D array 

Remarks

If a delimiter is not specified the function returns a 1D array with each element holding a line of the file - line endings can be any mix of @CR, @LF and @CRLF.

When a delimiter is specified the function tries to further split each line of the file - how this is done depends on the setting of the $FRTA_INTARRAYS flag.....

which is more or less what I described above and which seems perfectly clear to me.

Quote

It would be wonderful if the help file could include real-world-for-the-layman examples, in addition to the examples that, while craftily coded, sometimes require study and reverse-engineering.

Examples that require some study are usually the best in my experience as they force the potential user to give a modicum of thought as to the function usage. This tends to give better results than a simple "copy/paste" of a code immediately followed by a "WTF" as it then does not do what was expected.  What would you propose as a "real-world-for-the-layman example" in addition those currently used?

M23