timmy2

Seeking clarification about _FileReadToArray $sDelimiter

10 posts in this topic

I figure there must be a reason for _FileReadToArray to exist when FileReadToArray can already read an entire text file into an array, but I don't understand the _FileReadToArray examples in the Help File.  The threads about _FileReadToArray here in the forum all seem to end up using StringSplit to work with delimiters, yet _FileReadToArray has some sort of built-in delimiter capability. 

Here's where I was hoping _FileReadToArray might be applicable.  Let's assume the text file I wish to read into my script is like this:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Cras hendrerit tristique purus quis eleifend.
@@
Maecenas interdum dui in magna tincidunt, 
eget imperdiet tellus ornare.
  * Fusc
  * blandit
  * tellu
  * metus
Nulla nec blandit ligula. 
@@
Fusce nec rhoncus enim. Cras facilisis velit velit,
in fringilla felis porttitor sed. Cras vel nisi
lobortis, vestibulum justo quis, lacinia est.
Maecenas eu dolor ac est posuere semper a a enim.

I would like to import the text file into a 1D array, with the @@'s used as delimiters.  So in this example the resulting 1D array would have 3 elements in it. Note that I want to retain the formatting and line endings (CRLF's).

Can _FileReadToArray be used to accomplish this on its own?

Thank you.

Share this post


Link to post
Share on other sites



Hi,

Please check the syntax and description.

_FileReadToArray ( $sFilePath, ByRef $vReturn [, $iFlags = $FRTA_COUNT [, $sDelimiter = ""]] )
$sDelimiter - 

Used to further split each line of the file - e.g. reading CSV files into a 2D array.

 

So, it splits each line with the delimiter. The delimiter is not for the entire line.

Share this post


Link to post
Share on other sites

 

#include <Array.au3>
#include <File.au3>

Local $j = 1, $sLine = ""
Local $aRetArray, $aNewArray[2], $sFile = "c:\Temp\delimiter.txt"   ; <== lorum ipsum test file

; Read file with Count
_FileReadToArray ( $sFile,$aRetArray, $FRTA_COUNT, "" )

For $i = 1 to $aRetArray[0]
    If StringInStr ( $aRetArray[$i], "@@" ) Then
        AddLineToNewArray ( )
    Else
        $sLine &= $aRetArray[$i]
    EndIf
Next

; Add The Last part of the File to the NewArray
AddLineToNewArray ( )

; Display NewArray Result
_ArrayDisplay ( $aNewArray )

Func AddLineToNewArray ( )
        ReDim $aNewArray [$aNewArray [0] + 2]
        $aNewArray[$j] = $sLine
        $aNewArray[0] = $j
        $sLine = ""
        $j += 1
EndFunc

a Workaround.

Share this post


Link to post
Share on other sites

Another workaround:

#include <Array.au3>
#include <File.au3>

Global $aArray[1]
$sFile='lorem.txt'  ;note: if origin file is needed work with a copy 
_ReplaceStringInFile($sFile,@CRLF,'|')
_ReplaceStringInFile($sFile,'@@|',@CRLF)
_FileReadToArray($sFile,$aArray)
_ArrayDisplay($aArray,'"|" = @CRLF')

For $i = 1 To UBound($aArray)-1
    $aArray[$i]=StringReplace($aArray[$i],'|',@CRLF)
Next
_ArrayDisplay($aArray)

But you can use StringSplit for getting same result:

#include <Array.au3>
#include <File.au3>

Global $aArray[1]
$sFile='lorem.txt'
$sData=FileRead($sFile)
$aSplit=StringSplit($sData,'@@'&@CRLF, $STR_ENTIRESPLIT)
_ArrayDisplay($aSplit)

 

1 person likes this

Share this post


Link to post
Share on other sites
20 hours ago, Anoop said:

Hi,

Please check the syntax and description.

_FileReadToArray ( $sFilePath, ByRef $vReturn [, $iFlags = $FRTA_COUNT [, $sDelimiter = ""]] )
$sDelimiter - 

Used to further split each line of the file - e.g. reading CSV files into a 2D array.

 

So, it splits each line with the delimiter. The delimiter is not for the entire line.

@Anoop, what exactly is "each line of the file"? Does this mean up to the first ASCII line ending, like a CR? (I kinda doubt that because the delimiter can be defined.) So, in my example text file, why can't "@" or "@@" be delimiters? 

I see the other much-appreciated replies continue to use stringsplit, so back to my original question: what's the point of the delimiter option? I could must as easily use FileReadToArray and then use stringsplit.

Sorry (truly), but I am not getting it. 

Share this post


Link to post
Share on other sites

Hello,

As you have rightly said, _FileReadToArray is with FileReadToArray and StringSplit.

I have copied below the _FileReadToArray function.

Func _FileReadToArray($sFilePath, ByRef $vReturn, $iFlags = $FRTA_COUNT, $sDelimiter = "")
    ; Clear the previous contents
    $vReturn = 0

    If $iFlags = Default Then $iFlags = $FRTA_COUNT
    If $sDelimiter = Default Then $sDelimiter = ""

    ; Set "array of arrays" flag
    Local $bExpand = True
    If BitAND($iFlags, $FRTA_INTARRAYS) Then
        $bExpand = False
        $iFlags -= $FRTA_INTARRAYS
    EndIf
    ; Set delimiter flag
    Local $iEntire = $STR_CHRSPLIT
    If BitAND($iFlags, $FRTA_ENTIRESPLIT) Then
        $iEntire = $STR_ENTIRESPLIT
        $iFlags -= $FRTA_ENTIRESPLIT
    EndIf
    ; Set row count and split count flags
    Local $iNoCount = 0
    If $iFlags <> $FRTA_COUNT Then
        $iFlags = $FRTA_NOCOUNT
        $iNoCount = $STR_NOCOUNT
    EndIf

    ; Check delimiter
    If $sDelimiter Then
        ; Read file into an array
        Local $aLines = FileReadToArray($sFilePath)
        If @error Then Return SetError(@error, 0, 0)

        ; Get first dimension and add count if required
        Local $iDim_1 = UBound($aLines) + $iFlags
        ; Check type of return array
        If $bExpand Then ; All lines have same number of fields
            ; Count fields in first line
            Local $iDim_2 = UBound(StringSplit($aLines[0], $sDelimiter, $iEntire + $STR_NOCOUNT))
            ; Size array
            Local $aTemp_Array[$iDim_1][$iDim_2]
            ; Declare the variables
            Local $iFields, _
                    $aSplit
            ; Loop through the lines
            For $i = 0 To $iDim_1 - $iFlags - 1
                ; Split each line as required
                $aSplit = StringSplit($aLines[$i], $sDelimiter, $iEntire + $STR_NOCOUNT)
                ; Count the items
                $iFields = UBound($aSplit)
                If $iFields <> $iDim_2 Then
                    ; Return error
                    Return SetError(3, 0, 0)
                EndIf
                ; Fill this line of the array
                For $j = 0 To $iFields - 1
                    $aTemp_Array[$i + $iFlags][$j] = $aSplit[$j]
                Next
            Next
            ; Check at least 2 columns
            If $iDim_2 < 2 Then Return SetError(4, 0, 0)
            ; Set dimension count
            If $iFlags Then
                $aTemp_Array[0][0] = $iDim_1 - $iFlags
                $aTemp_Array[0][1] = $iDim_2
            EndIf
        Else ; Create "array of arrays"
            ; Size array
            Local $aTemp_Array[$iDim_1]
            ; Loop through the lines
            For $i = 0 To $iDim_1 - $iFlags - 1
                ; Split each line as required
                $aTemp_Array[$i + $iFlags] = StringSplit($aLines[$i], $sDelimiter, $iEntire + $iNoCount)
            Next
            ; Set dimension count
            If $iFlags Then
                $aTemp_Array[0] = $iDim_1 - $iFlags
            EndIf
        EndIf
        ; Return the array
        $vReturn = $aTemp_Array
    Else ; 1D
        If $iFlags Then
            Local $hFileOpen = FileOpen($sFilePath, $FO_READ)
            If $hFileOpen = -1 Then Return SetError(1, 0, 0)
            Local $sFileRead = FileRead($hFileOpen)
            FileClose($hFileOpen)

            If StringLen($sFileRead) Then
                $vReturn = StringRegExp(@LF & $sFileRead, "(?|(\N+)\z|(\N*)(?:\R))", 3)
                $vReturn[0] = UBound($vReturn) - 1
            Else
                Return SetError(2, 0, 0)
            EndIf
        Else
            $vReturn = FileReadToArray($sFilePath)
            If @error Then
                $vReturn = 0
                Return SetError(@error, 0, 0)
            EndIf
        EndIf

    EndIf
    Return 1
EndFunc   ;==>_FileReadToArray

 

Share this post


Link to post
Share on other sites

timmy2,

FileReadToArray is an internal (and hence very fast) AutoIt function which reads a file into an 1D array with each line (determined by the use of @CR, @LF or @CRLF EOL markers) in a separate element.

_FileReadToArray is a function within the standard File.au3 library which will also allow you to read a file into an array, but if used with the optional $sDelimiter parameter it also allows each line to be further split, resulting in a 2D array - as explained in the Help file, this is designed for use with CSV or similarly formatted files. This added functionality is written in AutoIt code and so the function is (as would be expected) slower that the internal function if all you need is to split the file at a standard EOL marker.

If you wish to split a file with no EOL markers but on a delimiter such as "@" or "@@" then you need to use StringSplit.

Clearer now?

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

Share this post


Link to post
Share on other sites

Thank you, @Melba23.  Yes, clearer now.  I think what left me in doubt is the statement, "If a delimiter is not specified... (EOL will be used)". I assumed that meant if I DO supply a delimiter it will be the sole delimiter, and therefore the function might split up the text in my example file.

I believe I recall seeing an old post by you asking for comments about this function's Help file.  I was tempted to add a comment (with all due respect - and some bowing and scraping because you've been extremely helpful in the past). It would be wonderful if the help file could include real-world-for-the-layman examples, in addition to the examples that, while craftily coded, sometimes require study and reverse-engineering.

I frequently refer to the help file to see if a function is applicable to my goal.  I'm shopping for the right "tool" and just want to know what a function does. The descriptions are sometimes a bit cryptic so I jump to the examples in hopes of understanding what the function does. If the example requires a time and mental investment, it is akin to an online shopper being confronted with riddles instead of quick answers. Truly, no offense intended. This is just feedback from someone who eventually "gets it", but sometimes needs a little elaboration.

Share this post


Link to post
Share on other sites
On 9/17/2016 at 3:28 AM, AutoBert said:

But you can use StringSplit for getting same result:

#include <Array.au3>
#include <File.au3>

Global $aArray[1]
$sFile='lorem.txt'
$sData=FileRead($sFile)
$aSplit=StringSplit($sData,'@@'&@CRLF, $STR_ENTIRESPLIT)
_ArrayDisplay($aSplit)

It can't get any easier than your suggested code.  Thank you @AutoBert !  Although not perfectly apt, the aphorism, "When you hear hoofbeats, think of horses not zebras" comes to mind.  I clearly over-thought this.

Share this post


Link to post
Share on other sites

timmy2,

Quote

I think what left me in doubt is the statement, "If a delimiter is not specified... (EOL will be used)".

I do not see any such comment in the Help file. What I see is:

$sDelimiter [optional] : Used to further split each line of the file - e.g. reading CSV files into a 2D array 

Remarks

If a delimiter is not specified the function returns a 1D array with each element holding a line of the file - line endings can be any mix of @CR, @LF and @CRLF.

When a delimiter is specified the function tries to further split each line of the file - how this is done depends on the setting of the $FRTA_INTARRAYS flag.....

which is more or less what I described above and which seems perfectly clear to me.

Quote

It would be wonderful if the help file could include real-world-for-the-layman examples, in addition to the examples that, while craftily coded, sometimes require study and reverse-engineering.

Examples that require some study are usually the best in my experience as they force the potential user to give a modicum of thought as to the function usage. This tends to give better results than a simple "copy/paste" of a code immediately followed by a "WTF" as it then does not do what was expected.  What would you propose as a "real-world-for-the-layman example" in addition those currently used?

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • You
      _FileReadToArray - CSV to Array
      By You
      CSV1:
      "Index","Name","Home Address","Last Love" "1","Rola Takizawa","Nữ hoàng, Nga Nhật","24/04/2016 22:37:00 PM" "2","Saori Hara","Hình Ảnh Nóng, Đức Nhật","21/03/2016 21:07:00 PM" "3","Ozawa","phổ biến nhất","23/04/2016 23:31:00 PM" "3","Akiho Ameri Ichinose","Kanagawa, Ameri, Ichinose","22/04/2016 21:50:00 PM"  
      CSV2:
      Index,Name,Home Address,Last Love 1,Takizawa,"Nữ hoàng, Nga Nhật","24/04/2016 22:37:00 PM" 2,"Saori Hara","Hình Ảnh Nóng, Đức Nhật","21/03/2016 21:07:00 PM" 3,Ozawa,"phổ biến nhất","23/04/2016 23:31:00 PM" 3,"Akiho Ameri Ichinose","Kanagawa, Ameri, Ichinose","22/04/2016 21:50:00 PM"  
      CSV3:
      Index,Name,Home Address,Last Love 1,Takizawa,"Nữ hoàng, Nga Nhật",24/04/2016 22:37:00 PM 2,Saori Hara,"Hình Ảnh Nóng, Đức Nhật",21/03/2016 21:07:00 PM 3,Ozawa,phổ biến nhất,23/04/2016 23:31:00 PM 3,Akiho Ameri Ichinose,"Kanagawa, Ameri, Ichinose",22/04/2016 21:50:00 PM ---------------------------------------------------------------------------------- Global $CSV3 = 'Index,Name,Home Address,Last Love' & @CRLF $CSV2 &= '1,Takizawa,"Nữ hoàng, Nga Nhật",24/04/2016 22:37:00 PM' & @CRLF $CSV2 &= '2,Saori Hara,"Hình Ảnh Nóng, Đức Nhật",21/03/2016 21:07:00 PM' & @CRLF $CSV2 &= '3,Ozawa,phổ biến nhất,23/04/2016 23:31:00 PM' & @CRLF $CSV2 &= '3,Akiho Ameri Ichinose,"Kanagawa, Ameri, Ichinose",22/04/2016 21:50:00 PM' & @CRLF  
      Does not work when have more ","
      #include <GUIConstantsEx.au3> #include <MsgBoxConstants.au3> #include <GUIListView.au3> #include <Array.au3> #include <File.au3> Global $CSV1 = '"Index","Name","Home Address","Last Love"' & @CRLF $CSV1 &= '"1","Rola Takizawa","Nữ hoàng, Nga Nhật","24/04/2016 22:37:00 PM"' & @CRLF $CSV1 &= '"2","Saori Hara","Hình Ảnh Nóng, Đức Nhật","21/03/2016 21:07:00 PM"' & @CRLF $CSV1 &= '"3","Ozawa","phổ biến nhất","23/04/2016 23:31:00 PM"' & @CRLF $CSV1 &= '"3","Akiho Ameri Ichinose","Kanagawa, Ameri, Ichinose","22/04/2016 21:50:00 PM"' & @CRLF Global $CSV2 = "Index,Name,Home Address,Last Love" & @CRLF $CSV2 &= "1,Takizawa,'Nữ hoàng, Nga Nhật','24/04/2016 22:37:00 PM'" & @CRLF $CSV2 &= "2,'Saori Hara','Hình Ảnh Nóng, Đức Nhật','21/03/2016 21:07:00 PM'" & @CRLF $CSV2 &= "3,Ozawa,'phổ biến nhất','23/04/2016 23:31:00 PM'" & @CRLF $CSV2 &= "3,'Akiho Ameri Ichinose','Kanagawa, Ameri, Ichinose','22/04/2016 21:50:00 PM'" & @CRLF Local $hOpen=FileOpen("CSV1.csv",128) FileWrite($hOpen,$CSV1) FileClose($hOpen) Local $hOpen=FileOpen("CSV2.csv",128) FileWrite($hOpen,$CSV2) FileClose($hOpen) Global $rArrayCSV1,$rArrayCSV2 Local $ArrayCSV1 = _FileReadToArray("CSV2.csv", $rArrayCSV1, Default,',') ConsoleWrite( $ArrayCSV1 & ' - ' & 'error = ' & @error & @CRLF) _ArrayDisplay($rArrayCSV1) Local $ArrayCSV2 = _FileReadToArray("CSV2.csv", $rArrayCSV2, Default,',') ConsoleWrite( $ArrayCSV2 & ' - ' & 'error = ' & @error & @CRLF) _ArrayDisplay($rArrayCSV2)  
    • TThomasson
      2D array help
      By TThomasson
      Hi everyone. New guy here. I'm still learning this awesome language and I'm unable to figure this one out from google searches. Heres my problem:
      I'm working on a small application to help users in my environment connect to wireless projectors. To keep this easily updated with new projectors I'm reading the room names and IP addresses from a csv file and putting them into a 2D array. (MeetingRoom1,xxx.xxx.xxx.xxx)
      So far I'm able to read the 0 column and display the room names in a combo box. Where I am stuck is how to take the user's room selection from the gui and associate it with an IP address in the array. After that point I've got things prepared to pass the address to the connection application.  
      Any help you all could provide would be greatly appreciated. 
    • simy8891
      Weird thing or just blind?
      By simy8891
      Hi guys,
      I'm either going crazy or AutoIT is..
      Basically I need to get to the parent folder of @scriptdir, to do that, I used:
      $parentFolder=StringLeft(@ScriptDir, StringInStr(@ScriptDir, "\", 0, -1))  So far, all good. When I use MsgBox to show $parentfolder, I'm indeed able to see the folder above @scriptdir. Now, I've added a subfolderfile to the string with:
      $PPID_Bulk_Big_printers_csv_path=$parentFolder&'Printers\printers_PPID_Big.csv'  And also here, all fine, MsgBox shows me the file location correctly.
      Now, for some weird reason, $PPID_Bulk_Big_printers_csv_path changes again!
      See the complete part of the script below:
      ; CSV to 2D array $parentFolder=StringLeft(@ScriptDir, StringInStr(@ScriptDir, "\", 0, -1)) ;Goes one folder up MsgBox(0,'',@ScriptDir&'----'&$parentFolder&'Printers\printers_PPID_Big.csv') $PPID_Bulk_Big_printers_csv_path=$parentFolder&'Printers\printers_PPID_Big.csv' ;CSV location Dim $PPID_Bulk_Big_printers $PPID_Bulk_Big_filereadtoarray_result = _FileReadToArray($PPID_Bulk_Big_printers_csv_path, $PPID_Bulk_Big_printers) ;$PPID_Bulk_Big_filereadtoarray_result will contain any error (if there's any) - $printers is an array which contains the list of printers MsgBox(0,'',$PPID_Bulk_Big_printers_csv_path) ; Still shows correct path ;Check if there was any error with the _FileReadToArray If $PPID_Bulk_Big_filereadtoarray_result <> 0 Then ;If no error reading the file ;Do Something Else ;If there was an error reading the CSV file MsgBox(0,'hhh',"msg="&$PPID_Bulk_Big_printers_csv_path) MsgBox(48,'Problem detected!','The file "'&$PPID_Bulk_Big_printers_csv_path&'" was not found. Software will exit!') Sleep (50) Exit EndIf So what's happening here is, I'm getting into the ELSE part of the script, so there was an error with reading the file and I'm getting indeed the message box with "Problem detected!" but NOT the message box above?? Why?
      Cheers
    • Command3r
      [HELP] File read all lines except last 1
      By Command3r
      Hello scripters i got another problem reading all lines in txt file except the last one.
      I dont want to write how many lines to read just reading all except the last whatever how many it is.

      My efforts:

      #include <file.au3> $file = @ScriptDir & "\x.txt" $ReadLines = FileReadLine($file) Local $Read _FileReadToArray($file, $Read) For $i = 1 To $Read - 1 ; with-1 or without-1 the result is the same MsgBox(0, $i, $Read - 1) Next
      Question here: "Why first msgbox title is "1" ? and how can i avoid that?"

      #include <file.au3> $file = @ScriptDir & "\x.txt" $ReadLines = FileReadLine($file) Local $Read _FileReadToArray($file, $Read) For $i = 1 To $Read MsgBox(0, $i, $Read[0] -1) ; here in msg's text resolved but in title not :D i think the problem from $i ?? Next