Read multiple files into 1 array

[GEOSoft]

Here is the gist of what I' m doing and what I need to do.

First I Use _FileReadToArray($File,$My_Array) ;; Not the right variable names

Then I run a function against that elements of $My_Array. It works like a charm because under normal circumstances all of the strings I need to work with are in 1 File.

Now I have a situation where the file has to be split into 2 or 3 files.

Getting a file list into an array is no problem. (although they are in separate folders) I would put the paths to all the files in the file list array.

I need to read all of the files into the same array'

I'm thinking along the lines of

$T_File = FileOpen(@TempDir & "\Temp.Txt, 1);; Only using FileOpen/Close for speed
For $x = 1 To $FileListArray[0]
  FileWrite($T_File, FileRead($FileListArray[$x]));; Should combine the files
Next
FileClose($T_File)
_FileReadToArray($T_File, $My_Array)
FileDelete($T_File) ;; Won't need it anymore
Function()

Any better ideas?

[LostUser]

You can try using the _ArrayAdd or check the _Array series of commands in the AutoIt help.

[GEOSoft]

You can try using the _ArrayAdd or check the _Array series of commands in the AutoIt help.

Way too slow.

I'd have to write the first file into an array and then the second into a second array and then use _arrayAdd() to move each element of array 2 into the first array and then repeat for each additional file. right now there are only 3 files but there may be more

[Stumpii]

Way too slow.

I'd have to write the first file into an array and then the second into a second array and then use _arrayAdd() to move each element of array 2 into the first array and then repeat for each additional file. right now there are only 3 files but there may be more

I think the way you have it is probably as fast as any other. You could create an array of arrays and then load each file's data into the array contained within each base arrays index.

[GEOSoft]

I think the way you have it is probably as fast as any other. You could create an array of arrays and then load each file's data into the array contained within each base arrays index.

Thanks for the input @Stumpii

Thats a good thought. After I slept on it I decided that FileWrite($Temp,File(FileRead($File[])) was probably as fast as it's going to get. As it stands right now there are just over 6000 lines to be read to the _FileToArray() function and I don't think that using multiple arrays will speed it up. Now if I can just speed up the functions that get called after the array is created, all will be well again.

[barple]

Thanks for the input @Stumpii

Thats a good thought. After I slept on it I decided that FileWrite($Temp,File(FileRead($File[])) was probably as fast as it's going to get. As it stands right now there are just over 6000 lines to be read to the _FileToArray() function and I don't think that using multiple arrays will speed it up. Now if I can just speed up the functions that get called after the array is created, all will be well again.

I just ran into the same problem with speed in combining two files, so I'd suggest:

CODE

#include <Process.au3>

_RunDOS("copy /Y $firstfile $tempfile") ; copy file to a temp file

_RunDOS("type $secondfile >> $tempfile) ; redirect text from second file to append into temp

If Not _FileReadToArray( $tempfile, $mynewarray) Then

MsgBox(0, "error reading file", "file")

Exit

EndIf

This is the fastest method I've found.

Barple

[PsaltyDS]

Deal with assembling the files as strings and only break them into an array once. It's bound to be much faster that way:

#include <array.au3> ; only for _ArrayDisplay()

Dim $avFiles[3] = ["C:\Temp\Temp1.txt", "C:\Temp\Temp2.txt", "C:\Temp\Temp3.txt" ]
Dim $sString = ""
For $n = 0 To UBound($avFiles) - 1
    $sString &= FileRead($avFiles[$n], FileGetSize($avFiles[$n]))
    If StringRight($sString, 2) <> @CRLF Then $sString &= @CRLF
Next
$avLines = StringSplit($sString, @CRLF, 1)

_ArrayDisplay($avLines, "Displaying " & $avLines[0] & " lines from " & UBound($avFiles) & " files.")

Edit: Fix missing paren and handle case where last line of any file has no newline. (Actually tested this time, after harassment by a certain smokey Moderator.)

Edited September 23, 2007 by PsaltyDS

[SmOke_N]

Deal with assembling the files as strings and only break them into an array once. It's bound to be much faster that way:

#include <array.au3> ; only for _ArrayDisplay()

Dim $avFiles[3] = ["C:\Temp1.txt", "C:\Temp2.txt", "C:\Temp3.txt" ]
Dim $sString = ""
For $n = 0 To UBound($avFiles) - 1
    $sString &= FileRead($avFiles[$n], FileGetSize($avFiles[$n])
Next
$avLines = StringSplit($sString, @CRLF, 1)

_ArrayDisplay($avLines, "Displaying " & $avLines[0] & " lines from " & UBound($avFiles) & " files.")oÝ÷ Ù(hºWi®Úny«­¢+ØÀÌØíÍMÑÉ¥¹µÀìô¥±I ÀÌØíÙ¥±ÍlÀÌØí¹t¤µÀì
I1

?

[PsaltyDS]

I claim the weekend, low-coffee, code excuse...

Still, I think the concept was the best way to go for the problem at hand.