Sign in to follow this  
Followers 0
GEOSoft

Read multiple files into 1 array

9 posts in this topic

Here is the gist of what I' m doing and what I need to do.

First I Use _FileReadToArray($File,$My_Array) ;; Not the right variable names

Then I run a function against that elements of $My_Array. It works like a charm because under normal circumstances all of the strings I need to work with are in 1 File.

Now I have a situation where the file has to be split into 2 or 3 files.

Getting a file list into an array is no problem. (although they are in separate folders) I would put the paths to all the files in the file list array.

I need to read all of the files into the same array'

I'm thinking along the lines of

$T_File = FileOpen(@TempDir & "\Temp.Txt, 1);; Only using FileOpen/Close for speed
For $x = 1 To $FileListArray[0]
  FileWrite($T_File, FileRead($FileListArray[$x]));; Should combine the files
Next
FileClose($T_File)
_FileReadToArray($T_File, $My_Array)
FileDelete($T_File) ;; Won't need it anymore
Function()

Any better ideas?


George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites



You can try using the _ArrayAdd or check the _Array series of commands in the AutoIt help.


Be open minded but not gullible.A hammer sees everything as a nail ... so don't be A tool ... be many tools.

Share this post


Link to post
Share on other sites

You can try using the _ArrayAdd or check the _Array series of commands in the AutoIt help.

Way too slow.

I'd have to write the first file into an array and then the second into a second array and then use _arrayAdd() to move each element of array 2 into the first array and then repeat for each additional file. right now there are only 3 files but there may be more


George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

Way too slow.

I'd have to write the first file into an array and then the second into a second array and then use _arrayAdd() to move each element of array 2 into the first array and then repeat for each additional file. right now there are only 3 files but there may be more

I think the way you have it is probably as fast as any other. You could create an array of arrays and then load each file's data into the array contained within each base arrays index.


“Give a man a script; you have helped him for today. Teach a man to script; and you will not have to hear him whine for help.”AutoIt4UE - Custom AutoIt toolbar and wordfile for UltraEdit/UEStudio users.AutoIt Graphical Debugger - A graphical debugger for AutoIt.SimMetrics COM Wrapper - Calculate string similarity.

Share this post


Link to post
Share on other sites

I think the way you have it is probably as fast as any other. You could create an array of arrays and then load each file's data into the array contained within each base arrays index.

Thanks for the input @Stumpii

Thats a good thought. After I slept on it I decided that FileWrite($Temp,File(FileRead($File[])) was probably as fast as it's going to get. As it stands right now there are just over 6000 lines to be read to the _FileToArray() function and I don't think that using multiple arrays will speed it up. Now if I can just speed up the functions that get called after the array is created, all will be well again.


George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

Thanks for the input @Stumpii

Thats a good thought. After I slept on it I decided that FileWrite($Temp,File(FileRead($File[])) was probably as fast as it's going to get. As it stands right now there are just over 6000 lines to be read to the _FileToArray() function and I don't think that using multiple arrays will speed it up. Now if I can just speed up the functions that get called after the array is created, all will be well again.

I just ran into the same problem with speed in combining two files, so I'd suggest:

CODE
#include <Process.au3>

_RunDOS("copy /Y $firstfile $tempfile") ; copy file to a temp file

_RunDOS("type $secondfile >> $tempfile) ; redirect text from second file to append into temp

If Not _FileReadToArray( $tempfile, $mynewarray) Then

MsgBox(0, "error reading file", "file")

Exit

EndIf

This is the fastest method I've found.

Barple

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

Deal with assembling the files as strings and only break them into an array once. It's bound to be much faster that way:

#include <array.au3> ; only for _ArrayDisplay()

Dim $avFiles[3] = ["C:\Temp\Temp1.txt", "C:\Temp\Temp2.txt", "C:\Temp\Temp3.txt" ]
Dim $sString = ""
For $n = 0 To UBound($avFiles) - 1
    $sString &= FileRead($avFiles[$n], FileGetSize($avFiles[$n]))
    If StringRight($sString, 2) <> @CRLF Then $sString &= @CRLF
Next
$avLines = StringSplit($sString, @CRLF, 1)

_ArrayDisplay($avLines, "Displaying " & $avLines[0] & " lines from " & UBound($avFiles) & " files.")

:)

Edit: Fix missing paren and handle case where last line of any file has no newline. (Actually tested this time, after harassment by a certain smokey Moderator.) :)

Edited by PsaltyDS

Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites

Deal with assembling the files as strings and only break them into an array once. It's bound to be much faster that way:

#include <array.au3> ; only for _ArrayDisplay()

Dim $avFiles[3] = ["C:\Temp1.txt", "C:\Temp2.txt", "C:\Temp3.txt" ]
Dim $sString = ""
For $n = 0 To UBound($avFiles) - 1
    $sString &= FileRead($avFiles[$n], FileGetSize($avFiles[$n])
Next
$avLines = StringSplit($sString, @CRLF, 1)

_ArrayDisplay($avLines, "Displaying " & $avLines[0] & " lines from " & UBound($avFiles) & " files.")oÝ÷ Ù(hºWi®Úny«­¢+ØÀÌØíÍMÑÉ¥¹µÀìô¥±I ÀÌØíÙ¥±ÍlÀÌØí¹t¤µÀì
I1
?

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

I claim the weekend, low-coffee, code excuse... :)

Still, I think the concept was the best way to go for the problem at hand.

:)


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0