Sign in to follow this  
Followers 0
RogueX20

Best Method: Loop or Array?

15 posts in this topic

#1 ·  Posted (edited)

Been learning a lot lately, but the learning curve is still high. Working hard to graduate :huh2: from Newb status.

Anyway with the help of a few folks I am almost done with a very useful app.

Now my question is, to loop or array?

I have a lot of files I go through. Actually a bunch of us. IT is still working on coming up with a way to get this done without using us.

Should I use a loop to go through each file at a time and read the specific line and write to a text(maybe excel) then go to the next file until I reach the last file.

Or should I use an array to read and capture all the specific lines and write to a text(or excel).

By the way what is the max an array can hold?

Edited by RogueX20

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

As usual - it depends.

Here (Datatypes and Ranges) or here (What are the current technical limits of AutoIt v3?) you can find more information about limits.

I don't think you will run into an AutoIt limit but rather memory or processing time constraints.

How many files do we talk about? What's the maximum size of a file in MB or number of lines?

If we have more information we can try to show you how to solve the problem.

Edited by water

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Not sure I understand the question "loop or array" as these have different purposes.

So you need to scan through a large number of files... but for what exactly?

The scan through a directory you use FileFindFirst()/FileFindNext().

J0s


Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites

By the way what is the max an array can hold?

From the Helpfile:

You can use up to 64 dimensions in an Array. The total number of entries cannot be greater than 2^24 (16 777 216).

As for loop or array, I couldn't suggest anything without seeing your code, I'd probably be wrong anyway..


- Bruce /*somdcomputerguy */  If you change the way you look at things, the things you look at change.

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

I haven't started on the code as far as getting the info from the group of files. I'm still reading up on on that. Wanted to know which one I should focus on. I will try the array route. The files can be anywhere from 5 thousand to 30 thousand. Reading the 4th line in the files which add up to anywhere between 85 to 100 characters.

This needs more work obviously but with this working I can build and build until it does what we need. I'm going to use if first for a while on the down low. :huh2:

#include <Array.au3>

$file = FileOpen("file1210.txt", 0)

; Check if file opened for reading OK
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Read in lines of text until the EOF is reached

    $line = FileReadLine($file,4)
    ;$len = Stringlen($line)

    MsgBox(0, "Line read: ", $line)
    ;msgbox(0, "Length: ",$len)

FileClose($file)



$aArray = StringRegExp($line, "Department (.+) has the following charge of (.+) for (.+) purchased on (.+),(.+) released to (.+)", 3)
_ArrayDisplay($aArray)
Edited by RogueX20

Share this post


Link to post
Share on other sites

Not sure I understand the question "loop or array" as these have different purposes.

So you need to scan through a large number of files... but for what exactly?

The scan through a directory you use FileFindFirst()/FileFindNext().

J0s

Need to retrieve a specific line from these files and paste to a main file. I was going to work with array but I think the FileFindFirst and FileFindNext functions might be easier for me at this point in time.

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

I'm guessing here a little (if I understood the question correctly), but I think it all depends on how many lines you need to read and the size of the file. Reading the whole file to an array can be useful if you need to parse several lines with multiple passes or a similar batch process. If it is just one line you want to read from each file, then I don't see any point in reading the whole file. You will probably want to use loops in either case. :huh2:

Edit

Until I read your last post, I see I didn't quite understand what you were asking.

Edited by czardas

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

Need to retrieve a specific line from these files and paste to a main file. I was going to work with array but I think the FileFindFirst and FileFindNext functions might be easier for me at this point in time.

When it it a fixed record number that needs to be read you can use FileReadLine(name,RecNr) for each file found in the FileFindNextFile() loop. Edited by Jos

Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites

I'm guessing here a little (if I understood the question correctly), but I think it all depends on how many lines you need to read and the size of the file. Reading the whole file to an array can be useful if you need to parse several lines with multiple passes or a similar batch process. If it is just one line you want to read from each file, then I don't see any point in reading the whole file. You will probably want to use loops in either case. :huh2:

Edit

Until I read your last post, I see I didn't quite understand what you were asking.

I'm reading a particular line, not the entire file. Working on the loop method right now. This is all very new to me, but I think I can get this part done in a little bit. Thanks

Share this post


Link to post
Share on other sites

Okay I get it. The while loop in the example for FileFindNextFile is a good example to work with. You just need to add a few lines of code inside that loop. If it doesn't work, post the code and we'll try and fix it.

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

Okay I get it. The while loop in the example for FileFindNextFile is a good example to work with. You just need to add a few lines of code inside that loop. If it doesn't work, post the code and we'll try and fix it.

I'm stuck. I need to have the name of the file copied and written into the first column & row and have the line that is read, written in the same row for that file. The line that is read is broken up into an array. Got help on the array part from another member and thanks to that member! Need only certain parts of the line. Only getting the first part of each array. I see you can choose which part as well, but I need the entire array.

I need to learn how to have the array write across the column and not down the row as well.

#include <Excel.au3>
#include <Array.au3>

$sFilePath1 = @ScriptDir & "\MainFile.xlsx"
$oExcel = _ExcelBookOpen($sFilePath1)

If @error = 1 Then
    MsgBox(0, "Error!", "Unable to Create the Excel Object")
    Exit
ElseIf @error = 2 Then
    MsgBox(0, "Error!", "File does not exist")
    Exit
EndIf


$search = FileFindFirstFile("y*.txt")
$i = 0


If $search = -1 Then
MsgBox(0, "Error", "No files/directories matched the search pattern")
Exit
EndIf

While 1


$file = FileFindNextFile($search)
$fileRead = FileOpen($file, 0)
$line = FileReadLine($fileRead,4)
;testing to see what shows up in the line
MsgBox(0, "Line read: ", $line)
;ie - Department XYZ has the following charge of 112.00 for Widget purchased on Oct 22, 2010 released to John Doe
$aArray = StringRegExp($line, "Department (.+) has the following charge of (.+) for (.+) purchased on (.+) released to (.+)", 3)
_ArrayDisplay($aArray)
If @error Then ExitLoop

$i = ($i + 1)
;looking to copy the name of the file and the info in the array into the excel file
$writevalue = $aArray & $file

_ExcelWriteCell($oExcel, $writevalue, $i, 1)
WEnd

FileClose($search)

Think I need to do some more reading on excel. Need to take a break first. Get a bite to eat and maybe catch a movie to get a mental break. This is literally like learning a language. :huh2:

Edited by RogueX20

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

You forgot to close $fileRead :huh2:

Edit:

And you are not checking if there's no more files for FileFindNextFile, you need to add something like "If @error Then ExitLoop", see helpfile.

And that FileOpen probably should't be there.

And checking @error for _ArrayDisplay is kinda weird.

Edited by AdmiralAlkex

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

You forgot to close $fileRead ;)

Edit:

And you are not checking if there's no more files for FileFindNextFile, you need to add something like "If @error Then ExitLoop", see helpfile.

And that FileOpen probably should't be there.

And checking @error for _ArrayDisplay is kinda weird.

Ok thanks for the input. Things are going to look somewhat sloppy. It will get cleaner I guess with time. :huh2:

Edited by RogueX20

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

Roquex20,

Also,

This code

$writevalue = $aArray & $file
creates an array named $writevalue. The concatenation appears to do nothing..

The following code may be closer th what you want.

$writevalue = _arraytostring($aArray) &  $file

kylomas

Edited by kylomas

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

Roquex20,

Also,

This code

$writevalue = $aArray & $file
creates an array named $writevalue. The concatenation appears to do nothing..

The following code may be closer th what you want.

$writevalue = _arraytostring($aArray) &  $file

kylomas

Thanks. This is getting me closer. Going to read up on sending arrays to excel in a particular order. I would like each value in the array to land in its own cell. I tried to save as a CSV and import using | as the delimeter, but it doesn't work.

This is getting me closer though, as I realize I need to work more with the array functions.

******* update ******

Found where to add a delimeter. Will work with that to see if I can get it to save in a way I can re-import into excel into is own column.

******* started new thread ******

http://www.autoitscript.com/forum/index.php?app=forums&module=post&section=post&do=new_post&f=2

Edited by RogueX20

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0