Sign in to follow this  
Followers 0
nssatomic

FileReadLine issue with Progress

14 posts in this topic

Hello hopefully i'm posting this right. I'm having an issue with my script where it wants to read the same line over and over again for the number of lines I have in a file. Once it goes through this it will go to the next line and repeat the process. Your thoughts?

#include <File.au3>
#include <GuiStatusBar.au3>
#include <GUIConstantsEx.au3>
#include <ProgressConstants.au3>

$Source      = "C:\file.txt"
$total       = _FileCountLines($Source)
$F           = FileOpen($Source, 0)

$answer = MsgBox(4, "FileCopy", "Copying files. Note: Make sure you have C:\output.txt ready. Continue?")

If $answer = 7 Then
    MsgBox(0, "Setup cancelled", "Click OK to exit out of this window.")
    Exit
EndIf

If $Source = -1 Then
    MsgBox(0, "Error", "C:\output.txt not found or is unreadable.")
    Exit
EndIf

$GUI         = GUICreate("Registry Delete",400,60,-1,-1)
$GUIProgress = GUICtrlCreateProgress (10,10,380,40)
GUISetState (@SW_SHOW)

while 1
    $line = FileReadLine($F)
    If @error = -1 Then ExitLoop
    For $count = 0 To $total
    $iPercentage = ($count/$total)*100
    filecopy($line, "C:\Temp\test")
    GUICtrlSetData($GUIProgress, $iPercentage)
    Sleep(2000)
    MsgBox(0, "Progress", "Copied file "& $line) ; added this temporarily to see what is happening
Next

WEnd

FileClose($F)

Share this post


Link to post
Share on other sites



nssatomic,

First, welcome to the AutoIt forums. A good start - some code to work on and a clear question. I wish some other newcomers would do the same. :D

Your problem is the unneccesary For...Next loop. FileReadLine will automatically move onto the next line, so you do not need to set up a loop to run through the file yourself. However, you do need a variable to count the lines as you go through so you can get a percentage for your progressbar. Take a look at this much reduced version of your code (I have only cut things out for clarity - not because they were wrong!):

#include <File.au3>
#include <GUIConstantsEx.au3>

$Source      = "M:\Program\Au3 Scripts\Texts\Newbie.txt"
$total       = _FileCountLines($Source)
$F           = FileOpen($Source, 0)

$GUI         = GUICreate("Registry Delete",400,60,-1,-1)
$GUIProgress = GUICtrlCreateProgress (10,10,380,40)
GUISetState (@SW_SHOW)

$count = 0

While 1
    $line = FileReadLine($F)
    If @error = -1 Then ExitLoop
    ;For $count = 0 To $total ; <<<<<<<<<<<<<<<<<<<<<<<<  not needed
    $count += 1
    $iPercentage = ($count/$total)*100
    ;filecopy($line, "C:\Temp\test")
    GUICtrlSetData($GUIProgress, $iPercentage)
    Sleep(200)  ; I have cut the delay down a bit or it takes forever to finish!!!!!!!!!!
    ;MsgBox(0, "Progress", "Copied file "& $line) ; added this temporarily to see what is happening
    ;Next  ; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< not needed

WEnd

FileClose($F)

I hope everything is clear - please ask if not.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

That damn FileReadLine() shold be removed from the language. It's the slowest possible way there is to read a file line by line. It's only decent use is when you want to read a particular line and you know what the line number is so it cann be called using the actual line number.

If you have to read the file one line at a time then use _FileReadToArray() or even just a simple SRE coupled with FileRead()

#include<array.au3> ;; Used for testing with _ArrayDisplay() only
$aArray = _StringToArray2("C:\file.txt")
_ArrayDisplay($aArray) ;; display the results of the 0 based array
Func _StringToArray2($sStr)
   If FileExists($sStr) Then $sStr = StringStripWS(FileRead($sStr), 2)
   If $sStr = "" Then Return SetError(1) ;; No string content
   $sRegEx = "(?i)(.+)\v?"
   $aRegEx = StringRegExp($sStr, $sRegEx,3)
   If @Error Then Return SetError(2);; No RegExp match
   Return $aRegEx
EndFunc

EDIT: Please note that

i > the array returned is zero based so watch that in your loop

ii > _StringToArray2() is virtually a replacement for _FileReadToArray() except that it will accept a string input as well as a file path\name

Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

George,

Noted. Although I am not too convinced by "just a simple SRE coupled with FileRead()". :D

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

George,

Noted. Although I am not too convinced by "just a simple SRE coupled with FileRead()". :D

M23

Maybe this will convince you

Func Test($sFile)
    If Not FileExists($sFile) Then Return SetError(1)
    Return StringRegExp(FileRead($sFile), "(.+)\v?", 3)
EndFunc

EDIT:

BTW, if you want to do the same thing but ignore blank lines just use this one. NOTE: NOT to be used with Progress

Func Test($sFile)
    If NOT FileExists($sFile) Then Return SetError(1)
    Return StringRegExp(FileRead($sFile), "([[:print:]]+)\v?", 3)
EndFunc
Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

That damn FileReadLine() shold be removed from the language. It's the slowest possible way there is to read a file line by line. It's only decent use is when you want to read a particular line and you know what the line number is so it cann be called using the actual line number.

If you have to read the file one line at a time then use _FileReadToArray() or even just a simple SRE coupled with FileRead()

I would agree that _FileReadToArray() is the the simplest way to load a file fast that you want to process line by line. However I would disagree with comment that FileReadLine() shold be removed from the language. I do find that I often have to resort to using FileReadLine() (with a file handle) when working with very large files (20 - 40 million lines and up to 1GB) on a 32bit OS you just can't install enough RAM to to be able to use the _FileReadToArray() method.


"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

I would agree that _FileReadToArray() is the the simplest way to load a file fast that you want to process line by line. However I would disagree with comment that FileReadLine() shold be removed from the language. I do find that I often have to resort to using FileReadLine() (with a file handle) when working with very large files (20 - 40 million lines and up to 1GB) on a 32bit OS you just can't install enough RAM to to be able to use the _FileReadToArray() method.

That comment is from frustration caused by the number of people who attempt to use it where it is not even close to being a reasonable solution and with how slow that function is to begin with. I see far too many posters attempting to use it in the wrong circumstances. When feasable, an array is the better (and much faster) method to use. Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

George,

OK, I am convinced! :D

I am reminded of one of Malkey's recent posts in the Examples forum where he mentions "thinking outside the [...] box" when using SREs. I find the "Could I use an SRE here?" part of the learning curve as steep as the actual SRE construction - which why I so appreciate helpful pointers such as yours above. Thanks.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Of course the flip side of that coin is "am I over using SREs?" That's also a big danger with them. I've also noticed that a lot of UDFs get overused to the point where someone #include<>s a huge UDF when they really only needed one of the functions (10 or 12 lines) that could probably have been written into the script with 4 or 5 lines of code.

I'm currently toying with the idea of taking each function from the UDFs (SREs to the rescue again) and turning them into individual scripts. Nothing is wrong with that concept, it's still reusable code, and of course it will mean writing more lines like #include<_ArrayAdd.au3> but it will still be a lot less code than including files like WinAPI.au3 or some of those other monsters.


George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

George,

Absolutely in agreement on the include files. I always regard Misc.au3 is a particularly bad case, although I agree that WinAPI.au3 must be pretty high up the list.

However, before I compile anything I always use the /StripOnly parameter in Obfuscator to remove unused functions and variables from the includes. That way I can get just the functions I want and not the massive overhead.

I realise this is probably teaching you to suck eggs, but I thought I might mention it for the enlightenment of any new members reading our conversation!

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

George,

Absolutely in agreement on the include files. I always regard Misc.au3 is a particularly bad case, although I agree that WinAPI.au3 must be pretty high up the list.

However, before I compile anything I always use the /StripOnly parameter in Obfuscator to remove unused functions and variables from the includes. That way I can get just the functions I want and not the massive overhead.

I realise this is probably teaching you to suck eggs, but I thought I might mention it for the enlightenment of any new members reading our conversation!

M23

I'm that person who steadfastly doesn't use Scite remember. I don't like things being added to my scripts and I don't like apps making my decisions for me and I particularily don't like apps that hijack my file associations without asking me first.

Edit, I should add that the other method I'm looking at does virtually the same as Obfuscators /StripOnly except that it reads the script and decides what functions to include

Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

George,

Ah yes, I forgot about you and and the SciTE package. :D

I await the first instalment of the GEOSoft includes with interest. If I can be of any help.....

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

George,

Ah yes, I forgot about you and and the SciTE package. :D

I await the first instalment of the GEOSoft includes with interest. If I can be of any help.....

M23

I already have the SRE that gets the functions and creates the files. I just have to do the one that looks for other functions that are called and then adds #include<funcname.au3> to the top of each file. In fact I may not even give them the au3 extension, maybe something like .ah

EDIT: I guess I should point out AGAIN that SciTe is a great package and in fact I do use it when I'm running in Win 7, only because I haven't installed any other editors there yet.

Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

nssatomic,

First, welcome to the AutoIt forums. A good start - some code to work on and a clear question. I wish some other newcomers would do the same. :D

Your problem is the unneccesary For...Next loop. FileReadLine will automatically move onto the next line, so you do not need to set up a loop to run through the file yourself. However, you do need a variable to count the lines as you go through so you can get a percentage for your progressbar. Take a look at this much reduced version of your code (I have only cut things out for clarity - not because they were wrong!):

#include <File.au3>
#include <GUIConstantsEx.au3>

$Source      = "M:\Program\Au3 Scripts\Texts\Newbie.txt"
$total       = _FileCountLines($Source)
$F           = FileOpen($Source, 0)

$GUI         = GUICreate("Registry Delete",400,60,-1,-1)
$GUIProgress = GUICtrlCreateProgress (10,10,380,40)
GUISetState (@SW_SHOW)

$count = 0

While 1
    $line = FileReadLine($F)
    If @error = -1 Then ExitLoop
    ;For $count = 0 To $total ; <<<<<<<<<<<<<<<<<<<<<<<<  not needed
    $count += 1
    $iPercentage = ($count/$total)*100
    ;filecopy($line, "C:\Temp\test")
    GUICtrlSetData($GUIProgress, $iPercentage)
    Sleep(200)  ; I have cut the delay down a bit or it takes forever to finish!!!!!!!!!!
    ;MsgBox(0, "Progress", "Copied file "& $line) ; added this temporarily to see what is happening
    ;Next  ; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< not needed

WEnd

FileClose($F)

I hope everything is clear - please ask if not.

M23

Thanks that worked great!

@ GEOSoft: I'll play around with _StringToArray. I'm sure it will come in handy when I have several thousand lines to read from. I'll try to work up a script and post it if I have any issues.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0