deleting blank lines?

[Dr.Chi]

Can any of you AutoIt gurus out there help me?

I need a script that can take a file, and delete all the empty lines in it.

I have a lot of PHP pages for my site and for some reason (or something I did) there are a lot of unessessary line feeds (carrage returns, whatever) in the pages.

In general there's 4 extra returns between each line of text. However, when I was writing a script to remove them I noticed that on occasion (and randomly) there would be more or less than 4. So my simple ("delete 4 times") AutoIt script won't work.

Can someone help me figure out a way to just remove any lines in a file that has no text in it?

[seandisanti]

Can any of you AutoIt gurus out there help me?

I need a script that can take a file, and delete all the empty lines in it.

I have a lot of PHP pages for my site and for some reason (or something I did) there are a lot of unessessary line feeds (carrage returns, whatever) in the pages.

In general there's 4 extra returns between each line of text.  However, when I was writing a script to remove them I noticed that on occasion (and randomly) there would be more or less than 4.  So my simple ("delete 4 times") AutoIt script won't work.

Can someone help me figure out a way to just remove any lines in a file that has no text in it?

<{POST_SNAPBACK}>

there's probably a UDF for this somewhere, but the way i do it is i'll open the file in read mode, read it line by line with filereadline(), and put the lines i want into another file (or array) and then i'll either close the file and reopen it in write mode(if i have all of the data in an array) or i'll write the lines i want to a new temp file that i can have the script rename to the old filename when it completes. Wasn't sure if you were just looking for a way to do it, or the code to do it, so i went with the easy way out. if you need more lemme know

[Dr.Chi]

But how do you tell it what lines you want (i.e. Non-blank lines?)

I can't do it manually, cause the files are generally very long, like 40-50 pages of code (PHP files for a forum and website).

This may be a little over my head, or I'm just not thinking of it right...I'll probably need someone to help me with the code.

[seandisanti]

But how do you tell it what lines you want (i.e. Non-blank lines?)

I can't do it manually, cause the files are generally very long, like 40-50 pages of code (PHP files for a forum and website).

This may be a little over my head, or I'm just not thinking of it right...I'll probably need someone to help me with the code.

<{POST_SNAPBACK}>

i'd never suggest doing things 'manually'. All you have to do is have a condition that is evaluated as each line is read in... example:

$myfile = FileOpen("C:\blah.txt",0); opens specified file for read access.
$temp = fileopen("c:\temp.txt",2); creates and opens a temp file, or overwrites data if temp file already exists

while 1;infinite loop so that it doesn't matter how large the file is (within language limits, see faq)
$line = FileReadLine($myfile);reads a line and assigns it to a variable
if $line = -1 then ExitLoop;if line couldn't be read because we've read them all, then we can stop the loop
if StringLen($line) > 3 Then;we want atleast 3 characters per line, 
    FileWriteLine($temp,$line);so we write the ones that have 3 to the temp file
EndIf
WEnd
FileClose($myfile);if you open a file close it, if you don't know why, feel free to experiment
FileClose($temp)
FileMove("c:\temp.txt","c:\blah.txt");this just renames the temp file to the original.  you could assign this to a variable and check the value to make sure it completes successfully, but i didn't....

there ya go, i even commented this time

[Dr.Chi]

thanks!

I'll try it out and let you know the results.

(I appreciate the comments)

[seandisanti]

thanks!

I'll try it out and let you know the results.

(I appreciate the comments)

<{POST_SNAPBACK}>

np, glad i could help

[Dr.Chi]

one more thing.

I wanted a progress bar, and with the code I have, it appears, but it doesn't increase. It stays at 0% then closes at the end.

expandcollapse popup

AutoItSetOption ( "TrayIconDebug", 1)

$file2edit = InputBox("Question", "Which file do you want to edit?", "e:\_temp\index.php", "")
If @Error = 1 Then Exit
$mincharcount = InputBox("Question", "What's the mininum number of characters to check for on each line?", "1", "")
If @Error = 1 Then Exit

; ==============================================================================
$count_file = Fileopen($file2edit,0)
$totalcount = 0
While 1
  $myline = FileReadline($count_file)
  If $myline = -1 then ExitLoop
  If @error = -1 Then ExitLoop
  $totalcount = $totalcount + 1
 ;Msgbox(0,"Line Count","Total Number of Lines" & $totalcount,1)
WEnd
fileclose($count_file)
;Msgbox(0,"Line Count","Total Number of Lines" & $totalcount)
; ==============================================================================


ProgressOn ("Deleting Blank Lines", "", "0%", -1, -1, 16 )

$myfile = FileOpen($file2edit,0); opens specified file for read access.
$temp = fileopen("c:\temp.txt",2); creates and opens a temp file, or overwrites data if temp file already exists

while 1;infinite loop so that it doesn't matter how large the file is (within language limits, see faq)
$currentline = 0
$line = FileReadLine($myfile);reads a line and assigns it to a variable
if $line = -1 then ExitLoop;if line couldn't be read because we've read them all, then we can stop the loop
If @error = -1 Then ExitLoop
if StringLen($line) > $mincharcount Then;we want atleast x number characters per line, 
    FileWriteLine($temp,$line);so we write the ones that have 3 to the temp file
EndIf
$currentline = $currentline + 1
$currentprogress = $currentline / $totalcount * 100
ProgressSet($currentprogress)
WEnd
ProgressOff()
FileClose($myfile);if you open a file close it, if you don't know why, feel free to experiment
FileClose($temp)
;FileMove("c:\temp.txt",$file2edit);this just renames the temp file to the original.  you could assign this to a variable and check the value to make sure it completes successfully, but i didn't....

I thought the line....

$currentprogress = $currentline / $totalcount * 100

would have done it. But it doesn't.

[seandisanti]

while 1;infinite loop so that it doesn't matter how large the file is (within language limits, see faq)

$currentline = 0

there's your issue right there. switch the position of those two lines and you'll be all set. currently you're reseting the counter on every iteration.

***edit***

oh yeah, you also want to change your ProgressSet() call too, it should read...

ProgressSet($currentprogress,Round($currentprogress,0) & " %")

the way you have it:

ProgressSet($currentprogress)

only updates the bar, not the label displaying the percentage... Edited July 28, 2005 by cameronsdad

[blindwig]

Do you realize that your routine actually reads the file twice? Once just to count lines and then once to process?

I think it would be better to just track your progress as you go:

$TotalSize = FileGetSize($File2Edit)
$TotalRead = 0
while 1;infinite loop so that it doesn't matter how large the file is (within language limits, see faq)
    $line = FileReadLine($myfile);reads a line and assigns it to a variable
    if $line = -1 then ExitLoop;if line couldn't be read because we've read them all, then we can stop the loop
    If @error = -1 Then ExitLoop
    $TotalRead = $TotalRead + StringLen($line) + 2;assume 2-byte EOL
    if StringLen($line) > $mincharcount Then;we want atleast x number characters per line, 
        FileWriteLine($temp,$line);so we write the ones that have 3 to the temp file
    EndIf
    $currentprogress = $TotalRead / $TotalSize * 100
    ProgressSet($currentprogress,Round($currentprogress,0) & " %")
WEnd

Edit: corrected code

Edited July 28, 2005 by blindwig

[Dr.Chi]

there's your issue right there. switch the position of those two lines and you'll be all set.  currently you're reseting the counter on every iteration.

Perfect, thanks! (I should have caught that..I'm an idiot)

blindwig: I made the changes you suggested (it makes sense to try to do it all while the file is open once) but I can't seem to get it to work. It runs the script alright, but when it gets to 100% it just keeps counting more and more I stopped the script at almost 200%.

It was stuck in an infinate loop.

What's throwing me off in your suggestion is you're dividing a StringLen (which counts characters in a line, right?) with a FileGetSize (which counts the size in bytes). Which doesn't seem to jive right in my head, wouldn't comparing character count and file byte size be like apples and oranges?

[blindwig]

Perfect, thanks!  (I should have caught that..I'm an idiot)

blindwig:   I made the changes you suggested (it makes sense to try to do it all while the file is open once) but I can't seem to get it to work.  It runs the script alright, but when it gets to 100% it just keeps counting more and more I stopped the script at almost 200%.

It was stuck in an infinate loop.

What's throwing me off in your suggestion is you're dividing a StringLen (which counts characters in a line, right?) with a FileGetSize (which counts the size in bytes).  Which doesn't seem to jive right in my head, wouldn't comparing character count and file byte size be like apples and oranges?

<{POST_SNAPBACK}>

RE: Infinate loop - yes, oops, please move the "$TotalSize = " line to below the "If @Error" line. I edited my previous post to reflect the change.

RE: Progress - no, that is correct. A textfile is typically 1 byte per character, and I'm assuming 2 bytes per EOL.

It's possible to have a uni-code text file (which would be 2 bytes per character), but the high number of #0 characters would break your routine anyway.

It's also possible to have a 1 byte EOL, but since that margin of error is so small, it's hardly worth considering. Unfortunately there is currently no way of knowing the size of the EOL (without writing your own function) - I suggested a while back to add that feature to the FileReadLine function, but didn't get much of a responce from the developers.

Edited July 28, 2005 by blindwig

[Dr.Chi]

I see. Consider me educated

Thanks everyone for all your help.

In case anyone wants it below is the final code I used (worked perfectly).

Some things I changed:

I make it call a text file with the path/filenames of all the files you want it to check (for batch checking) which you make. To get the file list you can either type it up or run "dir /o /b /s >..\filelist.txt" at a cmd prompt.

Also I added the current file being processed to the progress bar label.

expandcollapse popup

;
; AutoIt 3
; Language:   English
; Platform:    Win2k,Xp
;
; Script Function:
;   Delete Blank Lines in a file or multiple files
;

AutoItSetOption ( "TrayIconDebug", 1)

$mincharcount = 0

;  This Section grabs the names in the file.
; -----------------------------------------------
$WhichFile = InputBox("Question", "Whats the filename with the list of files in it?", "c:\files.txt", "")
$listofiles = FileOpen($WhichFile, 0)


; Check if file opened for reading OK
; -----------------------------------------------
If $listofiles = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Read in lines of text until the EOF is reached
; -----------------------------------------------
While 1
    $listofilesline = FileReadLine($listofiles)
    If $listofilesline = -1 then ExitLoop
    If @error = -1 Then ExitLoop

; What to do for each item in the file list...
; -----------------------------------------------
    $file2edit = $listofilesline
    
        ProgressOn ("Deleting Blank Lines", "", "0%", -1, -1, 16 )
        
        $myfile = FileOpen($file2edit,0);opens specified file for read access.
        $temp = fileopen("c:\temp.txt",2);creates and opens a temp file, or overwrites data if temp file already exists
        
        $TotalSize = FileGetSize($file2edit)
        $TotalRead = 0
        while 1 ;infinite loop
            $line = FileReadLine($myfile);reads a line and assigns it to a variable
            if $line = -1 then ExitLoop  ;if line couldn't be read because we've read them all, then we can stop the loop
            If @error = -1 Then ExitLoop
            $TotalRead = $TotalRead + StringLen($line) + 2 ;assume 2-byte EOL
            if StringLen($line) > $mincharcount Then  ;we want atleast x number characters per line, 
                FileWriteLine($temp,$line)  ;so we write the ones that have 3 to the temp file
            EndIf
            $currentprogress = $TotalRead / $TotalSize * 100
            ProgressSet($currentprogress,Round($currentprogress,0) & " % - File: " & $file2edit)
        WEnd
        SLEEP (500)
        ProgressOff()
        FileClose($myfile);if you open a file close it, if you don't know why, feel free to experiment
        FileClose($temp)
        FileMove("c:\temp.txt",$file2edit,1);this just renames the temp file to the original.  you could assign this to a variable and check the value to make sure it completes successfully, but i didn't....
        SLEEP (700)
WEnd
FileClose($listofiles)
Msgbox(0,"Finished","Done")