Sign in to follow this  
Followers 0

Remove Blank Lines in a file

22 posts in this topic

Posted · Report post

Hey everyone,

I'm back again with more problems ^^'

I have an application that opens a file to read the stuff in there.

What I want it to do beforehand is to remove blank lines from the File.

Here is what I got

	 $read=FileRead($listFull)
	 FileDelete($listFull)
	 Do
		 $read=StringReplace($read,@CRLF&@CRLF,@CRLF)
	 Until @extended=0
	 FileWrite($listFull,$read)

It actually works quite well, but it runs into an error if the first line of the document is a blank line

Share this post


Link to post
Share on other sites



Posted · Report post

DemonWareXT,

Perhaps like this: :graduated:

#include <File.au3>
#include <Array.au3>

Global $aLines

_FileReadToArray("Test.txt", $aLines)

For $i = $aLines[0] To 1 Step -1
	If $aLines[$i] = "" Then
		_ArrayDelete($aLines, $i)
	EndIf
Next

_FileWriteFromArray("NewTest.txt", $aLines, 1)

Note you have to run the delete code from the end of the array or you run into errors with the count when you delete lines. ;)

M23

Share this post


Link to post
Share on other sites

Posted · Report post

#include<Array.au3>
#include<File.au3>
Global $pfad = @ScriptDir & '\blank.txt', $lines_A
_FileReadToArray($pfad, $lines_A)
For $i = UBound($lines_A) - 1 To 1 Step -1
If Not StringRegExp($lines_A[$i], '[^\s]', 0) Then _ArrayDelete($lines_A, $i)
Next
_FileWriteFromArray($pfad, $lines_A, 1)

Share this post


Link to post
Share on other sites

Posted (edited) · Report post

This will be faster for big files:

#include <File.au3>
#include <Array.au3>

Global $aLines

_FileReadToArray("Test.txt", $aLines)

$sLines = _ArrayToString($aLines, "|", 1)
$sLines = StringReplace($sLines, "||", "|")
$aLines = StringSplit($sLines, "|")

_FileWriteFromArray("NewTest.txt", $aLines, 1)

There's no need to call _ArrayDelete which redims the array on every call (and therefor is slow).

Or you could use StringRegExpReplace:

$sContent = FileRead("Test.txt")
$sContent = StringRegExpReplace($sContent, "^\s*$", "")
FileWrite("NewTest.txt", $sContent)

But I'm not very much into RegularExpressions, so that needs to be tested. :graduated:

Edited by hannes08

Share this post


Link to post
Share on other sites

Posted · Report post

Thanks a lot M23 works like a charm :graduated:

Share this post


Link to post
Share on other sites

Posted (edited) · Report post

hannes08,

Unfortunately your solution will not work if there are multiple blank lines - it only repalces pairs with singles. :graduated:

However, your "No ReDim" point is very valid and so I have come up with this:

$sText = FileRead("Test.txt")

$sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "\A\v|\v\Z", "")

$hFile = FileOpen("NewText.txt", 2)
FileWrite($hFile, $sText)
FileClose($hFile)

which seems to fit the bill. The inner SRE repaces all multiple EOLs with a single EOL, while the outer removes any remaining leading and trailing EOLs (if they were multiple they have already been reduced to a single by the first SRE). ;)

M23

Edit: See below for a corrected version of the SRE. :)

Edited by Melba23

Share this post


Link to post
Share on other sites

Posted · Report post

@M23

your second version doesn't seem to work either, I get the original problem again, if there is a blank on the first line then it will crash... or maybe I'm to daft to implement it xD

Share this post


Link to post
Share on other sites

Posted · Report post

DemonWareXT,

As you might expect it works fine for me or I would not have posted it! :)

And I cannot see why a leading EOL would crash the code - or do you just mean that it does not remove the blank? :graduated:

Could you please post the code you are using and example of a file which crashes it - I will take a look. ;)

M23

Share this post


Link to post
Share on other sites

Posted (edited) · Report post

@M23

it doesn't leave the first line blank, it enters a number in there?

well if you really wanna see my whole code xD

(I did put your first code in there again tho)

Edit: I hope you can read it, somehow the code shows up crazy on ma PC, so I atached the script too

#include <GuiConstants.au3>
#include <Array.au3>
#include <file.au3>
#include <ButtonConstants.au3>
#include <GUIListBox.au3>
;include <GUIConstantsEx.au3>
;#include <WindowsConstants.au3>
;#Include <GuiScrollBars.au3>
;~ HotKeySet("^w", "tag")
HotKeySet("^q", "lTag")
HotKeySet("{ESC}", "errorH")
;;;;HotKeySet("{DEL}", "del")
 
;**************************
;Global Variables
;**************************
$listFull = @ScriptDir & 'list.csv'
;$listShort = @ScriptDir & 'list short names.txt'
;$listWhole = @ScriptDir & 'people.csv'
$error = 0
Do
;clean up the lists
;~   $read=FileRead($listFull)
;~   FileDelete($listFull)
;~   Do
;~	$read=StringReplace($read,@CRLF&@CRLF,@CRLF)
;~   Until @extended=0
;~   FileWrite($listFull,$read)
;call function to clear blank lines in text
clearBlank()
 
 
;**************************
;read array from text file
;**************************
;read the whole array
Dim $Whole
If Not _FileReadToArray($listFull,$Whole) Then
	MsgBox(4096,"Error", " Error reading Array	error:" & @error)
	Exit
   EndIf
   ;sort the array
  _ArraySort($Whole)
;write array back to csv
  _FileWriteFromArray($listFull,$Whole, 1)
  ;For $i = 1 to UBound($Whole)-1
  ;Next
Dim $helper[2] = [0,0]
Dim $PeopleFull[1]
Dim $PeopleShort[1]
For $i = 1 to UBound($Whole)-1
  $helper = StringSplit($Whole[$i],';', 1)
  ReDim $PeopleFull[UBound($PeopleFull)+1]
  ReDim $PeopleShort[UBound($PeopleShort)+1]
  $PeopleFull[$i] = $helper[1]
  $PeopleShort[$i] = $helper[2]
  ;MsgBox(0,"",$helper[1])
  ;MsgBox(0,"",$helper[2])
Next
 
 
;**************************
;Create the GUI
;**************************
;Do while loop so we can rebuild the GUI if need be.
;main GUI window
$Form1 = GUICreate("Auswählen",570,550,192,114)
;some textlables to eplain whats going on
GUICtrlCreateLabel("1. Select people, use shift and ctrl", 10, 480)
GUICtrlCreateLabel("2. Go to G+ get cursor in input dialog", 10, 500)
GUICtrlCreateLabel("3. CTRL+Q to start Tagging", 10, 520)
;Button that adds ponies to the list
$add = GUICtrlCreateButton("Add", 350, 520,50)
;Input Box
$Input1 = GUICtrlCreateInput("", 200, 520, 150, 25)
;some textlables to explain whats going on
GUICtrlCreateLabel("| Fomrate like this: Name;Shortname", 200, 480)
GUICtrlCreateLabel("| Use with caution", 200, 500)
;Button to delete people
$del = GUICtrlCreateButton("Delete Selected", 400, 520)
 
; Create an array to hold the checkbox ControlIds - make it the same size as the $PeopleFull array
;Global $Person[UBound($PeopleFull)]
;Global $del[UBound($PeopleFull)]
;Some variables to count stuff
$count = 0
$counter = 0
;if retry = 1 the do while loop will end
$retry = 1
;List of people
$ListPeople = GUICtrlCreateList("", 10, 10, 550, 450, BitOR($LBS_STANDARD, $LBS_EXTENDEDSEL))
;$ListPeople = GUICtrlCreateList("", 10, 10, 300, 450, BitOR($GUI_SS_DEFAULT_LIST,$LBS_MULTIPLESEL))
For $i = 1 to UBound($PeopleFull)-1
  GUICtrlSetData($ListPeople, $PeopleFull[$i])
Next
;**************************
;Create GUI Actions
;**************************
;Set the GUI to be shown
GUISetState(@SW_SHOW)
  Do
   $msg = GUIGetMsg()
   Switch $msg
	Case $add
	If GUICtrlRead ($Input1) = "" Then
	  MsgBox(0,'Niet','Please... Input!')
	 Else
	  add()
	 EndIf
	Case $del
	 del()
   EndSwitch
 
  Until $msg = $GUI_EVENT_CLOSE
Until $retry=1
;Functions
;~ Func tag()
;~  ;First sleep 5seconds so the user can place the cursor to the right place
;~  ;Sleep(4000)
;~  $error = 0
;~  For $i = 1 to UBound($PeopleFull)-1
;~   If $error = 1 Then
;~   ElseIf GuiCTRLRead ($Person[$i]) = $GUI_CHECKED Then
;~	ClipPut($PeopleFull[$i])
;~	;Send("{+}")
;~	Send("^v")
;~	Sleep(1000)
;~	Send("{ENTER}")
;~	Sleep(500)
;~   EndIf
;~  Next
;~ EndFunc
Func lTag()
$items = _GUICtrlListBox_GetSelItems($ListPeople)
Opt("SendKeyDelay",40)
For $i = 1 to UBound($items)-1
  If $error = 1 Then
  Else
   $item = $items[$i]+1
   Send("{+}");
   Send($PeopleShort[$item])
   Sleep(1000)
   Send("{ENTER}")
   Sleep(500)
  EndIf
Next
Opt("SendKeyDelay",5)
Send("{CTRLUP}")
;Send("{SHIFTUP}")
;Send("{ALTUP}")
$error = 0
EndFunc
Func clearBlank()
  Global $aLines
_FileReadToArray($listFull, $aLines)
For $i = $aLines[0] To 1 Step -1
  If $aLines[$i] = "" Then
   _ArrayDelete($aLines, $i)
  EndIf
Next
_FileWriteFromArray($listFull, $aLines, 1)
EndFunc
Func add()
;$counter = $counter
ReDim $Whole[UBound($Whole)+1]
$Whole[$counter] = GUICtrlRead ($Input1)
_ArraySort($Whole)
_FileWriteFromArray($listFull,$Whole, 1)
$helper = StringSplit(GUICtrlRead ($Input1),';', 1)
ReDim $PeopleFull[UBound($PeopleFull)+1]
ReDim $PeopleShort[UBound($PeopleShort)+1]
$PeopleFull[$i] = $helper[1]
$PeopleShort[$i] = $helper[2]
GUICtrlSetData($ListPeople, $helper[1])
_ArraySort($PeopleFull)
_ArraySort($PeopleShort)
EndFunc
Func del()
$items = _GUICtrlListBox_GetSelItems($ListPeople)
For $i = 1 to UBound($items)-1
  $item = $items[$i]+1
  $PeopleFull[$item] = ""
  $PeopleShort[$item] = ""
  $Whole[$item] = ""
  ;MsgBox(0,"",$PeopleShort[1])
Next
 
_ArraySort($PeopleFull)
_ArraySort($PeopleShort)
_ArraySort($Whole)
_FileWriteFromArray($listFull,$Whole, 1)
clearBlank()
GUICtrlSetData($ListPeople, "")
For $i = 1 to UBound($PeopleFull)-1
  GUICtrlSetData($ListPeople, $PeopleFull[$i])
Next
EndFunc
;debuging
Func errorH()
$error=1
;Exit
EndFunc

The list behind it is called list.csv and is built up like this:

name;name2

autoTag_V2.5.au3

Edited by DemonWareXT

Share this post


Link to post
Share on other sites

Posted · Report post

DemonWareXT,

That post did not work. :graduated:

M23

Share this post


Link to post
Share on other sites

Posted · Report post

@M23

just realised that xD

Attached the script

the list is built up like this:

list.csv

blablabla;blablabla

autoTag_V2.5.au3

Share this post


Link to post
Share on other sites

Posted · Report post

DemonWareXT,

Works just fine for me (but then I would say that! ;)):

(list.csv) - not in the file but the forum software does not show the leading blank line if I do not put it in!

blablabla;blablabla2
blablabla;blablabla3



blablabla;blablabla7
blablabla;blablabla8
blablabla;blablabla9
blablabla;blablabla10



blablabla;blablabla14
blablabla;blablabla15
blablabla;blablabla16
blablabla;blablabla17


blablabla;blablabla20
blablabla;blablabla21

blablabla;blablabla23
blablabla;blablabla24



blablabla;blablabla28
blablabla;blablabla29

becomes

(list.csv)
blablabla;blablabla2
blablabla;blablabla3
blablabla;blablabla7
blablabla;blablabla8
blablabla;blablabla9
blablabla;blablabla10
blablabla;blablabla14
blablabla;blablabla15
blablabla;blablabla16
blablabla;blablabla17
blablabla;blablabla20
blablabla;blablabla21
blablabla;blablabla23
blablabla;blablabla24
blablabla;blablabla28
blablabla;blablabla29

And the code does not crash in that function. Are you sure that is where the error occurs? :graduated:

M23

Share this post


Link to post
Share on other sites

Posted (edited) · Report post

@M23

I sent you the code where I implemented your first try which worked perfectly^^

either you implement your second version or I'll send the file again with error xD

Edit: attached it again

autoTag_V2.5.au3

Edited by DemonWareXT

Share this post


Link to post
Share on other sites

Posted · Report post

Just to ask, does StringStripWS() work for the same problem or does it only work on spaces?

Share this post


Link to post
Share on other sites

Posted · Report post

Just to ask, does StringStripWS() work for the same problem or does it only work on spaces?

StringStripWS() is only used to strip of white spaces, so I don't think it would work

Share this post


Link to post
Share on other sites

Posted · Report post

DemonWareXT,

Try this SRE - I think it solves the problem:

$sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "(^\v*)|(\v*\Z)", "")

I was only removing the CR of a CRLF. When I opened the file in NotePad to check it did not honour the remaining 0A with a new line - it was only when I looked with a hex editor I saw the problem. :graduated:

As for the added number I think it happens when you sort your array - try this code:

_ArraySort($Whole, 0, 1)

Must rush to the dentist. :)

Try those out and I will look again when I return. ;)

M23

Coder07 likes this

Share this post


Link to post
Share on other sites

Posted · Report post

@M23

Now it works, you're an hero :graduated:

about the number problem, I think this only happened because there was some array size confusion, which now doesn't happen any more since the line removing works correctly now

Share this post


Link to post
Share on other sites

Posted · Report post

DemonWareXT,

Excellent news. :graduated:

And I only have to take some antibiotics for a while, so a result for both of us! ;)

M23

Share this post


Link to post
Share on other sites

Posted · Report post

Just at a quick glance this may also work.

 
$sText = StringRegExpReplace($sText, "(?m:^)\h*(\r\n|\r|\n)", "")

Share this post


Link to post
Share on other sites

Posted · Report post

Is this working for you?

 
$sNew = StringReplace($text, Chr(10) & Chr(10), "")
If Not @extended Then $sNew = StringReplace($text, Chr(13) & Chr(10) & Chr(13) & Chr(10), "")
ConsoleWrite($sNew & @LF)

Br,

UEZ

Share this post


Link to post
Share on other sites

Posted (edited) · Report post

DemonWareXT,

Try this SRE - I think it solves the problem:

$sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "(^\v*)|(\v*\Z)", "")
I was only removing the CR of a CRLF. When I opened the file in NotePad to check it did not honour the remaining 0A with a new line - it was only when I looked with a hex editor I saw the problem. :graduated:

As for the added number I think it happens when you sort your array - try this code:
_ArraySort($Whole, 0, 1)
Must rush to the dentist. :)

Try those out and I will look again when I return. ;)

M23

 

you saved my life. Thnx a lot!

i'm newbie at SRE. Can i tell that that code works like this:

1- Group and convert any horizontal line feed to @CRLF.

2- Remove @CRLF if it is located at beginning or ending of file.

?

i was using these:

Func __RemoveExtraLines($string)
   Local $i, $s = "", $split
   $split = StringSplit($string, @CRLF, 0)
   For $i = 1 To $split[0]
      If $split[$i] = @CR Or $split[$i] = @CRLF Or $split[$i] = "" Or $split[$i] = @LF Then ContinueLoop
   $s = $s & $split[$i] & @CRLF
   Next
   $s = StringReplace($s, @CRLF, "", -1)
   Return ($s)
EndFunc   ;==>__RemoveExtraLines

waste of time but it works too ^^

@Never say never

your code works too if string doesn't have (for example) @CRLF + @CR in a row.

Edited by Coder07

Share this post


Link to post
Share on other sites

Posted · Report post

Please don't necro old posts. It seems you're not getting off to a good start here.

#include <Constants.au3>

Local $sData = FileRead(@ScriptFullPath)


$sData = StringRegExpReplace($sData, '(?m:^\s*[\r\n])', '') ; Empty lines. By guinness.


MsgBox($MB_SYSTEMMODAL, '', $sData)


Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Recently Browsing   0 members

    No registered users viewing this page.