Jump to content

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here. X
X


Photo

Remove Blank Lines in a file


  • Please log in to reply
21 replies to this topic

#1 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 09:24 AM

Hey everyone,

I'm back again with more problems ^^'

I have an application that opens a file to read the stuff in there.
What I want it to do beforehand is to remove blank lines from the File.

Here is what I got

     $read=FileRead($listFull)      FileDelete($listFull)      Do          $read=StringReplace($read,@CRLF&@CRLF,@CRLF)      Until @extended=0      FileWrite($listFull,$read)


It actually works quite well, but it runs into an error if the first line of the document is a blank line







#2 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 09:32 AM

DemonWareXT,

Perhaps like this: :graduated:
#include <File.au3> #include <Array.au3> Global $aLines _FileReadToArray("Test.txt", $aLines) For $i = $aLines[0] To 1 Step -1     If $aLines[$i] = "" Then         _ArrayDelete($aLines, $i)     EndIf Next _FileWriteFromArray("NewTest.txt", $aLines, 1)

Note you have to run the delete code from the end of the array or you run into errors with the count when you delete lines. ;)

M23
StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#3 Xenobiologist

Xenobiologist

    Xx Code~Mega xX

  • MVPs
  • 4,830 posts

Posted 27 September 2011 - 09:37 AM

#include<Array.au3> #include<File.au3> Global $pfad = @ScriptDir & '\blank.txt', $lines_A _FileReadToArray($pfad, $lines_A) For $i = UBound($lines_A) - 1 To 1 Step -1 If Not StringRegExp($lines_A[$i], '[^\s]', 0) Then _ArrayDelete($lines_A, $i) Next _FileWriteFromArray($pfad, $lines_A, 1)

Scripts & functions Organize Includes Let Scite organize the include files *new

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times


#4 hannes08

hannes08

    my oh my

  • Active Members
  • PipPipPipPipPipPip
  • 944 posts

Posted 27 September 2011 - 09:56 AM

This will be faster for big files:

#include <File.au3> #include <Array.au3> Global $aLines _FileReadToArray("Test.txt", $aLines) $sLines = _ArrayToString($aLines, "|", 1) $sLines = StringReplace($sLines, "||", "|") $aLines = StringSplit($sLines, "|") _FileWriteFromArray("NewTest.txt", $aLines, 1)


There's no need to call _ArrayDelete which redims the array on every call (and therefor is slow).

Or you could use StringRegExpReplace:
$sContent = FileRead("Test.txt") $sContent = StringRegExpReplace($sContent, "^\s*$", "") FileWrite("NewTest.txt", $sContent)

But I'm not very much into RegularExpressions, so that needs to be tested. :graduated:

Edited by hannes08, 27 September 2011 - 10:02 AM.

Regards,Hannes
Spoiler

#5 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 09:58 AM

Thanks a lot M23 works like a charm :graduated:

#6 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 11:01 AM

hannes08,

Unfortunately your solution will not work if there are multiple blank lines - it only repalces pairs with singles. :graduated:

However, your "No ReDim" point is very valid and so I have come up with this:
$sText = FileRead("Test.txt") $sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "\A\v|\v\Z", "") $hFile = FileOpen("NewText.txt", 2) FileWrite($hFile, $sText) FileClose($hFile)

which seems to fit the bill. The inner SRE repaces all multiple EOLs with a single EOL, while the outer removes any remaining leading and trailing EOLs (if they were multiple they have already been reduced to a single by the first SRE). ;)

M23

Edit: See below for a corrected version of the SRE. :)

Edited by Melba23, 27 September 2011 - 03:21 PM.

StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#7 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 11:08 AM

@M23
your second version doesn't seem to work either, I get the original problem again, if there is a blank on the first line then it will crash... or maybe I'm to daft to implement it xD

#8 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 11:29 AM

DemonWareXT,

As you might expect it works fine for me or I would not have posted it! :)

And I cannot see why a leading EOL would crash the code - or do you just mean that it does not remove the blank? :graduated:

Could you please post the code you are using and example of a file which crashes it - I will take a look. ;)

M23
StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#9 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 11:34 AM

@M23


it doesn't leave the first line blank, it enters a number in there?

well if you really wanna see my whole code xD
(I did put your first code in there again tho)
Edit: I hope you can read it, somehow the code shows up crazy on ma PC, so I atached the script too
Plain Text         
#include <GuiConstants.au3> #include <Array.au3> #include <file.au3> #include <ButtonConstants.au3> #include <GUIListBox.au3> ;include <GUIConstantsEx.au3> ;#include <WindowsConstants.au3> ;#Include <GuiScrollBars.au3> ;~ HotKeySet("^w", "tag") HotKeySet("^q", "lTag") HotKeySet("{ESC}", "errorH") ;;;;HotKeySet("{DEL}", "del")   ;************************** ;Global Variables ;************************** $listFull = @ScriptDir & 'list.csv' ;$listShort = @ScriptDir & 'list short names.txt' ;$listWhole = @ScriptDir & 'people.csv' $error = 0 Do ;clean up the lists ;~   $read=FileRead($listFull) ;~   FileDelete($listFull) ;~   Do ;~  $read=StringReplace($read,@CRLF&@CRLF,@CRLF) ;~   Until @extended=0 ;~   FileWrite($listFull,$read) ;call function to clear blank lines in text clearBlank()     ;************************** ;read array from text file ;************************** ;read the whole array Dim $Whole If Not _FileReadToArray($listFull,$Whole) Then     MsgBox(4096,"Error", " Error reading Array  error:" & @error)     Exit    EndIf    ;sort the array   _ArraySort($Whole) ;write array back to csv   _FileWriteFromArray($listFull,$Whole, 1)   ;For $i = 1 to UBound($Whole)-1   ;Next Dim $helper[2] = [0,0] Dim $PeopleFull[1] Dim $PeopleShort[1] For $i = 1 to UBound($Whole)-1   $helper = StringSplit($Whole[$i],';', 1)   ReDim $PeopleFull[UBound($PeopleFull)+1]   ReDim $PeopleShort[UBound($PeopleShort)+1]   $PeopleFull[$i] = $helper[1]   $PeopleShort[$i] = $helper[2]   ;MsgBox(0,"",$helper[1])   ;MsgBox(0,"",$helper[2]) Next     ;************************** ;Create the GUI ;************************** ;Do while loop so we can rebuild the GUI if need be. ;main GUI window $Form1 = GUICreate("Auswählen",570,550,192,114) ;some textlables to eplain whats going on GUICtrlCreateLabel("1. Select people, use shift and ctrl", 10, 480) GUICtrlCreateLabel("2. Go to G+ get cursor in input dialog", 10, 500) GUICtrlCreateLabel("3. CTRL+Q to start Tagging", 10, 520) ;Button that adds ponies to the list $add = GUICtrlCreateButton("Add", 350, 520,50) ;Input Box $Input1 = GUICtrlCreateInput("", 200, 520, 150, 25) ;some textlables to explain whats going on GUICtrlCreateLabel("| Fomrate like this: Name;Shortname", 200, 480) GUICtrlCreateLabel("| Use with caution", 200, 500) ;Button to delete people $del = GUICtrlCreateButton("Delete Selected", 400, 520)   ; Create an array to hold the checkbox ControlIds - make it the same size as the $PeopleFull array ;Global $Person[UBound($PeopleFull)] ;Global $del[UBound($PeopleFull)] ;Some variables to count stuff $count = 0 $counter = 0 ;if retry = 1 the do while loop will end $retry = 1 ;List of people $ListPeople = GUICtrlCreateList("", 10, 10, 550, 450, BitOR($LBS_STANDARD, $LBS_EXTENDEDSEL)) ;$ListPeople = GUICtrlCreateList("", 10, 10, 300, 450, BitOR($GUI_SS_DEFAULT_LIST,$LBS_MULTIPLESEL)) For $i = 1 to UBound($PeopleFull)-1   GUICtrlSetData($ListPeople, $PeopleFull[$i]) Next ;************************** ;Create GUI Actions ;************************** ;Set the GUI to be shown GUISetState(@SW_SHOW)   Do    $msg = GUIGetMsg()    Switch $msg     Case $add     If GUICtrlRead ($Input1) = "" Then       MsgBox(0,'Niet','Please... Input!')      Else       add()      EndIf     Case $del      del()    EndSwitch     Until $msg = $GUI_EVENT_CLOSE Until $retry=1 ;Functions ;~ Func tag() ;~  ;First sleep 5seconds so the user can place the cursor to the right place ;~  ;Sleep(4000) ;~  $error = 0 ;~  For $i = 1 to UBound($PeopleFull)-1 ;~   If $error = 1 Then ;~   ElseIf GuiCTRLRead ($Person[$i]) = $GUI_CHECKED Then ;~  ClipPut($PeopleFull[$i]) ;~  ;Send("{+}") ;~  Send("^v") ;~  Sleep(1000) ;~  Send("{ENTER}") ;~  Sleep(500) ;~   EndIf ;~  Next ;~ EndFunc Func lTag() $items = _GUICtrlListBox_GetSelItems($ListPeople) Opt("SendKeyDelay",40) For $i = 1 to UBound($items)-1   If $error = 1 Then   Else    $item = $items[$i]+1    Send("{+}");    Send($PeopleShort[$item])    Sleep(1000)    Send("{ENTER}")    Sleep(500)   EndIf Next Opt("SendKeyDelay",5) Send("{CTRLUP}") ;Send("{SHIFTUP}") ;Send("{ALTUP}") $error = 0 EndFunc Func clearBlank()   Global $aLines _FileReadToArray($listFull, $aLines) For $i = $aLines[0] To 1 Step -1   If $aLines[$i] = "" Then    _ArrayDelete($aLines, $i)   EndIf Next _FileWriteFromArray($listFull, $aLines, 1) EndFunc Func add() ;$counter = $counter ReDim $Whole[UBound($Whole)+1] $Whole[$counter] = GUICtrlRead ($Input1) _ArraySort($Whole) _FileWriteFromArray($listFull,$Whole, 1) $helper = StringSplit(GUICtrlRead ($Input1),';', 1) ReDim $PeopleFull[UBound($PeopleFull)+1] ReDim $PeopleShort[UBound($PeopleShort)+1] $PeopleFull[$i] = $helper[1] $PeopleShort[$i] = $helper[2] GUICtrlSetData($ListPeople, $helper[1]) _ArraySort($PeopleFull) _ArraySort($PeopleShort) EndFunc Func del() $items = _GUICtrlListBox_GetSelItems($ListPeople) For $i = 1 to UBound($items)-1   $item = $items[$i]+1   $PeopleFull[$item] = ""   $PeopleShort[$item] = ""   $Whole[$item] = ""   ;MsgBox(0,"",$PeopleShort[1]) Next   _ArraySort($PeopleFull) _ArraySort($PeopleShort) _ArraySort($Whole) _FileWriteFromArray($listFull,$Whole, 1) clearBlank() GUICtrlSetData($ListPeople, "") For $i = 1 to UBound($PeopleFull)-1   GUICtrlSetData($ListPeople, $PeopleFull[$i]) Next EndFunc ;debuging Func errorH() $error=1 ;Exit EndFunc



The list behind it is called list.csv and is built up like this:
name;name2

Attached Files


Edited by DemonWareXT, 27 September 2011 - 11:36 AM.


#10 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 11:36 AM

DemonWareXT,

That post did not work. :graduated:

M23
StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#11 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 11:39 AM

@M23
just realised that xD
Attached the script

the list is built up like this:

list.csv
blablabla;blablabla

Attached Files



#12 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 11:49 AM

DemonWareXT,


Works just fine for me (but then I would say that! ;)):
Plain Text         
(list.csv) - not in the file but the forum software does not show the leading blank line if I do not put it in! blablabla;blablabla2 blablabla;blablabla3 blablabla;blablabla7 blablabla;blablabla8 blablabla;blablabla9 blablabla;blablabla10 blablabla;blablabla14 blablabla;blablabla15 blablabla;blablabla16 blablabla;blablabla17 blablabla;blablabla20 blablabla;blablabla21 blablabla;blablabla23 blablabla;blablabla24 blablabla;blablabla28 blablabla;blablabla29

becomes
(list.csv) blablabla;blablabla2 blablabla;blablabla3 blablabla;blablabla7 blablabla;blablabla8 blablabla;blablabla9 blablabla;blablabla10 blablabla;blablabla14 blablabla;blablabla15 blablabla;blablabla16 blablabla;blablabla17 blablabla;blablabla20 blablabla;blablabla21 blablabla;blablabla23 blablabla;blablabla24 blablabla;blablabla28 blablabla;blablabla29

And the code does not crash in that function. Are you sure that is where the error occurs? :graduated:

M23
StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#13 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 11:51 AM

@M23
I sent you the code where I implemented your first try which worked perfectly^^

either you implement your second version or I'll send the file again with error xD

Edit: attached it again

Attached Files


Edited by DemonWareXT, 27 September 2011 - 11:56 AM.


#14 Akarillon

Akarillon

    Wayfarer

  • Active Members
  • Pip
  • 96 posts

Posted 27 September 2011 - 12:07 PM

Just to ask, does StringStripWS() work for the same problem or does it only work on spaces?
Challenge accepted!

#15 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 12:12 PM

Just to ask, does StringStripWS() work for the same problem or does it only work on spaces?


StringStripWS() is only used to strip of white spaces, so I don't think it would work

#16 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 12:57 PM

DemonWareXT,

Try this SRE - I think it solves the problem:
$sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "(^\v*)|(\v*\Z)", "")

I was only removing the CR of a CRLF. When I opened the file in NotePad to check it did not honour the remaining 0A with a new line - it was only when I looked with a hex editor I saw the problem. :graduated:

As for the added number I think it happens when you sort your array - try this code:
_ArraySort($Whole, 0, 1)

Must rush to the dentist. :)

Try those out and I will look again when I return. ;)

M23
  • Coder07 likes this
StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#17 DemonWareXT

DemonWareXT

    Seeker

  • Active Members
  • 10 posts

Posted 27 September 2011 - 01:09 PM

@M23

Now it works, you're an hero :graduated:
about the number problem, I think this only happened because there was some array size confusion, which now doesn't happen any more since the line removing works correctly now

#18 Melba23

Melba23

    I'm old - what's your excuse?

  • Moderators
  • 20,681 posts

Posted 27 September 2011 - 02:25 PM

DemonWareXT,

Excellent news. :graduated:

And I only have to take some antibiotics for a while, so a result for both of us! ;)

M23
StringSize - Automatically size controls to fit text                                                               ExtMsgBox - A user customisable replacement for MsgBox
Toast - Small GUIs which pop out of the Systray                                                                Marquee - Scrolling tickertape GUIs
Scrollbars - Automatically sized scrollbars with a single command                                   GUIFrame - Subdivide GUIs into many adjustable frames
GUIExtender - Extend and retract multiple sections within a GUI                                      NoFocusLines - Remove the dotted focus lines from buttons, sliders, radios and checkboxes
ChooseFileFolder - Single and multiple selections from specified path tree structure      Notify - Small notifications on the edge of the display
Date_Time_Convert - Easily convert date/time formats, including the language               GUIListViewEx - Insert, delete, move, drag, sort and edit ListView items

#19 GEOSoft

GEOSoft

    Sure I'm senile. What's your excuse?

  • MVPs
  • 10,573 posts

Posted 27 September 2011 - 04:09 PM

Just at a quick glance this may also work.
  $sText = StringRegExpReplace($sText, "(?m:^)\h*(\r\n|\r|\n)", "")

GeorgeQuestion about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.*** The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else."Old age and treachery will always overcome youth and skill!"

#20 UEZ

UEZ

    Never say never

  • MVPs
  • 4,778 posts

Posted 27 September 2011 - 05:53 PM

Is this working for you?

  $sNew = StringReplace($text, Chr(10) & Chr(10), "") If Not @extended Then $sNew = StringReplace($text, Chr(13) & Chr(10) & Chr(13) & Chr(10), "") ConsoleWrite($sNew & @LF)


Br,
UEZ

 
The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users