Delete double lines from txt file

[gempie 0]

Hi,

there is probably an simple answer to this, but how do i delete double lines from a text file?

Thanx for looking into this.

[Briegel 0]

I don't know if this is the best way, but I wshould try this:

_FileReadToArray

_ArraySort

compare always two side by side values from array

identify double lines

search double lines in the file and delete them

EDIT: grrrr, language

Edited July 18, 2006 by Briegel

[MHz 80]

Here is one way of just reading the file and looking at the line ends.

$file = 'c:\somefile.txt'
$contents = StringReplace(FileRead($file, FileGetSize($file)), @CRLF & @CRLF & @CRLF, @CRLF)
If @extended Then
    If FileMove($file, $file & '.bak') Then
        If FileWrite($file, $contents) Then
            FileDelete($file & '.bak')
        Else
            MsgBox(0x10, '', 'FileWrite failed')
        EndIf
    Else
        MsgBox(0x10, '', 'FileMove failed')
    EndIf
Else
    MsgBox(0x40, '', 'No double lines to cleanup')
EndIf

If the file is big, then an array method would perhaps be quicker.

Added missing @CRLF parameter as pointed out in next post.

Edited July 18, 2006 by MHz

[Briegel 0]

$contents = StringReplace(FileRead($file, FileGetSize($file)), @CRLF & @CRLF & @CRLF)

@MHz,

I tried your script and got this SciTE-error:

ERROR: StringReplace() [built-in] called with wrong number of args.

Edited July 18, 2006 by Briegel

[MHz 80]

Thanks Briegel,

Edited the intial code posted.

[gempie 0]

Thnx for your advise MHZ,

But your scripts doesn't seem to work. it only says No double lines to cleanup and then it ends.

any ideas guys?

[Briegel 0]

I exhibited you the way. Have a look at helpfile, search in this forum und try to write your own script step by step. Results can be displayed with 'MsgBox()'. If you need help then post your script and tell us what happens. You're happier if you managed it for oneself.

good luck

[MHz 80]

This is what I just tested with and is working

$file = 'c:\somefile.txt'
$contents = StringReplace(FileRead($file, FileGetSize($file)), @CRLF & @CRLF & @CRLF, @CRLF)
If @extended Then
    If FileMove($file, $file & '.bak') Then
        If FileWrite($file, $contents) Then
            FileDelete($file & '.bak')
        Else
            MsgBox(0x10, '', 'FileWrite failed')
        EndIf
    Else
        MsgBox(0x10, '', 'FileMove failed')
    EndIf
Else
    MsgBox(0x40, '', 'No double lines to cleanup')
EndIf
MsgBox(0, '', $contents)

somefile.txt

line1
line2


line5
line6

The double empty lines are removed.

[Briegel 0]

@MHz,

with my file (only for test with double lines ) it don't work.

First MsgBox: 'No double lines to cleanup'

Second MsgBox:

[boot loader]
timeout=0
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS

Lines 3, 4 and 7!

[gempie 0]

Hi guys,

i have this so far

$file = FileOpen("c:\patient.txt", 0)
If $file = -1 Then
    
    Exit
EndIf
$save = ""
; Read in lines of text until the EOF is reached
While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    If StringInStr($line, "long")Then $save = $save & $line & @CRLF
WEnd

FileClose($file)

MsgBox(48, "patient.txt", $save)

the double lines all start with "long". So in the end of my script $save has all the doubles behind each other. The only thing now is that i have to delete line 2,4,6,8 until EOF (or line 1,3,5,7 until EOF)

any ideas?

[jvanegmond 307]

@MHz,

I tried your script and got this SciTE-error:

ERROR: StringReplace() [built-in] called with wrong number of args.

If you call a FileRead, it reads the whole file by default, so you dont have to put in the number of characters.

[MHz 80]

@Manadar, if you are referring to the use of FileGetSize, then Autoit v3.1.1.0 needs it to read the whole file.

@gempie, look at using ContinueLoop to skip lines that are not wanted. Changes based with known info.

$file = FileOpen("c:\patient.txt", 0)
If $file = -1 Then
    
    Exit
EndIf
$save = ""
; Read in lines of text until the EOF is reached
While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    If StringLeft($line, 4) = "long" Then ContinueLoop
    $save = $save & $line & @CRLF
WEnd

FileClose($file)

MsgBox(48, "patient.txt", $save)

[Briegel 0]

Hi guys,

i have this so far

...

the double lines all start with "long". So in the end of my script $save has all the doubles behind each other. The only thing now is that i have to delete line 2,4,6,8 until EOF (or line 1,3,5,7 until EOF)

any ideas?

Great, but that the double lines starts with 'long' is new.

If you call a FileRead, it reads the whole file by default, so you dont have to put in the number of characters.

Sorry you're wrong this time (or do I misunderstand you?). The errormessage came from 'StringReplace()'.

Original from MHz:

$contents = StringReplace(FileRead($file, FileGetSize($file)), @CRLF & @CRLF & @CRLF & @CRLF)oÝ÷ ØGbµ¼6®¶­sbb33c¶6öçFVçG2Ò7G&æu&WÆ6RfÆU&VBb33c¶fÆRÂfÆTvWE6¦Rb33c¶fÆRÂ5$Äbfײ5$Äbfײ5$ÄbÂ5$Äb

[jvanegmond 307]

@Manadar, if you are referring to the use of FileGetSize, then Autoit v3.1.1.0 needs it to read the whole file.

From the helpfile:

FileRead ( filehandle or "filename" [, count] )

filehandle = The handle of a file, as returned by a previous call to FileOpen. Alternatively you may use a string filename as the first parameter.

count = [optional] The number of characters to read. Default read the entire file.

Or did i overlook something?

Edited July 18, 2006 by Manadar

[SmOke_N 212]

Great, but that the double lines starts with 'long' is new.

Sorry you're wrong this time (or do I misunderstand you?). The errormessage came from 'StringReplace()'.

Original from MHz:

$contents = StringReplace(FileRead($file, FileGetSize($file)), @CRLF & @CRLF & @CRLF & @CRLF)oÝ÷ ØGbµ¼6®¶­sbb33c¶6öçFVçG2Ò7G&æu&WÆ6RfÆU&VBb33c¶fÆRÂfÆTvWE6¦Rb33c¶fÆRÂ5$Äbfײ5$Äbfײ5$ÄbÂ5$Äb

No he's not wrong, for beta anyway... I believe the release version does need FileGetSize($file), when dealing with new people MHz will usually will write the code in release version because 90% haven't downloaded the beta yet.

[jvanegmond 307]

Right on the spot.

From release AutoIt's help file:

count = The number of characters to read.

[MHz 80]

Or did i overlook something?

You did. Look at 3.1.1.0 Release Helpfile on FileRead as I mentioned.

Edit:

Sorry. seem late at posting

Edited July 18, 2006 by MHz

[gempie 0]

Hi Guys,

Yhanx for all your efforts so far, thanx alot.

Oke, now i have 2 scripts:

1e

expandcollapse popup

$file = FileOpen("c:\patient.txt", 0)


If $file = -1 Then
    
    Exit
EndIf
$save = ""
; Read in lines of text until the EOF is reached
While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    If StringInStr($line, "long")Then $save = $save & $line & @CRLF
    msgbox(48,"test",$save)
WEnd

FileClose($file)
MsgBox(48, "patient.txt", $save)
FileWrite ("test.txt",$save)
sleep (2000)

$file2 = FileOpen("test.txt", 0)

If $file2 = -1 Then
    
    Exit
EndIf

For $i = 1 to 20 Step +2
    
$line2 = FileReadLine($file2,$i)
If @error = -1 Then ExitLoop
MsgBox(0, "Line read:", $line2)
FileWrite ("test2.txt",$line2 & @CRLF)
Next

FileClose($file2)

This leaves me with the file test2.txt which contains the single doubles (doubled out) i don't know how to say it in propper english

2e (which does basically the same as MHZ's)

$file = FileOpen("c:\patient.txt", 0)


If $file = -1 Then
    
    Exit
EndIf
$save = ""
; Read in lines of text until the EOF is reached
While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    If Not StringInStr($line, "long")Then $save = $save & $line & @CRLF
    msgbox(48,"test",$save)
WEnd

FileClose($file)
MsgBox(48, "patient.txt", $save)
FileWrite ("test3.txt",$save)

So this leaves me with 2 text files (test2.txt and test3.txt) which have to be joined together somehow. Any ideas on how to do this?

Thanx

[gempie 0]

Hey Guys,

Oke maybe someone can help me out with this idee. The doubles are allways following each other so i did this:

$file = FileOpen("test.txt", 0)

; Check if file opened for reading OK
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Read in lines of text until the EOF is reached

dim $line1, $line2
While 1
    $line1 = FileReadLine($file)
    If @error = -1 Then ExitLoop
    msgbox(0,"$line1=",$line1)
    If $line1 = $line2 then msgbox(48,"double","found")
    $line2 = FileReadLine($file)
    MsgBox(0,"$Line2=", $line2)
Wend

FileClose($file)

[SmOke_N 212]

Now that I think about this with a clearer head, I wrote a _FileDeleteLine() a while ago!

http://www.autoitscript.com/forum/index.ph...st&p=179430

It might help you with what your looking for.

Also, is this the type of approach you want.. If line 1 is blank and line 2 is blank then do something?

Dim $line1, $line2, $count = 0
While 1
    $count += 1
    $line1 = FileReadLine($file, $count)
    If @error = -1 Then ExitLoop
    $line2 = FileReadLine($file, $count + 1)
    If @error = -1 Then ExitLoop
    If StringStripWS($line1, 8) = '' And StringStripWS($line2, 8) = '' Then
        ;Do something
    EndIf
Wend