Jump to content
Sign in to follow this  
ffdshow

Delete duplicate lines from a file, but with some exceptions

Recommended Posts

ffdshow

A simple script like the next one can delete duplicate lines from files:

#include<Array.au3>
#include<File.au3>

$InputFile = "C:\Test.txt"

Local $aFile, $aArray
_FileReadToArray($InputFile, $aFile)
$aArray = _ArrayUnique($aFile, 1, 1)
_FileWriteFromArray($InputFile, $aArray, 1)

How can I preserve, for example the duplicate empty lines and the duplicate lines that contain a string like %16422% ?

Share this post


Link to post
Share on other sites
Malkey

This _ArrayUniqueEX() function appears to work.

#include<Array.au3>
#include<File.au3>

Local $aFile, $aArray
Local $InputFile = @ScriptDir & "\Test-2.txt"
Local $OutputFile = @ScriptDir & "\Test-2b.txt"

$aFile = FileReadToArray($InputFile)
;_ArrayDisplay($aFile)

$aArray = _ArrayUniqueEX($aFile)
;_ArrayDisplay($aArray)

_FileWriteFromArray($OutputFile, $aArray, 1)
ShellExecute($OutputFile)


Func _ArrayUniqueEX(Const ByRef $aArray)
    Local $sData = ChrW(160), $sSep = ChrW(160)
    For $i = 0 To UBound($aArray) - 1
        If $aArray[$i] = "" Or _                                                  ; Add blank lines.
                StringInStr($aArray[$i], "%16422%") <> 0 Or _                     ; Add lines with "%16422%" present.
                StringInStr($sData, ChrW(160) & $aArray[$i] & ChrW(160)) = 0 Then ; Add lines that do not already exist in $sData.
            $sData &= $aArray[$i] & $sSep
        EndIf
    Next
    Return StringSplit(StringTrimRight(StringTrimLeft($sData, 1), 1), $sSep, 0)
EndFunc   ;==>_ArrayUniqueEX
  • Like 1

Share this post


Link to post
Share on other sites
ffdshow

Yes, it's working. Thanks!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×