Sign in to follow this  
Followers 0
ffdshow

Delete duplicate lines from a file, but with some exceptions

3 posts in this topic

A simple script like the next one can delete duplicate lines from files:

#include<Array.au3>
#include<File.au3>

$InputFile = "C:\Test.txt"

Local $aFile, $aArray
_FileReadToArray($InputFile, $aFile)
$aArray = _ArrayUnique($aFile, 1, 1)
_FileWriteFromArray($InputFile, $aArray, 1)

How can I preserve, for example the duplicate empty lines and the duplicate lines that contain a string like %16422% ?

Share this post


Link to post
Share on other sites



This _ArrayUniqueEX() function appears to work.

#include<Array.au3>
#include<File.au3>

Local $aFile, $aArray
Local $InputFile = @ScriptDir & "\Test-2.txt"
Local $OutputFile = @ScriptDir & "\Test-2b.txt"

$aFile = FileReadToArray($InputFile)
;_ArrayDisplay($aFile)

$aArray = _ArrayUniqueEX($aFile)
;_ArrayDisplay($aArray)

_FileWriteFromArray($OutputFile, $aArray, 1)
ShellExecute($OutputFile)


Func _ArrayUniqueEX(Const ByRef $aArray)
    Local $sData = ChrW(160), $sSep = ChrW(160)
    For $i = 0 To UBound($aArray) - 1
        If $aArray[$i] = "" Or _                                                  ; Add blank lines.
                StringInStr($aArray[$i], "%16422%") <> 0 Or _                     ; Add lines with "%16422%" present.
                StringInStr($sData, ChrW(160) & $aArray[$i] & ChrW(160)) = 0 Then ; Add lines that do not already exist in $sData.
            $sData &= $aArray[$i] & $sSep
        EndIf
    Next
    Return StringSplit(StringTrimRight(StringTrimLeft($sData, 1), 1), $sSep, 0)
EndFunc   ;==>_ArrayUniqueEX
1 person likes this

Share this post


Link to post
Share on other sites

Yes, it's working. Thanks!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0