Jump to content

Delete duplicate lines from a file, but with some exceptions


Go to solution Solved by Malkey,

Recommended Posts

A simple script like the next one can delete duplicate lines from files:

#include<Array.au3>
#include<File.au3>

$InputFile = "C:\Test.txt"

Local $aFile, $aArray
_FileReadToArray($InputFile, $aFile)
$aArray = _ArrayUnique($aFile, 1, 1)
_FileWriteFromArray($InputFile, $aArray, 1)

How can I preserve, for example the duplicate empty lines and the duplicate lines that contain a string like %16422% ?

Link to comment
Share on other sites

  • Solution

This _ArrayUniqueEX() function appears to work.

#include<Array.au3>
#include<File.au3>

Local $aFile, $aArray
Local $InputFile = @ScriptDir & "\Test-2.txt"
Local $OutputFile = @ScriptDir & "\Test-2b.txt"

$aFile = FileReadToArray($InputFile)
;_ArrayDisplay($aFile)

$aArray = _ArrayUniqueEX($aFile)
;_ArrayDisplay($aArray)

_FileWriteFromArray($OutputFile, $aArray, 1)
ShellExecute($OutputFile)


Func _ArrayUniqueEX(Const ByRef $aArray)
    Local $sData = ChrW(160), $sSep = ChrW(160)
    For $i = 0 To UBound($aArray) - 1
        If $aArray[$i] = "" Or _                                                  ; Add blank lines.
                StringInStr($aArray[$i], "%16422%") <> 0 Or _                     ; Add lines with "%16422%" present.
                StringInStr($sData, ChrW(160) & $aArray[$i] & ChrW(160)) = 0 Then ; Add lines that do not already exist in $sData.
            $sData &= $aArray[$i] & $sSep
        EndIf
    Next
    Return StringSplit(StringTrimRight(StringTrimLeft($sData, 1), 1), $sSep, 0)
EndFunc   ;==>_ArrayUniqueEX
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...