RegExp for delete duplicate lines

[michaelslamet]

I got this code from PhoenixXL at this >post

#include <Array.au3>

Local $File = "First Line" & @CR & "Second Line" & @CRLF & "first line"
Local $aRegEx = StringRegExp( $File, '[^\v]+', 3 )
If @error Then Exit -1 ;No matches

_ArrayDisplay( $aRegEx )
$aRegEx = _ArrayUnique( $aRegEx )
_ArrayDelete( $aRegEx, 0 )
_ArrayDisplay( $aRegEx )

 

What is a correct RegExp pattern to let the code understand that "First Line" is the same (duplicate) with "first line     ", "    first      line", "firstline", "f i r s t l i n e" ?

Merry Christmas, Folks!

[Malkey]

Here are a couple of methods.

#include <Array.au3>

Local $File = "First Line" & @CR & "Second Line" & @CRLF & _
        "first line" & @CRLF & "First Line" & @CRLF & _
        "first line     " & @CRLF & "    first      line" & @CRLF & _
        "firstline" & @CRLF & "  second  line " & @CRLF & "f i r s t l i n e"

Local $aRegEx = StringRegExp($File, '\V+', 3)
If @error Then Exit -1 ;No matches

_ArrayDisplay($aRegEx)
Local $iUbnd = UBound($aRegEx) - 1
For $j = 0 To $iUbnd
    For $i = $iUbnd To $j Step -1
        If $i <> $j And StringCompare(StringStripWS($aRegEx[$i], 8), StringStripWS($aRegEx[$j], 8)) = 0 Then
            _ArrayDelete($aRegEx, $i)
            $iUbnd -= 1
        EndIf
    Next
Next

_ArrayDisplay($aRegEx)

and,

#include <Array.au3>

Local $File = "First Line" & @CR & "Second Line" & @CRLF & _
        "first line" & @CRLF & "First Line" & @CRLF & _
        "first line     " & @CRLF & "    first      line" & @CRLF & _
        "firstline" & @CRLF & "f i r s t l i n e"

Local $aRegEx = StringRegExp($File, '\V+', 3)
If @error Then Exit -1 ;No matches

_ArrayDisplay($aRegEx)
Local $iUbnd = UBound($aRegEx) - 1
Local $iCount = 0, $aNewArray[$iUbnd + 1]
For $j = 0 To $iUbnd ; step -1
    $sPattern = "(?im)(^\h*" & StringRegExpReplace(StringStripWS($aRegEx[$j], 8), "(\V+?)", "\1\\h*") & "(\v*|$))"
    ConsoleWrite($sPattern & @TAB & $aRegEx[$j] & @LF)
    If StringRegExp($File, $sPattern) Then
        $aNewArray[$iCount] = $aRegEx[$j]
        $File = StringRegExpReplace($File, $sPattern, "")
        $iCount += 1
    EndIf
    If $File = "" Then ExitLoop
Next

ReDim $aNewArray[$iCount]
_ArrayDisplay($aNewArray, "Unique")

[AZJIO]

RegExp is not the best for this task.