Clean up array from any duplicate elements: _ArrayRemoveDuplicates()

[uteotw]

Hi

I've been dealing with bigger arrays recently, and I need to remove duplicates elements from these arrays.

I wasn't satisfied with the simple,crappy and slow compare/delete routine I wrote.

So after searching the forums I found several good UDFs but the one from nitro322 and SmOke_N in this post http://www.autoitscript.com/forum/index.php?showtopic=7821 had the best performance.

So I decided take it and modify.

expandcollapse popup

;==================================================================
; Function Name:  _ArrayRemoveDuplicates()
;
; Description    :  Removes duplicate elements from an Array
; Parameter(s)   :  $avArray
;                   $iBase
;                   $iCaseSense
;                   $sDelimter
; Requirement(s) :  None
; Return Value(s):  On Success - Returns 1 and the cleaned up Array is set
;                   On Failure - Returns an -1 and sets @Error
;                        @Error=1 $avArray is not an array
;                        @Error=2 $iBase is different from 1 or 2
;                        @Error=3 $iCaseSense is different from 0 or 1
; Author         :  uteotw, but ALL the credits go to nitro322 and SmOke_N, see link below
; Note(s)        :  None
; Link           ;  http://www.autoitscript.com/forum/index.php?showtopic=7821
; Example        ;  Yes
;==================================================================
Func _ArrayRemoveDuplicates(ByRef $avArray, $iBase = 0, $iCaseSense = 0, $sDelimter = "")
    Local $sHold
    
    If Not IsArray($avArray) Then
        SetError(1)
        Return -1
    EndIf
    If Not ($iBase = 0 Or $iBase = 1) Then
        SetError(2)
        Return -1
    EndIf
    If $iBase = 1 AND $avArray[0] = 0 Then
        SetError(0)
        Return 0
    EndIf
    If Not ($iCaseSense = 0 Or $iCaseSense = 1) Then
        SetError(3)
        Return -1
    EndIf
    If $sDelimter = "" Then
        $sDelimter = Chr(01) & Chr(01)
    EndIf
 
    If $iBase = 0 Then
        For $i = $iBase To UBound($avArray) - 1
            If Not StringInStr($sDelimter & $sHold, $sDelimter & $avArray[$i] & $sDelimter, $iCaseSense) Then
                $sHold &= $avArray[$i] & $sDelimter
            EndIf
        Next
        $avNewArray = StringSplit(StringTrimRight($sHold, StringLen($sDelimter)), $sDelimter, 1)
        ReDim $avArray[$avNewArray[0]]
        For $i = 1 to $avNewArray[0]
            $avArray[$i-1] = $avNewArray[$i]
        Next
    ElseIf $iBase = 1 Then
        For $i= $iBase To UBound($avArray) - 1
            If Not StringInStr($sDelimter & $sHold, $sDelimter & $avArray[$i] & $sDelimter, $iCaseSense) Then
                $sHold &= $avArray[$i] & $sDelimter
            EndIf
        Next
        $avArray = StringSplit(StringTrimRight($sHold, StringLen($sDelimter)), $sDelimter, 1)
    EndIf

    Return 1
EndFuncoÝ÷ Øw«yªÞ²{¦¦W¬jëh×6#include<Array.au3>

; Example # 1 : using Base 0 array
Dim $avArray[10]
$avArray[0] = "JPM"
$avArray[1] = "STRING1"
$avArray[2] = "Jon"
$avArray[3] = "Larry"
$avArray[4] = "Jeremy"
$avArray[5] = "Valik"
$avArray[6] = "String1"     ; this element will be deleted from the updated array
$avArray[7] = "String2"
$avArray[8] = "JdeB"
$avArray[9] = "STRING2"     ; this element will be deleted from the updated array

_ArrayDisplay($avArray, "Example # 1 : Before")
_ArrayRemoveDuplicates($avArray)
_ArrayDisplay($avArray, "Example # 1 : After")


; Example # 2 : Base 1 array
Dim $avArray[11]
$avArray[0] = 10
$avArray[1] = "JPM"
$avArray[2] = "String1"
$avArray[3] = "Jon"
$avArray[4] = "Split/Split"
$avArray[5] = "Jeremy"
$avArray[6] = "Valik"
$avArray[7] = "String1"     ; this element will be deleted from the updated array
$avArray[8] = "String2"
$avArray[9] = "JdeB"
$avArray[10] = "STRING2"    ; this element will be deleted from the updated array

_ArrayDisplay($avArray, "Example # 2 : Before")
_ArrayRemoveDuplicates($avArray, 1)
_ArrayDisplay($avArray, "Example # 2 : After")


; Example # 3 : Base 1 array and Case Sensitive
Dim $avArray[11]
$avArray[0] = 10
$avArray[1] = "JPM"
$avArray[2] = "String1"
$avArray[3] = "Jon"
$avArray[4] = "Split/Split"
$avArray[5] = "Jeremy"
$avArray[6] = "Valik"
$avArray[7] = "String1"     ; this element will be deleted from the updated array
$avArray[8] = "String2"
$avArray[9] = "JdeB"
$avArray[10] = "STRING2"    ; this element will NOT be deleted from the updated array

_ArrayDisplay($avArray, "Example # 3 : Before")
_ArrayRemoveDuplicates($avArray, 1, 1)
_ArrayDisplay($avArray, "Example # 3 : After")


; Example # 4 : Base 1 array, Case Sensitive and using "/" as Delimiter
Dim $avArray[11]
$avArray[0] = 10
$avArray[1] = "JPM"
$avArray[2] = "String1"
$avArray[3] = "Jon"
$avArray[4] = "Split/Split" ; this element will become 2 elements in the updated array
$avArray[5] = "Jeremy"
$avArray[6] = "Valik"
$avArray[7] = "String1"     ; this element will be deleted from the updated array
$avArray[8] = "String2"
$avArray[9] = "JdeB"
$avArray[10] = "STRING2"    ; this element will NOT be deleted from the updated array

_ArrayDisplay($avArray, "Example # 4 : Before")
_ArrayRemoveDuplicates($avArray, 1, 1, "/")
_ArrayDisplay($avArray, "Example # 4 : After")

Hope this will useful.

uteotw

_ArrayRemoveDuplicates.au3

ArrayRemoveDuplicates.au3

Edited March 30, 2008 by uteotw

[Wooltown]

Nice idea :-)

Could be usefull, added it to my AutoIT favourites.