Array: Duplicate entries and sorting

[Lumbo]

Hi all, I'm new to the forum but I am using AutoIt since years.

I'm having some trouble with arrays, I need a fast solution for remove duplicate entries.

Example:

I have $FirstArray with 50.000 elements (many of these may be the same) to merge with $SecondArray which has 10.000 elements all different and alphabetically ordered but I want remove duplicate elements so I have a $FinalArray with both Arrays but no clones.

(I searched and found this topic but didn't help)

#include <Array.au3>
$FirstString = "john|tom|sam|mark|sam|betty|george|tony|tom|gus" ;Unsorted large array
$FirstArray = StringSplit($FirstString, "|")

$SecondString = "anna|betty|carl|donna|emily|fiona|gus|michael" ;Sorted array
$SecondArray = StringSplit($SecondString, "|")

Dim $FinalArray[50000] ;Not sure about this

; I thought something like this but the search has to be done for each array item and I can have more than 50.000 items
; Anyway this doesn't check for duplicate entries on $FirstArray ($SecondArray is already filtered and sorted)
For $i = 1 To UBound($FirstArray)-1
     If Not _ArrayBinarySearch($SecondArray, $FirstArray[$i]) Then
        _ArrayAdd($FinalArray, $FirstArray[$i])
     EndIf
Next

_ArraySort($FinalArray) ;Sort Array (I need $FinalArray sorted)
_ArrayDisplay($FinalArray) ;Show

(by the way this doesn't even work)

Thanks in advance for your help

Hope you understand me

Edited March 5, 2008 by Lumbo

[randallc]

Hi,

Probably a faster way to do this (native Obj Script Dictionary), but it will still be competitive;

; ArrayDupesEx.au3; v3.2.11.1 of AutoIt
; _ArrayDupes(ByRef $arrItemsF, $iDelete = 0, $iDetails = 0)
#include-once
#include<Array.au3>
#include<_ArraySortClib5v3.au3> ;@Saou
#include<string.au3>
#include"ArrayDupes8.au3"
$FirstString = "john|tom|sam|mark|sam|betty|george|tony|tom|gus" ;Unsorted large array
$FirstArray = _StringSplit($FirstString, "|"); v3.2.11.1 of AutoIt
_ArrayDisplay($FirstArray,"$FirstArray")

$SecondString = "anna|betty|carl|donna|emily|fiona|gus|michael" ;Sorted array
$SecondArray = _StringSplit($SecondString, "|")
_ArrayDisplay($SecondArray,"$SecondArray")
_ArrayConcatenate($SecondArray,$FirstArray)
_ArrayDisplay($SecondArray,"$SecondArray")

global $arrdupes0=_ArrayDupes( $SecondArray,  1,  0); _ArrayDupes(ByRef $arrItemsF, $iDelete = 0, $iDetails = 0)
_ArrayDisplay($arrdupes0,"$arrdupes0;Dupes only;   "&UBound($arrdupes0)-1)
_ArrayDisplay($SecondArray,"$SecondArray Not Sorted, Dupes Removed;   "&UBound($SecondArray)-1)
;_ArraySortClib(ByRef $Array, $iMode = 1, $fDescend = False, $iStart = 0, $iEnd = 0, $iColumn = 0, $iWidth = 0)
_ArraySortClib($SecondArray)

_ArrayDisplay($SecondArray,"$SecondArray Sorted;   "&UBound($SecondArray)-1)

Best, Randall

ArrayDupesEx2.zip

[herewasplato]

http://www.autoitscript.com/forum/index.ph...st&p=483184

...might work for you.

edit: Doh - did not see randallc's post

Edited March 5, 2008 by herewasplato

[Lumbo]

Great, ArrayDupes is what I searched for. I found it but could not understand how it works.

Your example made it clear.

Thank you both.