Output MultiD array with an ArrayUnique type function

[Champak]

The current arrayuniqe outputs a 1D array. of the unique rows based on a specific column in the multiD array. I need to output the exact same multiD array minus the duplicates. So 7 columns in, 7 columns out. How?

Thanks.

[jdelaney]

Define what is considered unique.

You could do an outer loop, that grabs the current record, and then an inner loop that re-loops through the array, to see if any dupes (logically skip the current outer loop value durring the inner loop)...if dupe is found, move all the records up, and re-dim.

I'd suggest moving from the ubound of the array to 0, on the outer loop.

Try it out, post some code.  Post your data.  Post something.

Edited February 13, 2014 by jdelaney

[BrewManNH]

Feed _ArrayUnique the array one column at a time. Then recombine when you're done.

[jdelaney]

That would work only if the uniquness is on columns, where there is no association of the values of rows.

I don't see why you would have a 2d array, at that point.

example:

Local $aArray[6][2] = [[1, "A"],[2, "B"],[3, "C"],[4, "D"],[5, "E"],[6, "E"]]
_ArrayDisplay($aArray, "$aArray")
$aNewArray1 = _ArrayUnique($aArray, 1, 0, 1) ;Using Default Parameters, with Case-Sensitivity
$aNewArray2 = _ArrayUnique($aArray, 2, 0, 1) ;Using Default Parameters, with Case-Sensitivity

If UBound($aNewArray1)>=UBound($aNewArray2) Then
    Local $combine[UBound($aNewArray1)][2]
Else
    Local $combine[UBound($aNewArray2)][2]
EndIf

For $i = 0 To UBound($aNewArray1) - 1
    $combine[$i][0]=$aNewArray1[$i]
    If $i < UBound($aNewArray2) Then
        $combine[$i][1]=$aNewArray2[$i]
    EndIf
Next
_ArrayDisplay($combine)

even though all rows are unique, relative to the other rows, you get this:

[0]|6|5
[1]|1|A
[2]|2|B
[3]|3|C
[4]|4|D
[5]|5|E
[6]|6|
 

The OP needs to give MUCH more information.

Edited February 13, 2014 by jdelaney

[Champak]

If I have an array that is [10][5] for example, and the sorting column might be the third column, and in that column there are four elements that are duplicates, the resulting array should spit out [7][5]. All other rows with the duplicate elements would be gone. I don't know how to explain it any better.

[jdelaney]

Provide us with 2 arrays.

One that's the original, and one that's the expected outcome.

[Champak]

 Element 3 and 5 are the same as 2 in column 2, so row 3 and 5 are removed.

#include <Array.au3>


Local $avArray1[6][4] = [ _
        ["String0", "SubString0", "2String0", "2SubString0"], _
        ["String1", "SubString1=====", "2String0", "2SubString0"], _
        ["String2", "SubString1=====", "2String0", "2SubString0"], _
        ["String3", "SubString3", "2String0", "2SubString0"], _
        ["String4", "SubString1=====", "2String0", "2SubString0"], _
        ["String5", "SubString5", "2String0", "2SubString0"]]

_ArrayDisplay($avArray1)

Local $avArray2[4][4] = [ _
        ["String0", "SubString0", "2String0", "2SubString0"], _
        ["String1", "SubString1=====", "2String0", "2SubString0"], _
        ["String3", "SubString3", "2String0", "2SubString0"], _
        ["String5", "SubString5", "2String0", "2SubString0"]]

_ArrayDisplay($avArray2)

[jdelaney]

This will work:

expandcollapse popup

#include <array.au3>
Local $aTemp[6][4] = [ _
        ["String0", "SubString0", "2String0", "2SubString0"], _
        ["String1", "SubString1=====", "2String0", "2SubString0"], _
        ["String2", "SubString1=====", "2String0", "2SubString0"], _
        ["String3", "SubString3", "2String0", "2SubString0"], _
        ["String4", "SubString1=====", "2String0", "2SubString0"], _
        ["String5", "SubString5", "2String0", "2SubString0"]]

$aUnique_2D = _ArrayUnique_2D($aTemp,1)
_ArrayDisplay($aUnique_2D)

Func _ArrayUnique_2D($aArray, $iColumn)
    
    ; Create array of survivors (unique)
    Local $aTemp[UBound($aArray)]
    For $i = 0 To UBound($aTemp)-1
        $aTemp[$i] = $i
    Next
    
    ; Remove dupes from survivng array
    For $i = 0 To UBound($aArray)-1
        $bFound = False
        For $j = UBound($aArray)-1 To 0 Step -1
            If $i = $j Or $aTemp[$j]="" Then ContinueLoop
            If $aArray[$i][$iColumn]=$aArray[$j][$iColumn] Then
                $aTemp[$j] = ""
                $aArray[$j][$iColumn] = "[ToBeDeleted]"
            EndIf
        Next
    Next

    ; Cleanup to drive survivors loop
    For $i = UBound($aTemp)-1 To 0 Step -1
        If String($aTemp[$i])="" Then
            _ArrayDelete($aTemp, $i)
        EndIf
    Next

    ; Move survivors to surviving rows
    For $i = 0 To UBound($aTemp) - 1
        If $i = $aTemp[$i] Then ContinueLoop ; no movement required, skip
        For $j = 0 To UBound($aArray,2)-1
            $aArray[$i][$j] = $aArray[$aTemp[$i]][$j]
        Next
    Next
    ReDim $aArray[UBound($aTemp)][UBound($aArray,2)]

    Return $aArray
EndFunc

output:

[0]|String0|SubString0|2String0|2SubString0
[1]|String1|SubString1=====|2String0|2SubString0
[2]|String3|SubString3|2String0|2SubString0
[3]|String5|SubString5|2String0|2SubString0
 

Edited February 13, 2014 by jdelaney

[Gianni]

another (shorter) way:

also included an example with an array of towns grabbed from internet:
in input array column 7 contains  duplicate rows (there are duplicate region names)
in output array only one region per row (there are no duplicates in column 7)

expandcollapse popup

#include <IE.au3>
#include <Array.au3>

Local $avArray1[6][4] = [ _
        ["String0", "SubString0", "2String0", "2SubString0"], _
        ["String1", "SubString1=====", "2String0", "2SubString0"], _
        ["String2", "SubString1=====", "2String0", "2SubString0"], _
        ["String3", "SubString3", "2String0", "2SubString0"], _
        ["String4", "SubString1=====", "2String0", "2SubString0"], _
        ["String5", "SubString5", "2String0", "2SubString0"]]

_ArrayDisplay($avArray1)
$avArray2 = _ArrayUnique_2D($avArray1, 2) ; filter on column 2 (1 based)
_ArrayDisplay($avArray2)

; grab a table of italy cities from web --------------------------------------
Local $oIE = _IECreate("http://en.wikipedia.org/wiki/List_of_cities_in_Italy")
Do
    Local $oTable = _IETableGetCollection($oIE, 0)
Until IsObj($oTable)
Local $aTableData = _IETableWriteToArray($oTable, 1)
_IEQuit($oIE)
_ArrayDelete($aTableData, 0) ; remove first row with column titles
; ----------------------------------------------------------------------------

_ArrayDisplay($aTableData, "List of some italian towns. Column 6 contains duplicates")
$aTableData2 = _ArrayUnique_2D($aTableData, 7) ; filter on column 7 (1 based)
_ArrayDisplay($aTableData2, "region are unique and not duplicate in column 7")


Func _ArrayUnique_2D($aTableData, $column)
    $Uniques = _ArrayUnique($aTableData, $column) ; unique elements from column $column
    ; _ArrayDisplay($Uniques,"only unique elements from specified column")
    ; create a new array with room for only unique rows
    Local $aTableData2[$Uniques[0]][UBound($aTableData, 2)] ; create a new array for output

    For $i = 1 To $Uniques[0] ; loop all rows of unique elements
        ; search first occurence of element $i in main table ; $ndx is row where is found
        $ndx = _ArraySearch($aTableData, $Uniques[$i], 0, 0, 0, 0, 1, $column - 1)

        ; copy data from all columns from source array to new array
        For $x = 0 To UBound($aTableData2, 2) - 1 ; loop all columns of row $ndx
            $aTableData2[$i - 1][$x] = $aTableData[$ndx][$x] ; copy data of row $ndx from main to new array
        Next
    Next
    Return $aTableData2
EndFunc   ;==>_ArrayUnique_2D

Edited February 14, 2014 by PincoPanco

[jdelaney]

Good call.  I wasted effort doing my own array unique.

[Malkey]

Here is another slightly smaller way.

_ArrayUnique function is not necessary when the _ArraySearch function is used.

#include <Array.au3>

Local $avArray1[6][4] = [ _
        ["String0", "SubString0", "2String0", "2SubString0"], _
        ["String1", "SubString1=====", "2String0", "2SubString0"], _
        ["String2", "SubString1=====", "2String0", "2SubString0"], _
        ["String3", "SubString3", "2String0", "2SubString0"], _
        ["String4", "SubString1=====", "2String0", "2SubString0"], _
        ["String5", "SubString5", "2String0", "2SubString0"]]

_ArrayDisplay($avArray1,"Original Array")

$aUnique_2D = _ArrayUnique_2D($avArray1, 1)
_ArrayDisplay($aUnique_2D,"Unique 2D Array")


; First column = 0, default.
Func _ArrayUnique_2D($aTableData, $column = 0)
    Local $iCount = 0
    Local $aTableData2[UBound($aTableData, 1)][UBound($aTableData, 2)] ; create a new array for output

    For $i = 0 To UBound($aTableData) - 1
        $ndx = _ArraySearch($aTableData2, $aTableData[$i][$column], 0, 0, 0, 0, 1, $column) ; Search new array
        If $ndx = -1 Then  ; If element does not exist, then, add new row to new array.
            For $x = 0 To UBound($aTableData2, 2) - 1 ; loop all columns of row $ndx
                $aTableData2[$iCount][$x] = $aTableData[$i][$x] ; copy data of row $ndx from old to new array
            Next
            $iCount += 1
        EndIf
    Next
    ReDim $aTableData2[$iCount][UBound($aTableData2, 2)]
    Return $aTableData2
EndFunc   ;==>_ArrayUnique_2D

[Gianni]

 

Here is another slightly smaller way.

_ArrayUnique function is not necessary when the _ArraySearch function is used.

 

nice, anyway I do not know if this is an improvement (especially on long arrays)

It seems that more steps and a longer loop are required

[Champak]

Thanks to all. I think something like this should definitely be included as a standard function.

[czardas]

Defining how a multidimensional unique array function should behave seems quite complicated. With a 2D array it's not so complicated - you simply eliminate rows or columns. Personally I would only consider a row to be unique if all subitems were also unique. This stricter definition would, at least, be easier to scale up to include more dimensions.

Edited February 15, 2014 by czardas