Remove duplicates in a array

[Valnurat]

How do I remove duplicates in an array with 2 columns?

 

I was looking at the _ArrayUnique, but it just return the value for 1 column.

Edited February 24, 2016 by Valnurat

[JohnOne]

It does not, show code, include test array.

Edited February 24, 2016 by JohnOne

[funkey]

https://www.autoitscript.com/forum/topic/44747-sort-1d2d-array-and-delete-doubles/

[Valnurat]

1 hour ago, JohnOne said:

It does not, show code, include test array.

I just looked at the autoit example.

I didn't do a testcode or something.

The autoit example shows that if you just use default in a 2d array it will just return 1 column or do I miss something?

[iamtheky]

No, there are just a metric shit ton of more questions when asking to simply dedup a 2d array.  So lets start, I'll provide an answer and you lead us down the bunny trail.  The biggest help would be the endgame, because I have to assume you want duplicates gone leaving the data in some sort of ingestable format. 

#include<array.au3>

local $a2D = [['1','2'],['3','4'],['4','5'],['5','6']]

_ArrayDisplay($a2D)

For $i = ubound($a2D) - 1 to 0 step -1
    For $c = 1 to 0 step -1
        $vSrch = _ArraySearch($a2D , $a2D[$i][$c] , 0 , $i - 1)
        If $vSrch > 1 then $a2D[$i][$c] = ""
    Next
Next

_ArrayDisplay($a2D)

 

[Valnurat]

Are you angry?

[BrewManNH]

I'm guessing he's just asking what the end result should look like, because there's a hundred ways of doing this and no one wants to waste time writing code when you want one of the other 99 ways it could be done.

Show us what the start array looks like, what you'd like the end array to look like, and then there's a goal to reach for. Also, don't change the parameters of your request in the middle of everything  because that's a sure way to cause anger issues. Make sure you know exactly what it is you need the code to do before you start asking for help, or writing the code.

[iamtheky]

Quote

Are you angry?

Nope, but inferring emotion in text is a good start at getting me there.  Focus on the task and less on the niceties, if you walked into a coding forum looking for people who are not completely socially inept, then thats on you.

[Valnurat]

Well, I did not expect I will get a code, but just someone who could put me in the right direction to solve it.

[mikell]

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

[iamtheky]

Sure, but there are dix who will tell you there a million ways to do something without giving you a single solution.  I prefer to be the dick who also provided a solution.  Are you not going to elaborate on what you would like the result to be?  My code > My Words

[danielcovington]

Which dimension of the array do you want to dedupe?

[czardas]

A very good question. I would recommend my ArrayWorkshop function for that, but it's not finished and only does 5 dimensions. Removing dupes based on the second dimension of a 2D array is equivalent to removing columns rather than rows. I suppose you could come up with a different interpretation, but I doubt I would be able to understand it.

[Valnurat]

28 minutes ago, mikell said:

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

I think I was just looking for a command to do the it, but I guess I need to come up with a solution to scan the array and remove duplicates by using what is already part of AutoIT commands.

[Valnurat]

I need to run through a path and find if there is duplicates.

My idea was to use _FileListToArray. With that I can get it 1D or 2D. If I use 1D I get the path and folder in 1 column, but if I use 2D I get the same, but in 2 columns.

I need the result in 2D, but without duplicates.

[czardas]

#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)

Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

Edited February 24, 2016 by czardas

[AutoBert]

11 hours ago, funkey said:

https://www.autoitscript.com/forum/topic/44747-sort-1d2d-array-and-delete-doubles/

 

[iamtheky]

1 hour ago, Valnurat said:

My idea was to use _FileListToArray. With that I can get it 1D or 2D.

FLTA returns a 1D array, FLATR returns a 1D,  so do you need as Czardas suggested?

1) a 2D array that looks like [Folder|File]

2) Where all filenames are unique without regard to folder location

[Valnurat]

10 hours ago, czardas said:

#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)

Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

You absolutely correct about windows won't allow it. In my company we have sites that have the backup folder located the same place. \\servername\sharename\%username%  I need to scan all locations for those sites. We have 3 sites (DKRO,DKVE,SESC) that use the same uncpath and then we have sites that use a uniqe uncpath, that's specified by the location of the OU the user is a member of. So I will end up be scanning 3 times the same uncpath. That is why I will have duplicates. I have done some code, but I was just looking for an easy way to solve it by using a fancy command.

Quote

Func _AllFolders()
    if $bDebugMode Then
        ConsoleWrite(@CRLF & "Collecting all folders." & @CRLF)
    Else
        _FileWriteLog($hFile, @CRLF & "Collecting all folders." & @CRLF)
    EndIf
    Local $aListOfFolders, $aTempListOfAllFolders[0]
    for $i = 0 to UBound($aAllBackupDestination) -1
        if $aAllBackupDestination[$i][0] <> "" then
            $aListOfFolders = _FileListToArray($aAllBackupDestination[$i][0],Default,2,True) ;Gives me 1D with path and folders
            _ArrayDelete($aListOfFolders,0)
            _ArrayConcatenate($aTempListOfAllFolders,$aListOfFolders)
        EndIf
    Next
    _ArrayDisplay($aListOfAllFolders,"$aListOfAllFolders")
    $aTempListOfAllFolders = _ArrayUnique($aTempListOfAllFolders)
    $aListOfAllFolders = _Array1to2($aTempListOfAllFolders)
EndFunc

Func _Array1to2($array)
    Local $aResult[UBound($array)][2], $aSplitAllFolders
    For $i = 0 To UBound($array) -1
        $aSplitAllFolders = StringSplit($array[$i],'\')
        if $aSplitAllFolders[0] = 5 Then
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4]
            $aResult[$i][1] = $aSplitAllFolders[5]
        Else
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4] & "\" & $aSplitAllFolders[5]
            $aResult[$i][1] = $aSplitAllFolders[6]
        EndIf
    Next
    Return $aResult
EndFunc

 

I'm NOT sure if this is fancy way of doing it, but it works fine.

[iamtheky]

So pull everything, but then only check against the portion of the string you need uniqueness on ?

like keeping this 1D and in the loop checking against the last split

#include<File.au3>
$aList = _FileListToArray(@ScriptDir , '*' , 2, True)

_ArrayDisplay($aList)

for $i = ubound($aList) - 1 to 1 step -1
    ;item to compare
    msgbox(0, '' , stringreverse(stringsplit(stringreverse($aList[$i]), "\" , 2)[0]))
next

 

Edited February 25, 2016 by iamtheky