Jump to content

Remove duplicates in a array


Valnurat
 Share

Recommended Posts

1 hour ago, JohnOne said:

It does not, show code, include test array.

I just looked at the autoit example.

I didn't do a testcode or something.

The autoit example shows that if you just use default in a 2d array it will just return 1 column or do I miss something?

Yours sincerely

Kenneth.

Link to comment
Share on other sites

No, there are just a metric shit ton of more questions when asking to simply dedup a 2d array.  So lets start, I'll provide an answer and you lead us down the bunny trail.  The biggest help would be the endgame, because I have to assume you want duplicates gone leaving the data in some sort of ingestable format. 

#include<array.au3>

local $a2D = [['1','2'],['3','4'],['4','5'],['5','6']]

_ArrayDisplay($a2D)

For $i = ubound($a2D) - 1 to 0 step -1
    For $c = 1 to 0 step -1
        $vSrch = _ArraySearch($a2D , $a2D[$i][$c] , 0 , $i - 1)
        If $vSrch > 1 then $a2D[$i][$c] = ""
    Next
Next

_ArrayDisplay($a2D)

 

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Link to comment
Share on other sites

I'm guessing he's just asking what the end result should look like, because there's a hundred ways of doing this and no one wants to waste time writing code when you want one of the other 99 ways it could be done.

Show us what the start array looks like, what you'd like the end array to look like, and then there's a goal to reach for. Also, don't change the parameters of your request in the middle of everything  because that's a sure way to cause anger issues. Make sure you know exactly what it is you need the code to do before you start asking for help, or writing the code.

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Link to comment
Share on other sites

Quote

Are you angry?

Nope, but inferring emotion in text is a good start at getting me there.  Focus on the task and less on the niceties, if you walked into a coding forum looking for people who are not completely socially inept, then thats on you.

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Link to comment
Share on other sites

Sure, but there are dix who will tell you there a million ways to do something without giving you a single solution.  I prefer to be the dick who also provided a solution.  Are you not going to elaborate on what you would like the result to be?  My code > My Words

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Link to comment
Share on other sites

A very good question. I would recommend my ArrayWorkshop function for that, but it's not finished and only does 5 dimensions. Removing dupes based on the second dimension of a 2D array is equivalent to removing columns rather than rows. I suppose you could come up with a different interpretation, but I doubt I would be able to understand it.

Link to comment
Share on other sites

28 minutes ago, mikell said:

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

I think I was just looking for a command to do the it, but I guess I need to come up with a solution to scan the array and remove duplicates by using what is already part of AutoIT commands.

Yours sincerely

Kenneth.

Link to comment
Share on other sites

I need to run through a path and find if there is duplicates.

My idea was to use _FileListToArray. With that I can get it 1D or 2D. If I use 1D I get the path and folder in 1 column, but if I use 2D I get the same, but in 2 columns.

I need the result in 2D, but without duplicates.

Yours sincerely

Kenneth.

Link to comment
Share on other sites

#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)


Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

Edited by czardas
Link to comment
Share on other sites

1 hour ago, Valnurat said:

My idea was to use _FileListToArray. With that I can get it 1D or 2D.

FLTA returns a 1D array, FLATR returns a 1D,  so do you need as Czardas suggested?

1) a 2D array that looks like [Folder|File]

2) Where all filenames are unique without regard to folder location

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Link to comment
Share on other sites

10 hours ago, czardas said:
#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)


Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

You absolutely correct about windows won't allow it. In my company we have sites that have the backup folder located the same place. \\servername\sharename\%username%  I need to scan all locations for those sites. We have 3 sites (DKRO,DKVE,SESC) that use the same uncpath and then we have sites that use a uniqe uncpath, that's specified by the location of the OU the user is a member of. So I will end up be scanning 3 times the same uncpath. That is why I will have duplicates. I have done some code, but I was just looking for an easy way to solve it by using a fancy command.

Quote
Func _AllFolders()
    if $bDebugMode Then
        ConsoleWrite(@CRLF & "Collecting all folders." & @CRLF)
    Else
        _FileWriteLog($hFile, @CRLF & "Collecting all folders." & @CRLF)
    EndIf
    Local $aListOfFolders, $aTempListOfAllFolders[0]
    for $i = 0 to UBound($aAllBackupDestination) -1
        if $aAllBackupDestination[$i][0] <> "" then
            $aListOfFolders = _FileListToArray($aAllBackupDestination[$i][0],Default,2,True) ;Gives me 1D with path and folders
            _ArrayDelete($aListOfFolders,0)
            _ArrayConcatenate($aTempListOfAllFolders,$aListOfFolders)
        EndIf
    Next
    _ArrayDisplay($aListOfAllFolders,"$aListOfAllFolders")
    $aTempListOfAllFolders = _ArrayUnique($aTempListOfAllFolders)
    $aListOfAllFolders = _Array1to2($aTempListOfAllFolders)
EndFunc

Func _Array1to2($array)
    Local $aResult[UBound($array)][2], $aSplitAllFolders
    For $i = 0 To UBound($array) -1
        $aSplitAllFolders = StringSplit($array[$i],'\')
        if $aSplitAllFolders[0] = 5 Then
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4]
            $aResult[$i][1] = $aSplitAllFolders[5]
        Else
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4] & "\" & $aSplitAllFolders[5]
            $aResult[$i][1] = $aSplitAllFolders[6]
        EndIf
    Next
    Return $aResult
EndFunc

 

I'm NOT sure if this is fancy way of doing it, but it works fine.

Yours sincerely

Kenneth.

Link to comment
Share on other sites

So pull everything, but then only check against the portion of the string you need uniqueness on ?

like keeping this 1D and in the loop checking against the last split

#include<File.au3>
$aList = _FileListToArray(@ScriptDir , '*' , 2, True)

_ArrayDisplay($aList)

for $i = ubound($aList) - 1 to 1 step -1
    ;item to compare
    msgbox(0, '' , stringreverse(stringsplit(stringreverse($aList[$i]), "\" , 2)[0]))
next

 

Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...