Jump to content
Valnurat

Remove duplicates in a array

Recommended Posts

Valnurat

How do I remove duplicates in an array with 2 columns?

 

I was looking at the _ArrayUnique, but it just return the value for 1 column.

Edited by Valnurat

Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
Valnurat
1 hour ago, JohnOne said:

It does not, show code, include test array.

I just looked at the autoit example.

I didn't do a testcode or something.

The autoit example shows that if you just use default in a 2d array it will just return 1 column or do I miss something?


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
iamtheky

No, there are just a metric shit ton of more questions when asking to simply dedup a 2d array.  So lets start, I'll provide an answer and you lead us down the bunny trail.  The biggest help would be the endgame, because I have to assume you want duplicates gone leaving the data in some sort of ingestable format. 

#include<array.au3>

local $a2D = [['1','2'],['3','4'],['4','5'],['5','6']]

_ArrayDisplay($a2D)

For $i = ubound($a2D) - 1 to 0 step -1
    For $c = 1 to 0 step -1
        $vSrch = _ArraySearch($a2D , $a2D[$i][$c] , 0 , $i - 1)
        If $vSrch > 1 then $a2D[$i][$c] = ""
    Next
Next

_ArrayDisplay($a2D)

 

  • Like 1

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
Valnurat

Are you angry?


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
BrewManNH

I'm guessing he's just asking what the end result should look like, because there's a hundred ways of doing this and no one wants to waste time writing code when you want one of the other 99 ways it could be done.

Show us what the start array looks like, what you'd like the end array to look like, and then there's a goal to reach for. Also, don't change the parameters of your request in the middle of everything  because that's a sure way to cause anger issues. Make sure you know exactly what it is you need the code to do before you start asking for help, or writing the code.

  • Like 1

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
iamtheky
Quote

Are you angry?

Nope, but inferring emotion in text is a good start at getting me there.  Focus on the task and less on the niceties, if you walked into a coding forum looking for people who are not completely socially inept, then thats on you.

  • Like 1

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
Valnurat

Well, I did not expect I will get a code, but just someone who could put me in the right direction to solve it.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
mikell

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

Share this post


Link to post
Share on other sites
iamtheky

Sure, but there are dix who will tell you there a million ways to do something without giving you a single solution.  I prefer to be the dick who also provided a solution.  Are you not going to elaborate on what you would like the result to be?  My code > My Words


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
danielcovington

Which dimension of the array do you want to dedupe?

Share this post


Link to post
Share on other sites
czardas

A very good question. I would recommend my ArrayWorkshop function for that, but it's not finished and only does 5 dimensions. Removing dupes based on the second dimension of a 2D array is equivalent to removing columns rather than rows. I suppose you could come up with a different interpretation, but I doubt I would be able to understand it.

Share this post


Link to post
Share on other sites
Valnurat
28 minutes ago, mikell said:

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

I think I was just looking for a command to do the it, but I guess I need to come up with a solution to scan the array and remove duplicates by using what is already part of AutoIT commands.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
Valnurat

I need to run through a path and find if there is duplicates.

My idea was to use _FileListToArray. With that I can get it 1D or 2D. If I use 1D I get the path and folder in 1 column, but if I use 2D I get the same, but in 2 columns.

I need the result in 2D, but without duplicates.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
czardas
#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)


Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

Edited by czardas
  • Like 1

Share this post


Link to post
Share on other sites
iamtheky
1 hour ago, Valnurat said:

My idea was to use _FileListToArray. With that I can get it 1D or 2D.

FLTA returns a 1D array, FLATR returns a 1D,  so do you need as Czardas suggested?

1) a 2D array that looks like [Folder|File]

2) Where all filenames are unique without regard to folder location


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
Valnurat
10 hours ago, czardas said:
#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)


Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

You absolutely correct about windows won't allow it. In my company we have sites that have the backup folder located the same place. \\servername\sharename\%username%  I need to scan all locations for those sites. We have 3 sites (DKRO,DKVE,SESC) that use the same uncpath and then we have sites that use a uniqe uncpath, that's specified by the location of the OU the user is a member of. So I will end up be scanning 3 times the same uncpath. That is why I will have duplicates. I have done some code, but I was just looking for an easy way to solve it by using a fancy command.

Quote
Func _AllFolders()
    if $bDebugMode Then
        ConsoleWrite(@CRLF & "Collecting all folders." & @CRLF)
    Else
        _FileWriteLog($hFile, @CRLF & "Collecting all folders." & @CRLF)
    EndIf
    Local $aListOfFolders, $aTempListOfAllFolders[0]
    for $i = 0 to UBound($aAllBackupDestination) -1
        if $aAllBackupDestination[$i][0] <> "" then
            $aListOfFolders = _FileListToArray($aAllBackupDestination[$i][0],Default,2,True) ;Gives me 1D with path and folders
            _ArrayDelete($aListOfFolders,0)
            _ArrayConcatenate($aTempListOfAllFolders,$aListOfFolders)
        EndIf
    Next
    _ArrayDisplay($aListOfAllFolders,"$aListOfAllFolders")
    $aTempListOfAllFolders = _ArrayUnique($aTempListOfAllFolders)
    $aListOfAllFolders = _Array1to2($aTempListOfAllFolders)
EndFunc

Func _Array1to2($array)
    Local $aResult[UBound($array)][2], $aSplitAllFolders
    For $i = 0 To UBound($array) -1
        $aSplitAllFolders = StringSplit($array[$i],'\')
        if $aSplitAllFolders[0] = 5 Then
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4]
            $aResult[$i][1] = $aSplitAllFolders[5]
        Else
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4] & "\" & $aSplitAllFolders[5]
            $aResult[$i][1] = $aSplitAllFolders[6]
        EndIf
    Next
    Return $aResult
EndFunc

 

I'm NOT sure if this is fancy way of doing it, but it works fine.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites
iamtheky

So pull everything, but then only check against the portion of the string you need uniqueness on ?

like keeping this 1D and in the loop checking against the last split

#include<File.au3>
$aList = _FileListToArray(@ScriptDir , '*' , 2, True)

_ArrayDisplay($aList)

for $i = ubound($aList) - 1 to 1 step -1
    ;item to compare
    msgbox(0, '' , stringreverse(stringsplit(stringreverse($aList[$i]), "\" , 2)[0]))
next

 

Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • Pagi
      By Pagi
       
      <a href="index.php=4143b27b41db0a40c499202865c3e24d" class="menuitem " target="\_self"> "Menu Item 1" </a>
      Can you tell me if it´s possible to click this menu button? If yes, how? Cause I have no idea, I think i should use _IETagNameGetCollection
    • mar3011
      By mar3011
      hello, I want to set in the inputbox field random key from the keyboard to which the function will be assigned, for example F4 function start notepad ALT + A function start notepad for one function. Thank you for any help.  
    • JuanFelipe
      By JuanFelipe
      Cambiar a español Hello friends, I need help with a problem I have, I want to know if there is any way to update a GUI already running, that is I have a GUI of a certain size and with a button I would like to enlarge it:
      example: $ form1 = GUICreate ("", 200,100,0,0)   When you press the button, the gui increases its size, and actuates the new size.
    • BogdanNicolescu
      By BogdanNicolescu
      While 1 / / / A whole bunch of codes found in help and here: https://www.autoitscript.com/autoit3/docs/ / / / WEnd OR:

       
      HotKeySet("{ESC}", "Terminate") While 1 MouseClick("Right",674,422) MouseClick("Left",673,447) Sleep(2000) / / / A whole bunch of codes found in help and here: https://www.autoitscript.com/autoit3/docs/ / / / MouseClick("Left",675,339) SLeep(3000) WEnd Func Terminate()     Exit 0 EndFunc  
      Sorry if i should't let this here to be found by newbies like me -.-'
    • santoshM
      By santoshM
      Dear friends i am facing a problem , i have created a window in autoit some time it is moving with the mouse curser even if iam not clicking the window 
×