Valnurat

Remove duplicates in a array

22 posts in this topic

#1 ·  Posted (edited)

How do I remove duplicates in an array with 2 columns?

 

I was looking at the _ArrayUnique, but it just return the value for 1 column.

Edited by Valnurat

Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites



1 hour ago, JohnOne said:

It does not, show code, include test array.

I just looked at the autoit example.

I didn't do a testcode or something.

The autoit example shows that if you just use default in a 2d array it will just return 1 column or do I miss something?


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites

No, there are just a metric shit ton of more questions when asking to simply dedup a 2d array.  So lets start, I'll provide an answer and you lead us down the bunny trail.  The biggest help would be the endgame, because I have to assume you want duplicates gone leaving the data in some sort of ingestable format. 

#include<array.au3>

local $a2D = [['1','2'],['3','4'],['4','5'],['5','6']]

_ArrayDisplay($a2D)

For $i = ubound($a2D) - 1 to 0 step -1
    For $c = 1 to 0 step -1
        $vSrch = _ArraySearch($a2D , $a2D[$i][$c] , 0 , $i - 1)
        If $vSrch > 1 then $a2D[$i][$c] = ""
    Next
Next

_ArrayDisplay($a2D)

 

1 person likes this

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

Are you angry?


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites

I'm guessing he's just asking what the end result should look like, because there's a hundred ways of doing this and no one wants to waste time writing code when you want one of the other 99 ways it could be done.

Show us what the start array looks like, what you'd like the end array to look like, and then there's a goal to reach for. Also, don't change the parameters of your request in the middle of everything  because that's a sure way to cause anger issues. Make sure you know exactly what it is you need the code to do before you start asking for help, or writing the code.

1 person likes this

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Quote

Are you angry?

Nope, but inferring emotion in text is a good start at getting me there.  Focus on the task and less on the niceties, if you walked into a coding forum looking for people who are not completely socially inept, then thats on you.

1 person likes this

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

Well, I did not expect I will get a code, but just someone who could put me in the right direction to solve it.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

Share this post


Link to post
Share on other sites

Sure, but there are dix who will tell you there a million ways to do something without giving you a single solution.  I prefer to be the dick who also provided a solution.  Are you not going to elaborate on what you would like the result to be?  My code > My Words


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

Which dimension of the array do you want to dedupe?

Share this post


Link to post
Share on other sites

A very good question. I would recommend my ArrayWorkshop function for that, but it's not finished and only does 5 dimensions. Removing dupes based on the second dimension of a 2D array is equivalent to removing columns rather than rows. I suppose you could come up with a different interpretation, but I doubt I would be able to understand it.

Share this post


Link to post
Share on other sites
28 minutes ago, mikell said:

To solve what exactly ? this needs precise requirements
i.e. If you have  [1,a] , [1,b] and [1,c] and arrayunique on the first column, which row do you want to keep ?
... etc
 

I think I was just looking for a command to do the it, but I guess I need to come up with a solution to scan the array and remove duplicates by using what is already part of AutoIT commands.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites

I need to run through a path and find if there is duplicates.

My idea was to use _FileListToArray. With that I can get it 1D or 2D. If I use 1D I get the path and folder in 1 column, but if I use 2D I get the same, but in 2 columns.

I need the result in 2D, but without duplicates.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)


Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

Edited by czardas
1 person likes this

Share this post


Link to post
Share on other sites
1 hour ago, Valnurat said:

My idea was to use _FileListToArray. With that I can get it 1D or 2D.

FLTA returns a 1D array, FLATR returns a 1D,  so do you need as Czardas suggested?

1) a 2D array that looks like [Folder|File]

2) Where all filenames are unique without regard to folder location


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
10 hours ago, czardas said:
#include 'ArrayWorkshop.au3'

_UniqueRegion($aArray)


Edit Actually, I just thought about this. There can't be any duplicate paths to files: Windows won't allow it. You must mean something more complicated, perhaps you mean duplicate file names regardless of folder location.

To do that, you could copy the array, remove duplicates from the 2nd column, search the original array for unique entries and create a new array with unique file names (any path). Of course a duplicate file name does not imply that the contents are the same. All the functions you need are in the help file starting with the prefix  '_Array'.

However if you were just giving an example of how you might use it, then you might like to try _UniqueRegion() with an example that contains fully duplicated rows. Suppose you combined data from several sources, then _UniqueRegion() might be suitable.

You absolutely correct about windows won't allow it. In my company we have sites that have the backup folder located the same place. \\servername\sharename\%username%  I need to scan all locations for those sites. We have 3 sites (DKRO,DKVE,SESC) that use the same uncpath and then we have sites that use a uniqe uncpath, that's specified by the location of the OU the user is a member of. So I will end up be scanning 3 times the same uncpath. That is why I will have duplicates. I have done some code, but I was just looking for an easy way to solve it by using a fancy command.

Quote
Func _AllFolders()
    if $bDebugMode Then
        ConsoleWrite(@CRLF & "Collecting all folders." & @CRLF)
    Else
        _FileWriteLog($hFile, @CRLF & "Collecting all folders." & @CRLF)
    EndIf
    Local $aListOfFolders, $aTempListOfAllFolders[0]
    for $i = 0 to UBound($aAllBackupDestination) -1
        if $aAllBackupDestination[$i][0] <> "" then
            $aListOfFolders = _FileListToArray($aAllBackupDestination[$i][0],Default,2,True) ;Gives me 1D with path and folders
            _ArrayDelete($aListOfFolders,0)
            _ArrayConcatenate($aTempListOfAllFolders,$aListOfFolders)
        EndIf
    Next
    _ArrayDisplay($aListOfAllFolders,"$aListOfAllFolders")
    $aTempListOfAllFolders = _ArrayUnique($aTempListOfAllFolders)
    $aListOfAllFolders = _Array1to2($aTempListOfAllFolders)
EndFunc

Func _Array1to2($array)
    Local $aResult[UBound($array)][2], $aSplitAllFolders
    For $i = 0 To UBound($array) -1
        $aSplitAllFolders = StringSplit($array[$i],'\')
        if $aSplitAllFolders[0] = 5 Then
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4]
            $aResult[$i][1] = $aSplitAllFolders[5]
        Else
            $aResult[$i][0] = "\\" & $aSplitAllFolders[3] & "\" & $aSplitAllFolders[4] & "\" & $aSplitAllFolders[5]
            $aResult[$i][1] = $aSplitAllFolders[6]
        EndIf
    Next
    Return $aResult
EndFunc

 

I'm NOT sure if this is fancy way of doing it, but it works fine.


Yours sincerely

Kenneth.

Share this post


Link to post
Share on other sites

#20 ·  Posted (edited)

So pull everything, but then only check against the portion of the string you need uniqueness on ?

like keeping this 1D and in the loop checking against the last split

#include<File.au3>
$aList = _FileListToArray(@ScriptDir , '*' , 2, True)

_ArrayDisplay($aList)

for $i = ubound($aList) - 1 to 1 step -1
    ;item to compare
    msgbox(0, '' , stringreverse(stringsplit(stringreverse($aList[$i]), "\" , 2)[0]))
next

 

Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • X_xkijux_x
      By X_xkijux_x
      Ok so im using this program. clarify for school. I would like to have a program that takes every word i type in microsoft word and searches on it in claryfi. is this possible. I would like to have the program not stopping me from typing more after that word. Like when i type a word it auto search it on clarify without making me stop typing and if i want to change i can do that and if im fine that word i can just keep typing and it will search after the next word. 
    • Miliardsto
      By Miliardsto
      Is this possible to make program works like in diagram? There is so much encryption methods and UDFs are they give security? If the $Address will be crypted with some encryption algorithm could be possible to get the value of $Address in easy way? Of course We know its easy too look into autoit code and get value of variable and so We dont want to anyone see $Address value - there are functions Could func1.au3, func2.au3.. be for example func1.php or other type of file? As we know .php files are unnable to see. For example func1 would be read by FileRead() and then Execute() Is this generally possible to do?
      Will it give secure?
      What do you think about? Maybe there is something wrong in this concept or missed.

    • JaredStroeb
      By JaredStroeb
      I need to open an application and send it keys in Session 0.  I have exe's working locally, I can see the application open in session 0 through interactive services, but my send commands do not go to the application.  The following article running-autoit-session-0, directs me to use ControlSend() however my application has no controls that can be identified by AutoIt v3 Window Info.  Get All Windows Controls also returned nothing for the application's handle.  
      What are my options?  
      Can this be done with AutoIt?
      Is there a different "relatively simple" library/tool to use? (I am familiar with C# if there is a viable library there )

       
    • nassausky
      By nassausky
      Hey all,
      Has anyone created any queries for youtube? I was thinking it might be something like this but can't quite figure it out:
      $oIE = _IECreateEmbedded() _IENavigate($oIE,"https://www.youtube.com") Local $oForms = _IETagNameGetCollection($oIE, "form") Local $sTxt = "" For $oForm In $oForms If $oForm.id = "masthead-search" Then $oInputs = _IETagNameGetCollection($oIE, "input") For $oInput In $oInputs If $oInput.id = "masthead-search-term" Then _IEFormElementSetValue($oInput, "cards") _IEFormSubmit($oForm) EndIf Next EndIf Next  
    • Skysnake
      By Skysnake
      Scite comes up occasional, almost incidentally, in some support topics.  I have not found a "Scite" topic in the forums, so this is where I will start.  I have recently been using Scite intensively, specifically with a large project and I have certain issues.  I know that Scite is not an AutoIt product, but it is so closely integrated to the core release that it makes sense to raise issues and request support here.  If this is not the right please refer me to the right forum.
      Thx
       
      Skysnake