Sign in to follow this  
Followers 0
BrewManNH

An improved _ArrayUnique

20 posts in this topic

The function _ArrayUnique in the Array UDF has some serious design flaws in it, not to mention it's incredibly slow. This is a rewrite of that function. Practically nothing from the original function has remained.

This version...

  • doesn't rely upon using _ArrayAdd, not using that shaved 4 seconds off when using this with a 4100+ row array.
  • has extensive bounds checking on all parameters of the function, something the original didn't do at all.
  • streamlines the error checking process so it returns quicker when something sent to it isn't set correctly, all done at the top of the function
  • doesn't use Dim
  • eliminates the $vDelim parameter, which even the documentation says about it, "However, cannot forsee its usefullness"
  • adds the ability to return a 2D array if sent a 2D array, and the $bReturn2 parameter is set to true. This will search the column requested, finding any duplicated entries, removing those duplicate entries, but returning all the rows that were found first. I don't know if anyone will find any use for this part of the function, but it was surprisingly easy to implement it, so I figured I'd leave it in.

This version is approximately 50 - 80% faster than the original, and won't crash your script if you send it the wrong values in the parameters. Please play around with this to see if you can break it, or improve it. There is an example script inside the archive that uses the MusicList file as input. The MusicList file is just a text file with 4104 lines and 6 columns, try running the new and original versions using the 6th column as a search and see the difference in execution time.

Returning a 2D array on a large array is going to take a lot longer than just returning a single dimension of the array. Using it on the 6th column of this array is still faster than returning a 1D array using the original function.

__ArrayUnique.zip

2 people like this

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites



Do you think you will update the UDF version with this one?


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

There was already an attempt by wraithdu -> '?do=embed' frameborder='0' data-embedContent>> and by me somewhere in this forum.

Maybe it is worth to look at all the versions and merge all into a new super-duper one to replace current one.

Br,

UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

And this one >> 

Well it's just an idea and would need some tweaking to match the similar functionality of _ArrayUnqiue().


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

And this one >> 

Well it's just an idea and would need some tweaking to match the similar functionality of _ArrayUnqiue().

Unfortunately, Jon has declared that Assign working with any text at all, instead of a string that would be a valid variable name, is a bug in the function.

Trak ticket : #2478


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

There was already an attempt by wraithdu -> '?do=embed' frameborder='0' data-embedContent>> and by me somewhere in this forum.

Maybe it is worth to look at all the versions and merge all into a new super-duper one to replace current one.

 

Damn, that one is fast.

 

BTW, there's a bug in the function I posted, too quick with the copy/paste.

; This line
    If ($iColumn < 1) Or ($iNumColumns = 0 And ($iColumn - 1 > $iNumColumns)) Or ($iNumColumns > 0 And ($iColumn -1 > $iNumColumns)) Then Return SetError(3, 0, 0)
; needs to be changed to this
    If ($iColumn < 1) Or ($iNumColumns = 0 And ($iColumn - 1 > $iNumColumns)) Or ($iNumColumns > 0 And ($iColumn > $iNumColumns)) Then Return SetError(3, 0, 0)

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

Unfortunately, Jon has declared that Assign working with any text at all, instead of a string that would be a valid variable name, is a bug in the function.

Trak ticket : #2478

 

That's not an issue at all. Just declare the binary of each string instead. Tweak the input for case insensitivity. The real issue with that technique is what I mentioned >here. If the technique was to be employed, I believe it would be wise to add a cautionary note about performance degradation.

Edited by czardas

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

A very good point you made in that post, which is probably why the original ArrayUnique slows to a crawl on large arrays with no duplicates.

I'm very surprised that the current version of _ArrayUnique ever made it into the Array UDF in it's broken state. I'm not even talking about the slowness, I'm talking about the fact that none of the parameters, are checked, the array is only checked to see if it's an array. It doesn't take much to make it error out when using it.

Edited by BrewManNH

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

You'd have to sort the array you're searching in, which in this case is the $T_Array, and you'd have to sort it after every insertion.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

I believe turning all rows to delimitered strings may be a viable option. I do this with the elements of a 1 dimensional array to achieve significantly faster results. The code is not currently reliable because the delimiters need to be tested.


Alternatively it may (I don't know) still be possible to use the variable declaration technique on portions of the array and test all portions against each other to limit the number of declarations. This approach might still beat the speed of all other methods hands down. Not an easy function to create, but it might be possible. Such a function would require extreme rigorous testing. You could remove dupes as you go using helper functions. For example parse the array 1000 rows at a time, starting with the last element, and use Redim after each batch of 1000.

On reflection this idea probably won't work (I need to think about this a while longer). The length of each row could still cause problems. See >here.

I also thought about combining two different approaches.

Edited by czardas

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

There are some issues with this implementation but this is how I always interpreted _ArrayUnique to work.  Am I right? (BrewManNH's MusicList is needed.)

#AutoIt3Wrapper_Testing=Y

#include <Array.au3>

_main()

Func _main()
  Local Const $array = _create_array()
 
  _ArrayDisplay($array)
 
  Local $timer = TimerInit()
 
  Local Const $unique_array1 = __ArrayUnique($array, 0)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array1)
 
  $timer = TimerInit()
 
  Local Const $unique_array2 = __ArrayUnique($array, 1)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array2)
 
  $timer = TimerInit()
 
  Local Const $unique_array3 = __ArrayUnique($array, 2)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array3)
 
  $timer = TimerInit()
 
  Local Const $unique_array4 = __ArrayUnique($array, 3)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array4)
 
  $timer = TimerInit()
 
  Local Const $unique_array5 = __ArrayUnique($array, 4)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array5)
 
  $timer = TimerInit()
 
  Local Const $unique_array6 = __ArrayUnique($array, 5)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array6)
EndFunc

Func __ArrayUnique(Const $array, Const $column = 0, Const $zero_base = True)
  If Not IsArray($array) Then Return SetError(1, 0, False)
 
  Local Const $array_size = UBound($array)
 
  Local $new_array = array_column_to_1dim($array, $column)
 
  _ArraySort($new_array)
 
  Local $unique_array[1] = ['']
 
  Local $k = 0
 
  Local $term = ''
 
  Local $last_term = ''
 
  For $i = 0 To $array_size - 1
    $term = $new_array[$i]
    
    If $last_term <> $term Then
      For $j = $i + 1 To $array_size - 1
        If $term <> $new_array[$j] Then
          $unique_array[$k] = $term
          
          $k += 1
            
          ReDim $unique_array[$k + 1]
          
          $last_term = $term
          
          ExitLoop
        EndIf
      Next
    EndIf
  Next
 
  Return $unique_array
EndFunc

Func array_column_to_1dim(Const $array, Const $column)
  Local Const $array_size = UBound($array)
 
  Local $new_array[$array_size] = ['']
 
  For $i = 0 To $array_size - 1
    $new_array[$i] = $array[$i][$column]
  Next
 
  Return $new_array
EndFunc

Func _create_array() ; BrewManNH
  Local Const $hFile = FileOpen(@ScriptDir & "\MusicList")
 
  Local Const $TempText = FileReadLine($hFile)
 
  Local $aTempText = StringSplit($TempText, '-')
 
  Local Const $Count = Number($aTempText[2]) ; get the song count from the music list header info
 
  Local $Array[$Count][6]
 
  Local $TmpLine = ''
    
  For $I = 0 To $Count
    $TmpLine = FileReadLine($hFile)
    
    If @error Then ExitLoop
    
    $aTmpLine = StringSplit($TmpLine, '|')
    
    For $X = 1 To 6
      $Array[$I][$X - 1] = $aTmpLine[$X]
    Next
  Next
 
  Return $Array
EndFunc

The last colum takes the longest time on my machine at 12 seconds.

Edited by jaberwocky6669

Share this post


Link to post
Share on other sites

Other than it sorting the array, that method is pretty fast, although I think that wraithdu's scripting dictionary version is going to be the big winner in speed.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

My timing tests using the musiclist posted in the first post, in order of speed.

 

Time taken = 0.0766695356048192 seconds - wraithdu's function

Time taken = 5.96837603512747 seconds - jabberwocky's function

Time taken = 30.0241667517623 seconds - My function

Time taken = 140.443000520299 seconds - Original function

I did notice a couple of items on your function jabberwocky, the column number doesn't match the original function's parameter, it assumes the first column is one and your's assumes 0. Also, the original returns the count in the 0 element of the array, where your's doesn't. Both are minor and easily fixed though.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

I tried it with an SQLite memory database, I can get a very consistent 1.3 seconds on every column, plus or minus a few milliseconds.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

it is faster than the current UDF.

is there are way it can return 2-dimensional array?

below are the array i got.

$aArray[10] = ["Alan", "James", "Alan"," Alan", "John", "John"," Alan", "James", "John"," John"]

when you use _ArrayUnique($aArray) /__ArrayUnique($aArray) - No matter fast or slow

Return array will be:-

$aArray[0] = "Alan"

$aArray[1] = "James"

$aArray[2] = "John"

Currently i am looking for a array unique that can also display the total number of the unique in 2D version like below:

 Row | Col0   | Col1

   0    | Alan   |   4

   1    | James|   2

   2    | John  |   4

Column 1 will show the number of the repeat in array.

Can anyone help me with this?

Your help will be very appreciated. thank you

Share this post


Link to post
Share on other sites

Very quickly, the code is executed based on Scripting.Dictionary.

Using Assign, Eval, IsDeclared makes the code more quickly, but has a problem with the interpretation of the character "[". Well if this function was performed within AutoIt3. We would get a higher speed.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • czardas
      By czardas
      Haven't had much time to code recently. However the following thread inspired me.
      The debate about linear, parallel and binary search methods was rather interesting and, in an attempt to be diplomatic, I decided to combine @jchd's suggestion with @LarsJ's binary search example. I decided that the binary search algorithm required modification to make it more linear. As usual, 'if you invent something, it probably already exists and if it already exists, it exists for a reason'. My first attempt was not all that good. The code worked but was really a mess. I blame peer pressure (to post an example of a parallel search method).  I will delete that old code in due course.
      With a little memory jogging and a glance at the help file, the solution turned out to be quite easy: I just needed a better understanding of Euler. Further modification will be needed to work with more complicated unicode strings. The output could be returned as an array or a delimitered string. I'm not so interested in those details. I'm just going to post the algorithm for now and anyone, who wants to, can modify it to suit their needs. Both arrays must contain at least 1 element.
      Local $aFoo = [0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,19,20,23,24,26,30,35,39,40,41] Local $aBar = [0,1,5,6,7,8,9,10,11,12,13,14,17,18,19,21,24,25,26,27,34,35,38,40] ParallelExponetialSearch($aFoo, $aBar) ; Compares two lists - returning positive matches. Each input array must be unique (individually) and in alphabetical order. Func ParallelExponetialSearch($aFoo, $aBar) Local $sFind, _ $iMin_F = -1, $iMax_F = UBound($aFoo) -1, $Lo_F = $iMin_F, $Hi_F, _ $iMin_B = -1, $iMax_B = UBound($aBar) -1, $Lo_B = $iMin_B, $Hi_B While $iMin_F < $iMax_F And $iMin_B < $iMax_B ; Toggle Arrays - Which array has most untested elements? This is the one we want to search next, ; so we can bypass more comparisons because (in theory) mismatches have a greater chance of being skipped. If $iMax_F - $iMin_F >= $iMax_B - $iMin_B Then ; $aFoo has more (or an equal number of) untested elements $Hi_F = $iMax_F $iMin_B += 1 $sFind = $aBar[$iMin_B] While $Lo_F < $Hi_F ; search $aFoo For $i = 0 To Floor(Log($Hi_F - $Lo_F) / Log(2)) $Lo_F = $iMin_F + 2^$i If $aFoo[$Lo_F] = $sFind Then $iMin_F = $Lo_F ; each match should be added to the output [perhaps an array] ConsoleWrite($sFind & " found at $aFoo[" & $Lo_F & "] = $aBar[" & $iMin_B & "]" & @LF) ExitLoop 2 ElseIf $aFoo[$Lo_F] > $sFind Then $Hi_F = $Lo_F -1 $iMin_F += Floor(2^($i -1)) $Lo_F = $iMin_F ContinueLoop 2 EndIf Next $iMin_F = $Lo_F ; minimum increment is one WEnd Else ; $aBar has more untested elements $Hi_B = $iMax_B $iMin_F += 1 $sFind = $aFoo[$iMin_F] While $Lo_B < $Hi_B ; search $aBar For $i = 0 To Floor(Log($Hi_B - $Lo_B) / Log(2)) $Lo_B = $iMin_B + 2^$i If $aBar[$Lo_B] = $sFind Then $iMin_B = $Lo_B ; each match should be added to the output [perhaps an array] ConsoleWrite($sFind & " found at $aFoo[" & $iMin_F & "] = $aBar[" & $Lo_B & "]" & @LF) ExitLoop 2 ElseIf $aBar[$Lo_B] > $sFind Then $Hi_B = $Lo_B -1 $iMin_B += Floor(2^($i -1)) $Lo_B = $iMin_B ContinueLoop 2 EndIf Next $iMin_B = $Lo_B ; minimum increment is one WEnd EndIf WEnd EndFunc ;==> ParallelExponetialSearch I hope this will be useful to someone. I believe it deserved a thread of its own!
    • RC86
      By RC86
      Afternoon!
      I have a requirement to return both a string and an array from a function so as a result I put them both into an array and returned that.  I can access them in their entirety after returning them but then I can't seem to access the array elements after this.  Should I be able to or is there a prettier way?
      #include <Array.au3> ;Memory info returned as a string and an array $memoryInfo = _getMemoryInfo() msgbox(0,"Memory Info",$memoryInfo[0]) _ArrayDisplay($memoryInfo[1],"Memory as an Array") Local $newArray[7] $memoryInfo[1] = $newArray msgbox(0,"Test element",$newArray[0]) _ArrayDisplay($newArray) Func _getMemoryInfo() Local $newArray[7] Local $array = MemGetStats() $newArray[0] = $array[0] ;% of memory in use $newArray[1] = Round($array[1]/1024 * 0.001,2) ;Total physical RAM $newArray[2] = Round($array[2]/1024 * 0.001,2) ;Availaible physical RAM $newArray[3] = Round($array[3]/1024 * 0.001,2) ;Total pagefile $newArray[4] = Round($array[4]/1024 * 0.001,2) ;Available pagefile $newArray[5] = Round($array[5]/1024 * 0.001,2) ;Total virtual $newArray[6] = Round($array[6]/1024 * 0.001,2) ;Available virtual $memoryUsage = $newArray[1] - $newarray[2] $pagefileUsage = $newArray[3] - $newarray[4] ;Output/Return Local $returnArray[2] $returnArray[0] = "Memory: " & $memoryUsage & " GB/" & $newArray[1] & " GB " & @CRLF & "Pagefile: " & $pagefileUsage & " GB/" & $newArray[3] & " GB " $returnArray[1] = $newArray return $returnArray EndFunc A bit messy but hopefully it's understandable what I'm trying to achieve.
      Thanks!
    • Jibberish
      By Jibberish
      I am having difficulties getting updated results from GUICtrlCreateInput. I wrote a small script to demonstrate what I am trying to do.
      This script reads a text file into an array and displays the Name and Number from the text file.
      The user can check a name and change the number, and the results are displayed in _ArrayDisplay.
      Only the Checked names are displayed in their original position. This is necessary for future
      plumbing.
      The problem with this script is the original number is returned, not the changed number. I cannot
      figure out how to get the updated number to be displayed.
      ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; ; This script reads a text file into an array and displays the Name and Number from the text file. ; The user can check a name and change the number, and the results are displayed in _ArrayDisplay. ; Only the Checked names are displayed in their original position. This is necessary for future ; plumbing. ; ; The problem with this script is the original number is returned, not the changed number. I cannot ; figure out how to get the updated number to be displayed. ; ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #include-once #include <MsgBoxConstants.au3> #include <array.au3> #include <Date.au3> #include <WinAPIFiles.au3> #include <AutoItConstants.au3> #include <Misc.au3> #include <File.au3> #include <GUIConstantsEx.au3> #include <StaticConstants.au3> Local $iMax = 6 ;Default number of Videos - 1 for the array counter Dim $aTextFile[$iMax][2] ; [start with 5 entries][Name, Number] Don't get confused! [Row][Column] Local $i = 0 Local $iLeft = 30 Local $iTop = 30 Local $sName Local $sNumber Local $aArray Local $iMaxCol = 5 Local $iRow = 0 Local $iCol = 0 Local $aNumberCount[$iMaxCol][2] $aArray1 = ReadFile() $aNamesNumbers = DisplayNames($aArray1) _ArrayDisplay($aNamesNumbers) ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; ; ReadFile() places the contents of the NameNumber.txt file in an array ; ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Func ReadFile() Local $sFileData = @ScriptDir & "\NameNumber.txt" Local $iStrReturn = 0 Local $aArrayFile Local $aTextFile[$iMaxCol][2] Local $aName Local $sCheckBox1 Const $iL = 10 ; Count from Left for GUI so all buttons line up Local $sFill Local $iA = 0, $iN = 0 ; Put the Names into an Array _FileReadToArray($sFileData, $aArrayFile) For $iA = 0 To $aArrayFile[0] ; Step through the array looking for Names If StringInStr($aArrayFile[$iA], "Name:") Then $aName = StringSplit($aArrayFile[$iA],":") $sFill = $aName[2] If UBound($aTextFile) <= $iRow Then ; Resize the array when $iRow is equal to the element count in the array to prevent subscript error ReDim $aTextFile[UBound($aTextFile) + 1][$iMaxCol] EndIf $aTextFile[$iRow][$iCol] = $sFill $iRow += 1 EndIf Next $iCol = 1 $iRow = 0 $sFill = "" For $iA = 1 To $aArrayFile[0] ; Step through the array looking for Numbers If StringInStr($aArrayFile[$iA], "Number:") Then $aName = StringSplit($aArrayFile[$iA],":") $sFill = $aName[2] If UBound($aTextFile) <= $iRow Then ; Resize the array when $iRow is equal to the element count in the array to prevent subscript error ReDim $aTextFile[UBound($aTextFile) + 1][$iMaxCol] EndIf $aTextFile[$iRow][$iCol] = $sFill $iRow += 1 EndIf Next Return $aTextFile EndFunc ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; ; Display the Name and Number ; ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Func DisplayNames($aArray1) Local $aName[$iMax] Local $aNumbers[$iMax] $iMMCount = UBound($aArray1) $iMMCount -=1 Local $iWidth = 300 Local $iLength = 300;$iMMCount * 30 GUISetFont(12) $hGUI = GUICreate("Edit / Update Number Test", $iWidth, $iLength, -1, -1) GUICtrlCreateLabel(" Name Number",1,5) ;$iTop += 30 ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; ; Display Name with a Checkbox. Only Checked Names should be saved. ; ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For $i = 0 to $iMMCount Step 1 $sNameText = $aArray1[$i][0] $iNameLength = StringLen($sNameText) $aName[$i] = GUICtrlCreateCheckbox($sNameText,$iLeft, $iTop) $iTop += 30 Next ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; ; Display the number from the text file below each name. Allow the user to change the number and display the ; new number. ; ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $iLeft = 60 GUIStartGroup() $iTop = 28 For $i = 0 to $iMMCount Step 1 $sNumberText = $aArray1[$i][1] $aNumberCount[$i][1] = $sNumberText $aNumberCount[$i][0] = GUICtrlCreateInput($sNumberText,$iLeft, $iTop, 50,18, $GUI_DOCKAUTO) GUICtrlSetPos($aNumberCount[$i][0],200) $iTop += 30 Next ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; ; Read the checked names and (possibly updated) number ; ;~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ $iMMCount = UBound($aNumberCount) $iMMNewCount = $iMMCount - 1 Local $aGUICheckbox[$iMMCount] Local $aCheckedNameNumber[$iMMCount][2] $iLeft = 30 Local $idCloseGUI = GUICtrlCreateButton("Close",$iLeft, $iTop) GUISetState(@SW_SHOW) While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $idCloseGUI For $i = 0 To $iMMNewCount Step 1 Switch GUICtrlRead($aName[$i]) Case $GUI_CHECKED $aCheckedNameNumber[$i][0] = $aArray1[$i][0] Case $GUI_UNCHECKED EndSwitch Switch GUICtrlRead($aNumberCount[$i][0]) Case $aCheckedNameNumber[$i][1] = GUICtrlRead($aNumberCount[$i][1]) EndSwitch Next ExitLoop EndSwitch WEnd For $i = 0 to $iMMNewCount step 1 GUICtrlRead($aNumberCount[$i][0]) If $aCheckedNameNumber[$i][0] <> "" Then $aCheckedNameNumber[$i][1] = $aNumberCount[$i][1] EndIf Next GUIDelete($hGUI) Return $aCheckedNameNumber EndFunc This is the text file I am reading. If you want to try this out put the NameNumber.txt file in your script directory. It's attached to the post.
      * This is a dummy file with a Name and Number * The only purpose of this file is to read the updated Number. Name:Taggart Number:916 Name:Mongo Number:90 Name:Hedley Lamarr Number:22 Name:Bart Number:9999 Name:The Waco Kid Number:2244 If I change the number, the original number is displayed at the end, not the updated/modified number. I need the modified number to be displayed.
      Thanks in advance for any assistance!
      Jibberish
      NameNumber.txt
    • Jibberish
      By Jibberish
      I am working on a Video Player Test Script. I am reading a text file into a 3d array to be displayed in a GUI.
      Array Content:
      [x][0]FileName.mp4
      [x][1]UsageCount -> Up to a 4 digit number
      [x][2]EnableUsageCount (True/False)
      In the GUI the user can check the filename box, edit the number of usages and check EnableUsageCount to turn on the UsageCount control using the UsageCount number for the maximum number of plays.
      To make this easier to deal with I have removed the EnableUsageCount section, and am just concentrating on getting a method to put the edited (or unchanged) UsageCount in the array. Currently nothing is read into the array, due to my using the wrong method in the Case statement.
      Here are code snippets of what I am trying to do:
      ; Snippets from script ; I read the text file at the bottom and put the .mp4 filenames in $aManifest[x][0] and UsageCount in $aManifest[x][1] ; Then I create a GUI to display the .mp4 filenames with checkboxes and the UsageCount to the right. UsageCount is editable by the user. ; If the filename is checked, I want to read the filename into $aCheckedVideos[x][0] and the updated UsageCount in $aCheckedVideos[x][1] ; The GUICtrlRead($aVideoName[$i]) with Case $GUI_CHECKED & UNCHECKED works for the checkboxes ; This section puts the filenames in the GUI with a checkbox For $i = 0 to $iMMCount Step 1 $sMP4Text = $aManifest[$i][0] $iMP4Length = StringLen($sMP4Text) $aVideoName[$i] = GUICtrlCreateCheckbox($sMP4Text,$iLeft, $iTop) $iTop += 30 Next ; This section reads numbers from the Manifest array, and I want to be able to change the number and have them saved. ; So the Case $GUI_CHECKED & UNCHECKED won't work here, and I can't figure out what I should be doing here. For $i = 0 to $iMMCount Step 1 $sUsageText = $aManifest[$i][1] $aUsageCount[$i] = GUICtrlCreateInput($sUsageText,$iLeft, $iTop, 50,18, $GUI_DOCKAUTO) GUICtrlSetPos($aUsageCount[$i],200) $iTop += 30 Next While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $idCloseGUI For $i = 0 To $iMMNewCount Step 1 Switch GUICtrlRead($aVideoName[$i]) Case $GUI_CHECKED $aCheckedVideos[$i][0] = $aManifest[$i][0] ; If checked, I put the results into a two D array Case $GUI_UNCHECKED ; where Video Name is $aCheckedVideos[x][0] EndSwitch Switch GUICtrlRead($aUsageCounter[$i]) Case $GUI_CHECKED $aCheckedVideos[$i][1] = $aUsageCount[$i] ;I want to put the text (numbers) in $aCheckedVideos[x][1] Case $GUI_UNCHECKED ;but what is returned is blank, probably due to EndSwitch ; $GUI_CHECKED being the wrong thing. Next ExitLoop EndSwitch WEnd ;The txt file I'm reading has the following: ;~ /** Title #1: Big Buck Bunny 1080p **/ ;~ "Name": "Big Buck Bunny", ;~ "URI": "..\\MediaFiles\\bbb_1080_60s.mp4", ;~ "UsageCount": 9999, ;~ "URI": "..\\MediaFiles\\bbb_1080_60s_enc1.mp4", ;~ "UsageCount": 45, ;~ "URI": "..\\MediaFiles\\bbb_1080_60s_enc1.mp4", ;~ "UsageCount": 2, ;~ /** Title #2: Tears of Steel 4K **/ ;~ "Name": "Tears of Steel 4K", ;~ "URI": "..\\MediaFiles\\tos_4K_60s_HEVC.mp4", ;~ "UsageCount": 9876, ;~ "URI": "..\\MediaFiles\\tos_4K_60s_HEVC_enc2.mp4", ;~ "UsageCount": 0, ;~ "URI": "..\\MediaFiles\\tos_4K_60s_HEVC_enc2.mp4", ;~ "UsageCount": 5, I am certain that the section
                          Switch GUICtrlRead($aUsageCounter[$i])
                              Case $GUI_CHECKED
                                  $aCheckedVideos[$i][1] = $aUsageCount[$i]    ;I want to put the text (numbers) in $aCheckedVideos[x][1]
                              Case $GUI_UNCHECKED                                ;but what is returned is blank, probably due to
                          EndSwitch                                            ; $GUI_CHECKED being the wrong thing.

      is wrong, and this is what I am looking for help with. Instead of $GUI_CHECKED what should I be looking for?
      The worst part of this is I had this working late last night, and then lost my changes and cannot for the life of me remember how I had this working.
      Help is truly appreciated!
      Jibberish
       
    • algiuxas
      By algiuxas
      Hello everybody, so I might found a bug in _ArraySearch:
       
      #include <Array.au3> Local $abc[2] = ["b", "b"] $h = "+---------------------------------+" & @CRLF ConsoleWrite($h) For $i = 0 To 1 $abc[0] = 0 out() $abc[0] = 1 out() $abc[0] = -1 out() $abc[0] = "Abc" out() $abc[1] = "Hello" Next Func txt() Return "$abc = [" & $abc[0] & ", " & $abc[1] & "]" & @CRLF & _ "> _ArraySearch($abc,""Hello"") = " EndFunc ;==>txt Func shouldoutput($ans) Return "_ArraySearch Should return: " & ($ans ? 1 : -1) EndFunc ;==>shouldoutput Func out() ConsoleWrite("> " & txt() & _ArraySearch($abc, "Hello") & @CRLF & "> " & shouldoutput($abc[1] == "Hello") & @CRLF & $h) EndFunc ;==>out Exit Sorry for this messy script