Jump to content
Sign in to follow this  
BrewManNH

An improved _ArrayUnique

Recommended Posts

BrewManNH

The function _ArrayUnique in the Array UDF has some serious design flaws in it, not to mention it's incredibly slow. This is a rewrite of that function. Practically nothing from the original function has remained.

This version...

  • doesn't rely upon using _ArrayAdd, not using that shaved 4 seconds off when using this with a 4100+ row array.
  • has extensive bounds checking on all parameters of the function, something the original didn't do at all.
  • streamlines the error checking process so it returns quicker when something sent to it isn't set correctly, all done at the top of the function
  • doesn't use Dim
  • eliminates the $vDelim parameter, which even the documentation says about it, "However, cannot forsee its usefullness"
  • adds the ability to return a 2D array if sent a 2D array, and the $bReturn2 parameter is set to true. This will search the column requested, finding any duplicated entries, removing those duplicate entries, but returning all the rows that were found first. I don't know if anyone will find any use for this part of the function, but it was surprisingly easy to implement it, so I figured I'd leave it in.

This version is approximately 50 - 80% faster than the original, and won't crash your script if you send it the wrong values in the parameters. Please play around with this to see if you can break it, or improve it. There is an example script inside the archive that uses the MusicList file as input. The MusicList file is just a text file with 4104 lines and 6 columns, try running the new and original versions using the 6th column as a search and see the difference in execution time.

Returning a 2D array on a large array is going to take a lot longer than just returning a single dimension of the array. Using it on the 6th column of this array is still faster than returning a 1D array using the original function.

__ArrayUnique.zip

  • Like 2

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
guinness

Do you think you will update the UDF version with this one?


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites
UEZ

There was already an attempt by wraithdu -> '?do=embed' frameborder='0' data-embedContent>> and by me somewhere in this forum.

Maybe it is worth to look at all the versions and merge all into a new super-duper one to replace current one.

Br,

UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites
guinness

And this one >> 

Well it's just an idea and would need some tweaking to match the similar functionality of _ArrayUnqiue().


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites
BrewManNH

And this one >> 

Well it's just an idea and would need some tweaking to match the similar functionality of _ArrayUnqiue().

Unfortunately, Jon has declared that Assign working with any text at all, instead of a string that would be a valid variable name, is a bug in the function.

Trak ticket : #2478


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
BrewManNH

There was already an attempt by wraithdu -> '?do=embed' frameborder='0' data-embedContent>> and by me somewhere in this forum.

Maybe it is worth to look at all the versions and merge all into a new super-duper one to replace current one.

 

Damn, that one is fast.

 

BTW, there's a bug in the function I posted, too quick with the copy/paste.

; This line
    If ($iColumn < 1) Or ($iNumColumns = 0 And ($iColumn - 1 > $iNumColumns)) Or ($iNumColumns > 0 And ($iColumn -1 > $iNumColumns)) Then Return SetError(3, 0, 0)
; needs to be changed to this
    If ($iColumn < 1) Or ($iNumColumns = 0 And ($iColumn - 1 > $iNumColumns)) Or ($iNumColumns > 0 And ($iColumn > $iNumColumns)) Then Return SetError(3, 0, 0)

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
czardas

Unfortunately, Jon has declared that Assign working with any text at all, instead of a string that would be a valid variable name, is a bug in the function.

Trak ticket : #2478

 

That's not an issue at all. Just declare the binary of each string instead. Tweak the input for case insensitivity. The real issue with that technique is what I mentioned >here. If the technique was to be employed, I believe it would be wise to add a cautionary note about performance degradation.

Edited by czardas

Share this post


Link to post
Share on other sites
BrewManNH

A very good point you made in that post, which is probably why the original ArrayUnique slows to a crawl on large arrays with no duplicates.

I'm very surprised that the current version of _ArrayUnique ever made it into the Array UDF in it's broken state. I'm not even talking about the slowness, I'm talking about the fact that none of the parameters, are checked, the array is only checked to see if it's an array. It doesn't take much to make it error out when using it.

Edited by BrewManNH

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
BrewManNH

You'd have to sort the array you're searching in, which in this case is the $T_Array, and you'd have to sort it after every insertion.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
czardas

I believe turning all rows to delimitered strings may be a viable option. I do this with the elements of a 1 dimensional array to achieve significantly faster results. The code is not currently reliable because the delimiters need to be tested.


Alternatively it may (I don't know) still be possible to use the variable declaration technique on portions of the array and test all portions against each other to limit the number of declarations. This approach might still beat the speed of all other methods hands down. Not an easy function to create, but it might be possible. Such a function would require extreme rigorous testing. You could remove dupes as you go using helper functions. For example parse the array 1000 rows at a time, starting with the last element, and use Redim after each batch of 1000.

On reflection this idea probably won't work (I need to think about this a while longer). The length of each row could still cause problems. See >here.

I also thought about combining two different approaches.

Edited by czardas

Share this post


Link to post
Share on other sites
jaberwacky

There are some issues with this implementation but this is how I always interpreted _ArrayUnique to work.  Am I right? (BrewManNH's MusicList is needed.)

#AutoIt3Wrapper_Testing=Y

#include <Array.au3>

_main()

Func _main()
  Local Const $array = _create_array()
 
  _ArrayDisplay($array)
 
  Local $timer = TimerInit()
 
  Local Const $unique_array1 = __ArrayUnique($array, 0)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array1)
 
  $timer = TimerInit()
 
  Local Const $unique_array2 = __ArrayUnique($array, 1)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array2)
 
  $timer = TimerInit()
 
  Local Const $unique_array3 = __ArrayUnique($array, 2)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array3)
 
  $timer = TimerInit()
 
  Local Const $unique_array4 = __ArrayUnique($array, 3)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array4)
 
  $timer = TimerInit()
 
  Local Const $unique_array5 = __ArrayUnique($array, 4)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array5)
 
  $timer = TimerInit()
 
  Local Const $unique_array6 = __ArrayUnique($array, 5)
 
  ConsoleWrite((TimerDiff($timer) / 1000) & @CRLF)
 
  _ArrayDisplay($unique_array6)
EndFunc

Func __ArrayUnique(Const $array, Const $column = 0, Const $zero_base = True)
  If Not IsArray($array) Then Return SetError(1, 0, False)
 
  Local Const $array_size = UBound($array)
 
  Local $new_array = array_column_to_1dim($array, $column)
 
  _ArraySort($new_array)
 
  Local $unique_array[1] = ['']
 
  Local $k = 0
 
  Local $term = ''
 
  Local $last_term = ''
 
  For $i = 0 To $array_size - 1
    $term = $new_array[$i]
    
    If $last_term <> $term Then
      For $j = $i + 1 To $array_size - 1
        If $term <> $new_array[$j] Then
          $unique_array[$k] = $term
          
          $k += 1
            
          ReDim $unique_array[$k + 1]
          
          $last_term = $term
          
          ExitLoop
        EndIf
      Next
    EndIf
  Next
 
  Return $unique_array
EndFunc

Func array_column_to_1dim(Const $array, Const $column)
  Local Const $array_size = UBound($array)
 
  Local $new_array[$array_size] = ['']
 
  For $i = 0 To $array_size - 1
    $new_array[$i] = $array[$i][$column]
  Next
 
  Return $new_array
EndFunc

Func _create_array() ; BrewManNH
  Local Const $hFile = FileOpen(@ScriptDir & "\MusicList")
 
  Local Const $TempText = FileReadLine($hFile)
 
  Local $aTempText = StringSplit($TempText, '-')
 
  Local Const $Count = Number($aTempText[2]) ; get the song count from the music list header info
 
  Local $Array[$Count][6]
 
  Local $TmpLine = ''
    
  For $I = 0 To $Count
    $TmpLine = FileReadLine($hFile)
    
    If @error Then ExitLoop
    
    $aTmpLine = StringSplit($TmpLine, '|')
    
    For $X = 1 To 6
      $Array[$I][$X - 1] = $aTmpLine[$X]
    Next
  Next
 
  Return $Array
EndFunc

The last colum takes the longest time on my machine at 12 seconds.

Edited by jaberwocky6669

Share this post


Link to post
Share on other sites
BrewManNH

Other than it sorting the array, that method is pretty fast, although I think that wraithdu's scripting dictionary version is going to be the big winner in speed.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
BrewManNH

My timing tests using the musiclist posted in the first post, in order of speed.

 

Time taken = 0.0766695356048192 seconds - wraithdu's function

Time taken = 5.96837603512747 seconds - jabberwocky's function

Time taken = 30.0241667517623 seconds - My function

Time taken = 140.443000520299 seconds - Original function

I did notice a couple of items on your function jabberwocky, the column number doesn't match the original function's parameter, it assumes the first column is one and your's assumes 0. Also, the original returns the count in the 0 element of the array, where your's doesn't. Both are minor and easily fixed though.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
BrewManNH

I tried it with an SQLite memory database, I can get a very consistent 1.3 seconds on every column, plus or minus a few milliseconds.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
alankam58

it is faster than the current UDF.

is there are way it can return 2-dimensional array?

below are the array i got.

$aArray[10] = ["Alan", "James", "Alan"," Alan", "John", "John"," Alan", "James", "John"," John"]

when you use _ArrayUnique($aArray) /__ArrayUnique($aArray) - No matter fast or slow

Return array will be:-

$aArray[0] = "Alan"

$aArray[1] = "James"

$aArray[2] = "John"

Currently i am looking for a array unique that can also display the total number of the unique in 2D version like below:

 Row | Col0   | Col1

   0    | Alan   |   4

   1    | James|   2

   2    | John  |   4

Column 1 will show the number of the repeat in array.

Can anyone help me with this?

Your help will be very appreciated. thank you

Share this post


Link to post
Share on other sites
AZJIO

Very quickly, the code is executed based on Scripting.Dictionary.

Using Assign, Eval, IsDeclared makes the code more quickly, but has a problem with the interpretation of the character "[". Well if this function was performed within AutoIt3. We would get a higher speed.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • Iceburg
      By Iceburg
      Hi everyone, I'm at best a noobie.  I have read through the Array helps, and specifically the 2D array help file, and I'm struggling to get my code working.
      I have an array that is read from a file, thats working great.  I'm trying to do some math on the array, so I can find the largest, average, lowest, day over day change %, etc.
      The array read working fine, I get 43 lines, line 0 is 44, and then I get data that looks like
      0519 $10,000
      0520 $10,001
      0521 $10,002
      The data in this array is a single 1D array, breaking it out into 2 columns so I can do the math is what I can get to happen.  
      How do I reference the array to store this data?  Second, how do I assign this data to the appropriate row/column?
      Thanks in advance.
      Dim $all_cash_amounts[UBound($aInput)][2] Dim $max_amount_in_account Dim $min_amount_in_account _FileReadToArray($LC_Check_file_path, $aInput) _ArrayDisplay($aInput) local $date = StringRegExp($aInput[1], "(\d\d\d\d)", 1) local $cash = StringRegExp($aInput[1], "\d+\s(-?[0-9\.\,]+)", 1) ConsoleWrite("Date is: " & $date & @CRLF) For $i = 1 To UBound($aInput)-1     $date = StringRegExp($aInput[$i], "(\d\d\d\d)", 1)     $all_cash_amounts[$i][2] = $date[$i][0], $cash[$i][1]      Next _ArrayDisplay($all_cash_amounts)  
    • OldGuyWalking
      By OldGuyWalking
      Given an array with multiple columns that is displayed in a listview,
       ===> What is the fastest/most efficient way to create and manage multiple filters and display results in ListView.
      I have a text file that loads into a listview that has string, numeric, and date columns.  The main file contains about 5100 rows. It's loaded into an array and (in this ListView) it's pre-filtered to display a range of rows based on a start and end date.  On the form I have menu options for various filters. (see below).
      I have options to filter on an "Air Date" column (=Today, >=Today, <=Today) and on a numeric field that is either 1 or 0 that indicate Active or Ended.

      For each filter option I have a prebuilt array that holds a subset of the main array based on a single filter.  For the list above I have the main Array and 5 additional arrays.  None of the arrays are updated since this is for "view only" purposes.  This is a short list and I could have done the filtering "live" but I have several of these forms and so kept the same functionality in each. I have another ListView that displays the complete 5100 row list with 3 filters that, when building the filters live was considerably slower than using prebuilt arrays.
      If I want to expand past simple single column filtering, using an array for each filter becomes cumbersome especially if I want to combine filters using AND & OR.
      The text file I'm working with has 16 columns. If I setup filters for 4 columns and include AND / OR capability that would require prebuilding 24 arrays to cover the various combinations.
      If using the slower method of building a filtered array in real time each time a different filter is selected is the only way to go with this then I'll live with it. It is less overhead. .
      Below is the code I'm currently using to "filter" an array.  My next change was going to add AND / OR functionality (see the info above the header for where I was going with this) .
      ; Description ...: Delete rows from an array and only keep rows that meet the crtieria of identified columns. ; ; Next Change: Add AND/OR to combine filters. Use array to hold multiple criteria and values? ; ; Local $aCriteria[][] = [["",$iColNbr1, $sOperator1, $vValue1], ["AND",$iColNbr2, $sOperator2, $vValue2], ["OR",$iColNbr3, $sOperator3, $vValue3]] ; The first set of criteria ["", $iColNbr1, $sOperator1, $vValue1] must start with a "". ; If anything is entered in that first parameter it will be ignored. ; If the first parameter in any additional criteria set is left blank, or it is not OR, it will default to AND. ; If $aArray is 1 dimension with more than one set of criteria, only the first set will be used. ; Any criteria that uses a column that is less than 0 or higher than the total number of columns in the array will return an error. ; ; Recognized data types for this function are: S (String), D (Date), N (Number). ; ; Recognized Operators are: "EQ", "NEQ", "IN", "GT", "GE", "LT", "LE", "BETWEEN". ; ****** Not all operators work with all data types. ; #FUNCTION# ==================================================================================================================== ; Name ..........: _ArrayFilter ; Description ...: Delete rows from an array and only keep rows that meet the crtieria of identified columns. ; Syntax ........: _ArrayFilter(Byref $aArray[, $iCol = 0[, $sOperator = "EQ"[, $vValue = ""[, $iOptionBase = 0]]]]) ; Parameters ....: $aArray - Array being filtered. ; $iCol - [optional] Column to filter. Default is 0. ; $sOperator - [optional] Operator. Default is "EQ". ; $vValue - [optional] Criteria to compare the column/row value against. ; $iOptionBase - [optional] Starting row. Default is 0. ; Return values .: None ; Author ........: OldGuyWalking ; Modified ......: ; Remarks .......: ; Related .......: ; Link ..........: ; Example .......: No ; =============================================================================================================================== Func _ArrayFilter(ByRef $aArray, $iCol = 0, $sOperator = "EQ", $vValue = "", $iOptionBase = 0) Local $hFunc = _ArrayFilter $vValue = StringStripWS($vValue, 3) If $vValue = "[Today]" Then $vValue = _NowCalcDate() EndIf Local $sMsg Local $sMsgHdr Local $n1 Local $sDeleteIndex Local $aDeleteIndex Local $iCnt = 0 Local $iRows Local $iColMax Local $iDim Local $sData Local $sVType Local $sDType Local $LBound Local $iDiff If $iOptionBase <> 0 Then $iOptionBase = 1 EndIf If _IsValueEmpty($aArray) Then Return SetError(1, 0, "") EndIf $iDim = UBound($aArray, $UBOUND_DIMENSIONS) If $iDim = 1 Then If $iCol <> 0 Then $iCol = 0 EndIf EndIf If $iDim = 2 Then $iColMax = UBound($aArray, $UBOUND_COLUMNS) - 1 If $iCol > $iColMax Or $iCol < 0 Then Return SetError(1, 0, "") EndIf EndIf If Not _IsBetween($iDim, 1, 2) Then ;############### MSG2 - START ############### $sMsgHdr = FuncName($hFunc) & " :Line: " & @ScriptLineNumber & " :Error= " & @error $sMsg = "Invalid Dimensioned Array. Must be a 1 or 2 dimensional array." MsgBox(0, $sMsgHdr, $sMsg) Return SetError(1, 0, "") ;############### MSG2 - END ############### EndIf ; Identify what the value is ; If it is not a String, Int, Number, or Date then skip. Select Case _DateIsValid($vValue) = 1 $sVType = "D" Case IsNumber($vValue) = 1 $sVType = "N" Case IsString($vValue) = 1 $sVType = "S" Case Else ;############### MSG2 - START ############### $sMsgHdr = FuncName($hFunc) & " :Line: " & @ScriptLineNumber & " :Error= " & @error $sMsg = "Comparison value must be a " & @CRLF & _ "1. Date in YYYY/MM/DD format " & @CRLF & _ "2. A string " & @CRLF & _ "3. A number " & @CRLF MsgBox(0, $sMsgHdr, $sMsg) Return SetError(1, 0, "") ;############### MSG2 - END ############### EndSelect $iCnt = 0 For $n1 = UBound($aArray) - 1 To $iOptionBase Step -1 If $iDim = 1 Then $sData = StringStripWS($aArray[$n1], 3) ElseIf $iDim = 2 Then $sData = StringStripWS($aArray[$n1][$iCol], 3) EndIf Select Case _DateIsValid($sData) = 1 $sDType = "D" Case IsNumber($sData) = 1 $sDType = "N" Case IsString($sData) = 1 $sDType = "S" Case Else $sDType = "U" EndSelect If _IsValueEmpty($sData) Then $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop ; $sDType = $sVType EndIf If Not _IsValueEmpty($sData) And $sDType <> $sVType Then $sDeleteIndex = $sDeleteIndex & $n1 & "," $iCnt += 1 ContinueLoop EndIf Select Case $sOperator = "EQ" Switch $sDType Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff = 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "S" If $sData = $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "N" If $sData = $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "NEQ" Switch $sDType Case "D" If $sData <> $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "S" If $sData <> $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "N" If $sData <> $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "IN" Switch $sDType Case "S" If StringInStr($sData, $vValue) Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "GT" Switch $sDType Case "N" If $sData > $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff > 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "GE" Switch $sDType Case "N" If $sData >= $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff >= 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "LT" Switch $sDType Case "N" If $sData < $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff < 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "LE" Switch $sDType Case "N" If $sData <= $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff <= 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch EndSelect Next If $iCnt > 0 Then _DeleteArrayRows($aArray, $sDeleteIndex) EndIf EndFunc ;==>_ArrayFilter Thanks in advance.
      OldGuyWalking
    • MyEarth
      By MyEarth
      Hello. I have a 1D array. Is made in this way:
      1. This is a line Messages: a message etc Context: a context etc 2. This is a line Messages: a message etc Context: a context etc 3. This is a line Messages: a message etc Correction: a correction etc Context: a context etc I need to make something like:
      1. This is a line|Messages: a message etc|Context: a context etc 2. This is a line|Messages: a message etc|Context: a context etc 3. This is a line|Messages: a message etc|Correction: a correction|Context: a context etc For exporting in another software. I need to split every time there is a number when the line start, can be 1. until something like 3.976. Since i don't know if there a 2 line after a number or 3 i have opened this thread. Thanks
    • FMS
      By FMS
      Hello,
      The last couple of day's I was searching on this forum for the best way to put array's inside array's.
      The best example's i found where a little outdatet (2010) whit a lot of pro's and con's.
      Now I've a big script where a lot of computations and big array's are involved, so speed is a big issue.
      Also I wanna try the script below but don't know iff speed is a problem this way or maybe there is a better way to do this.

      Does somebody know's the best way to put array's inside array's and get the data back from them?
      I've made an example of something I was thinking about.
      (maybe something totaly wrong but I'm open for sugestions)
      I'm doing it this way because I don't think I can access the data inside the array (and doing some calculations to it) some other way if the array is inside another array.
      Or is there?
      Thanks in advanced.
      #include <Array.au3> Global $aArray[Random(5,10,1)][Random(5,10,1)] Global $Holder[2][2] For $x = 0 To UBound($aArray,1) - 1 For $y = 0 To UBound($aArray,2) - 1 $aArray[$x][$y] = Round(Random(-1,1),4) Next Next $Holder[0][0] = $aArray $aArray = "" Global $get_array = $Holder[0][0] _ArrayDisplay($get_array)  
    • FMS
      By FMS
      Hello,
      I'm trying to randomly change some cells in a array on a given percentage.
      at this point I 've a array whit all 0's and want to change some cells to 1.
      I'm not shure how to do this in a good coding sort of way.
      Also maybe there is a build in function whish I'm not aware of.
      Does somebody know how I can do this in a easy way?
      I was trying to get the total count of cells and get the percentage of it.
      And was stuck when i wanna change the cell.
      please advice, thanks in advanced for your help.
      #include <Array.au3> Global $percentage = 0.2 Global $aArray[Random(10,30,1)][Random(10,30,1)] For $x = 0 To UBound($aArray,1) - 1 For $y = 0 To UBound($aArray,2) - 1 $aArray[$x][$y] = 0 Next Next _ArrayDisplay($aArray) randomize() _ArrayDisplay($aArray) Func randomize() Local $total_to_change = ((UBound($aArray,1) * UBound($aArray,2)) / 100) * $percentage ConsoleWrite( "$total_to_change = " & $total_to_change & @LF ) ConsoleWrite( "total in array = " & (UBound($aArray,1) * UBound($aArray,2)) & @LF ) EndFunc  
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.