Jump to content
Ascer

[UDF] ArrayList - actions on huge arrays

Recommended Posts

Ascer

1. Description.

  • Udf working with MSDN System.Collections.ArrayList.
  • Allow you to make fast operations on huge arrays, speed is even x10 better than basic _ArrayAdd. 
  • Not prefered for small arrays < 600 items.

2. Requirements

  • .NET Framework 1.1 - 4.5 (on this version Microsoft destroy old rules)
  • System Windows

3. Possibilities.

;===============================================================================================================
; UDF Name:         List.au3
;
; Date:             2018-02-17, 10:52
; Description:      Simple udf to create System Collections as ArrayList and make multiple actions on them.
;
; Function(s):      _ListCreate         -> Creates a new list
;                   _ListCapacity       -> Gets a list size in bytes
;                   _ListCount          -> Gets items count in list
;                   _ListIsFixedSize    -> Get bool if list if fixed size
;                   _ListIsReadOnly     -> Get bool if list is read only
;                   _ListIsSynchronized -> Get bool if list is synchronized
;                   _ListGetItem        -> Get item on index
;                   _ListSetItem        -> Set item on index
;
;                   _ListAdd            -> Add item at end of list
;                   _ListClear          -> Remove all list items
;                   _ListClone          -> Duplicate list in new var
;                   _ListContains       -> Get bool if item is in list
;                   _ListGetHashCode    -> Get hash code for list
;                   _ListGetRange       -> Get list with items between indexs
;                   _ListIndexOf        -> Get index of item
;                   _ListInsert         -> Insert a new item on index
;                   _ListInsertRange    -> Insert list into list on index
;                   _ListLastIndexOf    -> Get index last of item
;                   _ListRemove         -> Remove first found item
;                   _ListRemoveAt       -> Remove item in index
;                   _ListRemoveRange    -> Remove items between indexs
;                   _ListReverse        -> Reverse all items in list
;                   _ListSetRange       -> Set new value for items in range
;                   _ListSort           -> Sort items in list (speed of reading)
;                   _ListToString       -> Get list object name
;                   _ListTrimToSize     -> Remove unused space in list
;
; Author(s):        Ascer
;===============================================================================================================

4. Downloads

5. Examples

 

Edited by Ascer
  • Like 2

Share this post


Link to post
Share on other sites
Ascer
  • SpeedTest  ArrayAdd  vs  ListAdd
#include <Array.au3>
#include <List.au3>

;==> _ListAdd()

Local $iItems = 5000

ConsoleWrite("[_ListAdd]" & @CRLF)
ConsoleWrite("Start adding " & $iItems & " items using _ListAdd func..." & @CRLF)

Local $aListAdd = _ListCreate()
Local $iListAddTime = TimerInit()

For $i = 1 To $iItems
    _ListAdd($aListAdd, $i)
Next

ConsoleWrite("Adding has end. Time spent on this action is " & Int(TimerDiff($iListAddTime)) & " ms." & @CRLF)
ConsoleWrite("UBound of $aListAdd is " & _ListCount($aListAdd) & "." & @CRLF & @CRLF & @CRLF)

;==> _ArrayAdd()

ConsoleWrite("[_ArrayAdd]" & @CRLF)
ConsoleWrite("Start adding " & $iItems & " items using _ArrayAdd func..." & @CRLF)

Dim $aArrayAdd[0]
Local $iArrayAddTime = TimerInit()

For $i = 1 To $iItems
    _ArrayAdd($aArrayAdd, $i)
Next

ConsoleWrite("Adding has end. Time spent on this action is " & Int(TimerDiff($iArrayAddTime)) & " ms." & @CRLF)
ConsoleWrite("UBound of $aArrayAdd is " & UBound($aArrayAdd) & "." & @CRLF)

 

Edited by Ascer

Share this post


Link to post
Share on other sites
Ascer
  • TestSpeed ArraySearch vs ListIndexOf
#include <Array.au3>
#include <List.au3>

ConsoleWrite("Preparing array and list for later use..." & @CRLF)

Local $iSearchItem = 4999

Dim $aArray[0]

For $i = 0 To 5000
    _ArrayAdd($aArray, $i)
Next

Local $aList = _ListCreate()

For $i = 0 To 5000
    _ListAdd($aList, $i)
Next

ConsoleWrite("$aArray and $aList are ready." & @CRLF & @CRLF)

ConsoleWrite("Searching $aArray for item " & $iSearchItem & "..." & @CRLF)

Local $iArrayTime = TimerInit()
Local $iArrayFound = _ArraySearch($aArray, $iSearchItem)

If $iArrayFound <> - 1 Then
    ConsoleWrite("Found item in " & TimerDiff($iArrayTime) & " ms."  & @CRLF & @CRLF)
EndIf

ConsoleWrite("Searching $aList for item " & $iSearchItem & "..." & @CRLF)

Local $iListTime = TimerInit()
Local $vListFound = _ListIndexOf($aList, $iSearchItem)

If $vListFound <> Null Then
    ConsoleWrite("Found item in " & TimerDiff($iListTime) & " ms." & @CRLF)
EndIf

 

Share this post


Link to post
Share on other sites
Ascer
  • Basic usage - creating guild with members
#include <List.au3>

ConsoleWrite("Creating list for guild members..." & @CRLF)

Local $aMembers = _ListCreate()
Local $iTotalMembers = 1000

ConsoleWrite("Successfully created list." & @CRLF)

ConsoleWrite("Creating guild members..." & @CRLF)

For $i = 1 To $iTotalMembers
    Local $aMember = _CreateMember( _
        "Member_" & $i, _
        "Vocation_" & Random(1, 4, 1), _
        Random(50, 250, 1), _
        "Rank_" & Random(1, 3, 1) _
    )
    _ListAdd($aMembers, $aMember)
Next

ConsoleWrite("Successfully created " & $iTotalMembers & " members." & @CRLF)
ConsoleWrite("Successfully added members to list." & @CRLF)

ConsoleWrite("Preparing for display list..." & @CRLF)

Local $sDisplay

For $aMember In $aMembers
    $sDisplay &= "Browse Member..." & @CRLF

    $sDisplay &= @TAB & "Name = " & _ListGetItem($aMember, 0) & @CRLF
    $sDisplay &= @TAB & "Vocation = " & _ListGetItem($aMember, 1) & @CRLF
    $sDisplay &= @TAB & "Level = " & _ListGetItem($aMember, 2) & @CRLF
    $sDisplay &= @TAB & "Rank = " & _ListGetItem($aMember, 3) & @CRLF

Next

ConsoleWrite($sDisplay)

Func _CreateMember($sName, $sVocation, $iLevel, $sRank)
    Local $aMember = _ListCreate()
    _ListAdd($aMember, $sName)
    _ListAdd($aMember, $sVocation)
    _ListAdd($aMember, $iLevel)
    _ListAdd($aMember, $sRank)
    Return $aMember
EndFunc ;==>_CreateMember

 

Edited by Ascer

Share this post


Link to post
Share on other sites
junkew

Speed example results on my system but would be interesting to see all native AutoIt and .NET  collections compared (including maps in beta AutoIt)
http://geekswithblogs.net/BlackRabbitCoder/archive/2011/06/16/c.net-fundamentals-choosing-the-right-collection-class.aspx

 

[_ListAdd]
Start adding 5000 items using _ListAdd func...
Adding has end. Time spent on this action is 103 ms.
UBound of $aListAdd is 5000.


[_ArrayAdd]
Start adding 5000 items using _ArrayAdd func...
Adding has end. Time spent on this action is 3424 ms.
UBound of $aArrayAdd is 5000.
  • TestSpeed ArraySearch vs ListIndexOf
Preparing array and list for later use...
$aArray and $aList are ready.

Searching $aArray for item 4999...
Found item in 3.74687017624091 ms.

Searching $aList for item 4999...
Found item in 0.311995511007418 ms.

And this one some nice VB .NET steps to the arrays in post#2

 

Edited by junkew

Share this post


Link to post
Share on other sites
ptrex

Share this post


Link to post
Share on other sites
Ascer

@ptrex

I've added note about this in requiments.

Did you hear about OLEView.exe ? to check current available objects on your machine?

On my Windows 7 x64 and Windows Server 2008 R2. ArrayList working good.

What you tell about WIA.Vector ?  

 

Share this post


Link to post
Share on other sites
junkew
ptrex

Vector Object seems to be working fine on Win7 and 10 , Performance not tested ...

local $v

$v = ObjCreate("WIA.vector")

$v.Add(1)
$v.Add(42)
$v.Add(3)

$v.Remove(1)
$v.Remove(2)

ConsoleWrite("$v(1) = " & $v(1) & @CRLF)

$v.Clear

$v.Add("This")
$v.Add("Is")
$v.Add("Cool")

$v.Remove(1)
$v.Remove(1)

ConsoleWrite("$v(1) = " & $v(1) & @CRLF)

 

Share this post


Link to post
Share on other sites
Ascer

@ptrex

Take a look into this thread: 

 

Share this post


Link to post
Share on other sites
Earthshine

C# is my pick. Nothing you can’t do. Lists can be covert to array and back instantly. Tons of methods to use for anything you want. Now there is a new type Span<T> that lets us do even more in .net Core 2.1

https://msdn.microsoft.com/en-us/magazine/mt814808.aspx

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • DarkFingers1337
      By DarkFingers1337
      Hey,
      what is more efficient, using multiple arrays or one multidimensional array?
       
      ; Hamster data structure using multiple arrays Global $HamsterCount = 6 Global $HamsterX[$HamsterCount] Global $HamsterY[$HamsterCount] ;giving each hamster an x and y position For $i=0 To $HamsterCount-1 $HamsterX[$i] = 0 $HamsterY[$i] = 0 Next ; The same hamster data structure using one multidimensional array Global $HAMSTER_COUNT = 6 Global Enum $HAMSTER_X, _ $HAMSTER_Y, _ $HAMSTER_MAX Global $Hamsters[$HAMSTER_COUNT][$HAMSTER_MAX] ;giving each hamster an x and y position For $i=0 To $HAMSTER_COUNT-1 $Hamsters[$i][$HAMSTER_X] = 0 $Hamsters[$i][$HAMSTER_Y] = 0 Next  
    • Skysnake
      By Skysnake
      I know how JSON works. However JSON data is typically received from web based servers. 
       
      My question is:
      Has anyone considered to use, or currently actively using JSON as an internal data container? 
      Internal here as in an ordinary AutoIt data type, generated, populated and manipulated within an AutoIt script, without any external data sources. 
      The JSON and BinaryCall UDFs work well and appear the de facto defaults. Are there any thoughts on using JSON for internal variable values instead of the typical array? 
      Also, there can be situations where such a JSON collection contains an array and it will have to be parsed using typical array functions. 
      Currently I am working on a small project where I have to keep and use various identifiers from different sources. There are several ideas in this regard, some involving SQL child tables and arrays stored with keys (maps), which lead me to think that JSON may be a more suitable data container. It readily shrinks and stretches, allowing for dynamic data growth.  JSONs ability to add new repeating data groups on the fly makes it particularly powerful and appealing. 
      Ideas and suggestions welcome
      Skysnake
    • XinYoung
      By XinYoung
      Hello all! I hope everyone is enjoying their holiday festivities. 
      I'm working on a script that involves copying a string of text from an Excel workbook and searching for it in a particular website's search tool. If a result is found, it will do something. If not, it will do something else.
      So far, it can successfully execute the search -- and then it shows me the results in an array.
      Screenshot of the successful search:

      The search results in an array:

       
      Here's the code (sorry for all my comments):
      ;~ All the functions this app performs require the external files listed here. So, theyre "included". #include <ButtonConstants.au3> #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> #include <WinAPIFiles.au3> #include <Array.au3> #include <File.au3> #include <Excel.au3> #include <DateTimeConstants.au3> #include <MsgBoxConstants.au3> #include <WinAPIShellEx.au3> #include <Date.au3> #include <ComboConstants.au3> #include <Misc.au3> #include <WinAPIProc.au3> #include <WinAPISys.au3> #include <WinAPIConstants.au3> #include <Crypt.au3> #include <ColorConstants.au3> #include <guimenu.au3> #include <IE.au3> ;~ Kill all functions and close the app at anytime by pressing F4. HotKeySet("{F4}", "_Exit") ;~ Keep track whether or not a file is selected. When the program first opens, a file is currently not selected. Global $FileChosen = 0 ;~ The app must remember certain strings of text: ;~ 1. Login page Global $urlBBLogin = "website.com" ;~ 2. Credentials Global $bbUsername = "USER" Global $bbPassword = "PW" ;~ 3. Search page Global $urlBBCourseSearch = "website.com/search" ;~ When you launch the app, the UI gets built and is displayed to the user in the center of the screen. the "Function" buttons are disabled until a file is chosen. $MasterUI = GUICreate("Master Re-Creator", 469, 145, -1, -1) $Label1 = GUICtrlCreateLabel("Choose the Excel file", 8, 8, 103, 17) $Select = GUICtrlCreateButton("Select File", 16, 32, 75, 25) $FileName = GUICtrlCreateLabel("[No File Selected]", 104, 40, 88, 17) $Group1 = GUICtrlCreateGroup("Functions", 8, 72, 449, 65) $CheckCourse = GUICtrlCreateButton("Check Courses Exist", 24, 96, 123, 25) GUICtrlSetState(-1, $GUI_DISABLE) $DeleteCourse = GUICtrlCreateButton("Delete Courses", 168, 96, 123, 25) GUICtrlSetState(-1, $GUI_DISABLE) $CopyCourse = GUICtrlCreateButton("Copy Courses", 312, 96, 123, 25) GUICtrlSetState(-1, $GUI_DISABLE) GUICtrlCreateGroup("", -99, -99, 1, 1) GUISetState(@SW_SHOW) ;~ While the UI is open, it listens for triggers (in this case, button presses). While 1 $UI = GUIGetMsg() Select ;~ If the app is closed, the _Exit() function is performed (same function the F4 hotkey calls). Case $UI = $GUI_EVENT_CLOSE _Exit() ;~ The user has clicked the "Select File" button, the _LocateGetFileName() function is performed. Go there. Case $UI = $Select _LocateGetFileName() ;~ The user has clicked the "Check Courses Exist" button. Case $UI = $CheckCourse _CheckCourses() ;~ Other buttons are not ready EndSelect WEnd ;~ The user clicked the "Select File" button. This function will execute now. Func _LocateGetFileName() ;~ Prepare the app to take note of the details of a file. Local $sDrive = "", $sDir = "", $sFileName = "", $sExtension = "" ;~ Open a File Explorer to allow the user to select a file. Only Excel files are allowed to be chosen. Global $ChosenFileName = FileOpenDialog("Locate File", @DesktopDir, "Excel Files (*.xlsx)|Excel Macro Files (*.xlsm)", BitOR(1, 2), "") If @error Then Return 0 EndIf ;~ When an Excel file is selected, remember of the files location (path), file name, and file extension. $aPathSplit = _PathSplit($ChosenFileName, $sDrive, $sDir, $sFileName, $sExtension) ;~ Show me what file I selected in a Message Box. MsgBox(0, "Selected File", $sFileName) ;~ Display the chosen file name in the UI label (previously [No File Selected]) and make it green. GUICtrlSetData($FileName, "") $FileName = GUICtrlCreateLabel($sFileName, 104, 40) $FileName = GUICtrlSetColor($FileName, 0x32CD32) ;~ A file is now selected. The "Function" buttons are now enabled. Global $FileChosen = 1 GUICtrlSetState($CheckCourse, $GUI_ENABLE) GUICtrlSetState($DeleteCourse, $GUI_ENABLE) GUICtrlSetState($CopyCourse, $GUI_ENABLE) EndFunc ;==>_LocateGetFileName ;~ The user clicked the "Check Courses" button. This function will execute now. Func _CheckCourses() ;~ Disable the "Function" buttons again to prevent multiple processes. GUICtrlSetState($CheckCourse, $GUI_DISABLE) GUICtrlSetState($DeleteCourse, $GUI_DISABLE) GUICtrlSetState($CopyCourse, $GUI_DISABLE) ;~ Open a IE window and navigate to the login page. Global $oIE = _IECreate($urlBBLogin) ;~ Recognize the form on this page (login input boxes). Local $oForm = _IEFormGetObjByName($oIE, "login") Local $oTextLogin = _IEFormElementGetObjByName($oForm, "user_id") Local $oTextPass = _IEFormElementGetObjByName($oForm, "password") ;~ Enter the Automation user credentials into the form. _IEFormElementSetValue($oTextLogin, $bbUsername) _IEFormElementSetValue($oTextPass, $bbPassword) ;~ Click the Login button. _IEFormSubmit($oForm) ;~ Now that were logged in, navigate to the course search page. _IENavigate($oIE, $urlBBCourseSearch) ;~ Change the search criteria to "Course ID" _bbCourseSearchCategoryChange("Course ID") ;~ Open the selected Excel file Local $oAppl = _Excel_Open() Local $oWorkbook = _Excel_BookOpen($oAppl, $ChosenFileName, Default, Default, True) ;~ Copy just whats in cell A1 (for now) _Excel_RangeCopyPaste($oWorkbook.Worksheets(1), "A1") Global $WhatsCopied = ClipGet() ;~ Paste whats copied into the search text box and click submit Local $oForm = _IEGetObjByName($oIE, "courseManagerFormSearch") Local $oSearchString = _IEFormElementGetObjByName($oForm, "courseInfoSearchText") _IEFormElementSetValue($oSearchString, $WhatsCopied) _IEFormSubmit($oForm) ;~ Lets see what we got from the search Local $oBBTable = _IETableGetCollection($oIE, 2) Local $aBBTableData = _IETableWriteToArray($oBBTable) _ArrayDisplay($aBBTableData) EndFunc ;==>_CheckCourses ;~ This function allows changing the search criteria. Func _bbCourseSearchCategoryChange($sCategoryToSearch) Local $aSearchCategory[6] = ["Course ID", "Course Name", "Description", "Instructor", "Data Source Key", "Term"] Local $oForm = _IEGetObjByName($oIE, "courseManagerFormSearch") Local $oSearchCategory = _IEGetObjByName($oForm, "courseInfoSearchKeyString") _IEAction($oSearchCategory, "focus") _IEFormElementOptionSelect($oSearchCategory, $aSearchCategory[$sCategoryToSearch], 1, "byText") EndFunc ;==>_bbCourseSearchCategoryChange ;~ All exit commands, including F4, calls this function Func _Exit() Exit EndFunc ;==>_Exit  
      My main question is: How do I create an If... Then based on what is found in the search results? I need additional tasks to run if Col 1, Row 2 in the array contains the exact string I searched for. (Am I going about this the right way?)
      My next question (I might make a new thread for): How do I make the whole thing loop, as in, copy the next cell in the Excel sheet and do the whole thing over again until there's no more? I understand that a For/Next loop thingy would be used. I just don't know how. Loops are really confusing to me.
      Thank you all for your guidance and have a happy new year!
    • lattey
      By lattey
      hi,
      i have checkboxes and each checkbox that checked, i put in array. 
      now, im stuck on how to loop the checked array and store in in one variable. what i can do now, is only write the result into a text file. 
      below is the code:
      #include <GUIConstantsEx.au3> ;~ #include <MsgBoxConstants.au3> #include <ButtonConstants.au3> #include <Array.au3> Global $Count = 3 Global $CheckBoxP[$Count] Global $step[$Count] global $array1[1] Global $ExitResult $hGUI = GUICreate("Summary Steps", 500, 400) GUISetFont(12, 400, "Tahoma") GUICtrlCreateLabel( "Please Select the Summary Steps for Script Check", 70, 20) GUISetFont(10, 400, "Tahoma") Global $array_Pstep[3] = ["fix2","fix1","fix3"] global $step[3] = ["2","3","4"] $Spacing = 50 For $i = 0 To UBound($array_Pstep) - 1 $CheckBoxP[$i] = GUICtrlCreateCheckbox($array_Pstep[$i], 80, $Spacing + (20 * $i), 65, 17) Next $submit = GUICtrlCreateButton("Submit",180, 280, 80, 30) $exit = GUICtrlCreateButton("Exit",180, 320, 80, 30) GUISetState() While 1 $Msg = GUIGetMsg() Select case $Msg=$submit For $i = 0 To $Count - 1 If GUICtrlRead($CheckBoxP[$i]) = $GUI_CHECKED Then _ArrayAdd($array1, $step[$i]) EndIf Next Global $logfilerray = @WorkingDir & "\checkedlist.txt" FileDelete ($logfilerray) Global $readlogfile = FileOpen($logfilerray,1) for $a = 1 to UBound($array1) - 1 ;~ $var=$array1[$a] FileWriteLine($readlogfile,$array1[$a]) Next FileClose($readlogfile) Exit case $Msg=$exit $ExitResult = MsgBox(1,"Summary Step", "Continue to Exit ?") if $ExitResult = 1 Then ;ok Exit EndIf Exit EndSelect WEnd  
    • omicron
      By omicron
      How do you perform a nested loop function with a multidimensional array from 2 lists.
      for i in list1
      (open file) extract variable
          while open for i in list 2
          (open file2) extract variable
       
      var1 + var2 = (search term)

      The list sizes will more than likely consist of different lengths.
       
      What is the best approach to accomplishing this method?
             
×