Sign in to follow this  
Followers 0
Xerix

_arraydelete is slow

19 posts in this topic

#1 ·  Posted (edited)

Hello

 

I'm looking for a function that would do the same as _arraydelete but faster.

 

The problem of _arraydelete is that it is very slow on large array.

 

Would you have this kind of function?

 

Thank you.
Edited by Xerix

Share this post


Link to post
Share on other sites



No. Maybe there is a better approach to your specific context.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

How big an array?

Maybe a database would be better...delete statements take fractions of a second.


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites
Not so great than that.

 

About 5000 lines (in 2D)

 

Maybe a database would be better...delete statements take fractions of a second.

 

 

Probably, but I don't know how to use databases
 
But I think that an array of 5000 line is not unmanageable.

Share this post


Link to post
Share on other sites

_ArrayDelete isn't optimized, but it does the work it was designed to do. If you want a faster array delete type function you'll probably have to create your own.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

Are you using _ArrayDelete in a loop, and deleting a lot of lines?  

Share this post


Link to post
Share on other sites

Are you using _ArrayDelete in a loop, and deleting a lot of lines?  

Yes this is the case

Share this post


Link to post
Share on other sites

Give us an example.  You can create a function that deletes ALL at one time, which would cut down on the looping to reset the array each time.


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

Is your data persistant?


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

I made a function that delete all the data in one loop...the results are huge time savings, but only works on a 1D array, for now

#include <array.au3>
Local $aFixed[5000]
For $i = 0 To UBound($aFixed)-1
    $aFixed[$i]=$i+1
Next

$a_ArrayDelete=$aFixed
$a_ArrayDeleteMult=$aFixed

; Setup array to delete multiple items...sort descending, so you delete proper items
Local $aDeletes[50]
For $i = 0 To UBound($aDeletes)-1
    $aDeletes[$i] = $i*5
Next


; Time _arraydelete for 50 deletes
$iArrayDelete = TimerInit()
For $i = 0 To UBound($aDeletes)-1
    _ArrayDelete($a_ArrayDelete,$aDeletes[$i])
Next
ConsoleWrite("_arraydelete: " & TimerDiff($iArrayDelete) & @CRLF)

$iArrayDeleteMulti = TimerInit()
_ArrayDeleteMulti($a_ArrayDeleteMult,$aDeletes)
ConsoleWrite("_arraydeleteMulti: " & TimerDiff($iArrayDeleteMulti) & @CRLF)

;~ _ArrayDisplay($a_ArrayDelete)
;~ _ArrayDisplay($a_ArrayDeleteMult)

For $i = 0 To UBound($a_ArrayDeleteMult)-1
    If $a_ArrayDelete[$i] <> $a_ArrayDeleteMult[$i] Then ConsoleWrite("wrong" & @CRLF)
Next

Exit

Func _ArrayDeleteMulti(ByRef $avArray, $aDeletes)

    If Not IsArray($avArray) Then Return SetError(1, 0, 0)

    Local $iUBound = UBound($avArray, 1) - 1

    $iMoves = 0
    For $j = 0 To UBound($aDeletes)-1
        If $j = UBound($aDeletes)-1 Then
            $iMax = $iUBound-UBound($aDeletes)
        Else
            $iMax = $aDeletes[$j+1]
        EndIf
        For $i = $aDeletes[$j] To $iMax
            $avArray[$i] = $avArray[$i+$j+1]
        Next
    Next

    ReDim $avArray[UBound($avArray)-UBound($aDeletes)]

    Return True
EndFunc   ;==>_ArrayDelete

output:

_arraydelete: 568.492675738927
_arraydeleteMulti: 13.6767694278312

Edited by jdelaney

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

Here's a simple example of what I would like to make

The idea is to remove any lines or appears the word doc in the 4th column.

Well here is a quick example but the idea is that the doc may be found in any line.
Not necessarily as here where the doc are all before the txt.

Thanks for your help
 
#include "array.au3"


Global $array[5000][6]


For $i = 0 To 4500
$array[$i][0] = $i
$array[$i][1] = "file" & $i
$array[$i][2] = "date" & $i
$array[$i][3] = "doc"
$array[$i][4] = "name" & $i
$array[$i][5] = "color" & $i
Next


For $i = 4501 To 4999
$array[$i][0] = $i
$array[$i][1] = "file" & $i
$array[$i][2] = "date" & $i
$array[$i][3] = "txt"
$array[$i][4] = "name" & $i
$array[$i][5] = "color" & $i
Next

_ArrayDisplay($array)

$ext = _ArrayFindAll($array, "doc", 0, 0, 0, 0, 3)
For $i = UBound($ext) - 1 To 0 Step -1
_ArrayDelete($array, $ext[$i])
Next

_ArrayDisplay($array)

Share this post


Link to post
Share on other sites

Try this, you'll see that it's much faster. Although it does presuppose you know the dimensions of the array to check.

#include "array.au3"
Global $array[5000][6]

For $i = 0 To 4500
    $array[$i][0] = $i
    $array[$i][1] = "file" & $i
    $array[$i][2] = "date" & $i
    $array[$i][3] = "doc"
    $array[$i][4] = "name" & $i
    $array[$i][5] = "color" & $i
Next

For $i = 4501 To 4999
    $array[$i][0] = $i
    $array[$i][1] = "file" & $i
    $array[$i][2] = "date" & $i
    $array[$i][3] = "txt"
    $array[$i][4] = "name" & $i
    $array[$i][5] = "color" & $i
Next

_ArrayDisplay($array)

;~ $ext = _ArrayFindAll($array, "doc", 0, 0, 0, 0, 3)
;~ For $i = UBound($ext) - 1 To 0 Step -1
;~  _ArrayDelete($array, $ext[$i])
;~ Next
Local $aTemp[5000][6], $X = 0
For $i = 1 To UBound($array) - 1
    If $array[$i][3] <> "doc" Then
        For $j = 0 To 5
            $aTemp[$X][$j] = $array[$i][$j]
        Next
        $X += 1
    EndIf
Next
ReDim $aTemp[$X][6]
$array = $aTemp

_ArrayDisplay($array)

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

Thank you BrewManNH

So the idea is to transfer all the txt file in an another temporary array then replace this temporary array values in the main array.

 

And indeed it is extremely faster

 

Can you explain in $X += 1 the +=
I think have already asked the question in another message but I don't know where. :idiot:

 

Thank you

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

Just a small correction.

To find the exact number should we not have
For $i = 0 To UBound($array) - 1

Rather than

For $i = 1 To UBound($array) - 1

And maybe

Local $aTemp[UBound($array)][6], $X = 0
Edited by Xerix

Share this post


Link to post
Share on other sites

$X += 1
; is the same as
$X = $X + 1

Look at the 2nd parameter of UBound to set dimension to get the size. UBound($array, 1) 1st dimension (default), UBound($array, 2) 2nd dimension ...

:)

Share this post


Link to post
Share on other sites

Just a small correction.

 

To find the exact number should we not have

For $i = 0 To UBound($array) - 1
Yes, you're right, small error on my part.

Rather than

For $i = 1 To UBound($array) - 1
And maybe

Local $aTemp[UBound($array)][6], $X = 0
As I had said, this would only work if you knew what size the incoming array would be. It was just an example to get the OP started in the right direction. _ArrayDelete is terrible when used in a loop with a lot of deletions, but that isn't exactly how it was intended to be used.

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
OK, thank you all.

 

As we talked, what would be the code on this example with databases.

 

Thank you

Share this post


Link to post
Share on other sites

The Array.au3 I feel should be only used for simple array manipulation if you're scared to get your feet wet with using arrays as they are.


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0