Optimized Array.au3

[GEOSoft]

NEW

Array_benchmark.zip (for benchmarking)

Array_UDF.zip (for UDF submission)

In Array_UDF.zip, I've included a list of any possible compatibility problems (Compatibility Concerns.txtthat may arise in the transition from the current official Array.au3 and my version. I honestly tried to track down all changes, but I might have missed something along the way (I doubt it though). Even still, that's why these "new" UDFs need to be tested widely, so please test if you can? Thanks, and enjoy!

Changes since last time...

- $iEnd now defaults to 0 instead of -1 (minor change, but will probably clear up some backwards compatibility problems I forgot about)

- _ArrayDisplay() should now be backwards compatible (old parameters added back)

- Added _ArrayFindAll() [thanks GEOSoft]

- Added backwards searching capability to _ArraySearch() [thanks GEOSoft for the idea]

- Changed error code for _ArraySearch() and _ArraySort() just a bit more; now properly sets $iSubItem if out of bounds instead of simply failing

COMPATIBILITY  CONCERNS
-=-=-=-=-=-=-=-=-=-=-=-

_ArrayBinarySearch()
====================
- Changed the failure return value from "" to -1. I made this change for the sake of consistency with _ArraySearch()'s failure return value. In the context of returning index values, -1 seems like a better failure response than simply an empty string... Opinions?
_ArrayDelete()
==============
- I'm not really sure if this would cause a compatibility problem, but um... Instead of failing to delete the last item in an array, I made the function behave more like _ArrayPop() by setting the passed array to "" instead of leaving it alone.
_ArrayDisplay() [still up in the air about this one]
===============
- The only possible problem now is the redefinition of the third parameter, $i_ShowOver4000. I tend to prefer a soft, redefinable limit wherever possible over a hard 4000 limit, so I thought the function should allow the user to select the number of items instead of toggling between 4000 or all. Dunno, what do you guys say?
_ArrayMaxIndex() / _ArrayMinIndex()
===================================
- For the same reasons as with _ArrayBinarySearch(), I changed the failure return value to -1. Basically, if index, then return -1 on failure; if value, then return "" on failure; if miscellaneous, it's usually return 0 on failure. That's how I made my decisions for the return values anyway. IMHO, making everything consistent now makes it easier to maintain in the future, and makes it easier for people that use the functions regularly [since there's a pattern and all] :\
_ArrayPush()
============
- Well, the function originally used the return value as the error signal, which is probably a bad idea. I simply set it to return 0 and make proper use of error codes on failure.
_ArraySearch()
==============
- The old meanings for error codes 2-5 were deprecated, since they no longer apply
- Error code 2 now indicates that $avArray contains more dimensions than the function handles
- Error code 3 now indicates that the value wasn't found (basically the equivalent of the old error code 6)
_ArraySort()
============
- The $i_Dim parameter was removed entirely, and $i_SortIndex (now called $iSubItem) moves up one position to take its place...
_ArrayToString()
================
- Error code 2 removed due to bounds checking...
_ArrayTrim()
============
- Error codes 2-4 removed due to bounds checking...

Good work but there is still going to be a compatability problem. You should NOT change the existing error codes. Just remove those that are depricated.

For example. In _ArraySearch(), if the value is not found it should still return 6, not 3. People could be checking for Error code 6. With tose that are removed it doesn't matter if they are checking or not because there will be no error return.

Edit:: Also I just noticed that you removed some parameters and shifted the others up....... Instant compatibility problem. Leave them in but don't refer to them in the function. That will keep it compatible. I know it seems stupid to leave a redundant parm but in order for it to be a direct replacement for the existing function that's all you can do. The only way that you could safely use your changes would be to rename the file to array2.au3 or some such. That way they couls #include either array.au3 or array2.au3.

Your code woud break about 60% of my current AutoIt projects.

Edited August 29, 2007 by GEOSoft

[-Ultima-]

Also I just noticed that you removed some parameters and shifted the others up.

Other than _ArraySearch(), where have you seen this happen? JdeB has no problem with _ArraySearch() being modified as it has been (and would've done it on his own anyway). The list up above includes _ArrayDisplay() as a compatibility concern, but that's because I forgot to delete it from the list before uploading it I'm pretty sure all of these concerns have been discussed before; I'm just collecting them in a list like JdeB requested.

As for the error codes, I previously asked for suggestions on the matter, but no one responded to me then (before I made another release :|)... Thanks for the feedback though

I'd still like to see more feedback and opinions (which has been sorely lacking for a change as big as this; maybe that's a good sign?). Error codes are minor details that can be easily changed, so I'm not too worried about getting any possible changes to them in on time; what I'm most worried about is the reliability/stability of my code. So for that change, I'll wait for JdeB to look through the rest of the UDF. I'll still be actively reading this thread and considering any further suggestions, though (of course).

_{Edit: Made tone of post less menacing... wasn't my intention}

Edited August 30, 2007 by -Ultima-

[-Ultima-]

Found a silly bug in my rewrite of _ArrayDisplay() where it would never displayed more than 4000 items. I was calling the _GUICtrlListViewInsertItem() with incorrect arguments

Also, I just realized that _ArrayDisplay()'s first parameter might be another point of failure for some scripts. As it currently is, _ArrayDisplay() doesn't take the array in by reference; it takes it in by value. This stands in stark constrast with all the other _Array* functions, which is why I decided to change it.

I'll update the attachment later.

Edit:

Array_UDF.zip (for UDF submission)

Edited September 9, 2007 by -Ultima-

[-Ultima-]

I've updated the first post with more changes...

BOUNDS CHECKING

I changed the bounds checking a bit (again) because it was faulty for when $iEnd = $iStart. For example, if you used the 3-element array returned by StringSplit("a|b", "|") in _ArrayToString($avArray, "|", 1, $avArray[0]-1) would return "a|b" instead of the expected "a". The change might've improved the performance a bit for some functions in certain situations, as there is potentially 1 check less. The old bounds checking looked like this:

If $iEnd <= $iStart Or $iEnd > $iUBound Then $iEnd = $iUBound
    If $iStart < 0 Or $iStart > $iEnd Then $iStart = 0

The "new" bounds checking:

If $iEnd < 1 Or $iEnd > $iUBound Then $iEnd = $iUBound
    If $iStart < 0 Then $iStart = 0
    If $iStart > $iEnd Then Return SetError(1, 0, 0); different return value and error depending on function

I can't really call this bounds checking all that "new," as it looks reminiscent of the bounds checking that was already in place in some of the original functions. Turns out they did a better job than I had initially thought Anyway, all functions that have $iStart and $iEnd parameters now use this.

ERROR CODES

I think I reverted most of my error code changes, meaning error codes should now be backwards compatible. The error codes that I added were given numbers higher than the old error codes. For the documentation, I simply put a note in the @Error section that whatever deprecated error codes were deprecated, like so:

(2-4 = Deprecated error codes)

OTHER STUFF

- _ArrayConcatenate() now returns $avArrayTarget's new size instead of just 1

- _ArrayDisplay() now Au3Check.exe friendly again (whoops! )

- _ArrayFindAll() is actually faster now. I had misread my previous results as my implementation being faster than GEOSoft's. Um, after another rewrite of the function, the code is much simpler, and is just slightly faster than GEOSoft's (unreleased?) _ArrayFindAll() by around 3-5% (was slower by about 40-60% in the last rewrite; slip up!)

- _ArrayToClip() now relies entirely on _ArrayToString(). This change makes sense, as it was *exactly* the same as _ArrayToString() until the very end, where it simply copied the string into the clipboard.

Edited September 14, 2007 by -Ultima-

[randallc]

Also, I just realized that _ArrayDisplay()'s first parameter might be another point of failure for some scripts. As it currently is, _ArrayDisplay() doesn't take the array in by reference; it takes it in by value. This stands in stark constrast with all the other _Array* functions, which is why I decided to change it.

Hi,

I think byval is correct in the original version, else will a transposed view then change the original script array?...

Maybe OK in yours if you don't change the array for transpose or other reason, only make listview delimited strings array with different name...

Best, Randall

[-Ultima-]

Yep. I actually take the array in as a Const ByRef, so there's no way I'll end up modifying it (not even by mistake).

Edited September 15, 2007 by -Ultima-

[Jos]

Ok folks... Lets start to finalize this item and work towards using this set of Array.au3 UDFs replacing the current set.

What things still need to be discussed before we can move on?

Backward compatibility is close as far as I understand ... right ?

Are there any UDF's that still need to be discussed as far as functionality ?

When all is agreed upon I will need the _UDFName.txt and _UDFName.au3 files that are required for the generation of the Helpfile and examples.

Thoughts / remarks / suggestions / opinions ?

Thanks

Jos

[jackyyll]

Why is it that we can't define a variable as an empty array in autoit? :/ That'd be a very nice thing to have instead of having to do Dim $var you could instead do like $var = _ArrayNew() which creates an empty array. Dunno, just saying.. I'm used to Ruby now so i like either $var = [] or $var = Array.new

$0.02

[GaryFrost]

Ok folks... Lets start to finalize this item and work towards using this set of Array.au3 UDFs replacing the current set.

What things still need to be discussed before we can move on?

Backward compatibility is close as far as I understand ... right ?

Are there any UDF's that still need to be discussed as far as functionality ?

When all is agreed upon I will need the _UDFName.txt and _UDFName.au3 files that are required for the generation of the Helpfile and examples.

Thoughts / remarks / suggestions / opinions ?

Thanks

Jos

If you want Jos, they can send the au3 to me and I can set up the headers to work with Template Builder. Your choice.

[Jos]

If you want Jos, they can send the au3 to me and I can set up the headers to work with Template Builder. Your choice.

That is a good plan but why don't we just post it, explain how to use it and let the UDF submitters update their header ?

[Jos]

Why is it that we can't define a variable as an empty array in autoit? :/ That'd be a very nice thing to have instead of having to do Dim $var you could instead do like $var = _ArrayNew() which creates an empty array. Dunno, just saying.. I'm used to Ruby now so i like either $var = [] or $var = Array.new

$0.02

This is not related to this thread ... please make a new thread in the appropriate forum when you have questions or want to propose things for AutoIt3.

[-Ultima-]

Well, I have already been including all of the function documentations/examples in Array_UDF.zip since I started this little project, so that part should already be done (unless there are other suggestions that would cause them to be changed) That's, of course, assuming that the requirements haven't changed since this page was written O.o

@jackyyll: That's outside the scope of the UDFs (which is what this entire thread is all about), and wouldn't be possible without a core language change anyway. AutoIt has absolutely no concept of an empty array. An array must have at least one item; AutoIt will spit an error back at you if you try Dim $var[0].

Edit: Uh, I just noticed a small, silly bug in my _ArrayDisplay()... I didn't anchor $iCopy $hCopy to the bottom (so when you make the window taller, the button starts slipping behind the listview) -.- I'll update the attachments in a bit. Updated the attachments...

Edited October 17, 2007 by -Ultima-

[randallc]

Hi, [i am not sure that _ArrayDisplay is working; returning an empty Display for me on at least some scripts, where myversion does not..]

Copying everything when nothing is selected is the exact opposite of intuitive :|

I am not going to argue strongly about this, despite my wanting it to be easy to copy the whole lot as was previuous behaviour!; but then the code should reflect that;

Case $iCopy
                Local $sClip = ""
                $arSelItems = _GUICtrlListView_GetSelectedIndices($iListView, 1)
                    If not (IsArray($arSelItems)) or  $arSelItems[0]=0 Then
                    For $i = 0 To $iUBound - 1
                        $sClip &= $avArrayText[$i] & @CR
                    Next
                Else
                    For $i = 1 To UBound($arSelItems) - 1
                        $sClip &= $avArrayText[$arSelItems[$i]] & @CR
                    Next
                EndIf
                ClipPut($sClip)

Needs changing?

** I would be glad to see if the larger _Array.au3 UDF is worthwhile, that it be used, although it does not seem to me to add any sig functionality to _ArrayDisplay without my modifications; which

1. Make copying total Array to clipboard easy.

2. Allow including column headers piped into "$Title" parameter.

3. Avoid erratic display if many columns are used.

[EDIT -4. Height and Width of GUI mod in mine...]

A. If there is a problem with my modified script with the button, could that not just be modified?

B. Could the "Close" button be added back?...

I will send this to -Ultima- who has done all the hard work [and the Help templates are there...]; then there is only one person with whom to negotiate...

Best, randall

Edited October 17, 2007 by randallc

[randallc]

hi,

Here's a comparison script;

Randall

[-Ultima-]

I'll fix the copy behavior, but I'm not sure the extra feature (column header customization) should be included.

Anyone who wants to customize their column headers obviously need to display arrays as a prominent part of their program. The _ArrayDisplay() function, as I see it, should be a function purely to display arrays -- nothing more. Customizing column headers goes beyond that, and IMHO, beyond the scope of what a UDF should do. Anyone who needs to customize their column headers would be better off writing a better-integrated function than using an utility UDF.

Beyond that, though, I tend to disagree with retrofitting a feature onto the $sText parameter, as that results in something that looks more like an ugly hack addon instead of a feature (no offense intended at all). If anything, a new parameter should've been added instead of trying to make the $sTitle parameter do more than what it should be used for (setting the dialog title). Regardless, though, as I said, I'm not sure this kind of feature belongs in a standard UDF.

The point of my making all of these array changes was to overhaul the entire thing, since it has become somewhat dilapidated in terms of readability/quality, and speed. I wasn't really setting out to add new features (though I did add where I felt they would make the function more "complete"). As it stands, _ArrayDisplay() is already a very complete function (since it does what it was originally intended to do -- display an array), and then some (like transposing the display of an array). Adding any additional features would make it more "bloated" than it would increase the function's real utility value.

Regarding the width/height thing... varying the size depending on desktop resolution doesn't seem necessary to me, since the dialog is resizable for a reason.

As for erratic display... can you elaborate? Also, can you provide an example where the function isn't working properly?

Thanks for your continued feedback! You're the only person still replying in this thread that sorely needs more testing/replies (considering the impact that any inadvertently bad changes might have on the many other scripts that rely on the array UDFs)

Edit: Fixed the "copy all" behavior in attachment...

Edit: The "Close" button, IMO, isn't necessary, since you can already exit via the X in the titlebar, or press Esc.

Edited October 18, 2007 by -Ultima-

[randallc]

As for erratic display... can you elaborate? Also, can you provide an example where the function isn't working properly?

Hi,

Thanks for your thoughts; you are right, of course, i can just use my own..

Columns; any array with, say over 300 columns and without curtailing at 250; eg; [any array with 300 cols and use Helpfile for ArrayDisplay.. attached script also shows transpose not working; columns disrupt ar about 218 cols on mine?]

*** Mis-functioning; see my posted zip example [in previous post]; not working.... [only on some computers; only when over 4000 by the way?]

randall

Edited October 18, 2007 by randallc

[GEOSoft]

Just a quick point here.

I really think _ArrayDisplay should be removed from the array UDF and placed into a UDF of it's own. It's only used for testing anyway and has no practical purpose in a script that you may want to release. Perhaps a new Test.au3 UDF?

[Jos]

Just a quick point here.

I really think _ArrayDisplay should be removed from the array UDF and placed into a UDF of it's own. It's only used for testing anyway and has no practical purpose in a script that you may want to release. Perhaps a new Test.au3 UDF?

Nah... _ArrayDisplay(0 is an Array function and as such needs to be in Array.au3.. and of course not to forget the backwards compatibility

I do think that it should be as strait forward as possible without too much fancy stuff given that most people indeed use it for debugging purposes ...

Jos

[GEOSoft]

Nah... _ArrayDisplay(0 is an Array function and as such needs to be in Array.au3.. and of course not to forget the backwards compatibility

I do think that it should be as strait forward as possible without too much fancy stuff given that most people indeed use it for debugging purposes ...

Jos

It was just a thought. Since the GUI was added to it that function is just way too heavy. It's the main reason that I never #included Array.au3. I prefered to create a new one that I could use. I'm one of those people where virtually all of my scripts were compiled for distribution and I tried to keep the exe size to a minimum.

[Jos]

It was just a thought. Since the GUI was added to it that function is just way too heavy. It's the main reason that I never #included Array.au3. I prefered to create a new one that I could use. I'm one of those people where virtually all of my scripts were compiled for distribution and I tried to keep the exe size to a minimum.

What is the effect of removing the _ArrayDisplay() on the the size of your compiled script ?

One other thing not many people know is that you can run Obfuscator to strip all unused functions from the generated EXE by adding the following Directives in you script:

#Region;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Run_Obfuscator=y
#Obfuscator_Parameters=/cs 0 /cn 0 /cf 0 /cv 0 /sf 1 
#EndRegion;**** Directives created by AutoIt3Wrapper_GUI ****
#Include<array.au3>
Msgbox(0,"test","test")

in this case it reduced the EXE from 319 KB to 259 Kb

You need to use the latest Obfuscator available v 1.0.22 to avoid a bug that was introduced in one of th last version.

Edited October 18, 2007 by Jos