LarsJ

_ArrayDisplayEx based on virtual ListView

3 posts in this topic

#1 ·  Posted (edited)

I'm working on some code related to virtual ListViews. In this regard, I need to be able to show the contents of large arrays with the same ease as _ArrayDisplay can show the contents of smaller arrays. The arrays have to be shown as fast as possible. _ArrayDisplayEx is the answer.

The most fundamental difference between _ArrayDisplayEx based on a virtual ListView and the official version of _ArrayDisplay is, that a virtual ListView is depending on WM_NOTIFY messages and needs a function to handle these messages. Advantages and disadvantages of _ArrayDisplayEx:

Advantages

  • No limitation on the number of rows as the ListView can display. The array itself is the limitation.
  • The ListView shows up immediately. Because a virtual ListView is fed with data directly from the array, no time is spent on inserting rows in the ListView.

Disadvantages

  • _ArrayDisplayEx is depending on WM_NOTIFY message handlers registered with GUIRegisterMsg.
  • Because you can't use two different WM_NOTIFY message handlers registered with GUIRegisterMsg at the same time, this makes it difficult to use a WM_NOTIFY message handler in your own code.
  • This also makes it impossible to display two _ArrayDisplayEx windows side by side at the same time.
  • Significantly more work is carried out by _ArrayDisplayEx than _ArrayDisplay. _ArrayDisplayEx must constantly feed the ListView with data for example when you scroll up or down. This is done in the WM_NOTIFY message handlers.
  • Column widths are calculated based on the rows at the first page in the list view. This will not necessarily lead to correct column widths for all rows.

Don't use _ArrayDisplayEx if the array contains less than 10,000 rows.

 

The GUI
Most of the GUI code is copied directly from the official version of _ArrayDisplay. Credit goes to

; #FUNCTION# ===================================================================
; Author ........: randallc, Ultima
; Modified.......: Gary Frost (gafrost), Ultima, Zedna, jpm, Melba23, AZJIO, UEZ
; ==============================================================================

and especially Melba23, who has done most of the coding in the recent versions of the function.

 

Changes compared to _ArrayDisplay (3.3.14)
Changes in functionality and changes for parameters in function definition.

A new "Goto row" control is added to the left of "Copy" buttons. If there are many rows in the array, it can be difficult to navigate to a specific row with the scroll bar. It's much easier to enter the row number.

The dimensions label below the ListView that shows array dimensions is always shown. If a user function is specified the "Run User Func" button is always shown whether "Goto row" control or "Copy" buttons are shown or not. "Run User Func" button appears to the right of the dimensions label.

$iFlags parameter (new/modified in bold):

  •       0 = Left aligned text (default)
  •       1 = Transposes the array
  •       2 = Right aligned text
  •       4 = Centered text
  •       8 = Verbose: MsgBox on error, splash screen on lengthy commands
  •     16 = No "Copy" buttons displayed (page 2 in Examples.au3) 
  •     32 = No "Exit Script" button displayed (page 2 in Examples.au3)
  •     64 = No "Goto row" control displayed (page 2 in Examples.au3)
  •   128 = No "Row" column displayed in ListView (header / page 5)
  •   256 = Pass array and selection to user function (page 2)
  •   512 = Show types of special data (page 3)
  • 1024 = Half-height ListView (page 5)

Flag values to hide controls and "Row" column. The meaning of the values 16, 32 and 64 is changed compared to the official version. 128 is a new value.

When I specify a user function, it's rare that I need the array and selection. The default in _ArrayDisplayEx is not to pass array and selection to user function. If the array contains 100,000 rows the calculation of selected rows takes time. Set $iFlags = 256 to pass array and selection to user function.

When array and selection is passed to user function ($iFlags = 256), "Run User Func" button works in the same way as "Copy" buttons according to the selection. If no rows are selected, all rows are passed to the user function. Else selected rows are passed to the function.

If only rows 5 - 17 are visible in the ListView ($sArrayRange = "5:17"), you might only want to execute the user function for rows 5 - 17. For this purpose _ArrayDisplayEx settings can be passed to user function. Call _ArrayDisplayEx_LocalStorage (see below) to get the settings.

Set $iFlags = 512 to show types of special data. Special data includes arrays, functions, objects and structures (DllStructCreate). When the flag is set, the corresponding cells in the ListView are shown with the following text strings: {Array}, {Function}, {Object} and {Struct}. Without this flag data is shown as empty cells. The flag is not set as default. For large arrays the control of special data types is time consuming.

Because a lot of work is carried out in WM_NOTIFY functions to feed the ListView with data, you can set $iFlags = 1024 to use a half-height ListView. Then the ListView only have to be fed with half as many data. When you drag the scrollbar with the mouse, it's easier for the scrollbar to follow the mouse.

A new parameter $hWM_NOTIFY_Function = 0 is added as the last parameter. Because _ArrayDisplayEx is based on a virtual ListView it's depending on WM_NOTIFY messages and functions. If you are using a WM_NOTIFY function in your own script, you can use the $hWM_NOTIFY_Function parameter to restore your function when the code returns from _ArrayDisplayEx. See page 4 in Examples.au3.

 

Optimized for speed
In this version the code is optimized for speed and nothing else. This means that as many control statements as possible are calculated before the repetitive and fast call of WM_NOTIFY functions in response to LVN_GETDISPINFO notifications to feed the ListView with data. The disadvantage is, that there is a fairly large number of WM_NOTIFY functions. The reason for the size of the script.

 

Avoiding global variables
To avoid globals a function _ArrayDisplayEx_LocalStorage and static locals are used to transfer information from _ArrayDisplayEx to WM_NOTIFY functions. _ArrayDisplayEx_LocalStorage can be called e.g. in a user function to get the same information. See page 2 in Examples.au3.

Because of this usage of _ArrayDisplayEx_LocalStorage the keywords Const and ByRef can't be used in definition of _ArrayDisplayEx. Since _ArrayDisplayEx does not change the array this is not a problem. On the other hand it means that you can call the function in this way:

_ArrayDisplayEx( StringSplit( "string", "delimiters" ) )

 

Examples
Examples.au3 contains a bunch of small examples:

  • Page 1 demonstrates the usage of the $sArrayRange parameter
  • Page 2 shows the changes in relation to controls and user function
  • Page 3 demonstrates how to show types of special data
  • Page 4 shows how to restore a WM_NOTIFY function
  • Page 5 is a summary of parameters and flags

For the examples at page 1 (only page 1) you can click the "Run User Func" button to compare with the official version. To avoid influence of the WM_NOTIFY function (which would decrease performance significantly) _ArrayDisplay is executed in it's own process. The array is transferred to the new process through an object registered with AutoItObject (you can find a step-by-step demonstration of the technique here, and you can find a description in the AutoItObject thread).

Run this code to test _ArrayDisplayEx with a large array:

#include "ArrayDisplayEx.au3"

; One million rows, ten columns
Global $iRows = 1000000, $iCols = 10
Global $aArray[$iRows][$iCols]

; This takes some seconds
; It'll use some memory
For $i = 0 To $iRows - 1
  For $j = 0 To $iCols - 1
    $aArray[$i][$j] = $i & "/" & $j
  Next
Next
ConsoleWrite( "Array done ..." & @CRLF )

_ArrayDisplayEx( $aArray, "", "", 16, "", "", Default, 0xCCFFFF )
; 16       => remove "Copy" buttons
; 0xCCFFFF => alternating row color

Notice the "Array done ..." message in the console. The delay from the message shows up till you see the rows in the ListView, is the time it takes to create the GUI.

 

Zip
The zip contains four files:

  • ArrayDisplayEx.au3 - contains the _ArrayDisplayEx UDF
  • Examples.au3 - a bunch of small examples spread over five pages
  • Examples\AutoItObject.au3 by the AutoItObject team - used in examples at page 1
  • Examples\Original.au3 - used in examples at page 1

You need AutoIt 3.3.10 or later. Tested on Windows 7 64 bit and Windows XP 32 bit.

Comments are welcome. Let me know if there are any issues.

ArrayDisplayEx.7z

Updated zip at the bottom of next post.

Edited by LarsJ
Update in next post

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Since this example is so popular, I've decided to make an update. And the UDF is needed in Accessing AutoIt Variables.

There are two significant improvements in this update: Message handlers based on GUIRegisterMsg are replaced with message handlers based on subclassing. Instances of the array stored as local static variables are properly deleted.

 

GUIRegisterMsg
GUIRegisterMsg is an easy way to create a handler function for a Windows message eg. the WM_NOTIFY message. However, there are also some disadvantages associated with the use of GUIRegisterMsg.

With GUIRegisterMsg you can create only one handler function for a specific Windows message. If a WM_NOTIFY message handler is used in a UDF, it's difficult to use a WM_NOTIFY message handler in your own code.

A message handler registered with GUIRegisterMsg is a global message handler. In a GUI with two listviews where a WM_NOTIFY message handler is implemented for the first listview, but the message handler is not needed for the second listview, the message handler function will still be called for every WM_NOTIFY message from the second listview. And this cannot be prevented. If there are many rows in the second listview, this can be a serious performance issue.

In the examples on page 1 (Examples.au3 in zip in bottom of first post) you can click the "Run User Func" button to compare with the official version of _ArrayDisplay. Several workarounds are needed to prevent the global WM_NOTIFY message handlers in _ArrayDisplayEx to be called for every single row in _ArrayDisplay (which would reduce the performance of _ArrayDisplay significantly). First, _ArrayDisplay is executed in its own process. Secondly, the array is passed to _ArrayDisplay by means of functionality of AutoItObject.au3 (necessary because _ArrayDisplay runs in its own process).

In order to eliminate the drawbacks of WM_NOTIFY message handlers registered with GUIRegisterMsg the message handlers are implemented through subclassing.

 

Subclassing
Subclassing is implemented with the four functions SetWindowSubclass, GetWindowSubclass, RemoveWindowSubclass and DefSubclassProc (coded in WinAPIShellEx.au3).

Note that subclassing implemented with these four functions is not comparable with subclassing implemented through _WinAPI_SetWindowLong. Subclassing with the four functions is easy and error-free. Subclassing with _WinAPI_SetWindowLong is simply not working properly. If you're very careful it's possible to implement subclassing through _WinAPI_SetWindowLong in simple situations.

Message handlers based on subclassing can be created for a single control or window at a time. If you need to subclass several controls or windows you can use different message handlers and avoid messages being mixed together (separation of the messages takes place in ComCtl32.dll where the four functions are coded).

You can find more information about subclassing in Colors and fonts in custom drawn ListViews.

Replacing message handlers based on GUIRegisterMsg with message handlers based on subclassing is very simple as you can see by comparing the code in the new and the old zip.

Subclassing eliminates the first three disadvantages mentioned in the list in top of first post.

 

Static variables
To avoid using a global variable the array is passed to the WM_NOTIFY message handler as a local static variable. Because it's a static variable it must be deleted when it's not needed any more. That is when _ArrayDisplayEx is closed. I forgot to delete the local static array in the first version, but I've done it in this version.
 

Examples
Examples0.au3 in the new zip is the same as Examples.au3 in the old zip, but page 4 in the old examples about the WM_NOTIFY message handler is omitted. And all the workarounds in the old zip needed to compare the examples on page 1 with the official version of _ArrayDisplay are omitted too. Because message handling in the new UDF is based on subclassing the workarounds are not needed. AutoItObject.au3 is not included in the new zip.

Examples1.au3 shows that you can feed _ArrayDisplayEx directly with an array returned by a function.

Examples2.au3 shows that it's possible to display two _ArrayDisplayEx windows side by side at the same time if you click the "Run User Func" button.

Examples3.au3 - Examples5.au3 are examples with large arrays with 16,000,000 elements and 1, 4, and 16 columns, respectively. Examples3.au3 also demonstrates that even if the local static versions of the array are deleted when _ArrayDisplayEx is closed, the original array is not deleted. This is demonstrated by showing the array once more.

 

New zip
You need AutoIt 3.3.10 or later. Tested on Windows 10/7 32/64 bit and Windows XP 32 bit.

Comments are welcome. Let me know if there are any issues.

ArrayDisplayEx.7z

Edited by LarsJ
Zip updated
1 person likes this

Share this post


Link to post
Share on other sites

In the update monday WM_NOTIFY message handlers are implemented through subclassing.

If you click "Run User Func" button in Examples2.au3 two _ArrayDisplayEx windows are displayed at the same time and two WM_NOTIFY message handlers are running at the same time through two subclasses.

If you click "Exit Script" button in the window opened with "Run User Func" button, it's important that both subclasses are properly removed before the entire script exits. I forgot to do that in the monday update, and that's the reason for this new update.

To remove the subclasses two variables are needed for each subclass: $hGUI and $pNotifyFunc. Then a subclass can be removed in this way:

_ArrayDisplayEx_RemoveWindowSubclass( $hGUI, $pNotifyFunc, 1000 )

I can store all occurrences of $hGUI and $pNotifyFunc as local static variables in _ArrayDisplayEx_LocalStorage or I can store them in a global array.

Since accessing variables through a function takes much longer time than accessing variables through an array, I've decided to remove _ArrayDisplayEx_LocalStorage completely and use an array to store all variables that are used globally. In the new update the global array $aArrayDisplayEx_Info is introduced.

Because this example is all about displaying array data as fast as possible in a virtual listview, it seems to be a good idea to use an array instead of a function to store the global variables. Even if it's at the cost of a single global array.

In Examples6.au3 you can click "Run User Func" to open the official version of _ArrayDisplay. If you click "Exit Script" in _ArrayDisplay GUI, the subclass in _ArrayDisplayEx is properly removed and Examples6.au3 exits nicely.

Zip in post above updated.

1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now