_GUICtrlListView_AddArray() Bug ?

[taz742]

Well I've a problem with _GUICtrlListView_AddArray() when array is larger than 1000 items with last stable & beta version

So using 1000 items: it work (sometimes very slow but it work)

So using more than 1000 items:

• v3.3.0.0: work fine
• v3.3.2.0: work at very slow speed
• v3.3.4.0: Failed
• v3.3.5.1: Failed

Test yourself and if somebody know the reason: just let me know

expandcollapse popup

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
#include <ListviewConstants.au3>
#include <GuiListView.au3>
#include <GuiStatusBar.au3>
#include <Array.au3>
#include <Constants.au3>
#include <GUIConstantsEx.au3>
#include <Constants.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <ButtonConstants.au3>
Global $ProcessTime
Global $bSort = 0, $bColumn = 1
$GUI = GUICreate("ListView TEST", 600, 400, -1, -1, BitOR($WS_MINIMIZEBOX, $WS_MAXIMIZEBOX, $WS_CAPTION, $WS_POPUP, $WS_SYSMENU, $WS_SIZEBOX))
$Btn_1000 = GUICtrlCreateButton("Populate" & @CRLF & "ListView 1000", 150, 13, 100, 36, BitOR($BS_MULTILINE, $BS_CENTER))
GUICtrlSetResizing(-1, $GUI_DOCKALL)
GUICtrlSetCursor(-1, 0)
$Btn_1001 = GUICtrlCreateButton("Populate" & @CRLF & "ListView 1001", 350, 13, 100, 36, BitOR($BS_MULTILINE, $BS_CENTER))
GUICtrlSetResizing(-1, $GUI_DOCKALL)
GUICtrlSetCursor(-1, 0)
$Debug_LV = False
$ListView = GUICtrlCreateListView("column1|column2|column3|column4|column5|column6|column7|column8|column9", 1, 60, 598, 314, $LVS_REPORT)
_GUICtrlListView_SetExtendedListViewStyle($ListView, BitOR($LVS_EX_CHECKBOXES, $LVS_EX_FULLROWSELECT, $LVS_EX_GRIDLINES))
GUICtrlSetResizing($ListView, $GUI_DOCKBORDERS)
For $j = 1 To 8
    If $j = 4 Then ContinueLoop
    _GUICtrlListView_JustifyColumn($ListView, $j, 1)
Next
Dim $a_ctrl_status_part[7] = [115, 180, 255, 300, 335, 385, -1]
Dim $a_ctrl_status_text[10] = [@AutoItVersion, 'by "taz742"', "", "#Items", "Total", "", ""]
Global $ctrl_status = _GUICtrlStatusBar_Create($GUI, $a_ctrl_status_part, $a_ctrl_status_text)
_GUICtrlStatusBar_SetMinHeight($ctrl_status, 23)
GUIRegisterMsg($WM_SIZE, "_WM_SIZE")
GUIRegisterMsg($WM_NOTIFY, "_WM_NOTIFY")
GUISetState(@SW_SHOW, $GUI)
While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg;[0]
        Case $GUI_EVENT_CLOSE
            GUIRegisterMsg($WM_SIZE, "")
            GUIRegisterMsg($WM_NOTIFY, "")
            If $ProcessTime Then
                MsgBox("", "ProcessTime Results", "Current ProcessTime Results are copied to ClipBoard" & @CRLF & @CRLF & $ProcessTime)
            EndIf
            Exit
        Case $Btn_1000
            _display(1000)
        Case $Btn_1001
            _display(1001)
    EndSwitch
WEnd
Func _display($rows)
    $chrono = TimerInit()
    GUISetCursor(15, 1, $GUI)
    Local $a_filelist_ext = ""
    Dim $a_filelist_ext[$rows][9]
    For $x = 0 To $rows - 1
        $a_filelist_ext[$x][0] = Random(100000000, 9999999999, 1)
        $a_filelist_ext[$x][1] = Random(1, 365 * 2, 1)
        $a_filelist_ext[$x][2] = StringFormat("%04d/%02d/%02d %02d:%02d:%02d", Random(2009, 2010, 1), Random(1, 12, 1), Random(1, 31, 1), Random(0, 23, 1), Random(1, 59, 1), Random(1, 59, 1))
        $a_filelist_ext[$x][3] = Random(0, 4000, 1)
        $a_filelist_ext[$x][4] = StringFormat("%x", Random(3, 9999999999, 1))
        $a_filelist_ext[$x][5] = Random(1, 210000000, 1)
        $a_filelist_ext[$x][6] = Random(1, 9999999999)
        $a_filelist_ext[$x][7] = StringFormat("%08x", Random(3, 9999999999, 1))
        $a_filelist_ext[$x][8] = Random(3, 9999999999, 0)
    Next
    _ArraySort($a_filelist_ext, $bSort, 0, 0, 2)
    _UpdateColumnArrowSort(2)
ConsoleWrite('@@ Trace(63) :        _GUICtrlListView_BeginUpdate($ListView)'  & @crlf) ;### Trace Console
    _GUICtrlListView_BeginUpdate($ListView)
ConsoleWrite('>Error code: ' & @error & @crlf & @crlf & '@@ Trace(65) :     _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView))'  & @crlf) ;### Trace Console
    _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView))
ConsoleWrite('>Error code: ' & @error & @crlf & @crlf & '@@ Trace(67) :     _GUICtrlListView_AddArray($ListView, $a_filelist_ext)'  & @crlf) ;### Trace Console
    _GUICtrlListView_AddArray($ListView, $a_filelist_ext)
ConsoleWrite('>Error code: ' & @error & @crlf & @crlf & '@@ Trace(69) :     _GUICtrlListView_EndUpdate($ListView)'  & @crlf) ;### Trace Console
    _GUICtrlListView_EndUpdate($ListView)
ConsoleWrite('>Error code: ' & @error & @crlf & @CRLF)
    _UpdateColumnWidth($ListView)
    GUISetCursor(2, 0, $GUI)
    Local $autoitarch
    Switch @AutoItX64
        Case 0
            $autoitarch = "x86"
        Case 1
            $autoitarch = "x64"
    EndSwitch
    $ProcessTime &=  "[" & @AutoItVersion & "-" & $autoitarch & "] _display(" & $rows & ") : " & TimerDiff($chrono) & " ms" & @CRLF
    ClipPut($ProcessTime)
EndFunc   ;==>_display
Func _UpdateColumnWidth(ByRef $ListView)
    GUISetCursor(15, 1, $GUI)
    For $cw = 0 To _GUICtrlListView_GetColumnCount($ListView) - 1
        _GUICtrlListView_SetColumnWidth($ListView, $cw, $LVSCW_AUTOSIZE_USEHEADER)
    Next
    GUISetCursor(2, 0, $GUI)
EndFunc   ;==>_UpdateColumnWidth
Func _WM_NOTIFY($hWnd, $iMsg, $iwParam, $ilParam)
    #forceref $hWnd, $iMsg, $iwParam
    Local $hWndFrom, $iCode, $tNMHDR, $hWndListView
    $hWndListView = $ListView
    If Not IsHWnd($ListView) Then $hWndListView = GUICtrlGetHandle($ListView)
    $tNMHDR = DllStructCreate($tagNMHDR, $ilParam)
    $hWndFrom = HWnd(DllStructGetData($tNMHDR, "hWndFrom"))
    $iCode = DllStructGetData($tNMHDR, "Code")
    Switch $hWndFrom
        Case $hWndListView
            Switch $iCode
                Case $NM_DBLCLK
                    Local $tInfo = DllStructCreate($tagNMITEMACTIVATE, $ilParam)
                    Local $nIndex = DllStructGetData($tInfo, "Index")
                    Switch _GUICtrlListView_GetItemChecked($hWndFrom, $nIndex)
                        Case True
                            _GUICtrlListView_SetItemChecked($hWndFrom, $nIndex, False)
                        Case False
                            _GUICtrlListView_SetItemChecked($hWndFrom, $nIndex, True)
                    EndSwitch
                Case $LVN_ITEMCHANGED
                    Local $ItemsTotal = _GUICtrlListView_GetItemCount($hWndFrom)
                    If Not (_GUICtrlStatusBar_GetText($ctrl_status, 5) == $ItemsTotal) Then _GUICtrlStatusBar_SetText($ctrl_status, $ItemsTotal, 5)
            EndSwitch
    EndSwitch
    Return $__LISTVIEWCONSTANT_GUI_RUNDEFMSG
EndFunc   ;==>_WM_NOTIFY
Func _WM_SIZE()
    _GUICtrlStatusBar_Resize($ctrl_status)
    Return $GUI_RUNDEFMSG
EndFunc   ;==>_WM_SIZE
Func _UpdateColumnArrowSort($nColumn)
    ; bells and whistles (I):
    ; mark the sorted column with a grey rectangle
    GUICtrlSendMsg($ListView, $LVM_SETSELECTEDCOLUMN, GUICtrlGetState($ListView), 0)
    DllCall("user32.dll", "int", "InvalidateRect", "hwnd", GUICtrlGetHandle($ListView), "int", 0, "int", 1)
    ; bells and whistles (II):
    ; create an arrow in the listview header
    Local $iFormat
    Local Const $hHeader = _GUICtrlListView_GetHeader($ListView)
    ; clear existing arrows
    For $x = 0 To _GUICtrlHeader_GetItemCount($hHeader) - 1
        $iFormat = _GUICtrlHeader_GetItemFormat($hHeader, $x)
        If BitAND($iFormat, $HDF_SORTDOWN) Then
            _GUICtrlHeader_SetItemFormat($hHeader, $x, BitXOR($iFormat, $HDF_SORTDOWN))
        ElseIf BitAND($iFormat, $HDF_SORTUP) Then
            _GUICtrlHeader_SetItemFormat($hHeader, $x, BitXOR($iFormat, $HDF_SORTUP))
        EndIf
    Next
    ; set arrow in current column
;~                      Local $nColumn = GUICtrlGetState($ListView)
    $iFormat = _GUICtrlHeader_GetItemFormat($hHeader, $nColumn)
    If $bSort == 0 And $bColumn == $nColumn Then ; ascending
        _GUICtrlHeader_SetItemFormat($hHeader, $nColumn, BitOR($iFormat, $HDF_SORTUP))
    Else ; descending
        _GUICtrlHeader_SetItemFormat($hHeader, $nColumn, BitOR($iFormat, $HDF_SORTDOWN))
    EndIf
    $bColumn = $nColumn
EndFunc   ;==>_UpdateColumnArrowSort

Here my timer results (average of 10 run each) using previous script on different AutoIt version:

(would know why some of them are so slow or failed...)

/----------------------------------------------------+-----------------------------------------------------\
|                    AutoIt 32 bits                  |                    AutoIt 64 bits                   |
|----------------------------------------------------+-----------------------------------------------------|
| [3.3.0.0-x86] _display(1000) : 2436.50217791475 ms | [3.3.0.0-x64] _display(1000) :  1317.69232025496 ms |
| [3.3.0.0-x86] _display(1001) : 2179.04862337590 ms | [3.3.0.0-x64] _display(1001) :  1126.74880605477 ms |
|                                                    |                                                     |
| [3.3.2.0-x86] _display(1000) : 9651.61576311007 ms | [3.3.2.0-x64] _display(1000) : 63151.07472569010 ms |
| [3.3.2.0-x86] _display(1001) : 9662.51303599217 ms | [3.3.2.0-x64] _display(1001) : 63430.03399273480 ms |
|                                                    |                                                     |
| [3.3.4.0-x86] _display(1000) : 2524.11897690500 ms | [3.3.4.0-x64] _display(1000) : 2111.093483520410 ms |
| [3.3.4.0-x86] _display(1001) : Failed at 1000 rows | [3.3.4.0-x64] _display(1001) : Failed at 1000 rows  |
|                                                    |                                                     |
| [3.3.5.1-x86] _display(1000) : 2459.72735820223 ms | [3.3.5.1-x64] _display(1000) : 2065.324377480340 ms |
| [3.3.5.1-x86] _display(1001) : Failed at 1000 rows | [3.3.5.1-x64] _display(1001) : Failed at 1000 rows  |
\----------------------------------------------------+-----------------------------------------------------/

Edited February 6, 2010 by taz742

[taz742]

Maybe I've post in the wrong place:

_GUICtrlListView_AddArray() can be more suitable here "Graphical User Interface (GUI) Help and Support"...

[water]

Here are my results:

[3.3.4.0-x86] _display(1000) : 2513.00910669206 ms

[3.3.4.0-x86] _display(1001) : 2471.06300910441 ms

[3.3.5.0-x86] _display(1000) : 2490.60772553701 ms

[3.3.5.0-x86] _display(1001) : 2487.2158274627 ms

The only problem I'm aware of (and therefore changed the array.au3) was a bug in the internal function __Array_Combinations of Array.au3

I had to change line 1485 from "Return $i_Total" to "Return Round($i_Total)"

Edited February 6, 2010 by water

[PsaltyDS]

Your symptoms are invalid. The demo in the help file runs fine and uses 5000 rows. The 2D load loads 5000x4 in about 2.5sec for me.

Try the demo from the help file on your machine and then compare it to your script.

[taz742]

Your symptoms are invalid. The demo in the help file runs fine and uses 5000 rows. The 2D load loads 5000x4 in about 2.5sec for me.

Try the demo from the help file on your machine and then compare it to your script.

Damn you're right the demo from the help wronk like a charm:

2D loads 5000x4 in 2.14sec using v3.3.5.2

2D loads 5000x4 in 0.48sec using v3.3.4.0

so latest beta is around 4 times slower than stable version.... any reason?

BTW I find what cause my crash: It's the GUIRegisterMsg($WM_NOTIFY, "_WM_NOTIFY") line

So I've made some errors in the _WM_NOTIFY() func: if somebody have idea...

Edited February 6, 2010 by taz742

[Bowmore]

I've managed to get your script to crash intermittently  on either the 1000 or 1001 button after several runs close together or alternating the button used.

I believe that the problem may be that the $LVN_ITEMCHANGED messages in your _WM_NOTIFY function are not being processed fast enough.

When I commented out the following lines I could not get it to crash any more.

;~                 Case $LVN_ITEMCHANGED
;~                     Local $ItemsTotal = _GUICtrlListView_GetItemCount($hWndFrom)
;~                                      ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ItemsTotal = ' & $ItemsTotal & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console
;~                     If Not (_GUICtrlStatusBar_GetText($ctrl_status, 5) == $ItemsTotal) Then _GUICtrlStatusBar_SetText($ctrl_status, $ItemsTotal, 5)

 

 

[PsaltyDS]

You must be running x64, I can't test against 3.3.5.2 until Jon posts the fixed version for x86: RegDeleteKeyEx Issue

[taz742]

@Bowmore:

Thanks for feedback same result here after commented lines related to $LVN_ITEMCHANGED -> No crash at all.

Here my last result with 5000 rows :

[3.3.4.0-x86] _display(5000) : 2803.02598332793 ms

[3.3.4.0-x64] _display(5000) : 2490.92321654795 ms

[3.3.5.1-x86] _display(5000) : 11922.7872642731 ms !!!!

[3.3.5.1-x64] _display(5000) : 2535.45626555862 ms

[3.3.5.3-x86] _display(5000) : 32257.3882287615 ms !!!! from bad to worse

[3.3.5.3-x64] _display(5000) : 5542.96872995024 ms !!!! slower too

@PsaltyDS:

Yes I'm on Win7x64 but I work with AutoIt x86

I've revert back to 3.3.5.1 since 3.3.5.2 is reported buggy on x86

x86 Results are still 4x slower than 3.3.4.0 with the demo from the help or why my fixed script.

Edited February 6, 2010 by taz742