Treeview woes :(

[-Ultima-]

There seems to be a limit with how much data can be stored in treeviews, but I can't pinpoint it. After I pass the threshold, AutoIt simply crashes. Below, I've posted a test script with 4 cases that I've noticed...

Case 1 works as expected.

Case 2 is a variation of case 1, with the only changed things being the treeview items' strings. You'll notice that AutoIt crashes here.

Case 3 crashes AutoIt.

Case 4 is a simple variation of case 3, with the only difference being that I add lots of junk data to a variable before going into the same loop as in case 3. For some odd reason, that one simple change seems to prevent the crash. In case you're wondering, that 699911 is necessary, because AutoIt will crash if even one byte less than that amount is added to the $memorypad string (yes, I had to test a bunch of numbers to get to this number ). Increasing the number of items added to the treeview will cause another crash, and the only way to fix that up is to increase the $memorypad size. *shrug*

Quick edit: It seems the $memorypad size thing may vary, so maybe increasing the upper limit for the loop to 700000 might help if case 4 continues to crash?

You can change which case you're testing in the first line of the script. So yeah, I'm not sure what's up, and hesitate to report it as a bug without knowing for sure first :\

And just for the record, I would use GUICtrlCreateTreeViewItem() instead of _GUICtrlTreeViewInsertItem(), but for some reason, the former seems to work less reliably than the latter in adding items. The program I'm writing deals with LOTS of data being added to a treeview (in case it weren't already obvious ), and for some reason, GUICtrlCreateTreeViewItem() sometimes fails to add items properly when there is a lot of data to be added. I've spent a good many hours trying to debug this and track down the issue as to why GUICtrlCreateTreeViewItem() just doesn't work sometimes, but haven't been successful.

expandcollapse popup

Global $case = 1
; $case = 1 --> no crash
; $case = 2 --> crash
; $case = 3 --> crash
; $case = 4 --> no crash

#NoTrayIcon
#Include <Array.au3>
#Include <GUIConstants.au3>
#Include <GUITreeView.au3>

Opt("GUIOnEventMode", 1)

Global $title = "Test"
Global $guiMinWidth = 450
Global $guiMinHeight = 275

Global $GUI = GUICreate($title, $guiMinWidth, $guiMinHeight, DEFAULT, DEFAULT, $WS_SIZEBOX + $WS_MINIMIZEBOX + $WS_MAXIMIZEBOX, $WS_EX_ACCEPTFILES)
GUISetFont(8, 400, "", "MS Shell Dlg")

GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit")

Global $Tree = GUICtrlCreateTreeView(0, 0, $guiMinWidth, $guiMinHeight - 20, $GUI_SS_DEFAULT_TREEVIEW + $TVS_NONEVENHEIGHT, $WS_EX_CLIENTEDGE)

GUISetState()

;===============================================================================
; ACTUAL CODE [START]
;===============================================================================

Switch $case
    Case 1      ; working =-=-=-=-=-
        For $i = 1 to 750
            Local $item = _GUICtrlTreeViewInsertItem($Tree, "Item " & $i)
            For $j = 1 To 5
                Local $subitem = _GUICtrlTreeViewInsertItem($Tree, "Subitem " & $j, $item)
                For $k = 1 To 5
                    _GUICtrlTreeViewInsertItem($Tree, "Sub-subitem " & $k, $subitem)
                Next
            Next
        Next

    Case 2      ; NOT working =-=-=-=-=-
        For $i = 1 to 750
            Local $item = _GUICtrlTreeViewInsertItem($Tree, "Itemmmmmm " & $i)
            For $j = 1 To 5
                Local $subitem = _GUICtrlTreeViewInsertItem($Tree, "Subitemmmmmm " & $j, $item)
                For $k = 1 To 5
                    _GUICtrlTreeViewInsertItem($Tree, "Sub-subitemmmmmm " & $k, $subitem)
                Next
            Next
        Next

    Case 3      ; NOT working =-=-=-=-=-
        For $i = 1 to 1000
            Local $item = _GUICtrlTreeViewInsertItem($Tree, "Item " & $i)
            For $j = 1 To 5
                Local $subitem = _GUICtrlTreeViewInsertItem($Tree, "Subitem " & $j, $item)
                For $k = 1 To 5
                    _GUICtrlTreeViewInsertItem($Tree, "Sub-subitem " & $k, $subitem)
                Next
            Next
        Next

    Case 4      ; working =-=-=-=-=-
        Global $memorypad = ""
        For $i = 1 to 699911
            $memorypad &= " "
        Next
    
        For $i = 1 to 1000
            Local $item = _GUICtrlTreeViewInsertItem($Tree, "Item " & $i)
            For $j = 1 To 5
                Local $subitem = _GUICtrlTreeViewInsertItem($Tree, "Subitem " & $j, $item)
                For $k = 1 To 5
                    _GUICtrlTreeViewInsertItem($Tree, "Sub-subitem " & $k, $subitem)
                Next
            Next
        Next
EndSwitch

;===============================================================================
; ACTUAL CODE [END]
;===============================================================================

While 1
    Sleep(1000)
WEnd

Func _Exit()
    Exit
EndFunc

Edit: Changed all instances of "listview" to "treeview"... Stupid typo

Edited June 13, 2007 by -Ultima-

[PaulIA]

When I run your script, it crashes every time. I put a ConsoleWrite in to see what item it crashes on:

Case 1: 696

Case 2: 637

Case 3: 696

Case 4: 696

When I run your test using the Auto3Lib TreeView functions, all cases work:

expandcollapse popup

#include <A3LTreeView.au3>

Opt("GUIOnEventMode", 1)

$hGUI  = GUICreate("Test", 500, 400)
$hTree = _TreeView_Create($hGUI, 2, 2, 496, 396)
GUISetState()
GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit")

$iCase = 4

Switch $iCase
  case 1
    _TreeView_BeginUpdate($hTree)
    for $iI = 1 to 750
      $hItem = _TreeView_Add($hTree, 0, "Item " & $iI)
       for $iJ = 1 To 5
         $hSubItem = _TreeView_AddChild($hTree, $hItem, "SubItem " & $iJ)
         for $iK = 1 To 5
           _TreeView_AddChild($hTree, $hSubItem, "Sub-SubItem " & $iK)
         next
      next
    next
    _TreeView_EndUpdate($hTree)
    ConsoleWrite("Done" & @CR)
  case 2
    _TreeView_BeginUpdate($hTree)
    for $iI = 1 to 750
      $hItem = _TreeView_Add($hTree, 0, "Itemmmmmm " & $iI)
       for $iJ = 1 To 5
         $hSubItem = _TreeView_AddChild($hTree, $hItem, "SubItemmmmmm " & $iJ)
         for $iK = 1 To 5
           _TreeView_AddChild($hTree, $hSubItem, "Sub-SubItemmmmmm " & $iK)
         next
      next
    next
    _TreeView_EndUpdate($hTree)
    ConsoleWrite("Done" & @CR)
  case 3
    _TreeView_BeginUpdate($hTree)
    for $iI = 1 to 1000
      $hItem = _TreeView_Add($hTree, 0, "Item " & $iI)
       for $iJ = 1 To 5
         $hSubItem = _TreeView_AddChild($hTree, $hItem, "SubItem " & $iJ)
         for $iK = 1 To 5
           _TreeView_AddChild($hTree, $hSubItem, "Sub-SubItem " & $iK)
         next
      next
    next
    _TreeView_EndUpdate($hTree)
    ConsoleWrite("Done" & @CR)
  case 4
    Global $memorypad = ""
    for $i = 1 to 699911
      $memorypad &= " "
    next

    _TreeView_BeginUpdate($hTree)
    for $iI = 1 to 1000
      $hItem = _TreeView_Add($hTree, 0, "Item " & $iI)
       for $iJ = 1 To 5
         $hSubItem = _TreeView_AddChild($hTree, $hItem, "SubItem " & $iJ)
         for $iK = 1 To 5
           _TreeView_AddChild($hTree, $hSubItem, "Sub-SubItem " & $iK)
         next
      next
    next
    _TreeView_EndUpdate($hTree)
    ConsoleWrite("Done" & @CR)
EndSwitch

While 1
    Sleep(1000)
WEnd

Func _Exit()
    Exit
EndFunc

[-Ultima-]

Thanks for the quick response, Paul

Indeed, Auto3Lib works beyond the limits that the standard UDFs seem to have, but at the same time, it's not a simple drop-in solution, and it requires a bunch of changes in my actual code to get it working (I'm a neat/consistency freak when it comes to my code, so using Auto3Lib in one place while using the standard UDFs in others irks me somewhat ). Creating the treeview using _TreeView_Create(), for example, is different from creating it using GUICtrlCreateTreeView() (obviously ), and I can't just use _TreeView_AddChild() on any regular treeview created by the standard AutoIt way (otherwise, it complains about an invalid window handle), which is a tad bit annoying.

Another thing is, when I create the treeview using _TreeView_Create(), it seems to want to be on top of EVERYTHING, which is bad, since I use tab controls in my program, and it's blocking every other tab's contents, no matter which tab I switch to.

Much more importantly, though, is the fact that Auto3Lib's implementation appears to be slower than the UDF's -- much slower (the only reason I decided to test was that it was noticable enough a difference). When I was measuring the time on my actual application (not just the script up above), _GUICtrlTreeViewInsertItem() took about 3-4 seconds to complete the treeview filling process, while Auto3Lib's _TreeView_AddChild() took approximately 22 seconds. That's about 5-7x slower In case 1 from the above script, my version of the script took about 9 seconds to fill the treeview, while the Auto3Lib version took 22 seconds (a little bit more than twice the amount of time). I got a similar result with case 4 (Auto3Lib's implementation being a bit more than 2x slower -- 14 seconds vs 32 seconds). My actual program makes use of recursion while filling the treeview, and isn't as simple as just a few nested for loops, which might explain the speed disparity between the test case and the realworld case.

I really do appreciate your help, and would like to say "keep up the good work on Auto3Lib" Without a doubt, it's an awesome and very comprehensive library, but unfortunately, it can't seem to fulfill my needs at the moment

In the end, I guess this confirms that there *is* a bug with AutoIt's native/UDF implementation, *somewhere*, and that it's not some inherent treeview limitation... right? Does this mean it's safe for me to post it as a bug report in the appropriate forum now?

Thanks again!

Edit: Oh, and I was just retesting GUICtrlCreateTreeViewItem(), and it seems that it doesn't want to add additional treeview items to a treeview after a certain amount, even after you clear the treeview, and that's the thing I was running into when I said it sometimes doesn't work. The exact limit is 4096; GUICtrlCreateTreeViewItem() won't add any more treeview items to a treeview after you add the 4095th one, regardless of whether it was cleared using _GUICtrlTreeViewDeleteAllItems().

_GUICtrlTreeViewInsertItem() and _TreeView_AddChild() obviously don't see any problems with adding more than 4095 treeview items, and _TreeView_AddChild() clearly shows that there *is* no side-effect from doing so (memory usage was normal, as was application responsiveness after the treeview was filled). Interestingly, GUICtrlCreateTreeViewItem() seemed to be *much* faster than even _GUICtrlTreeViewInsertItem() in filling up the first 4095 treeview items. After several tests, I found that it's 7x faster than _GUICtrlTreeViewInsertItem() -- consistently too It would be awesome if that 4096 limitation were removed from GUICtrlCreateTreeViewItem()...

As a side note, I do believe GUICtrlCreateListViewItem() suffers from the same problem; even if it gets cleared by _GUICtrlListViewDeleteAllItems(), it'll still refuse to add anything to a listview after 4095 were added to it previously... Argh. Luckily, I didn't run into the same crashing bug using _GUICtrlListViewInsertItem() as I ran into using _GUICtrlTreeViewInsertItem(), but I guess that's because I wasn't adding as many listview items as I was treeview items.

Edit: Eh, just remembered that there was a 4096 control limitation built into AutoIt, right? That seems to be the cause of the GUICtrlCreateTreeViewItem() and GUICtrlCreateListViewItem() limitations. Grr... That limitation really needs to go

Edited June 16, 2007 by -Ultima-

[PaulIA]

Thanks for the quick response, Paul

Indeed, Auto3Lib works beyond the limits that the standard UDFs seem to have, but at the same time, it's not a simple drop-in solution, and it requires a bunch of changes in my actual code to get it working (I'm a neat/consistency freak when it comes to my code, so using Auto3Lib in one place while using the standard UDFs in others irks me somewhat ). Creating the treeview using _TreeView_Create(), for example, is different from creating it using GUICtrlCreateTreeView() (obviously ), and I can't just use _TreeView_AddChild() on any regular treeview created by the standard AutoIt way (otherwise, it complains about an invalid window handle), which is a tad bit annoying.

Another thing is, when I create the treeview using _TreeView_Create(), it seems to want to be on top of EVERYTHING, which is bad, since I use tab controls in my program, and it's blocking every other tab's contents, no matter which tab I switch to.

Much more importantly, though, is the fact that Auto3Lib's implementation appears to be slower than the UDF's -- much slower (the only reason I decided to test was that it was noticable enough a difference). When I was measuring the time on my actual application (not just the script up above), _GUICtrlTreeViewInsertItem() took about 3-4 seconds to complete the treeview filling process, while Auto3Lib's _TreeView_AddChild() took approximately 22 seconds. That's about 5-7x slower In case 1 from the above script, my version of the script took about 9 seconds to fill the treeview, while the Auto3Lib version took 22 seconds (a little bit more than twice the amount of time). I got a similar result with case 4 (Auto3Lib's implementation being a bit more than 2x slower -- 14 seconds vs 32 seconds). My actual program makes use of recursion while filling the treeview, and isn't as simple as just a few nested for loops, which might explain the speed disparity between the test case and the realworld case.

I really do appreciate your help, and would like to say "keep up the good work on Auto3Lib" Without a doubt, it's an awesome and very comprehensive library, but unfortunately, it can't seem to fulfill my needs at the moment

In the end, I guess this confirms that there *is* a bug with AutoIt's native/UDF implementation, *somewhere*, and that it's not some inherent treeview limitation... right? Does this mean it's safe for me to post it as a bug report in the appropriate forum now?

Thanks again!

Edit: Oh, and I was just retesting GUICtrlCreateTreeViewItem(), and it seems that it doesn't want to add additional treeview items to a treeview after a certain amount, even after you clear the treeview, and that's the thing I was running into when I said it sometimes doesn't work.

There is always a trade off in everything you do. In my tests, the AutoIt TreeView functions are about twice as fast. But they don't work on external controls and Auto3Lib does. That's why Auto3Lib is a bit slower. Oddly enough, I've been working on some code that will make Auto3Lib as fast (if not faster) than the AutoIt functions. But, to each his own. Anyway, the Auto3Lib TreeView doesn't crash AutoIt just shy of 1000 nodes, so I think I'll keep using Auto3Lib.

Edit: Actually, after looking some more at the code, there is a bug in the TreeView module that is causing the slow down. When fixed, the case 1 test runs in 6 seconds on my machine, which is 3 seconds faster than your time listed above. I'll put out the new code with the next release (probably this weekend).

Edited June 12, 2007 by PaulIA

[randallc]

Hi,

GUICtrlCreateTreeViewItem() seemed to be *much* faster than even _GUICtrlTreeViewInsertItem() in filling up the first 4095

yes, you will see that is why Iarranged in this viewto put in the first 4000, arbitrarily,with create,then change over to insert each time if there are alreadythat many objects arranged.[that is in my list view functions]

awesome if that 4096 limitation were removed

interesting, I understood that was aWindows problem ;not built into AutoIt as you say?

_GUICtrlListViewDeleteAllItems(), it'll still refuse to add anything to a listview after 4095 were added to it previously

I understand that is expected behaviour;I think I got around that at times in listview,by drawing a new GUI with a new name, then deleting the previous one (releases all the objects) adding a new list view again. I think I managed to do that without too much flicker at times! at times I think it means its best to have some separate GUIs with the list views,so I don't need to delete everything at once.

Best, Randall

[-Ultima-]

http://www.autoitscript.com/forum/index.ph...809&hl=4096

Definitely an AutoIt limitation. As I said, _GUICtrlTreeViewInsertItem() can blast right through the limit just fine, but that's because it doesn't use GUICtrlCreateTreeViewItem() to do its dirty work -- it creates the treeview items directly via API calls.

I'd considered deleting/recreating the treeview, but the fact of the matter is, I use *other* controls besides just the treeview, including two other listviews, one of which can conceivably contain thousands of listview items. The 4096 limitation is a global AutoIt limitation, meaning it's shared among every single control created by AutoIt. Since the treeview needs to be able to contain thousands at any given time as well, deleting isn't a viable option anyway, since I'll run straight up to the limit again when I refill the treeview using GUICtrlCreateTreeViewItem(). Besides that, even without the limit (using _GUICtrlTreeViewInsertItem()), I run into a crash problem not existent in Auto3Lib's implementation.

Thanks for the input

@PaulIA: That's great news I might test it if I have the time, but I still think the built-in implementation needs to be fixed as well

Edited June 13, 2007 by -Ultima-

[randallc]

I run into a crash problem not existent in Auto3Lib's implementation.

Yes, that is interesting; it may be equivalent to the column headers in Listview being limited to 196-250 somewhere, and I can't get an answer whether that is an AutoIt limitation.

I don't think the 4096 barrrier is an AutoIt limitation for control number, though a lot of people have thought so; that link you refer to is just someone else's opinion.

The issue, I believe, is whether you want to use AutoIt's limited [but faster] "Create" functions; as each line uses an object handle, whereas "Insert" does not, so does not chew through the handles.

Best, randall

[PS -maybe I'm wrong; see Listview maximum 4096 lines?]

Edited June 13, 2007 by randallc

[-Ultima-]

Yeah, it's not an opinion, and it's a known fact that there's a deliberate limitation set on the number of controls able to be created in AutoIt. You'll also notice that everyone in that thread was saying "the 4096 limitation," not just saying any ol' limitation. There is another link in that thread I linked to that clearly shows Jon saying how he decides the limit. This limitation can't be a Windows limit at all -- I've been saying all along that the regular Windows API calls for creating listview/treeview items already can go past that limit with ease. Each and every listview item and treeview item created via GUICtrlCreate{List|Tree}ViewItem() counts as one additional item toward that limit...

Again, this is a known AutoIt limitation. It's an ugly limitation at that. I disagree with jpm when he says AutoIt should be used to create small GUIs. AutoIt is pretty much evolving (or already has evolved?) into a very full and mature language, and the control limitation is entirely superfluous, as having more than 4096 controls doesn't affect much adversely. At the very least, listview/treeview/list items shouldn't count toward the limit :|

Anyhow, this is going off-topic already, so...

Edited June 13, 2007 by -Ultima-