Lupo73 Posted November 15, 2008 Share Posted November 15, 2008 I'm working of this application: http://sourceforge.net/projects/arcthemall/It's a multi-archiver in UPX, ZIP and 7z formats.It works loading in a listview all files and folders dropped by the user and then process them at the end of the loading. I saw that if I drop a folder with many files (about 1000 for example), It loads first few files faster and then progressively slower (also 1 second per file). So it could need also some minutes to load all that files. Can I improve the code to make this process of loading faster? thanks! SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
martin Posted November 15, 2008 Share Posted November 15, 2008 I'm working of this application: http://sourceforge.net/projects/arcthemall/It's a multi-archiver in UPX, ZIP and 7z formats.It works loading in a listview all files and folders dropped by the user and then process them at the end of the loading. I saw that if I drop a folder with many files (about 1000 for example), It loads first few files faster and then progressively slower (also 1 second per file). So it could need also some minutes to load all that files. Can I improve the code to make this process of loading faster? thanks!The slow down is caused by windows trying to redraw the list view every time an item is added. You can stop windows doing that with _GUICtrlListView_BeginUpdatethen use_GUICtrlListView_EndUpdatewhen you have addd all the items. Serial port communications UDF Includes functions for binary transmission and reception.printing UDF Useful for graphs, forms, labels, reports etc.Add User Call Tips to SciTE for functions in UDFs not included with AutoIt and for your own scripts.Functions with parameters in OnEvent mode and for Hot Keys One function replaces GuiSetOnEvent, GuiCtrlSetOnEvent and HotKeySet.UDF IsConnected2 for notification of status of connected state of many urls or IPs, without slowing the script. Link to comment Share on other sites More sharing options...
Lupo73 Posted November 16, 2008 Author Share Posted November 16, 2008 The slow down is caused by windows trying to redraw the list view every time an item is added. You can stop windows doing that with _GUICtrlListView_BeginUpdatethen use_GUICtrlListView_EndUpdatewhen you have addd all the items.It makes the process a few faster, but loading many items during the process listview is not correctly visible.. SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
torels Posted November 16, 2008 Share Posted November 16, 2008 the only thing I can think of is adding a "Loading" label and maybe a graphical element (an avi or GDI+ animation) to make the thing less boring Some Projects:[list][*]ZIP UDF using no external files[*]iPod Music Transfer [*]iTunes UDF - fully integrate iTunes with au3[*]iTunes info (taskbar player hover)[*]Instant Run - run scripts without saving them before :)[*]Get Tube - YouTube Downloader[*]Lyric Finder 2 - Find Lyrics to any of your song[*]DeskBox - A Desktop Extension Tool[/list]indifference will ruin the world, but in the end... WHO CARES :P---------------http://torels.altervista.org Link to comment Share on other sites More sharing options...
ProgAndy Posted November 16, 2008 Share Posted November 16, 2008 (edited) you should also use _GUICtrlListView_SetItemCount to allocate the memory first Edited November 16, 2008 by ProgAndy *GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes Link to comment Share on other sites More sharing options...
Lupo73 Posted January 8, 2009 Author Share Posted January 8, 2009 I have tried with both _GUICtrlListView_BeginUpdate and _GUICtrlListView_SetItemCount methods, but the process gain only a few seconds (but the process for 350 files is more than 2 minutes long). It's strange because other software are able to load long lists really faster, so could be a way to do it. Can someone help me? Thanks! SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
mikiutama Posted January 8, 2009 Share Posted January 8, 2009 (edited) ........... post deleted.......... Edited January 10, 2009 by mikiutama Link to comment Share on other sites More sharing options...
torels Posted January 8, 2009 Share Posted January 8, 2009 Excuse me could you please post the code that's giving you problems ? I have some problems downloading the zip file from sourceforge anyway... I wrote a piece of code that scans ~1000 files, gets info on the artist, album and track title and sorts them in a treeview in about 2 or 3 minutes I know it's still slow... but I think that what you need is faster. I also believe that updating a listview is faster than updating a treeview Some Projects:[list][*]ZIP UDF using no external files[*]iPod Music Transfer [*]iTunes UDF - fully integrate iTunes with au3[*]iTunes info (taskbar player hover)[*]Instant Run - run scripts without saving them before :)[*]Get Tube - YouTube Downloader[*]Lyric Finder 2 - Find Lyrics to any of your song[*]DeskBox - A Desktop Extension Tool[/list]indifference will ruin the world, but in the end... WHO CARES :P---------------http://torels.altervista.org Link to comment Share on other sites More sharing options...
HAL9000 Posted January 8, 2009 Share Posted January 8, 2009 I'm working of this application: http://sourceforge.net/projects/arcthemall/It's a multi-archiver in UPX, ZIP and 7z formats.It works loading in a listview all files and folders dropped by the user and then process them at the end of the loading. I saw that if I drop a folder with many files (about 1000 for example), It loads first few files faster and then progressively slower (also 1 second per file). So it could need also some minutes to load all that files. Can I improve the code to make this process of loading faster? thanks!you host your app on sourgeforce.netbut where is the code? Link to comment Share on other sites More sharing options...
Lupo73 Posted January 18, 2009 Author Share Posted January 18, 2009 Sorry for the late, I was away in this period. @ HAL9000: source code is available on the page I linked (Download -> Browse all packages), in a separated zip file for each release. @ torels: I use listview... the code is about 1500 lines, this is the function that load files and folders in the listview: expandcollapse popupFunc Add($temp, $mainf) Local $search, $file, $size, $subtemp, $main, $dir, $dupli $i = 0 _GUICtrlListView_BeginUpdate($hListViewH) If $mainf = 2 Then ; load only one file $main = StringTrimLeft($temp, StringLen(Cut($temp))) $size = FileGetSize($temp) $dir = StringTrimRight(Cut($temp), 1) & "\" $dupli = DuplicateCheck($dir, $main) $item = GUICtrlCreateListViewItem($main & "|" & $dir & "|" & ConvSize($size) & "| - ", $hListView) If $dupli = 1 Then GUICtrlSetBkColor($item, 0xFFBFBF) GUICtrlCreateListViewItem("0|1", $hListViewH) Else GUICtrlCreateListViewItem("0|0", $hListViewH) EndIf Else ; load main folder If $mainf = 0 Then $mainf = 1 $main = StringTrimLeft($temp, StringLen(Cut($temp))) $size = DirGetSize($temp & "\" & $file) $dir = Cut($temp) $dupli = DuplicateCheck($dir, $main) $item = GUICtrlCreateListViewItem($main & "|" & $dir & "|" & ConvSize($size) & "| - ", $hListView) If $dupli = 1 Then GUICtrlSetBkColor($item, 0xFFBFBF) GUICtrlCreateListViewItem("1|1", $hListViewH) Else GUICtrlSetBkColor($item, 0xE5F7FF) ;GUICtrlSetBkColor($item, 0xC6F0FF) GUICtrlCreateListViewItem("1|0", $hListViewH) EndIf EndIf ; load files $search = FileFindFirstFile($temp & "\*.*") While 1 $file = FileFindNextFile($search) If @error Then _GUICtrlListView_EndUpdate($hListViewH) ExitLoop EndIf $i = $i + 1 $attrib = FileGetAttrib($temp & "\" & $file) If Not StringInStr($attrib, "D") Then $size = FileGetSize($temp & "\" & $file) $dir = $temp & "\" $dupli = DuplicateCheck($dir, $file) $item = GUICtrlCreateListViewItem($file & "|" & $dir & "|" & ConvSize($size) & "| - ", $hListView) If $dupli = 1 Then GUICtrlSetBkColor($item, 0xFFBFBF) GUICtrlCreateListViewItem("0|1", $hListViewH) Else GUICtrlCreateListViewItem("0|0", $hListViewH) EndIf EndIf WEnd FileClose($search) ; load folders $search = FileFindFirstFile($temp & "\*.*") While 1 $file = FileFindNextFile($search) If @error Then _GUICtrlListView_EndUpdate($hListViewH) ExitLoop EndIf $i = $i + 1 $attrib = FileGetAttrib($temp & "\" & $file) If StringInStr($attrib, "D") Then $size = DirGetSize($temp & "\" & $file) $dir = $temp & "\" $dupli = DuplicateCheck($dir, $file) $item = GUICtrlCreateListViewItem($file & "|" & $dir & "|" & ConvSize($size) & "| - ", $hListView) If $dupli = 1 Then GUICtrlSetBkColor($item, 0xFFBFBF) GUICtrlCreateListViewItem("0|1", $hListViewH) Else GUICtrlSetBkColor($item, 0xE5F7FF) GUICtrlCreateListViewItem("0|0", $hListViewH) EndIf $subtemp = $temp & "\" & $file Add($subtemp, $mainf) EndIf WEnd _GUICtrlListView_EndUpdate($hListViewH) FileClose($search) EndIf EndFunc SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
Lupo73 Posted January 22, 2009 Author Share Posted January 22, 2009 ..I forgot: in the code I use two listview, one showed that include some info like file name, directory, size.. the other one hidden that include some info about files loaded. Any advice? SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
ResNullius Posted January 23, 2009 Share Posted January 23, 2009 Have you looked at _GUICtrlListView_AddArray() ? It's very fast at adding items to a listview, but you'll need to make your file list an array first. Possible that your array building might offset the speed of the _GuiCtrlListView_AddArray, but you never know. Link to comment Share on other sites More sharing options...
Lupo73 Posted January 28, 2009 Author Share Posted January 28, 2009 thanks, I'll try it SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
PsaltyDS Posted January 28, 2009 Share Posted January 28, 2009 thanks, I'll try it You should put some debug points in there to collect the time it takes to do things. I'll bet you lunch that the real time-waster is DuplicateCheck(). Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
Lupo73 Posted January 29, 2009 Author Share Posted January 29, 2009 (edited) Yes! Eureca ^^ ..it seem to be the big problem. I have to run the duplicate check only at the end of the loading process, not each time a file is loaded. Can you help me to create a function that, given a listview, check if there are lines that are duplicate? thank you very much! This is my idea (but need to be completed, or changed ^^): Func DuplicateCheck() Local $nIt, $iI, $k, $h, $a, $b, $aa, $bb ; $a[3] = file name $b[3] = directory $nIt = _GUICtrlListView_GetItemCount($hListView) For $k = 0 To $nIt $a = _GUICtrlListView_GetItem($hListView, $k, 0) $b = _GUICtrlListView_GetItem($hListView, $k, 1) For $h = 0 To $nIt If Not $h = $k Then $aa = _GUICtrlListView_GetItem($hListView, $h, 0) $bb = _GUICtrlListView_GetItem($hListView, $h, 1) If $bb[3] = $b[3] Then If $aa[3] = $a[3] Then ; here I need to add a code that change the background color of the corresponding line ; and change a parameter in a second listview that has the same number of items EndIf EndIf EndIf Next Next EndFunc Edited January 29, 2009 by Lupo73 SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
PsaltyDS Posted January 29, 2009 Share Posted January 29, 2009 Yes! Eureca ^^ ..it seem to be the big problem.I have to run the duplicate check only at the end of the loading process, not each time a file is loaded. Can you help me to create a function that, given a listview, check if there are lines that are duplicate? thank you very much!First, you owe me lunch... Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
Lupo73 Posted January 29, 2009 Author Share Posted January 29, 2009 I have added a comment in the previous post (I didn't see your answer).. what I need (but I'm not able to create it) is a function that start at the end of the adding process, when the whole listview is loaded, and check the list only one time for duplicates. When the function find a duplicate item, it change the background of the item line and change also an item in a second listview. In particular this second listview is an hidden listview that contains parameters of files loaded in the main listview (so it has the same number and order of lines of the main one)... and I need that if a duplicate is found, in this second listview the corresponding item is changed from 0 to 1. SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
PsaltyDS Posted January 30, 2009 Share Posted January 30, 2009 (edited) I have added a comment in the previous post (I didn't see your answer)...I don't see where I get lunch out of this... what I need (but I'm not able to create it) is a function that start at the end of the adding process, when the whole listview is loaded, and check the list only one time for duplicates. When the function find a duplicate item, it change the background of the item line and change also an item in a second listview. In particular this second listview is an hidden listview that contains parameters of files loaded in the main listview (so it has the same number and order of lines of the main one)... and I need that if a duplicate is found, in this second listview the corresponding item is changed from 0 to 1. The nested For/Next loops are slowest possible way to do this. You can do it with only pass through both listviews: All you have to do is assemble a string of used values, with a delimiter like "|" and just use StringInStr() to test each value against that list for duplicates: Func DuplicateCheck() Local $sSearch = "|" For $k = 0 To _GUICtrlListView_GetItemCount($hListView) - 1 $sLVData = _GUICtrlListView_GetItemText($hListView, $k, 1) If StringInStr($sSearch, "|" & $sLVData & "|") Then ; Stuff to do if duplicate... EndIf $sSearch &= $sLVData & "|" Next EndFunc This method is quick enough on each item that it could be removed from the function and done in line with populating the listviews. P.S. I note you didn't account for ListView item numbers being 0-based, while the sub-items are 1-based. You should check on that. Edit: Typo, should be _GUICtrlListView_GetItemText() Edited January 30, 2009 by PsaltyDS Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
Lupo73 Posted January 30, 2009 Author Share Posted January 30, 2009 I have made some changes to the code. I have removed the two listviews, now there is only one listview that for each element loaded use a line (4 columns) + a parameter (using this function _GUICtrlListView_SetItemParam). So now the process of adding items is very fast, but at the end start the DuplicateCheck() that needs many time to do the checking. I have created this function to do the check: Func DuplicateCheck() Local $nIt, $iI, $k, $h, $param Local $a, $b, $aa, $bb $nIt = _GUICtrlListView_GetItemCount($hListView) For $k = 0 To $nIt - 1 $a = _GUICtrlListView_GetItemText($hListView, $k, 0) $b = _GUICtrlListView_GetItemText($hListView, $k, 1) $param = _GUICtrlListView_GetItemParam($hListView, $k) If Not ($param = 2) Then For $h = 0 To $nIt - 1 If Not ($h = $k) Then $aa = _GUICtrlListView_GetItemText($hListView, $h, 0) $bb = _GUICtrlListView_GetItemText($hListView, $h, 1) If $bb = $b Then If $aa = $a Then _GUICtrlListView_SetItemParam($hListView, $h, 2) ;GUICtrlSetBkColor($iI, 0xFFBFBF) EndIf EndIf EndIf Next EndIf Next EndFunc You said your function is really faster, can you explain me how to implement it? I didn't understand what it does.. I need to do something like the function I post (it receive the whole listview and have to detect if there are duplicate items in it..checking the file name, first element of each line, and the directory, second element of each line). Thanks! SFTPEx, AutoCompleteInput, _DateTimeStandard(), _ImageWriteResize(), _GUIGraduallyHide(): some AutoIt functions. Lupo PenSuite: all-in-one and completely free selection of portable programs and games. DropIt: a personal assistant to automatically manage your files. ArcThemALL!: application to multi-archive your files and folders. Link to comment Share on other sites More sharing options...
PsaltyDS Posted January 30, 2009 Share Posted January 30, 2009 I have made some changes to the code. I have removed the two listviews, now there is only one listview that for each element loaded use a line (4 columns) + a parameter (using this function _GUICtrlListView_SetItemParam). So now the process of adding items is very fast, but at the end start the DuplicateCheck() that needs many time to do the checking. I have created this function to do the check: You said your function is really faster, can you explain me how to implement it? I didn't understand what it does.. I need to do something like the function I post (it receive the whole listview and have to detect if there are duplicate items in it..checking the file name, first element of each line, and the directory, second element of each line). Thanks! It would be something like this, even simpler since there is only one listview now: Func DuplicateCheck() Local $sSearch = "|" For $i = 0 To _GUICtrlListView_GetItemCount($hListView) - 1 If StringInStr($sSearch, "|" & _GUICtrlListView_GetItemText($hListView, $i, 0) & "|") Then ; Stuff to do if duplicate... Else ; Not duplicate, add to list $sSearch &= $sLVData & "|" EndIf Next EndFunc But it's simple enough now that it doesn't need to be a function (unless you will be doing it from other places in the script). You can just put this check in with the same code that populates the listview, checking for duplicates as it adds them. Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now