Sign in to follow this  
Followers 0
garbb

Reading listview w/o high CPU usage

7 posts in this topic

I have a script that periodically reads a listview from another application and checks for changes. I have noticed that when the listview has a large number of rows in it (>500), autoit will have high CPU usage whenever it reads the listview's data.

Here is my test code (which also generates a listview in a GUI to read):

#include <GUIListView.au3>

$iListviewColumns = 26
$iListviewItems = 500   ;rows

;create test GUI
$hGUI = GUICreate("ListView", 1000, 400)
$hListView = GUICtrlCreateListView("", 2, 2, 994, 400)
;add columns to listview
for $i = 1 to $iListviewColumns
    _GUICtrlListView_AddColumn($hListView, "column" & $i, 75)
Next
;add items to listview
for $i = 1 to $iListviewItems
    $sItemText = "item." & $i & "|"
    For $j = 2 to $iListviewColumns
        $sItemText &= "col." & $i & $j & "|"
    Next
    GUICtrlCreateListViewItem($sItemText, $hListView)
Next
GUISetState()

;main loop to poll listview data
while 1
    $data = _GUICtrlListView_CreateArray($hListView)
    sleep(3000)
wend

Func _GUICtrlListView_CreateArray($hListView)
    Local $iColumnCount = _GUICtrlListView_GetColumnCount($hListView), $iItemCount = _GUICtrlListView_GetItemCount($hListView)
    Local $aColumns = 0, $aReturn[$iItemCount + 1][$iColumnCount] = [[$iItemCount, $iColumnCount, '']]

    For $i = 0 To $iItemCount - 1
        For $j = 0 To $iColumnCount - 1
            ;##### switch between these two lines to see the difference in CPU usage (for me the 1st line uses much more CPU)
            $aReturn[$i + 1][$j] = _GUICtrlListView_GetItemText($hListView, $i, $j)
;~          $aReturn[$i + 1][$j] = ControlListView($hGUI, '', $hListView, 'GetText', $i, $j)
        Next
    Next
    Return $aReturn
EndFunc   ;==>_GUICtrlListView_CreateArray

The _GUICtrlListView_CreateArray function in there is just a stripped down version of what guinness made >here. I was experimenting with changing the line that actually reads the listview data (you can see the two lines I was switching between(one is commented out)).

If you run the script and look in windows task manager you can see the CPU spike (on my computer with a quad core it spikes to around 25% for the 1st line and 14% for the second line). Note that I believe for a quad core system and a process that only uses one core like autoit, 25% is basically maxing the CPU.

I searched around and couldn't seem to find a better/more efficient way to read the listview that doesn't use so much CPU. Does anyone know a better way? Is this possible with autoit?

Share this post


Link to post
Share on other sites



When the program reads the contents of the ListView, trying to do this as fast as possible.
Therefore, using all available resources.

Tip:
Add Sleep(10) inside the function For .... Next.

For $i = 0 To $iItemCount - 1
    Sleep(10)
        For $j = 0 To $iColumnCount - 1
            ;##### switch between these two lines to see the difference in CPU usage (for me the 1st line uses much more CPU)
            $aReturn[$i + 1][$j] = _GUICtrlListView_GetItemText($hListView, $i, $j)
;~          $aReturn[$i + 1][$j] = ControlListView($hGUI, '', $hListView, 'GetText', $i, $j)
        Next
    Next

mLipok


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

In my example I have 26 columns * 500 rows = 13000 items. If there is a 10ms delay per item then reading the entire listview will take 130 seconds or a little over 2 minutes.

This will reduce the CPU usage but taking that long to read the listview is not feasible for my application.

With my original code it takes a little over 400ms to read the entire list. I would actually like it to be faster than that but I am not sure if that is possible.

Actually, your logic is not bad. I might be able to change how my script works to be more tolerant of it taking longer to read the list but 2 minutes is too long. The problem is that I think that autoit's sleep() has a minimum of 10ms.

Edited by garbb

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Have you considered using _GUICtrlListView_GetItemTextArray instead of doing it one by one?

I didn't.
 
Unfortunately I just tried it and the results were the same as when I tried _GUICtrlListView_GetItemText() because looking in the UDF's I can see that _GUICtrlListView_GetItemTextArray() calls _GUICtrlListView_GetItemTextString() which calls _GUICtrlListView_GetItemText(). So it's basically all the same code...
Edited by garbb

Share this post


Link to post
Share on other sites

Only other thing I can think of is using a shorter (high precision) sleep.

For example swap Sleep(10) for

$Timer = TimerInit()
Do
Until TimerDiff($Timer) >= 0.01

There is a high precision sleep using dllcall, that might be better if you can find it.


AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

I have a script that periodically reads a listview from another application and checks for changes.

Why not wait for the change and then get the items at one go. This would take the same CPU but only at the time when update occurs, or maybe you can modify it to suit you rather better.

 

The following way to proceed, may help,

  1. For a third party app, to monitor messages you would have to use
    • _WinAPI_SetWindowsHookEx with $WH_CALLWNDPROCRET
    • Hook the ListView
  2. Monitor message LVM_SETITEM LVM_INSERTITEM
  3. Respond to the message, and do your stuff.

 

Example in C++

In the moment I can't try for if it works. 

Regards :)

Phoenix XL

Edited by PhoenixXL

My code:

PredictText: Predict Text of an Edit Control Like Scite. Remote Gmail: Execute your Scripts through Gmail. StringRegExp:Share and learn RegExp.

Run As System: A command line wrapper around PSEXEC.exe to execute your apps scripts as System (LSA). Database: An easier approach for _SQ_LITE beginners.

MathsEx: A UDF for Fractions and LCM, GCF/HCF. FloatingText: An UDF for make your text floating. Clipboard Extendor: A clipboard monitoring tool. 

Custom ScrollBar: Scroll Bar made with GDI+, user can use bitmaps instead. RestrictEdit_SRE: Restrict text in an Edit Control through a Regular Expression.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0