Jump to content
Sign in to follow this  
hou

Slowdown / Garbage Collection

Recommended Posts

hou

Hello AutoIt Fans out there!

I am an newbie in programming with AutoIt and experimented with some of its features and it seems to be a real goodie for my utility pack as programmer.

My first little programm handles a lot of string data, batch processing files. Watching the debug output I noticed a real slowdown of the running script and did some investigation on it.

First I noticed the usage of ReDim in some of the UDF packages. AutoIt itself cant be blamed for that slowdown.

Second, and that's the point here: By assigning values -explicit or implicit- autoit seems to slow down.

To speedup my script I did the following modification to _StringRepeat() in <Strings.au3>:

CODE

; *** Experimental speedup on _StringRepeat() function ********************

Case Else

While $iRepeatCount > 1

If mod($iRepeatCount,2)=1 Then

$sResult &= $sString

$iRepeatCount -= 1

EndIf

if $iRepeatCount>1 Then

$sString &= $sString

$iRepeatCount /= 2

EndIf

WEnd

$sResult &= $sString

;************************************original code commented out here

;For $iCount = 1 To $iRepeatCount

; $sResult &= $sString

;Next ;

;*******************************************************************

Wondering if this would speedup my code something I did run this test (_StringRepeatOld() is the original thing):

CODE

dim $j, $i, $s, $t, $t1, $t2

for $j=1 to 10

$t= TimerInit()

for $i=1 to 1000

$s= _StringRepeat( "A", $i )

if $i<>StringLen($s) then ConsoleWrite( StringFormat( "error: %d / %d " & @CRLF, $i, StringLen($s) ) )

Next

$t1= TimerDiff($t)

$t= TimerInit()

for $i=1 to 1000

$s= _StringRepeatOld( "A", $i )

if $i<>StringLen($s) then ConsoleWrite( StringFormat( "error: %d / %d " & @CRLF, $i, StringLen($s) ) )

Next

$t2= TimerDiff($t)

ConsoleWrite( StringFormat( "%2d: %5.2d / %5.2d ", $j, $t1, $t2 ) & @CRLF )

Next

Exit

here the output of this little test

1: 445 / 4430

2: 450 / 4560

3: 444 / 18220

4: 1823 / 21104

5: 2470 / 18651

6: 442 / 4703

7: 1613 / 21741

8: 1820 / 21545

9: 2494 / 13307

10: 618 / 4492

as one can see there is

1) a real good speedup with the modification of _StringRepeat() :)

2) AutoIt slows overall slows down, speeds up 1 time, slows down ... :mellow:

Is there any possibility to avoid this behavior? Is it possible to force the garbage collection, that obviously triggered

between 5/6 and 9/10, manually? Are there any other hints?

Thx and cu

hou

BTW: this is my first time using this BB stuff - hope it is in a readable format. Otherwise I apologize for it :()

Share this post


Link to post
Share on other sites
PsaltyDS

Hello AutoIt Fans out there!

I am an newbie in programming with AutoIt and experimented with some of its features and it seems to be a real goodie for my utility pack as programmer.

My first little programm handles a lot of string data, batch processing files. Watching the debug output I noticed a real slowdown of the running script and did some investigation on it.

First I noticed the usage of ReDim in some of the UDF packages. AutoIt itself cant be blamed for that slowdown.

Second, and that's the point here: By assigning values -explicit or implicit- autoit seems to slow down.

To speedup my script I did the following modification to _StringRepeat() in <Strings.au3>:

CODE

; *** Experimental speedup on _StringRepeat() function ********************

Case Else

While $iRepeatCount > 1

If mod($iRepeatCount,2)=1 Then

$sResult &= $sString

$iRepeatCount -= 1

EndIf

if $iRepeatCount>1 Then

$sString &= $sString

$iRepeatCount /= 2

EndIf

WEnd

$sResult &= $sString

;************************************original code commented out here

;For $iCount = 1 To $iRepeatCount

; $sResult &= $sString

;Next ;

;*******************************************************************

Wondering if this would speedup my code something I did run this test (_StringRepeatOld() is the original thing):

CODE

dim $j, $i, $s, $t, $t1, $t2

for $j=1 to 10

$t= TimerInit()

for $i=1 to 1000

$s= _StringRepeat( "A", $i )

if $i<>StringLen($s) then ConsoleWrite( StringFormat( "error: %d / %d " & @CRLF, $i, StringLen($s) ) )

Next

$t1= TimerDiff($t)

$t= TimerInit()

for $i=1 to 1000

$s= _StringRepeatOld( "A", $i )

if $i<>StringLen($s) then ConsoleWrite( StringFormat( "error: %d / %d " & @CRLF, $i, StringLen($s) ) )

Next

$t2= TimerDiff($t)

ConsoleWrite( StringFormat( "%2d: %5.2d / %5.2d ", $j, $t1, $t2 ) & @CRLF )

Next

Exit

here the output of this little test

1: 445 / 4430

2: 450 / 4560

3: 444 / 18220

4: 1823 / 21104

5: 2470 / 18651

6: 442 / 4703

7: 1613 / 21741

8: 1820 / 21545

9: 2494 / 13307

10: 618 / 4492

as one can see there is

1) a real good speedup with the modification of _StringRepeat() ;)

2) AutoIt slows overall slows down, speeds up 1 time, slows down ... :(

Is there any possibility to avoid this behavior? Is it possible to force the garbage collection, that obviously triggered

between 5/6 and 9/10, manually? Are there any other hints?

Thx and cu

hou

BTW: this is my first time using this BB stuff - hope it is in a readable format. Otherwise I apologize for it ;))

That's a cute trick to speed up the_StringRepeat() function ! :)

I don't like your demo, because it assumes modifying the UDF before testing. So I tried it this way, with the original _StringRepeat() first, then your modified version as _StringRepeatNew():

#include <String.au3>

Dim $s, $t, $t1, $t2, $sMsg = ""

For $j = 1 To 20
    $t = TimerInit()
    For $i = 1 To 1000
        $s = _StringRepeat("A", $i)
        If $i <> StringLen($s) Then ConsoleWrite(StringFormat("error: %d / %d " & @CRLF, $i, StringLen($s)))
    Next
    $t1 = TimerDiff($t)

    $t = TimerInit()
    For $i = 1 To 1000
        $s = _StringRepeatNew("A", $i)
        If $i <> StringLen($s) Then ConsoleWrite(StringFormat("error: %d / %d " & @CRLF, $i, StringLen($s)))
    Next
    $t2 = TimerDiff($t)

    $sMsg &= StringFormat("%2d: %5.2d / %5.2d ", $j, $t1, $t2) & @CRLF
Next

ConsoleWrite($sMsg)

Func _StringRepeatNew($sString, $iRepeatCount)
;==============================================
; Local Constant/Variable Declaration Section
;==============================================
    Local $sResult
    
    Select
        Case Not StringIsInt($iRepeatCount)
            SetError(1)
            Return ""
        Case StringLen($sString) < 1
            SetError(1)
            Return ""
        Case $iRepeatCount <= 0
            SetError(1)
            Return ""
        Case Else
            While $iRepeatCount > 1
                If Mod($iRepeatCount, 2) = 1 Then
                    $sResult &= $sString
                    $iRepeatCount -= 1
                EndIf
                If $iRepeatCount > 1 Then
                    $sString &= $sString
                    $iRepeatCount /= 2
                EndIf
            WEnd
            $sResult &= $sString
            Return $sResult
    EndSelect
EndFunc  ;==>_StringRepeatNew

Note that it does not use ConsoleWrite() until the end because that function is slow itself.

My results were:

>Running:(3.2.12.1):C:\Program Files\AutoIt3\autoit3.exe "C:\temp\Test\Test1.au3"   
 1:   977 /   124 
 2:   964 /   126 
 3:   956 /   123 
 4:   956 /   123 
 5:   955 /   123 
 6:   954 /   123 
 7:   955 /   126 
 8:   956 /   124 
 9:   953 /   127 
10:  1021 /   131 
11:  1119 /   131 
12:   953 /   130 
13:   951 /   128 
14:   957 /   127 
15:   957 /   126 
16:   960 /   124 
17:   950 /   124 
18:   952 /   123 
19:   952 /   124 
20:   951 /   123 
+>13:34:34 AutoIT3.exe ended.rc:0
+>13:34:35 AutoIt3Wrapper Finished
>Exit code: 0   Time: 23.306

These results are much more consistent than yours. Please try again with my version and see if you still get that increase in time.

:mellow:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
hou

That's a cute trick to speed up the_StringRepeat() function ! :)

I don't like your demo, because it assumes modifying the UDF before testing. So I tried it this way, with the original _StringRepeat() first, then your modified version as _StringRepeatNew():

~~~cut~~~

...

~~~cut~~~

My results were:

>Running:(3.2.12.1):C:\Program Files\AutoIt3\autoit3.exe "C:\temp\Test\Test1.au3"   
 1:   977 /   124 
 2:   964 /   126 
 3:   956 /   123 
 4:   956 /   123 
 5:   955 /   123 
 6:   954 /   123 
 7:   955 /   126 
 8:   956 /   124 
 9:   953 /   127 
10:  1021 /   131 
11:  1119 /   131 
12:   953 /   130 
13:   951 /   128 
14:   957 /   127 
15:   957 /   126 
16:   960 /   124 
17:   950 /   124 
18:   952 /   123 
19:   952 /   124 
20:   951 /   123 
+>13:34:34 AutoIT3.exe ended.rc:0
+>13:34:35 AutoIt3Wrapper Finished
>Exit code: 0   Time: 23.306

These results are much more consistent than yours. Please try again with my version and see if you still get that increase in time.

:mellow:

Thanks for the response and sorry my answer took that long.

You are right saying that my demo is not done the right way around new/old ... - Didn't think of other people when I tested.

I did it your way and got ( taking a 40 rounds to make it real clear ):

CODE

1: 2365 / 290

2: 2294 / 301

3: 2297 / 293

4: 2323 / 287

5: 2303 / 287

6: 2371 / 293

7: 7503 / 1254

8: 9922 / 1261

9: 10179 / 1237

10: 11188 / 1255

11: 9928 / 1243

12: 7461 / 350

13: 2454 / 372

14: 2572 / 322

15: 2580 / 321

16: 5441 / 1279

17: 14545 / 1288

18: 11745 / 1731

19: 15495 / 1929

20: 9482 / 387

21: 2682 / 316

22: 2574 / 339

23: 8679 / 2519

24: 23955 / 2449

25: 14224 / 2105

26: 8438 / 293

27: 2814 / 296

28: 2607 / 294

29: 2912 / 2931

30: 16053 / 1358

31: 12680 / 10333

32: 15563 / 626

33: 2315 / 295

34: 2307 / 287

35: 2484 / 295

36: 9825 / 3152

37: 20392 / 1323

38: 10611 / 1273

39: 10625 / 1286

40: 4322 / 302

I think *seeing* this behaviour is caused by my machine having only 256MB memory available or cause of 1GHz PIII, on the other hand the used string space per round (SumOf(1...1000) about 500.000 bytes + overhead) is'nt really that much and there is no swapping to see on the disks. May be this example is a little bit extreme on using temp/hidden vars but nevertheless I would like things to behave a little bit more deterministic :(

Execution time goes up and down a lot. So I will not get a real chance to guarantie reaction times.

I fear users will be confused if the expected time goes by and the result is missing (For example the program reacts 10 times within a 2 or 3 seconds, then the user will not expect a reaction after 20 to 30 seconds. Inspite he/she will try to kill that app or do other things to get back and produce more confusion).

Are there tricks to force the reorganization that took place after several rounds to balance the throughput?

Share this post


Link to post
Share on other sites
PsaltyDS

Thanks for the response and sorry my answer took that long.

You are right saying that my demo is not done the right way around new/old ... - Didn't think of other people when I tested.

I did it your way and got ( taking a 40 rounds to make it real clear ):

CODE
1: 2365 / 290

2: 2294 / 301

3: 2297 / 293

4: 2323 / 287

5: 2303 / 287

6: 2371 / 293

7: 7503 / 1254

8: 9922 / 1261

9: 10179 / 1237

10: 11188 / 1255

11: 9928 / 1243

12: 7461 / 350

13: 2454 / 372

14: 2572 / 322

15: 2580 / 321

16: 5441 / 1279

17: 14545 / 1288

18: 11745 / 1731

19: 15495 / 1929

20: 9482 / 387

21: 2682 / 316

22: 2574 / 339

23: 8679 / 2519

24: 23955 / 2449

25: 14224 / 2105

26: 8438 / 293

27: 2814 / 296

28: 2607 / 294

29: 2912 / 2931

30: 16053 / 1358

31: 12680 / 10333

32: 15563 / 626

33: 2315 / 295

34: 2307 / 287

35: 2484 / 295

36: 9825 / 3152

37: 20392 / 1323

38: 10611 / 1273

39: 10625 / 1286

40: 4322 / 302

I think *seeing* this behaviour is caused by my machine having only 256MB memory available or cause of 1GHz PIII, on the other hand the used string space per round (SumOf(1...1000) about 500.000 bytes + overhead) is'nt really that much and there is no swapping to see on the disks. May be this example is a little bit extreme on using temp/hidden vars but nevertheless I would like things to behave a little bit more deterministic :mellow:

Execution time goes up and down a lot. So I will not get a real chance to guarantie reaction times.

I fear users will be confused if the expected time goes by and the result is missing (For example the program reacts 10 times within a 2 or 3 seconds, then the user will not expect a reaction after 20 to 30 seconds. Inspite he/she will try to kill that app or do other things to get back and produce more confusion).

Are there tricks to force the reorganization that took place after several rounds to balance the throughput?

I don't think this is an AutoIt issue any more. The variability you are seeing is likely a performance issue with your particular machine under the load of your particular collection of software. AutoIt itself does not introduce the variability you are seeing.

Your version of _StringRepeat() is still much faster than the current UDF's, and I hope you submit it as a change.

:(


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×