Sign in to follow this  
Followers 0
neilontherock

Why does my script use so much memory

7 posts in this topic

#1 ·  Posted (edited)

Hi,

I have a script written that connects to a website using _IECreate() then use _IENavigate to loop through multiple pages. On each page I use _IELinkGetCollection to obtain all the links on each page. For each valid link I then use InetGet to download a temp file.

As my script runs it starts off at about 28k but as it runs memory keeps growing to about 1g.

I do make a call to _IEQuit after I process each search. About 30 web pages are processed per search.

Does anyone know why this is happening and what/if anything I can do to free memory?

Edited by neilontherock

Share this post


Link to post
Share on other sites



Because you aren't ever clearing up the memory used when it loads all those webpages.

Ascend4nt has a thread:

http://www.autoitscript.com/forum/index.php?showtopic=115352

with a udf full of functions to handle processes. You can use that to make a call to your autoit script to free its working load set.

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Hi,

I have a script written that connects to a website using _IECreate() then use _IENavigate to loop through multiple pages. On each page I use _IELinkGetCollection to obtain all the links on each page. For each valid link I then use InetGet to download a temp file.

As my script runs it starts off at about 28k but as it runs memory keeps growing to about 1g.

I do make a call to _IEQuit after I process each search. About 30 web pages are processed per search.

Does anyone know why this is happening and what/if anything I can do to free memory?

Waouh 1 Go ! Posted Image

Show your script for find where is the problem

Or try

_ReduceMemory(ProcessExists(@ScriptName))


Func ReduceMemory($i_PID = -1)
    If $i_PID <> -1 Then
        Local $ai_Handle = DllCall("kernel32.dll", 'int', 'OpenProcess', 'int', 0x1f0fff, 'int', False, 'int', $i_PID)
        Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', $ai_Handle[0])
        DllCall('kernel32.dll', 'int', 'CloseHandle', 'int', $ai_Handle[0])
    Else
        Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', -1)
    EndIf
    
    Return $ai_Return[0]
EndFunc;==> ReduceMemory()
Edited by wakillon

AutoIt 3.3.14.2 X86 - SciTE 3.6.0WIN 8.1 X64 - Other Example Scripts

Share this post


Link to post
Share on other sites

Waouh 1 Go ! Posted Image

Show your script for find where is the problem

Or try

_ReduceMemory(ProcessExists(@ScriptName))


Func ReduceMemory($i_PID = -1)
    If $i_PID <> -1 Then
        Local $ai_Handle = DllCall("kernel32.dll", 'int', 'OpenProcess', 'int', 0x1f0fff, 'int', False, 'int', $i_PID)
        Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', $ai_Handle[0])
        DllCall('kernel32.dll', 'int', 'CloseHandle', 'int', $ai_Handle[0])
    Else
        Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', -1)
    EndIf
    
    Return $ai_Return[0]
EndFunc;==> ReduceMemory()

Thanks, I tried your code and it doesn't help. I even wrote code to clean up the "Temporary Internet Files\Content.IE5\" folder as the write runs and that doesn't help either. I know that is working because I'm able to see the files getting deleted as the script executes.

The script starts of with about 30k and within 30 minutes it's shows 1.5 g. This is running on windows 2000 server.

When the memory reaches about 1.3 g, the INetGet function starts failing and 0 bytes is returned.

Doesn't make sense.

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Hi - yes this does work but it does not reduce Virtual Memory or perhaps the amount of VM that Windows task manager is showing is wrong because my script is deleting the temp files being created by the INetGet function calls.

Thanks for sharing, now if I could only free the Virtual Memory.

Edited by neilontherock

Share this post


Link to post
Share on other sites

Hi - yes this does work but it does not reduce Virtual Memory or perhaps the amount of VM that Windows task manager is showing is wrong because my script is deleting the temp files being created by the INetGet function calls.

Thanks for sharing, now if I could only free the Virtual Memory.

Hi, I have found the problem. What is happening is the following folders are eating up Virtual Memory as the script runs and is never deleted. This has nothing to do with Autoit and just the way Internet Explorer cauches data.

So I created to constants in my code based on the running user;

$WindowTempIEPath1 = "C:\Documents and Settings\<user>\Local Settings\Temp\Temporary Internet Files\Content.IE5\"

$WindowTempIEPath2 = "C:\Documents and Settings\<user>\Local Settings\Temporary Internet Files\Content.IE5\"

...and then called the function with each constant after the script processed a web page. Works like a charm!!!

RecursiveFileSearch (WindowTempIEPath1 )

RecursiveFileSearch (WindowTempIEPath2 )

Func RecursiveFileSearch ($startDir, $depth = 0)

If $depth = 0 Then Global $RFSstring = ""

$search = FileFindFirstFile($startDir & "\*.*")

If @error Then

Return

EndIf

;Search through all files and folders in directory

While 1

$next = FileFindNextFile($search)

If @error Then ExitLoop

;If folder, recurse

If StringInStr(FileGetAttrib($startDir & "\" & $next), "D") Then

RecursiveFileSearch ($startDir & "\" & $next, $depth + 1)

Else

$RFSstring &= $startDir & "\" & $next & "*"

Local $result = FileDelete($startDir & "\" & $next )

EndIf

WEnd

FileClose($search)

If $depth = 0 Then Return StringSplit(StringTrimRight($RFSstring, 1), "*")

EndFunc

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0