Memory leak - error allocating memory

[vulcan4d]

I have a function where I'm hitting some sort of memory leak and for the life of me I can't figure it out.  I'm hoping someone more experienced can spot the error easily as I've been struggling for days.  Between 1-2hours of running the script it gives me the "Error Allocating Memory" message in Scite.  The system is not running out of memory with 32GB and Scite doesn't grow much while running so I'm hitting some kind of limit here.  

The script is meant to analyze the trading market.  A file exists that is constantly being updated called current.txt.  The script reads the stats from this file, makes a copy called previous for comparison and runs it through an analyzer.  The analyzer I've confirmed has no issues however something in the functions below is causing the issue:

readstats() - reads the files then calls the searcharray() function to find the indicator name by variable
searcharray() - function searches the array for the variable
calcdir() - calculates the direction of the trend/indicator

I'm at a complete loss right now without completely re-writing this.  Would someone be so kind is to skim through the code and let me know if there is an obvious fault?  I've pasted what is relevant below, any help is greatly appreciated.

 

expandcollapse popup

Global $loop = 0;
Global $time = "Time"
Global $volume = "Volume"
Global $vwap = "VWAP"
Global $wtbull = "WT Bullish Divergence"
Global $buysellc = "Buy and sell circle"
Global $buyc = "Buy circle"
Global $sellc = "Sell circle"
Global $divbuy = "Divergence buy circle"
Global $kite1 = "1st timeframe"
Global $kite2 = "2nd timeframe"
Global $kite3 = "3rd timeframe"
Global $kite4 = "4th timeframe"
Global $plot = "Plot"
Global $plot2 = "Plot2"
Global $stochK = "Stoch K"
Global $stochD = "Stoch D"
Global $mf = "MF"
Global $close = "Close"
Global $ema13 = "4EMA 3"
Global $ema21 = "4EMA 6"
Global $ema55 = "4EMA 13"
Global $cArray[];
Global $pArray[];
Global $loop = 0;
Global $timechange = "";
Global $timerun = 0;
Global $strategy1 = "";
Global $bidding = "no";
Global $vwapangle = 0;

Global $filecurrent = @ScriptDir & "\Files\current.txt"
Global $fileprevious = @ScriptDir & "\Files\previous.txt"

start();
FileDelete($fileprevious);
Filecopy($filecurrent,$fileprevious,1);

Do
readstats();
analysis();
sleep(100);
Until $loop = 10;

Func start();
    $dll = DllOpen("user32.dll")
    While NOT _IsPressed("01", $dll)
       Sleep (50)
    WEnd
    Global $mousecurrent = MouseGetPos()
    WinActivate("[CLASS:MozillaWindowClass]", "")
    DllClose($dll)
EndFunc

Func readstats()
    FileOpen($filecurrent, 0)
    _FileReadToArray($filecurrent,$cArray)
    FileClose($filecurrent)
    FileOpen($fileprevious, 0)
    _FileReadToArray($fileprevious,$pArray)
    FileClose($fileprevious)

    Global $find_close = searcharray($close,$cArray);
    Global $pfind_close = searcharray($close,$pArray);
    Global $closedir = calcdir($pfind_close,$find_close);

    Global $find_ema13 = searcharray($ema13,$cArray);
    Global $find_ema21 = searcharray($ema21,$cArray);
    Global $find_ema55 = searcharray($ema55,$cArray);

    Global $find_time = searcharray($time,$cArray);
    Global $pfind_time = searcharray($time,$pArray);

    Global $find_kite1 = searcharray($kite1,$cArray);
    Global $pfind_kite1 = searcharray($kite1,$pArray);
    Global $kite1dir = calcdir($pfind_kite1,$find_kite1);

    Global $find_kite2 = searcharray($kite2,$cArray);
    Global $pfind_kite2 = searcharray($kite2,$pArray);
    Global $kite2dir = calcdir($pfind_kite2,$find_kite2);

    Global $find_vwap = searcharray($vwap,$cArray);
    Global $pfind_vwap = searcharray($vwap,$pArray);
    Global $vwapdir = calcdir($pfind_vwap,$find_vwap);
    Global $vwapangle = Round(calcvwapangle(($find_vwap-$pfind_vwap),5),2);

    Global $find_buysellc = searcharray($buysellc,$cArray);

    Global $find_buyc = searcharray($buyc,$cArray); -107

    Global $find_sellc = searcharray($sellc,$cArray); 105

    Global $find_mf = searcharray($mf,$cArray);
    Global $pfind_mf = searcharray($mf,$pArray);
    Global $mfdir = calcdir($pfind_mf,$find_mf);

    Global $find_stochk = searcharray($stochK,$cArray);
    Global $find_stochd = searcharray($stochD,$cArray);

    Global $find_divbuy = searcharray($divbuy,$cArray); -106
    Global $prefind_divbuy = searcharray($divbuy,$pArray); -106

    Global $find_plot1 = searcharray($plot,$cArray);
    Global $pfind_plot1 = searcharray($plot,$pArray);

    Global $find_plot2 = searcharray($plot2,$cArray);
    Global $pfind_plot2 = searcharray($plot2,$pArray);

        Global $plot1dir = calcdir(Number($pfind_plot1),Number($find_plot1));
        If $find_plot2 <> "n/a" OR $pfind_plot2 <> "n/a" Then
            Global $plot2dir = calcdir(Number($pfind_plot2),Number($find_plot2));
        Else
            Global $plot2dir = "n/a";
        EndIf

    If $find_time <> $pfind_time AND $timerun = 0 Then; time change on first run, create a temp file from current
        $timechange = $find_time;
        $timerun = 1;
        filecopy($filecurrent,$filetemp,1);
    EndIf
    If $timechange <> $find_time AND $timerun = 1 Then; second time change since start
        filecopy($filetemp,$fileprevious,1);
        $timerun = 0;
    EndIf

    Global $emastatus = "";
    If $find_close > $find_ema55 Then
        $emastatus = "yes"
    Else
        $emastatus = "no"
    EndIf

    tooltip("Close: "&$find_close&", "&$pfind_close&", "&$closedir& _
        @CRLF&"Time: "&$find_time&", "&$pfind_time& _
        @CRLF&"Vwap: "&$find_vwap&", "&$pfind_vwap&", "&$vwapdir& _
        @CRLF&"Kite1: "&$find_kite1&", "&$pfind_kite1&", "&$kite1dir& _
        @CRLF&"Kite2: "&$find_kite2&", "&$pfind_kite2&", "&$kite2dir& _
        @CRLF&"MF: "&$find_mf&", "&$pfind_mf&", "&$mfdir& _
        @CRLF&"Plot1: "&$find_plot1&", "&$pfind_plot1&", "&$plot1dir& _
        @CRLF&"Plot2: "&$find_plot2&", "&$pfind_plot2&", "&$plot2dir& _
        @CRLF&"BuySellC: "&$find_buysellc& _
        @CRLF&"VwapAng: "&$vwapangle& _
        @CRLF&"Strat1: "&$strategy1& _
        @CRLF&"Bidding: "&$bidding& _
        @CRLF&"Ema55: "&$find_ema21);
EndFunc

Func calcdir($previous,$current); calculates the direction of the trend
    If $previous = "n/a" OR $current = "n/a" Then
        Local $updown = "n/a";
    ElseIf $previous > $current Then
        Local $updown = "down";
    ElseIf $previous < $current Then
        Local $updown = "up";
    ElseIf $previous = $current Then
        Local $updown = "steady";
    Else
        Local $updown = "n/a";
    EndIf
    Return $updown;
EndFunc

Func searcharray($indicator,$array)
    $ivalue = 0;
    If $indicator = "Plot2" Then
        Local $linenum = _ArraySearch($array,"Plot", 0, 0, 1, 1)
    Else
        Local $linenum = _ArraySearch($array,$indicator, 0, 0, 1, 1)
    EndIf
    If Not @error Then
        ;MF edit
        If $indicator = "MF" Then; if MF go to next line
            $linenum = $linenum + 1;
        EndIf

        ;Plot edit
        If $indicator = "Plot" Then;
            Local $linenum2 = $linenum + 1;
            Local $plot1 = _ArraySearch($array,$indicator, $linenum2, 0, 1, 1); continue search from previous find + 1;
            $ivalue = StringReplace($array[$plot1],$indicator,"");
        ElseIf $indicator = "Plot2" Then
            $indicator = stringtrimright($indicator,1)
            Local $plot2 = _ArraySearch($array,$indicator, 0, 0, 1, 1,0); searches from the end to beginning;
            $ivalue = StringReplace($array[$plot2],$indicator,"");
            $indicator = "Plot2"
        Else
            $ivalue = StringReplace($array[$linenum],$indicator,"");
        EndIf

        ;Volume Edit
        ;If $indicator = $volume AND StringRight($ivalue,1) = "K" Then; if Volume has K, remove letter
        ;   $ivalue = StringTrimRight($ivalue,1)
        ;EndIf

        If $ivalue = "−0.00" Then;
            $ivalue = 0;
        EndIF

        If stringleft($ivalue,1) = "−" Then
            $ivalue = stringreplace($ivalue,"−","-");
        EndIf

        If $ivalue = "n/a" OR $indicator = $time Then
            $ivalue = string($ivalue);
        Else; if it's a number, convert to number (excludes time)
            $ivalue = number($ivalue);
        EndIf
    Else
        $ivalue = "n/a";
    EndIf
return $ivalue;
EndFunc

 

Edited December 25, 2020 by vulcan4d

[GokAy]

Hey,

I remembered reading about this not long ago, so searched again. I am not sure if your issue is related. And I am not experienced enough to help, however for others that will help, can you post how large your files are getting?

Maybe related to this? https://www.autoitscript.com/trac/autoit/ticket/257

Couple more links:

https://www.autoitscript.com/forum/topic/136847-solved-string-size-limitation/

https://www.autoitscript.com/forum/topic/179096-autoit-memory-allocation-bug/

In case related: https://www.autoitscript.com/forum/topic/131315-accumulating-memory-usage/

[Jos]

4 hours ago, vulcan4d said:

Between 1-2hours of running the script it gives me the "Error Allocating Memory" message in Scite. 

SciTE or do you mean AutoIt3 ?

[Nine]

I think the culprit is there :

FileOpen($filecurrent, 0)
    _FileReadToArray($filecurrent,$cArray)
    FileClose($filecurrent)
    FileOpen($fileprevious, 0)
    _FileReadToArray($fileprevious,$pArray)
    FileClose($fileprevious)

You are opening file (and not keeping its handle) and you are NOT closing the file because of bad format.  All that for nothing since you do not need a file handle to do _FileReadToArray...

[JockoDundee]

Although I agree with @Nine that File handles are being unnecessarily opened and then not getting closed, it should be noted that the $loop variable never increments; so it would seemingly run forever, since it never gets to ten.

Is this the intention?

 

[Nine]

2 hours ago, JockoDundee said:

Is this the intention?

I was supposing it was the intention, since he wants to run it for a long time.  It crashes after 1-2 hours.  So count should be way higher than 10 after that period of time.

[JockoDundee]

2 hours ago, Nine said:

So count should be way higher than 10 after that period of time.

I think it’s no higher than 0.  

The only statements I see regarding $loop are:

Global $loop = 0
...
Global $loop = 0
...
Until $loop = 10

 

[Nine]

Sigh...You don't understand me.  It is intentional because he doesn't want to get to 10 (he should have said False, it would have made this more clear).  It is a fast growing loop.  In a sec it would have reached 10.  But he wants to run it for hours and hours.  The problem is not achieving 10.  The problem is that he keeps creating handles and never closes them.  For hours, probably tens of thousands of handles never closed, that is the real issue.

And please don't explain me how a loop should be like...

Edited December 26, 2020 by Nine

[JockoDundee]

 

50 minutes ago, Nine said:

The problem is that he keeps creating handles and never closes them.  For hours, probably tens of thousands of handles never closed, that is the real issue.

 

I’ve already agreed with this, so I’m not sure why you act like I’m denying your analysis.
At the same time, I think it’s a fair observation for me to point out when there’s a clear outer loop that doesn’t increment; he obviously once intended it to be a fixed duration program.

If I had posted code with such a loop, I would hope someone would point it out to me.