[solved] Autoit Error Allocating Memory

[goldenix]

You know hjsplit right? well this is more user friendly version that joins all files in scriptdir with just double click this exe once compiled.

My 001,002 etc files are 300MB each & I have 2GB of memory & when I run my script I get that error while its reading first file. My output file is btw 0 bytes long so its not even writing anything into the file.

what may be the problem? I have run his script before & it was working fine.

I think the problem lies hire but im not too sure cuz it has worked before, I even removed 3 zeroes from end & still same error:

$data = FileRead($file_read, 1000000000)

expandcollapse popup

#Include <File.au3>

$FileList=_FileListToArray(@ScriptDir,'*.001',1) ; files only
If @Error=4 Then
    MsgBox (0,"","No Files Found.")
    Exit
EndIf

For $i = 1 To $FileList[0]

    _Join(StringTrimRight($FileList[$i],4)) ;keep filename only
Next

Func _Join($file)

;## loop counts number of files: 001,002 etc.
    $count = 1
    While 1

        $zeros = _add_zeros($count, '000')
        $zefile = $file & '.' & $zeros
        If not FileExists($zefile) Then Exit

        ConsoleWrite($zefile & @CRLF)
        ToolTip('Reading: ' & $zefile,0,0)

        $file_read = FileOpen($zefile, 0)
        $file_write = FileOpen($file, 1)

        While 1
            $data = FileRead($file_read, 1000000000)
            If @error Then ExitLoop
            FileWrite($file_write, $data)
        WEnd

        $count = $count + 1
    WEnd
EndFunc

;~ ==========================================================================
;~ if Input 1 & mask 000, then output = 001 etc.
;~ ==========================================================================
Func _add_zeros($count, $mask)

    $result = $mask & $count

    $resLenght  = StringLen($result)
    $masklenght = StringLen($mask)
    $countlenght = StringLen($count)

    If $countlenght > $masklenght Then
        ConsoleWrite('Error: You have exeeded mask char lenght 000 > $mask' & ' ' & $result & @CRLF)
        Exit
    EndIf

    If $resLenght > $masklenght Then
        $CutLenght = $resLenght - $masklenght
            $result = StringTrimLeft($result, $CutLenght)
    EndIf

    Return $result
EndFunc

Edited August 8, 2010 by goldenix

[jaberwacky]

Works fine for me without any modifications.

I created two new files in the script dir and changed the extensions to 001, 002 such that I had two files named: "New Text document.001" with contents of "asdf" and "New Text document.002" with contents of "qwerty".

The result was "New Text Document" without an extension with contents, "asdfqwerty".

Is that expected behaviour?

A problem may be that you don't have admin rights?

Also, how many files do you want to open? A mere six files will equal 1800MB thats 1.8GB, other running programs are probably taking up the rest and more. Wait, nvm, I forgot that it fails on the first file...

I made a script that would make a large file (23MB but I got tired of waiting) and yeap sure nuff, don't work...

Ok, I coded this up so far. Try it when you get back...

expandcollapse popup

#AutoIt3Wrapper_AU3Check_Parameters=-w 1 -w 2 -w 3 -w 4 -w 6 -d

#RequireAdmin

;~ #include <Array.au3>
#include <File.au3>

HotKeySet("{ESC}", "term")

OnAutoItExitRegister("_Exit")

Global Const $STRMAX = 2147483647

Global $FileList = _FileListToArray(@ScriptDir, "*.00*", 1) ; files only

If @error = 4 Then
    MsgBox(0, "", "No Files Found.")
    Exit
EndIf

;~ _ArrayDisplay($FileList)

Global $zeros, $zefile, $file_read, $file_write, $data, $file, $result

;## loop counts number of files: 001,002 etc.
For $count = 1 To $FileList[0]

    $file = StringTrimRight($FileList[$count], 4) ; keep filename only
    $zeros = _add_zeros($count, '000')
    $zefile = $file & '.' & $zeros
    
    ConsoleWrite($zefile & @LF)

    $file_read = FileOpen($zefile, 0)
    $file_write = FileOpen($file, 2)

    While 1

        ToolTip('Reading: ' & $zefile, 0, 0)

        ; $STRMAX is the maximum amount of characters that a string can contain.
        $data = FileRead($file_read, $STRMAX)

        If @error = -1 Then ExitLoop

        $result = FileWrite($file_write, $data)

        If $result = 0 Then
            ConsoleWrite("There was an error writing to " & $file & '.' & @LF)
            Exit
        EndIf
    WEnd
Next

;~ ==========================================================================
;~ if Input 1 & mask 000, then output = 001 etc.
;~ ==========================================================================
Func _add_zeros(Const $count, Const $mask)

    Local $result = $mask & $count
    Local Const $maskLength = StringLen($mask)
    Local Const $countLength = StringLen($count)

    If $countLength > $maskLength Then
        ConsoleWrite('Error: You have exeeded mask char length 000 > $mask ' & $result & @LF)
        Exit
    EndIf

    Local Const $resLength = StringLen($result)

    If $resLength > $maskLength Then
        Local $CutLength = $resLength - $maskLength
        $result = StringTrimLeft($result, $CutLength)
    EndIf

    Return $result
EndFunc ;==>_add_zeros

Func term()
    Exit
EndFunc ;==>term

Func _Exit()
    If $file_read <> 0 Then FileClose($file_read)
    If $file_write <> 0 Then FileClose($file_write)
EndFunc

Is that closer to what you want?

Edited August 7, 2010 by jaberwocky6669

[goldenix]

The result was "New Text Document" without an extension with contents, "asdfqwerty".

Is that expected behaviour?

Yes its expected behavior cuz if before we have myfile.rar.001 & myfile.rar.002 then result must be myfile.rar

I removed more zeros from the end & kept only 9 chars -> 100000000 & it started working again. And the joining speed is not bad, I guess autoit reads whole file into ram first & only then enters the loop & starts writing the file. Guess I had too little free memory.

btw where did you get this logical number from: ? is the number of chars autoit reads at once, why exactely 2147483647?

$STRMAX = 2147483647

Another question, how many Bytes is 1 char?

Edited August 8, 2010 by goldenix

[Ascend4nt]

goldenix, look at the ranges listed in the chart in Language Reference - Datatypes. That's where you'll find the value 2147483647. As for the size of a character - 2 bytes for Unicode character, 1 byte for ANSI/ASCII characters.

[goldenix]

I still get error allocating memory with this magical number of yours, even tho i have over a gig free memory, but after I put back 100000000 instead 2147483647 it works. well ty anyway.