BinaryToString() Error

[AutID]

Hello,

I have a database of 36mb in a .txt file and it is binary. I am trying to read the file and then write it in another file returned in ANSI.

However I am having an error saying this.

It's an AutoIt msgbox saying "Error allocating memory".

the code is this:

FileWrite($file, BinaryToString($read))

Any ideas?

Cheers

EDIT:

Hmmmm, it could be the File functions giving this error. ConsoleWrite gives a good return...sometimes...

Filewrite gives a true return but it doesn't write the read from the previous file... weird, it doesn't work everytime though...

Edited August 20, 2013 by AutID

[czardas]

There's some information >here. There are probably a few questions like this on the forum, so do a search.

Edited August 20, 2013 by czardas

[AutID]

I already looked at these posts. And other similar. I have a simple consowrite without the binary return on scite running and it is still running 25 minutes now...

[czardas]

Well perhaps writing so much data to Scite console is not a good idea. I just did a quick test on multiple writes to a binary file and it worked fine. First try to achieve what you want with a small file and if it works then step it up to the big test. I split the binary using the following function after removing "0x" prefix. Then I wrote 512 bytes at a time to a file using a For loop.

;

Func _StringEqualSplit($sString, $iNumChars) ; Returns a 0 based array
    If Not IsString($sString) Or $sString = "" Then Return SetError(1, 0, 0)
    If Not IsInt($iNumChars) Or $iNumChars < 1 Then Return SetError(2, 0, 0)
    Return StringRegExp($sString, "(?s).{1," & $iNumChars & "}", 3)
EndFunc ; _StringEqualSplit

;

I haven't tried the method on a large file. I hope this information helps.

Edited August 20, 2013 by czardas

[jchd]

AutId,

Unless you machine is really very short in memory I don't believe it could be that instruction causing the error.

Post your code, the error is most probably somewhere else.

Edited August 20, 2013 by jchd

[trancexx]

That would be weird but sometimes it's possible. Like jchd says the error could be caused by something else. For example this simple code can cause issues depending how much RAM your system has:

DllStructCreate("byte[1234567865465]")

Try lowering the number to get rough idea of how much is max for your script.

[czardas]

I was doubting myself about this after I ran further tests this morning. Apologies for misleading anyone.

[AutID]

8GB RAM

#include <File.au3>

$file = _FileCreate(@ScriptDir & "\database.txt")
$read = FileRead(@ScriptDir & "\database 2012.txt")
$write = FileWrite($file, BinaryToString($read))

code is very simple. I changed the .sql file of the database to .txt but problem is still the same.

As I looked around people having similar problems, File functions are propably causing this error. If instead of using FileWrite I do a consolewrite then it works fine but it take about 2 hours to consolewrite the entire file. When I change the .sql to .txt the txt file became 261 MB...

[FireFox]

You open the file in text read mode, not in binary mode.

[FireFox]

$read = FileRead(@ScriptDir & "\database 2012.txt")
$write = FileWrite($file, $read)

;OR
$hFile = FileOpen(@ScriptDir & "\database 2012.txt", $FO_BINARY)
$read = FileRead($hFile)
FileClose($hFile)
$write = FileWrite($file, BinaryToString($read))

[AutID]

You open the file in text read mode, not in binary mode.

I also tried a fileopen to force in a binary mode* but the same error message comes directly...

Edited August 20, 2013 by AutID

[FireFox]

You will have to process the file by chunks.

[jchd]

I don't get it. Either your file is a genuine .sql file, that is a text file in which case you don't have to process it as binary, or it is a true database and I wonder what processing you intend to apply to it.

Nevertheless, FileRead-ing a 36Mb binary shouldn't burst in you face.  Which AutoIt version are you using and do you run for x86 or x64?

BTW, this seems to work flawlessly over a 36Mb binary file (an old SQLite database, actually) and produces a ... 36 Mb file (unsurprisingly):

$hread = FileOpen(@ScriptDir & "\test.db3", 16)
$read = FileRead($hread)
FileClose($hread)
$hfile = FileOpen(@ScriptDir & "\binarbug.bin", 2)
$write = FileWrite($hfile, BinaryToString($read))
$read = 0
FileClose($hfile)

The whole process is nothing more than a sophisticated NOP operation, or rather an overly complexified file copy, BTW.

May I ask what the actual need is?

Edited August 20, 2013 by jchd

[AutID]

I don't get it. Either your file is a genuine .sql file, that is a text file in which case you don't have to process it as binary, or it is a true database and I wonder what processing you intend to apply to it.

 

Nevertheless, FileRead-ing a 36Mb binary shouldn't burst in you face.  Which AutoIt version are you using and do you run for x86 or x64?

 

BTW, this seems to work flawlessly over a 36Mb binary file (an old SQLite database, actually) and produces a ... 36 Mb file (unsurprisingly):

$hread = FileOpen(@ScriptDir & "\test.db3", 16)
$read = FileRead($hread)
FileClose($hread)
$hfile = FileOpen(@ScriptDir & "\binarbug.bin", 2)
$write = FileWrite($hfile, BinaryToString($read))
$read = 0
FileClose($hfile)

The whole process is nothing more than a sophisticated NOP operation, or rather an overly complexified file copy, BTW.

 

May I ask what the actual need is?

the .sql file was 36MB. As soon as I opened and saved it in Notepad it turned to 261MB. As I am understanding the filewrite times out or the data written in the FileWrite function is greater than it is supposed to be.

It is a true database.

ConsoleWrite managed to turn it to string (BinaryToString) and write it after some hours. I thought I could then copy it from the consolewrite and paste it in a file manually but copy doesn't seem to work on the consolewrite because the amount of the lines... 800k+ lines of consolewrite...

I will try split the database in parts and try again.

[BrewManNH]

Why are you trying to read a DB file using FileRead, you're not going to get much usable information out of doing that.

[jchd]

AutId,

Instead of talking about what you think it does wrong, did you try my code with your own 36 Mb file (fairly similar to the one I used)?

Did you think twice about what it actually does (hint: nothing but copy the file verbatim)?

Did you bother to even explain in plain english what you really want to achieve?

Edited August 20, 2013 by jchd