AutID Posted August 20, 2013 Share Posted August 20, 2013 (edited) Hello,I have a database of 36mb in a .txt file and it is binary. I am trying to read the file and then write it in another file returned in ANSI.However I am having an error saying this.It's an AutoIt msgbox saying "Error allocating memory".the code is this:FileWrite($file, BinaryToString($read))Any ideas?CheersEDIT:Hmmmm, it could be the File functions giving this error. ConsoleWrite gives a good return...sometimes...Filewrite gives a true return but it doesn't write the read from the previous file... weird, it doesn't work everytime though... Edited August 20, 2013 by AutID https://iblockify.wordpress.com/ Link to comment Share on other sites More sharing options...
czardas Posted August 20, 2013 Share Posted August 20, 2013 (edited) There's some information >here. There are probably a few questions like this on the forum, so do a search. Edited August 20, 2013 by czardas operator64 ArrayWorkshop Link to comment Share on other sites More sharing options...
AutID Posted August 20, 2013 Author Share Posted August 20, 2013 I already looked at these posts. And other similar. I have a simple consowrite without the binary return on scite running and it is still running 25 minutes now... https://iblockify.wordpress.com/ Link to comment Share on other sites More sharing options...
czardas Posted August 20, 2013 Share Posted August 20, 2013 (edited) Well perhaps writing so much data to Scite console is not a good idea. I just did a quick test on multiple writes to a binary file and it worked fine. First try to achieve what you want with a small file and if it works then step it up to the big test. I split the binary using the following function after removing "0x" prefix. Then I wrote 512 bytes at a time to a file using a For loop. ; Func _StringEqualSplit($sString, $iNumChars) ; Returns a 0 based array If Not IsString($sString) Or $sString = "" Then Return SetError(1, 0, 0) If Not IsInt($iNumChars) Or $iNumChars < 1 Then Return SetError(2, 0, 0) Return StringRegExp($sString, "(?s).{1," & $iNumChars & "}", 3) EndFunc ; _StringEqualSplit ; I haven't tried the method on a large file. I hope this information helps. Edited August 20, 2013 by czardas operator64 ArrayWorkshop Link to comment Share on other sites More sharing options...
jchd Posted August 20, 2013 Share Posted August 20, 2013 (edited) AutId, Unless you machine is really very short in memory I don't believe it could be that instruction causing the error. Post your code, the error is most probably somewhere else. Edited August 20, 2013 by jchd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
trancexx Posted August 20, 2013 Share Posted August 20, 2013 That would be weird but sometimes it's possible. Like jchd says the error could be caused by something else. For example this simple code can cause issues depending how much RAM your system has: DllStructCreate("byte[1234567865465]") Try lowering the number to get rough idea of how much is max for your script. ♡♡♡ . eMyvnE Link to comment Share on other sites More sharing options...
czardas Posted August 20, 2013 Share Posted August 20, 2013 I was doubting myself about this after I ran further tests this morning. Apologies for misleading anyone. operator64 ArrayWorkshop Link to comment Share on other sites More sharing options...
AutID Posted August 20, 2013 Author Share Posted August 20, 2013 8GB RAM#include <File.au3> $file = _FileCreate(@ScriptDir & "\database.txt") $read = FileRead(@ScriptDir & "\database 2012.txt") $write = FileWrite($file, BinaryToString($read))code is very simple. I changed the .sql file of the database to .txt but problem is still the same.As I looked around people having similar problems, File functions are propably causing this error. If instead of using FileWrite I do a consolewrite then it works fine but it take about 2 hours to consolewrite the entire file. When I change the .sql to .txt the txt file became 261 MB... https://iblockify.wordpress.com/ Link to comment Share on other sites More sharing options...
FireFox Posted August 20, 2013 Share Posted August 20, 2013 You open the file in text read mode, not in binary mode. Link to comment Share on other sites More sharing options...
FireFox Posted August 20, 2013 Share Posted August 20, 2013 $read = FileRead(@ScriptDir & "\database 2012.txt") $write = FileWrite($file, $read) ;OR $hFile = FileOpen(@ScriptDir & "\database 2012.txt", $FO_BINARY) $read = FileRead($hFile) FileClose($hFile) $write = FileWrite($file, BinaryToString($read)) Link to comment Share on other sites More sharing options...
AutID Posted August 20, 2013 Author Share Posted August 20, 2013 (edited) You open the file in text read mode, not in binary mode.I also tried a fileopen to force in a binary mode* but the same error message comes directly... Edited August 20, 2013 by AutID https://iblockify.wordpress.com/ Link to comment Share on other sites More sharing options...
FireFox Posted August 20, 2013 Share Posted August 20, 2013 You will have to process the file by chunks. Link to comment Share on other sites More sharing options...
jchd Posted August 20, 2013 Share Posted August 20, 2013 (edited) I don't get it. Either your file is a genuine .sql file, that is a text file in which case you don't have to process it as binary, or it is a true database and I wonder what processing you intend to apply to it. Nevertheless, FileRead-ing a 36Mb binary shouldn't burst in you face. Which AutoIt version are you using and do you run for x86 or x64? BTW, this seems to work flawlessly over a 36Mb binary file (an old SQLite database, actually) and produces a ... 36 Mb file (unsurprisingly): $hread = FileOpen(@ScriptDir & "\test.db3", 16) $read = FileRead($hread) FileClose($hread) $hfile = FileOpen(@ScriptDir & "\binarbug.bin", 2) $write = FileWrite($hfile, BinaryToString($read)) $read = 0 FileClose($hfile) The whole process is nothing more than a sophisticated NOP operation, or rather an overly complexified file copy, BTW. May I ask what the actual need is? Edited August 20, 2013 by jchd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
AutID Posted August 20, 2013 Author Share Posted August 20, 2013 I don't get it. Either your file is a genuine .sql file, that is a text file in which case you don't have to process it as binary, or it is a true database and I wonder what processing you intend to apply to it. Nevertheless, FileRead-ing a 36Mb binary shouldn't burst in you face. Which AutoIt version are you using and do you run for x86 or x64? BTW, this seems to work flawlessly over a 36Mb binary file (an old SQLite database, actually) and produces a ... 36 Mb file (unsurprisingly):$hread = FileOpen(@ScriptDir & "\test.db3", 16) $read = FileRead($hread) FileClose($hread) $hfile = FileOpen(@ScriptDir & "\binarbug.bin", 2) $write = FileWrite($hfile, BinaryToString($read)) $read = 0 FileClose($hfile)The whole process is nothing more than a sophisticated NOP operation, or rather an overly complexified file copy, BTW. May I ask what the actual need is?the .sql file was 36MB. As soon as I opened and saved it in Notepad it turned to 261MB. As I am understanding the filewrite times out or the data written in the FileWrite function is greater than it is supposed to be.It is a true database.ConsoleWrite managed to turn it to string (BinaryToString) and write it after some hours. I thought I could then copy it from the consolewrite and paste it in a file manually but copy doesn't seem to work on the consolewrite because the amount of the lines... 800k+ lines of consolewrite...I will try split the database in parts and try again. https://iblockify.wordpress.com/ Link to comment Share on other sites More sharing options...
BrewManNH Posted August 20, 2013 Share Posted August 20, 2013 Why are you trying to read a DB file using FileRead, you're not going to get much usable information out of doing that. If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays. - ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script. - Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label. - _FileGetProperty - Retrieve the properties of a file - SciTE Toolbar - A toolbar demo for use with the SciTE editor - GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI. - Latin Square password generator Link to comment Share on other sites More sharing options...
jchd Posted August 20, 2013 Share Posted August 20, 2013 (edited) AutId, Instead of talking about what you think it does wrong, did you try my code with your own 36 Mb file (fairly similar to the one I used)? Did you think twice about what it actually does (hint: nothing but copy the file verbatim)? Did you bother to even explain in plain english what you really want to achieve? Edited August 20, 2013 by jchd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now