(Edited from original. Please note that I AM NOT AN AUTOIT EXPERT. I write code using Autoit frequently but I am no expert, especially when it comes to I/O. So any remarks that start with "Why did you..." can be answered by referring to the first sentence. This project was done in Autoit because of an interface I built to display the data.)
Attached is a program and ascii input file I wrote to read stock price data, convert it to binary and then read it back into the program in binary. The goal was to show increased performance for reading the files in binary and provide a demo on how to read/write binary for int32, int64, double and strings for anyone who might find it helpful. The results on my PC show the following:
Time to read ascii file only: 456.981951167202
Ascii read & process time: 6061.83075631701
Binary write file time: 14787.9184635239
Time just to read binary file: 42.418867292311
Binary read and process time: 4515.16129830537
A couple things to note:
1) The 32 MB ascii file took 10x longer to read than the 15 MB binary file. Not entirely sure why. Both were read into a buffer.
2) The Binary write takes a long time but I made no effort to optimize this because the plan was to write this file one time only so I don't mind if it takes longer to write this file. I care much more about how long it takes to read the file because I will be reading it many times.
3) There was a modest gain in converting the ascii file to binary in terms of file size and reading speed.
So big picture... not sure it's worth the effort to convert the files to binary even though most of the data is numerical data in the binary file. That was actually surprising as I expected there would be more of a difference. Any ideas on how to get the binary data to read at a faster rate would be great.
I had a need, and so I developed.
Before I did that though, I did a quick search of the forum, and found a few things, but none seemed to suit what I really wanted ... though to be honest, I did not investigate them deeply or search extensively. In any case, I am not very good at modifying the code of others, nor do I enjoy doing so, so in the end I decided it was best to start from scratch ... re-inventing the wheel perhaps, but a wheel I feel a certain degree of comfort with. I also thought about Maps, but don't know enough.
_IniString Functions (was Ini_InMem)
Ini in memory
Settings INI - Read/Write once to the disk
INI File Processing Functions
... sure to be others.
As with all my stuff, feel free to use (no guarantees) or modify ... just give me credit where due.
I always go for the quick & simple approach, as those who know me here, are well aware, so without a doubt, my code could be improved - sped up, RegExp used, etc. So feel free to do that, and convert into a proper UDF, if you want, following best practices etc, as I am sure others will appreciate it.
Recently, I have come upon a need to extract data from an approximately 50 Mb downloaded Index file, just extracting the few elements I need, and storing them in an INI file. It is quite a time consuming process, especially on my underpowered Netbook, which is where I am using the program I built. I have been considering ways to speed up things. Two good sounding methods, out of several possibles, have come to mind - 1) RAM Disk and 2) Virtual INI processing. That last being a better option, when sharing code (program) with others.
Another method, which I am currently using, that did give me some measurable benefit, was to download the almost 50 Mb Index file in Stages (i.e. 30 x 1.4 Mb roughly), extracting data from each in turn. I have also considered extracting to 10 (or 11) separate INI files, rather than the single large INI file (4~5 Mb), as things noticeably slow up as the INI file increases in size ... obviously due to writing time.
I can't really speed up the extraction process, but that last method may speed up the writing stage. However, it would require significant program changes, and perhaps not gain me much benefit. To complicate things, there are too many Section names (Index ID's) for the IniReadSectionNames command, so I have had to split them off into their own (read/write) line-by-line index file, and create code to deal with duplicates. When read, Index ID's and Titles populate a Listbox control in a user GUI.
So at this point in time, it seems best to use the Virtual INI approach to gain a significant reduction in time taken. For my own specific use, with my slow Netbook, I may also use a RAM Disk.
Due to the aforementioned INI issues, and lack of available specific data, probably because it is always program concentric, I have created an ASCII Checker program, which I guess you can say, is AutoIt concentric. Because others may also find it handy, I am providing it here. NOTE - As yet, I am only aware that Escape characters exist, but have not investigated or catered for them in any way.
The Virtual_INI_UDF is still a work in progress, though I have done most of the functions now, with the working but incomplete _Ini_Test function, still requiring changes etc from the information gleaned by the ASCII Checker program.
Here's something to play with meanwhile.
Updated files (24th February 2017)
Example.au3 (23rd February 2017)
All pretty basic, but managed to load my 4 Mb INI file (seemingly ok, but took a few seconds) ... though I have only worked with much smaller testing ones so far.
P.S. While I have done a fair amount of testing, it has been pretty basic, and mostly limited to the examples provided. Testing has definitely not been extensive.
Bitmap2AscII use Lucida Console font with a size set to 8, so Windows 8/8.1 users need to change their notepad font and size for get a correct display.
Image Rescale slider is only available when saving as image.
A click on the "PreviewEdit" open AscII string in Notepad.
You can save as Text, Html or Image (add the extension you want)
Each setting change is immediately applied.
Downloads available in the download section
Hope you like it !
I do not like Charmap, it's too small and you can't test a character with another font, you only get all chars for each font so it's not very handy.
SpecialCharactersViewer permit with the Segoe UI Symbol font to display a maximum ( not all! ) of Ascii and Unicode characters.
Simple click on a char and the corresponding Chr or ChrW code is displayed.
Free to you to choose another font for see if the selected character can be used with.
Windows XP do not have Segoe UI Symbol font, so it's more for Win7/Win8 users...
Previous downloads : 100
Source : SpecialCharactersViewer v18.104.22.168.au3
Executable : SpecialCharactersViewer.exe.html
(Once this html file downloaded, double click on it for start the download)
Will be added to the next version of SciTE Hopper.
Hope it can help !