Recently Browsing 0 members
No registered users viewing this page.
(Edited from original. Please note that I AM NOT AN AUTOIT EXPERT. I write code using Autoit frequently but I am no expert, especially when it comes to I/O. So any remarks that start with "Why did you..." can be answered by referring to the first sentence. This project was done in Autoit because of an interface I built to display the data.)
Attached is a program and ascii input file I wrote to read stock price data, convert it to binary and then read it back into the program in binary. The goal was to show increased performance for reading the files in binary and provide a demo on how to read/write binary for int32, int64, double and strings for anyone who might find it helpful. The results on my PC show the following:
Time to read ascii file only: 456.981951167202
Ascii read & process time: 6061.83075631701
Binary write file time: 14787.9184635239
Time just to read binary file: 42.418867292311
Binary read and process time: 4515.16129830537
A couple things to note:
1) The 32 MB ascii file took 10x longer to read than the 15 MB binary file. Not entirely sure why. Both were read into a buffer.
2) The Binary write takes a long time but I made no effort to optimize this because the plan was to write this file one time only so I don't mind if it takes longer to write this file. I care much more about how long it takes to read the file because I will be reading it many times.
3) There was a modest gain in converting the ascii file to binary in terms of file size and reading speed.
So big picture... not sure it's worth the effort to convert the files to binary even though most of the data is numerical data in the binary file. That was actually surprising as I expected there would be more of a difference. Any ideas on how to get the binary data to read at a faster rate would be great.
I'm trying to read a binary file to an array but couln't get it to work.
Also I coul not find any help in the forum around this subject whish was helpfull.
Is there any way it could be done?
I tried a lot of ways but maybe somebody know's the right way?
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 #include <File.au3> #include <Array.au3> #include <AutoItConstants.au3> Local $in=FileOpen("TEST_labels.idx1-ubyte",16) ; 16+0=Read binary Local $data = FileRead($in) Local $FileArray = BinaryToString($data,4) ;~ $FileArray = StringSplit($BinarydData, @CRLF, 1+2) ;~ Local $FileArray = StringRegExp($BinarydData, "[^\r\n]+", 3) FileClose($in) _ArrayDisplay($FileArray,"$FileArray","",32) MsgBox($MB_SYSTEMMODAL, "", "$FileArray = " & $FileArray )
I'm too stupid to see my mistake:
To investigate the internal "dictionary" of TIFF files I'd like to read in the files in binary mode and to check, if there are more than one pages "in" this TIFF.
Notepad++, "View as Hex" is presenting the first bytes as "49 49 2a 20 08 20 20 20 12" for the TIF attached to this posting
The "TIFF Header Format" is easy:
Offset 00h, 2 Byte = Byte Order, "II"=intel, "MM"=motorola. (I = 0x49)
Offset 02h, 2 Byte = Version Nr.
Offset 04h, 4 Byte = pointer to first IFD entry
Description of TIFF header: https://www.awaresystems.be/imaging/tiff/faq.html#q3
Howto read and analyse the binary content correctly? This is my messy, not operational code:
$sampleTiff="H:\daten\tif\11\11\111111.TIF" $h=FileOpen($sampleTiff,16) $content=FileRead($h) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = ' & $content & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console FileClose($h) $type=VarGetType($content) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $type = ' & $type & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console $ToString=BinaryToString($content) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ToString = ' & $ToString & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console ConsoleWrite(@CRLF & @CRLF) $content=StringTrimLeft($content,2) ; cut off the leading "0x" ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = ' & $content & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console for $i = 1 to 8 step 8 $next=StringMid($content,$i,2) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $next = ' & $next & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console $Chr=BinaryToString($next) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $Chr = ' & $Chr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console ConsoleWrite(@CRLF & "---" & @CRLF) Next Regards, Rudi.
It's now possible to see the best compression ratio using LZMA, LZNT and Base64 compressions with differents combinations.
Nothing too complicate, you drag'n drop a file on the picture and script Test all compression types and return the ratios.
( Test duration depends of file size, slowest compression is LZNT, but all decompressions are fast )
Free to you after, to choose the compression(s) you want...
Yes, LZMA needs a dll ( embedded & compressed in script ) but brings a powerfull compression.
It opens scite with your file compressed to an au3 script with or without decompression function as you want.
Hold Left Shift key when clicking button for just copy script to clipboard.
Use the 3 compressions at a time works but doesn't give a good ratio, that's why i don't display it.
Usefull for little files you want include in your scripts !
No externals files needed, they are already in script.
Previous downloads : 1103
Source and Executable
BinaryToAu3Kompressor will be added to the next version of >SciTEHopper
Thanks to Ward for his >Base64.au3 and LZMA.au3, and trancexx for his >LZNT functions and his >Base64Decode function.