WeMartiansAreFriendly Posted June 29, 2009 Posted June 29, 2009 (edited) I have a 65 MB file and I'd like to read a number of bytes in increments until the end of file. Reading the whole thing at once is just not reasonable (memory and speed wise) Something like this.. the third parameter is what has been read so far, so it starts reading from there. pseudo code. :/ Local $iBuffer = 1024;read 1024 bytes or 1 kb at a time Local $sFile = FileOpen("test2.txt", 0) Local $iChunk = 0 Local $LastReadBytes = 0 ; Read a number bytes in increments of text until the EOF is reached While 1 $iChunk += $iBuffer $sChunk = FileRead($sFile, $iChunk, $LastReadBytes) $LastReadBytes = $iChunk If @error = -1 Then ExitLoop MsgBox(0, "Line read:", $sChunk) Wend FileClose($sFile) Any help is appreciated. Edited June 29, 2009 by Kastout Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()
Moderators SmOke_N Posted June 29, 2009 Moderators Posted June 29, 2009 Thanubis said: It sounds to me like you need the WinAPI functions. These four specifically._WinAPI_CreateFile - Open the File_WinAPI_SetFilePointer - Set the point where you want to start reading_WinAPI_ReadFile - Read the file_WinAPI_CloseHandle - Close the fileI don't believe _WinAPI_SetFilePointer is necessary there, you're reading, not writing. Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.
WeMartiansAreFriendly Posted June 29, 2009 Author Posted June 29, 2009 (edited) Ok. How can I accurately tell when I've reached the end of the file and stop reading?#include <WinAPI.au3> Local $sData = "" Local $sFile = "test2.txt" Local $iBufferSize = 1024;read 1024 bytes or 1 kb at a time Local $iBytesRead Local $iChunk = 0 Local $tBuffer = DllStructCreate("char["& $iBufferSize &"]");set buffer Local $pBuffer = DllStructGetPtr($tBuffer) Local $hFile = _WinAPI_CreateFile($sFile, 2, 2);open file Local $iSize = _WinAPI_GetFileSizeEx($hFile) While 1 $bRead = _WinAPI_ReadFile($hFile, $pBuffer, $iBufferSize, $iBytesRead) If Not $bRead Then ExitLoop $iChunk += $iBytesRead;this adds up to 841 not the full data If $iChunk >= $iSize then ExitLoop;<< if all the data has been read, i'd like to exit the loop $sData = BinaryToString(DllStructGetData($tBuffer, 1)) ConsoleWrite( $iBytesRead &")"&@lf& $sData &@lf) WEnd _WinAPI_CloseHandle($hFile)Here's the contents of test2.txt (a smaller example than 65 MB)CODE1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000 Edited June 29, 2009 by Kastout Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()
Moderators SmOke_N Posted June 29, 2009 Moderators Posted June 29, 2009 Did you ever check the value of $bRead ... if it fails exit and close the handle? Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.
WeMartiansAreFriendly Posted June 29, 2009 Author Posted June 29, 2009 SmOke_N said: Did you ever check the value of $bRead ... if it fails exit and close the handle?I just edited the code from my previous post. Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()
water Posted June 29, 2009 Posted June 29, 2009 A file read by fileopen/fileread/fileclose maintains its read position until closed. So just read on until you get @error = -1. This works fine for me:Local $iBuffer = 1024 ;read 1024 bytes or 1 kb at a time Local $sFile = FileOpen("test.txt", 0) ; Read a number bytes in increments of text until the EOF is reached While 1 $sChunk = FileRead($sFile, $iBuffer) If @error = -1 Then ExitLoop MsgBox(0, "Line read:", $sChunk) Wend MsgBox(0, "Line read:", $sChunk) ; Display alst read line FileClose($sFile) My UDFs and Tutorials: Reveal hidden contents UDFs: Active Directory (NEW 2024-07-28 - Version 1.6.3.0) - Download - General Help & Support - Example Scripts - Wiki ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts OutlookEX (2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki PowerPoint (2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki Task Scheduler (2022-07-28 - Version 1.6.0.1) - Download - General Help & Support - Wiki Standard UDFs: Excel - Example Scripts - Wiki Word - Wiki Tutorials: ADO - Wiki WebDriver - Wiki
WeMartiansAreFriendly Posted June 29, 2009 Author Posted June 29, 2009 (edited) Ok here's what I came up with.. It's ugly and bloated -- but after a few tests, I believe it works. Thanks for getting me on the right path. I'm sure someone will come up with a better solution expandcollapse popup#include <WinAPI.au3> Local $sData = "" Local $sFile = "test2.txt" Local $iBufferSize = 1024;read 1024 bytes or 1 kb at a time Local $iBytesRead Local $iChunk = 0 Local $tBuffer = DllStructCreate("char["& $iBufferSize &"]");set buffer Local $pBuffer = DllStructGetPtr($tBuffer) Local $hFile = _WinAPI_CreateFile($sFile, 2, 2);open file Local $iSize = _WinAPI_GetFileSizeEx($hFile) Local $bEOF = False ConsoleWrite("Filesize: " & $iSize &" bytes = ") ConsoleWrite(Ceiling($iSize / $iBufferSize) &" x "& $iBufferSize &" chunks " & @LF) While 1 $bRead = _WinAPI_ReadFile($hFile, $pBuffer, $iBufferSize, $iBytesRead) If Not $bRead Then ExitLoop If ($iChunk + $iBytesRead) >= $iSize then ; reached end of file -- trim off the excess fat $sData = DllStructGetData($tBuffer, 1) $sData = BinaryMid($sData, 1, ($iSize-$iChunk));keep the remaining bytes $sData = BinaryToString($sData) $bEOF = True Else ; deliver the data $iChunk += $iBytesRead $sData = BinaryToString(DllStructGetData($tBuffer, 1)) EndIf ConsoleWrite($sData) If $bEOF Then ExitLoop WEnd _WinAPI_CloseHandle($hFile) Edited June 29, 2009 by Kastout Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()
WeMartiansAreFriendly Posted June 29, 2009 Author Posted June 29, 2009 (edited) water said: A file read by fileopen/fileread/fileclose maintains its read position until closed. So just read on until you get @error = -1. This works fine for me:Local $iBuffer = 1024 ;read 1024 bytes or 1 kb at a time Local $sFile = FileOpen("test.txt", 0) ; Read a number bytes in increments of text until the EOF is reached While 1 $sChunk = FileRead($sFile, $iBuffer) If @error = -1 Then ExitLoop MsgBox(0, "Line read:", $sChunk) Wend MsgBox(0, "Line read:", $sChunk) ; Display alst read line FileClose($sFile) Well now.. that couldn't be any more simpler. It's even in the help file (I'm more senile then I thought, nice.) Thanks XD Edited June 29, 2009 by Kastout Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now