Sign in to follow this  
Followers 0
smashly

File split ?

8 posts in this topic

I'm trying to read a file over 350mb.

I only need to read the first 7000000 characters at the beginning of the file and I have no problem.

But trying to read the characters at the end of a file I'm having no success.

Is there an easy method to either split my input file to more feasable sizes to read (without reading the whole file begiining to end)?

or a way to read a file from the end backwards (without reading the whole file from beginning end) ?

Any input is apreciated.

Yes I have been reading the Autoit help file as well as forum searches but I'm no closer then when I started researching :/ (started 4 days ago)

Thank You

N

Cheers

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

350MB is quite a loot..:P Is it plain text?

Think you have to dig up some of the File UDF's that are lying around. I know I have seen some signatures with file udf's in them. Or wrap up the native API's yourself.

If _FileCountLines are fast enough you might use something like this?

NOTE: I have not tested the code.

Func FileTail($file, $lines)
Local $count = _FileCountLines($file)
Local $i, $data
For $i = $count - $lines to $count
    $data += filereadline($file, $count)
    If @error then ExitLoop
Next
Return $data
EndFunc

EDIT: Added $data declaration.

Edited by Uten

Share this post


Link to post
Share on other sites

Thank You for the response Uten.

I have tried _FileCountLines udf, but it fails as the file is to big I gather.

I was reading binary data (even though FileRead says it's only for text), I find FileRead works fine for binary data as long as the file being read isn't to big.

I was extracting *.png files out of a file that can be anything from 100MB upto 1.6GB by using the header and footer of the *.png and using StringInStr, StringTrimLeft or Right and writing the data to a new file (new png is only about 200-300kb)

As the file strucure of the files I'm working with only have the *.png's in the first 7000000 characters and in the last 7000000 characters of the file and the data in the middle is redundant to me, I was hoping there was a way to read from the end back without actually having to parse the input file from the beginning.

Gather I'll have to look into api udfs abit more to get it sussed out.

PS. I'm still using your tray TrayMenuEx.au3 udf, one of my favorite udfs :P

Thank You

Cheers

Share this post


Link to post
Share on other sites

Hi,

I think the newest version of my Tail works;

eg this reads the thirds of a bmp in any order, then puts the splices back together;

Best, Randall

;apitry.au3
#include "APITailRW.au3"
Local $sDeskFile = @DesktopDir & "\Rhododendron.bmp", $s_File = @WindowsDir & "\Rhododendron.bmp"
FileDelete($sDeskFile)
$i_FileSize = FileGetSize($s_File)
;======Read parts or tail of binary file =====in any order=================================
$h_File = _FileOpenAPI ($s_File) ;_APIFileOpen
$s_data3 = _FileReadAPI ($h_File, Int($i_FileSize / 3) + 2, 2 * Int($i_FileSize / 3), 1)    ; read 3rd third file
$s_data2 = _FileReadAPI ($h_File, Int($i_FileSize / 3), Int($i_FileSize / 3), 1)            ; read 2nd third file
$s_data1 = _FileReadAPI ($h_File, Int($i_FileSize / 3), 0, 1)                               ; read 1st third file
$s_data = $s_data1 & $s_data2 & $s_data3                                                    ; concatenate in correct order
_FileCloseAPI ($h_File);_APIFileClose
;======Save binary file to desktop and open======================================
$ha_File = FileOpen($sDeskFile, 18)
FileWrite($ha_File, $s_data)
FileClose($ha_File)
ShellExecuteWait($sDeskFile)
FileDelete($sDeskFile)
;============================================

Share this post


Link to post
Share on other sites

Hi,

I think the newest version of my Tail works;

eg this reads the thirds of a bmp in any order, then puts the splices back together;

Best, Randall

;apitry.au3
#include "APITailRW.au3"
Local $sDeskFile = @DesktopDir & "\Rhododendron.bmp", $s_File = @WindowsDir & "\Rhododendron.bmp"
FileDelete($sDeskFile)
$i_FileSize = FileGetSize($s_File)
;======Read parts or tail of binary file =====in any order=================================
$h_File = _FileOpenAPI ($s_File) ;_APIFileOpen
$s_data3 = _FileReadAPI ($h_File, Int($i_FileSize / 3) + 2, 2 * Int($i_FileSize / 3), 1)    ; read 3rd third file
$s_data2 = _FileReadAPI ($h_File, Int($i_FileSize / 3), Int($i_FileSize / 3), 1)            ; read 2nd third file
$s_data1 = _FileReadAPI ($h_File, Int($i_FileSize / 3), 0, 1)                               ; read 1st third file
$s_data = $s_data1 & $s_data2 & $s_data3                                                    ; concatenate in correct order
_FileCloseAPI ($h_File);_APIFileClose
;======Save binary file to desktop and open======================================
$ha_File = FileOpen($sDeskFile, 18)
FileWrite($ha_File, $s_data)
FileClose($ha_File)
ShellExecuteWait($sDeskFile)
FileDelete($sDeskFile)
;============================================
Thanks RandallC! I'm not the original poster but you helped me out a ton also. I am working on a utility to split MP3's using CUE files and tagging them at the same time (mostly to help organize my massive music collection) and after the new binary functions were added my FileOpen (for binary files) pretty much stopped working with the new version 3.2.4.9. Your (and others) code saved the day..

Thanks again!

-J

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0