footswitch Posted September 6, 2007 Share Posted September 6, 2007 (edited) Hello folks!So... the title says it all...I have, let's say, a 100 MB binary file.How to remove (for instance) the first 512 bytes of that file? And leave the rest of the file intact.Can someone help me on this?Any time and/or effort is much appreciated.Thanks in advance! footswitch Edited September 7, 2007 by footswitch Link to comment Share on other sites More sharing options...
evilertoaster Posted September 6, 2007 Share Posted September 6, 2007 You will need to use FileRead() to read the file in in any case. psudo code might be somthing like- FileWrite("output",StringTrimLeft(FileRead("input",4),512)) I'm a little rusty on the current state of autoit's binary reading/writing but something like that should work Link to comment Share on other sites More sharing options...
footswitch Posted September 7, 2007 Author Share Posted September 7, 2007 You will need to use FileRead() to read the file in in any case. psudo code might be somthing like- FileWrite("output",StringTrimLeft(FileRead("input",4),512)) I'm a little rusty on the current state of autoit's binary reading/writing but something like that should workYou're evil If you insist so much in reading the whole file, then I must say... the file is 2GB long. Now what? Stick to the FileRead? Lol don't take me wrong, but if I wanted to use FileRead() I'd use it and wouldn't ask how to do this without having to copy the whole file to memory. Thanks for your attention though. Regards footswitch Link to comment Share on other sites More sharing options...
MHz Posted September 7, 2007 Share Posted September 7, 2007 You're evil If you insist so much in reading the whole file, then I must say... the file is 2GB long. Now what? Stick to the FileRead?Lol don't take me wrong, but if I wanted to use FileRead() I'd use it and wouldn't ask how to do this without having to copy the whole file to memory. Thanks for your attention though.RegardsfootswitchIt looks like FileRead() in the example by evilertoaster only does the 1st 4 chars. Link to comment Share on other sites More sharing options...
footswitch Posted September 7, 2007 Author Share Posted September 7, 2007 (edited) It looks like FileRead() in the example by evilertoaster only does the 1st 4 chars. His answer isn't a solution. His intent was, I think, to open the file in binary read mode and then extract 512 bytes from it. You need to open the file in binary read mode first. So the correct coding would be (please note that I could be wrong with these +1 and -$chunk_size, but it must be close to this): ; this will read a file to memory, remove the first 256 bytes from it and save the result to a new file $file_src="sourcefile.bin" ; source file $file_dest="destfile.bin" ; destination file (chunked) $chunk_size=256 ; number of bytes to remove, thus the $start value for BinaryMid() (+1 to [b]start[/b], for instance, in byte no. 257) $count=FileGetSize($file_src)-$chunk_size ; this will be the new filesize, thus the $count value for BinaryMid() $handle_src=FileOpen($file_src,16) ; open in binary read mode $handle_dest=FileOpen($file_dest,16+2) ; open new file in binary WRITE mode - erase previous contents if they exist FileWrite($handle_dest,BinaryMid(FileRead($handle_src),$chunk_size+1,$count)) ; read the whole file with FileRead() and return chunked file with BinaryMid(), writing it to the new file with FileWrite() ; close files after using them FileClose($handle_src) FileClose($handle_dest) Just to remind you, this can't be done with a 2GB file EDIT: Some more explanations introduced Edited September 7, 2007 by footswitch Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted September 7, 2007 Moderators Share Posted September 7, 2007 You're only native option then would be to read it line by line... 2GB file... Yeah, see you next week Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
PsaltyDS Posted September 7, 2007 Share Posted September 7, 2007 Looked interesting, so I gave it shot with a demo. The input file was 512MB in the test and took 42 seconds to break it into six files: Dim $sRead = "C:\Temp\InputFile.bin", $iFile = 0, $EOF = 0, $OneMeg = 1024 * 1024, $Timer = TimerInit() $hRead = FileOpen($sRead, 0 + 16) ; Binary read Do $iFile += 1 $hWrite = FileOpen($sRead & ".Part" & $iFile, 2 + 16) ; Binary overwrite For $n = 1 To 100 $binData = FileRead($hRead, $OneMeg) ; 1MB blocks $EOF = @error FileWrite($hWrite, $binData) If $EOF Then ExitLoop Next FileClose($hWrite) Until $EOF FileClose($hRead) $Timer = TimerDiff($Timer) $sMsg = "Completed breaking up file " & $sRead & " into 100MB parts." & @CRLF $sMsg &= Round(FileGetSize($sRead) / $OneMeg, 2) & "MB file broken up into " & $iFile & " parts." & @CRLF $sMsg &= "Processed in " & Round($Timer / 1000, 2) & " seconds." MsgBox(64, "Finished", $sMsg) Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
ptrex Posted September 7, 2007 Share Posted September 7, 2007 @all Something from old box : expandcollapse popup; File Splitter 1.0 = Origional Script ; -by Andrew Dunn (Hallman) ; File Splitter 1.1 = GUI modified, added functions ; -smashly ; New Progress Bars, modified split/join function ; -Hallman ; Fixed Broken drop files & hide n show gui for join & resized/moved progress window. ; -smashly #include <GUIConstants.au3> #include <file.au3> Opt("GUIOnEventMode", 1) ;1 = enable $apw = 330 $aph = 134 $MainWindow = GUICreate("File Splitter 1.1", $apw, $aph, @DesktopWidth - $apw - 10, @DesktopHeight - $aph - 65, -1, $WS_EX_ACCEPTFILES) WinSetOnTop($MainWindow, "", 1) ; Set the gui as always on top GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit") GUICtrlCreateGroup("Drop or Browse for a file to Split or Join", 10, 10, 310, 114) $InFile = GUICtrlCreateInput("", 20, 30, 190, 20) $OutDirectory = GUICtrlCreateInput("", 20, 60, 190, 20) $InBrowse = GUICtrlCreateButton("Input File", 220, 30, 90, 20) GUICtrlSetOnEvent($InBrowse, "_handler") $OutBrowse = GUICtrlCreateButton("Output Directory", 220, 60, 90, 20) GUICtrlSetOnEvent($OutBrowse, "_handler") $SplitInto = GUICtrlCreateLabel("Split File Into:", 22, 93, 64, 20) $InSize = GUICtrlCreateInput("100", 90, 90, 60, 20) $MB = GUICtrlCreateCombo("", 160, 90, 50, 21) GUICtrlSetData($MB, "MB|KB|B", "MB") $SplitJoin = GUICtrlCreateButton("", 220, 90, 90, 20) GUICtrlSetOnEvent($SplitJoin, "_handler") GUICtrlSetState($InFile, $GUI_DROPACCEPTED) GUICtrlSetState($OutDirectory, $GUI_DROPACCEPTED) GUISetOnEvent($GUI_EVENT_DROPPED, "_filedrop") GUISetState(@SW_SHOW,$MainWindow) ; Old Progress Bar ;~ $progressbar = GUICtrlCreateProgress (20,121,290,17,$PBS_SMOOTH) ;~ $percent = GUICtrlCreateLabel("", 155, 123, 30, 12, $SS_CENTER) ;~ GUICtrlSetFont($percent,8,500) ;~ GUICtrlSetBkColor($percent,0xffffff) ; ----- Create Progress GUI ----------------------------------------------------------------- $Progress_Win = GUICreate("file progress ...", $apw, $aph, @DesktopWidth - $apw - 10, @DesktopHeight - $aph - 65, $WS_CAPTION) $Total_Percent_Label = GUICtrlCreateLabel("Total Percent Done = 0%", 10, 10, 300, 15) $Total_Percent_Progress = GUICtrlCreateProgress(10, 25, 310, 25) GUICtrlSetLimit(-1, 100, 0) $File_Number_Label = GUICtrlCreateLabel("File 0 Of 0", 10, 65, 300, 15) $File_Percent_Progress = GUICtrlCreateProgress(10, 80, 310, 25) GUICtrlSetLimit(-1, 100, 0) $File_Percent_Label = GUICtrlCreateLabel("Percent = 0", 10, 105, 300, 15) ; havn't done anything with this yet ;~ $Cancel_Btn = GUICtrlCreateButton("Cancel", 10, 130, 100, 20) GUISetState(@SW_HIDE, $Progress_Win) ; -------------------------------------------------------------------------------------------- _StartUpDisable() While 1 Sleep(10) WEnd Func _handler() Select Case @GUI_CtrlId = $InBrowse ;Input File browse button, same sorta filtering as dropfile, so the gui will show the right controls depending on file opened. $Select_File = FileOpenDialog("Open ...", "::{20D04FE0-3AEA-1069-A2D8-08002B30309D}", "All Files (*.*)", 1) If Not @error Then Dim $szDrive, $szDir, $szFName, $szExt $FileType = _PathSplit($Select_File, $szDrive, $szDir, $szFName, $szExt) If StringLeft($FileType[4], 3) <> ".FS" Then GUICtrlSetData($InFile, $Select_File) GUICtrlSetData($OutDirectory, $FileType[1] & StringTrimRight($FileType[2], 1)) EnableSplitCtrls() ElseIf $FileType[4] = ".FS1" And $FileType[4] <> "" Then GUICtrlSetData($InFile, $Select_File) $sp = StringSplit($FileType[2], "\") $i = $sp[0] - 1 $LastString = StringLen($sp[$i]) $DirOut = StringTrimRight($FileType[1] & $FileType[2], $LastString + 2) GUICtrlSetData($OutDirectory, $DirOut) EnableJoinCtrls() ElseIf StringLeft($FileType[4], 3) = ".FS" And StringRight($FileType[4], 2) <> "S1" Then GUICtrlSetData($InFile, "", "") GUICtrlSetData($OutDirectory, "", "") _StartUpDisable() EndIf EndIf Case @GUI_CtrlId = $OutBrowse ; Output directory browse button. $OutputPath = FileSelectFolder("Choose the ISO folder you want to share.", "", 1) If Not @error Then GUICtrlSetData($OutDirectory, $OutputPath) EndIf Case @GUI_CtrlId = $SplitJoin ;Split or Join button depending what file is dropped/opened If GUICtrlRead($SplitJoin) = "Split File" Then GUICtrlSetState($SplitJoin, $GUI_DISABLE) Dim $szDrive, $szDir, $szFName, $szExt $OutFile = _PathSplit(GUICtrlRead($InFile), $szDrive, $szDir, $szFName, $szExt) $OutFile2 = GUICtrlRead($OutDirectory) $n = 0 $outpath = $OutFile2 & '\Split' & $n & "_" & $OutFile[3] & $OutFile[4] Do $n = $n + 1 $exist1 = FileExists($outpath) $outpath = $OutFile2 & '\Split' & $n & "_" & $OutFile[3] & $OutFile[4] $exist1 = FileExists($outpath) Until $exist1 = 0 GUISetState(@SW_HIDE, $MainWindow) $File_Split = Split_File(GUICtrlRead($InFile), $outpath & "\", $OutFile[3] & $OutFile[4]) GUISetState(@SW_SHOW, $MainWindow) If $File_Split = -1 Then MsgBox(0, "error", "The size you inputed for the split size is larger than the file itself!") GUICtrlSetState($SplitJoin, $GUI_ENABLE) ElseIf $File_Split = -2 Then MsgBox(0, "error", "Unable to open the source file.") GUICtrlSetState($SplitJoin, $GUI_ENABLE) ElseIf $File_Split = -3 Then MsgBox(0, "error", "Unable to to create output files.") GUICtrlSetState($SplitJoin, $GUI_ENABLE) ElseIf $File_Split = 1 Then MsgBox(0, "Done!", 'Done splitting the file. The pieces have been saved in: ' & @CRLF & @CRLF & $outpath & '\') GUICtrlSetState($SplitJoin, $GUI_ENABLE) EndIf ElseIf GUICtrlRead($SplitJoin) = "Join File" Then GUICtrlSetState($SplitJoin, $GUI_DISABLE) Dim $szDrive, $szDir, $szFName, $szExt $Path_Split = _PathSplit(GUICtrlRead($InFile), $szDrive, $szDir, $szFName, $szExt) $Temp_Split = StringSplit($Path_Split[3], ".") $Real_File_Name = "" For $i = 1 To ($Temp_Split[0] - 1) Step 1 $Real_File_Name &= $Temp_Split[$i] & "." Next $Real_File_Name = StringTrimRight($Real_File_Name, 1) $OutFile2 = GUICtrlRead($OutDirectory) $n = 0 $outpath = $OutFile2 & '\Join' & $n & "_" & $Real_File_Name Do $n = $n + 1 $exist1 = FileExists($outpath) $outpath = $OutFile2 & '\Join' & $n & "_" & $Real_File_Name $exist1 = FileExists($outpath) Until $exist1 = 0 GUISetState(@SW_HIDE, $MainWindow) $File_Join = Join_File(GUICtrlRead($InFile), $outpath & "\" & $Real_File_Name, $Temp_Split[$Temp_Split[0]]) GUISetState(@SW_SHOW, $MainWindow) If $File_Join = -1 Then MsgBox(0, "error", "Unable to open the output file.") GUICtrlSetState($SplitJoin, $GUI_ENABLE) ElseIf $File_Join = -2 Then MsgBox(0, "error", "Unable to open the file part one.") GUICtrlSetState($SplitJoin, $GUI_ENABLE) ElseIf $File_Join = 1 Then MsgBox(0, "Done!", 'Done joining the file. The file has been saved in: ' & @CRLF & @CRLF & $outpath & '\') GUICtrlSetState($SplitJoin, $GUI_ENABLE) EndIf EndIf EndSelect EndFunc ;==>_handler Func _filedrop() ;Just some drop file filtering so the gui will offer the right controls for the file dropped Dim $szDrive, $szDir, $szFName, $szExt $FileType = _PathSplit(@GUI_DragFile, $szDrive, $szDir, $szFName, $szExt) If @GUI_DropId = $InFile Then If $FileType[4] <> "" And StringLeft($FileType[4], 3) <> ".FS" Then GUICtrlSetData($OutDirectory, $FileType[1] & StringTrimRight($FileType[2], 1)) EnableSplitCtrls() ElseIf $FileType[4] = ".FS1" And $FileType[4] <> "" Then $sp = StringSplit($FileType[2], "\") $i = $sp[0] - 1 $LastString = StringLen($sp[$i]) $DirOut = StringTrimRight($FileType[1] & $FileType[2], $LastString + 2) GUICtrlSetData($OutDirectory, $DirOut) EnableJoinCtrls() ElseIf $FileType[4] = "" Or StringLeft($FileType[4], 3) = ".FS" And StringRight($FileType[4], 2) <> "S1" Then GUICtrlSetData($InFile, "", "") GUICtrlSetData($OutDirectory, "", "") _StartUpDisable() EndIf ElseIf @GUI_DropId = $OutDirectory Then If $FileType[4] <> "" Then GUICtrlSetData($OutDirectory, "", "") ElseIf $FileType[4] = "" Then GUICtrlSetData($OutDirectory, "", "") GUICtrlSetData($OutDirectory, @GUI_DragFile) EndIf EndIf EndFunc ;==>_filedrop Func _StartUpDisable() ;Disable n hide some controls so a user can only do things in the right order. GUICtrlSetState($SplitInto, $GUI_DISABLE + $GUI_HIDE) GUICtrlSetState($InSize, $GUI_DISABLE + $GUI_HIDE) GUICtrlSetState($MB, $GUI_DISABLE + $GUI_HIDE) GUICtrlSetState($SplitJoin, $GUI_DISABLE + $GUI_HIDE) EndFunc ;==>_StartUpDisable Func EnableSplitCtrls() ;Show n Enable some controls if the user drops/opens a file to split. GUICtrlSetState($SplitInto, $GUI_ENABLE + $GUI_SHOW) GUICtrlSetState($InSize, $GUI_ENABLE + $GUI_SHOW) GUICtrlSetState($MB, $GUI_ENABLE + $GUI_SHOW) GUICtrlSetState($SplitJoin, $GUI_ENABLE + $GUI_SHOW) GUICtrlSetData($SplitJoin, "Split File") EndFunc ;==>EnableSplitCtrls Func EnableJoinCtrls() ;Show n enable some controls if a user drops/opens .FS1 for join. GUICtrlSetState($SplitInto, $GUI_DISABLE + $GUI_HIDE) GUICtrlSetState($InSize, $GUI_DISABLE + $GUI_HIDE) GUICtrlSetState($MB, $GUI_DISABLE + $GUI_HIDE) GUICtrlSetState($SplitJoin, $GUI_ENABLE + $GUI_SHOW) GUICtrlSetData($SplitJoin, "Join File") EndFunc ;==>EnableJoinCtrls Func Split_File($Source, $Output_Folder, $Base_Name) ; Reset progress to 0 in case the user has split a file before this GUICtrlSetData($Total_Percent_Label, "Total Percent Done = 0%") GUICtrlSetData($Total_Percent_Progress, 0) GUICtrlSetData($File_Number_Label, "File 0 Of 0") GUICtrlSetData($File_Percent_Progress, 0) GUICtrlSetData($File_Percent_Label, "Percent = 0") ; Check if the file is smaller than what the user inputed to be the split size If GUICtrlRead($MB) = "MB" And (FileGetSize($Source) / 1048576) <= GUICtrlRead($InSize) Then Return -1 If GUICtrlRead($MB) = "KB" And (FileGetSize($Source) / 1024) <= GUICtrlRead($InSize) Then Return -1 If GUICtrlRead($MB) = "B" And FileGetSize($Source) <= GUICtrlRead($InSize) Then Return -1 ; open the file that will be split (source file) $Open_File = FileOpen($Source, 0) If $Open_File = -1 Then Return -2 ; show progress bar GUISetState(@SW_SHOW, $Progress_Win) ; get the size of the source file $File_Size = FileGetSize($Source) ; Calculate the total number of files the source file will be split into $Number_Of_Files = "" If GUICtrlRead($MB) = "MB" Then $Number_Of_Files = Ceiling($File_Size / (GUICtrlRead($InSize) * 1048576)) ElseIf GUICtrlRead($MB) = "KB" Then $Number_Of_Files = Ceiling($File_Size / (GUICtrlRead($InSize) * 1024)) ElseIf GUICtrlRead($MB) = "B" Then $Number_Of_Files = Ceiling($File_Size / GUICtrlRead($InSize)) EndIf ; convert the split size form to bytes $Split_Size = "" If GUICtrlRead($MB) = "MB" Then $Split_Size = (GUICtrlRead($InSize) * 1048576) ElseIf GUICtrlRead($MB) = "KB" Then $Split_Size = (GUICtrlRead($InSize) * 1024) ElseIf GUICtrlRead($MB) = "B" Then $Split_Size = GUICtrlRead($InSize) EndIf ; create a variable to hold the current file number being wrote $Cur_File_Number = 0 Sleep(1000) $Last_Per_Wrote = 0 ; loop through all the file pieces (except for the last one since it wont be the same size as the others) and write it's chunk of the source files data For $i = 1 To $Number_Of_Files - 1 $Cur_File_Number = $i If $i = 1 Then $Open_Cur_File = FileOpen($Output_Folder & $Base_Name & "." & $Number_Of_Files & ".FS" & $i, 10) Else $Open_Cur_File = FileOpen($Output_Folder & $Base_Name & ".FS" & $i, 10) EndIf If $Open_Cur_File = -1 Then FileClose($Open_File) DirRemove(StringTrimRight($Output_Folder, 1), 1) Return -3 EndIf GUICtrlSetData($File_Number_Label, "File " & $i & " Of " & $Number_Of_Files) If $Split_Size >= 20000 Then $Loop_Amount = Floor($Split_Size / 20000) $Loop_To_Num = ($Loop_Amount * 20000) For $ii = 20000 To $Loop_To_Num Step 20000 FileWrite($Open_Cur_File, FileRead($Open_File, 20000)) $Cur_File_Percent = Round((100 / $Loop_To_Num) * $ii, 0) If $Cur_File_Percent <> $Last_Per_Wrote Then GUICtrlSetData($File_Percent_Progress, $Cur_File_Percent) GUICtrlSetData($File_Percent_Label, "Percent = " & $Cur_File_Percent) $Last_Per_Wrote = $Cur_File_Percent EndIf Next FileWrite($Open_Cur_File, FileRead($Open_File, $Split_Size - (20000 * $Loop_Amount))) Else FileWrite($Open_Cur_File, FileRead($Open_File, $Split_Size)) EndIf GUICtrlSetData($File_Percent_Label, "Percent = 100") GUICtrlSetData($File_Percent_Progress, 100) FileClose($Open_Cur_File) ; calculate total percent done $Cur_Total_Per = Round((100 / $Number_Of_Files) * $i, 0) GUICtrlSetData($Total_Percent_Label, "Total Percent Done = " & $Cur_Total_Per & "%") GUICtrlSetData($Total_Percent_Progress, $Cur_Total_Per) ;----------------------------- Next ; calculate the total number of the source files data already wrote in the split files $Bytes_Wrote = $Cur_File_Number * $Split_Size ; calculate the number bytes left over that still need to be wrote $Left_Over_Bytes = $File_Size - $Bytes_Wrote ; Open the last split file $Open_Cur_File = FileOpen($Output_Folder & $Base_Name & ".FS" & ($Cur_File_Number + 1), 10) If $Open_Cur_File = -1 Then FileClose($Open_File) DirRemove(StringTrimRight($Output_Folder, 1), 1) Return -3 EndIf GUICtrlSetData($File_Number_Label, "File " & $Number_Of_Files & " Of " & $Number_Of_Files) ; Write the rest of the bytes to the last file part If $Left_Over_Bytes > 20000 Then $Loop_Amount = Floor($Left_Over_Bytes / 20000) $Loop_To_Num = ($Loop_Amount * 20000) For $ii = 20000 To $Loop_To_Num Step 20000 FileWrite($Open_Cur_File, FileRead($Open_File, 20000)) $Cur_File_Percent = Round((100 / $Loop_To_Num) * $ii, 0) If $Cur_File_Percent <> $Last_Per_Wrote Then GUICtrlSetData($File_Percent_Progress, $Cur_File_Percent) GUICtrlSetData($File_Percent_Label, "Percent = " & $Cur_File_Percent) $Last_Per_Wrote = $Cur_File_Percent EndIf Next FileWrite($Open_Cur_File, FileRead($Open_File, $Left_Over_Bytes - (20000 * $Loop_Amount))) Else FileWrite($Open_Cur_File, FileRead($Open_File, $Left_Over_Bytes)) EndIf GUICtrlSetData($File_Percent_Progress, 100) GUICtrlSetData($File_Percent_Label, "Percent = 100") GUICtrlSetData($Total_Percent_Label, "Total Percent Done = 100%") GUICtrlSetData($Total_Percent_Progress, 100) Sleep(1000) ; close files FileClose($Open_Cur_File) FileClose($Open_File) GUISetState(@SW_HIDE, $Progress_Win) ; return success Return 1 EndFunc ;==>Split_File Func Join_File($File_Part_One_Path, $Output_File, $Number_Of_Pieces) ; Show progress bar ; Open the output file $Save_File = FileOpen($Output_File, 10) If $Save_File = -1 Then Return -1 ; Reset progress to 0 in case the user has split a file before this GUICtrlSetData($Total_Percent_Label, "Total Percent Done = 0%") GUICtrlSetData($Total_Percent_Progress, 0) GUICtrlSetData($File_Number_Label, "File 0 Of 0") GUICtrlSetData($File_Percent_Progress, 0) GUICtrlSetData($File_Percent_Label, "Percent = 0") ; Show progress bar GUISetState(@SW_SHOW, $Progress_Win) ; Open part one of the split file in read mode $Open_PartOne_File = FileOpen($File_Part_One_Path, 0) If $Open_PartOne_File = -1 Then FileClose($Save_File) GUISetState(@SW_HIDE, $Progress_Win) Return -2 EndIf $Part_One_Size = FileGetSize($File_Part_One_Path) GUICtrlSetData($File_Number_Label, "File 1 Of " & $Number_Of_Pieces) ; Write the contents of part one to the output file If $Part_One_Size < 20000 Then FileWrite($Save_File, FileRead($Open_PartOne_File)) Else $Last_Per_Wrote = 0 $Loop_Amount = Floor($Part_One_Size / 20000) $Loop_To_Num = ($Loop_Amount * 20000) For $ii = 20000 To $Loop_To_Num Step 20000 FileWrite($Save_File, FileRead($Open_PartOne_File, 20000)) $Cur_File_Percent = Round((100 / $Loop_To_Num) * $ii, 0) If $Cur_File_Percent <> $Last_Per_Wrote Then GUICtrlSetData($File_Percent_Progress, $Cur_File_Percent) GUICtrlSetData($File_Percent_Label, "Percent = " & $Cur_File_Percent) $Last_Per_Wrote = $Cur_File_Percent EndIf Next FileWrite($Save_File, FileRead($Open_PartOne_File, $Part_One_Size - $Loop_To_Num)) GUICtrlSetData($File_Percent_Label, "Percent = 100") GUICtrlSetData($File_Percent_Progress, 100) EndIf FileClose($Open_PartOne_File) ; calculate total percent done $Cur_Total_Per = Round((100 / $Number_Of_Pieces) * 1, 0) GUICtrlSetData($Total_Percent_Label, "Total Percent Done = " & $Cur_Total_Per & "%") GUICtrlSetData($Total_Percent_Progress, $Cur_Total_Per) ;----------------------------- ; Set variable to the base file name of all the file parts $Base_Split_File_path = StringReplace($File_Part_One_Path, "." & $Number_Of_Pieces & ".FS1", "") ; Loop through all the file pieces writing their contents to the output file For $i = 2 To $Number_Of_Pieces Step 1 $Open_Part_File = FileOpen($Base_Split_File_path & ".FS" & $i, 0) If $Open_Part_File = -1 Then FileClose($Save_File) GUISetState(@SW_HIDE, $Progress_Win) MsgBox(0, "Error", "Unable to locate """ & $Base_Split_File_path & ".FS" & $i & """. Make sure it's in the same directory as part one.") Return 0 EndIf $Part_Size = FileGetSize($Base_Split_File_path & ".FS" & $i) GUICtrlSetData($File_Number_Label, "File " & $i & " Of " & $Number_Of_Pieces) If $Part_Size < 20000 Then FileWrite($Save_File, FileRead($Open_Part_File)) Else $Last_Per_Wrote = 0 $Loop_Amount = Floor($Part_Size / 20000) $Loop_To_Num = ($Loop_Amount * 20000) For $ii = 20000 To $Loop_To_Num Step 20000 FileWrite($Save_File, FileRead($Open_Part_File, 20000)) $Cur_File_Percent = Round((100 / $Loop_To_Num) * $ii, 0) If $Cur_File_Percent <> $Last_Per_Wrote Then GUICtrlSetData($File_Percent_Progress, $Cur_File_Percent) GUICtrlSetData($File_Percent_Label, "Percent = " & $Cur_File_Percent) $Last_Per_Wrote = $Cur_File_Percent EndIf Next FileWrite($Save_File, FileRead($Open_Part_File, $Part_Size - $Loop_To_Num)) GUICtrlSetData($File_Percent_Label, "Percent = 100") GUICtrlSetData($File_Percent_Progress, 100) EndIf FileClose($Open_Part_File) ; calculate total percent done $Cur_Total_Per = Round((100 / $Number_Of_Pieces) * $i, 0) GUICtrlSetData($Total_Percent_Label, "Total Percent Done = " & $Cur_Total_Per & "%") GUICtrlSetData($Total_Percent_Progress, $Cur_Total_Per) ;----------------------------- Next ; close the output file FileClose($Save_File) GUISetState(@SW_HIDE, $Progress_Win) ; return Success Return 1 EndFunc ;==>Join_File Func _Exit() Exit EndFunc ;==>_Exit I didn't make as you can see. But nevertheless to good to stay under the carpet Regards, ptrex Contributions :Firewall Log Analyzer for XP - Creating COM objects without a need of DLL's - UPnP support in AU3Crystal Reports Viewer - PDFCreator in AutoIT - Duplicate File FinderSQLite3 Database functionality - USB Monitoring - Reading Excel using SQLRun Au3 as a Windows Service - File Monitor - Embedded Flash PlayerDynamic Functions - Control Panel Applets - Digital Signing Code - Excel Grid In AutoIT - Constants for Special Folders in WindowsRead data from Any Windows Edit Control - SOAP and Web Services in AutoIT - Barcode Printing Using PS - AU3 on LightTD WebserverMS LogParser SQL Engine in AutoIT - ImageMagick Image Processing - Converter @ Dec - Hex - Bin -Email Address Encoder - MSI Editor - SNMP - MIB ProtocolFinancial Functions UDF - Set ACL Permissions - Syntax HighLighter for AU3ADOR.RecordSet approach - Real OCR - HTTP Disk - PDF Reader Personal Worldclock - MS Indexing Engine - Printing ControlsGuiListView - Navigation (break the 4000 Limit barrier) - Registration Free COM DLL Distribution - Update - WinRM SMART Analysis - COM Object Browser - Excel PivotTable Object - VLC Media Player - Windows LogOnOff Gui -Extract Data from Outlook to Word & Excel - Analyze Event ID 4226 - DotNet Compiler Wrapper - Powershell_COM - New Link to comment Share on other sites More sharing options...
footswitch Posted September 7, 2007 Author Share Posted September 7, 2007 (edited) You're only native option then would be to read it line by line... 2GB file... Yeah, see you next week LOL yeah that's the point looolll(...)Since we're talking about binary R/W I think this translates to using some sort of buffer and progressively copy stuff from the old to the new file. Even if that was possible, it can be compared to a FileCopy(), and that's the exact opposite of what I want to do. It still would take a lot of time (because of HD Read/Write and script's memory operations) and it would be extremely stupid I'm looking for a sort of FileMove( $file_src, $file_dest, $byte_start, $byte_end).Yeah, basically that's it. Wow, pretty simple to tell, ain't it? EDIT: Wow, wait let me check out those replies Edited September 7, 2007 by footswitch Link to comment Share on other sites More sharing options...
evilertoaster Posted September 7, 2007 Share Posted September 7, 2007 (edited) since you want to copy data you kinda have to read it... I guess what you're asking is for a way to read the start bytes of a file without ever opening it...the problem there is I think most windows API/standard read/writes open the file first .soooo..... an alterntive is then to simply raw read the bytes off the hard drive. If you look here - http://www.autoitscript.com/forum/index.php?showtopic=44841 it shows a way to data directly form the hard drive. You'll need to find the offset for your file and start reading it that way.Really though...Opening the file in autoit is pretty fast (especially with all the speed imporvments in the latest versions). Edited September 7, 2007 by evilertoaster Link to comment Share on other sites More sharing options...
PsaltyDS Posted September 7, 2007 Share Posted September 7, 2007 Since we're talking about binary R/W I think this translates to using some sort of buffer and progressively copy stuff from the old to the new file. Even if that was possible, it can be compared to a FileCopy(), and that's the exact opposite of what I want to do. It still would take a lot of time (because of HD Read/Write and script's memory operations) and it would be extremely stupid I'm looking for a sort of FileMove( $file_src, $file_dest, $byte_start, $byte_end).I guess I'm missing something about your requirements. What about the code I posted above doesn't work for you? Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted September 7, 2007 Moderators Share Posted September 7, 2007 What about the code I posted above doesn't work for you?Everything works well for him I'm sure until he gets to Dim and probably stops failing after $sMsg) (j/k) Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
footswitch Posted September 7, 2007 Author Share Posted September 7, 2007 since you want to copy data you kinda have to read it... I guess what you're asking is for a way to read the start bytes of a file without ever opening it...the problem there is I think most windows API/standard read/writes open the file first .soooo..... an alterntive is then to simply raw read the bytes off the hard drive. If you look here - http://www.autoitscript.com/forum/index.php?showtopic=44841 it shows a way to data directly form the hard drive. You'll need to find the offset for your file and start reading it that way.Really though...Opening the file in autoit is pretty fast (especially with all the speed imporvments in the latest versions).Alright, if there's still someone who didn't understand what I'm looking for (although I believe to have explained myself, apart from my english which isn't that perfect when I intend to explain stuff ), here it goes.I don't know how filesystems work, but I do imagine that files have "FileStart" and "FileEnd" pointers. So:- Name any file you want;- Remove some bytes from the beggining of the file. That is, I believe, change the "FileStart" pointer to another location further in the file and "get rid of"/"ignore" the data left behind.I've already tried to explain this three times, and I really believe I've made myself clear.I DON'T CARE how the file is to be handled When I'm doing something, the thing I'm looking for is results. I'm also NOT looking for things impossible to accomplish. I only analyse the task and try to get an approximate idea of how long it might take to accomplish and how effective it should be.I guess I'm missing something about your requirements. What about the code I posted above doesn't work for you? I didn't fully test your script (which, by the way, isn't that useless ), but basically it copies data. If you're still wondering what's wrong with that, imagine that your file is 4GB long. It takes a long time and spends a lot of resources doing something that could be done in less than one second with really few operations. Not even to talk about data integrity issues.When I code something, I always TRY to use these rules:- Make it logical (why copy millions of bytes from one side to the other when all you want to do is remove a dozen of bytes from there?)- Make it ready for the ultimate stretching of its functionality.- Make it fool-proof (always expect a nasty user to wreck it by doing something unexpected)So, your script might be pretty cool for a 100 MB file. But if you're dealing with 4GB... Not recommended at all Guys, I really appreciate your efforts Gosh do I feel so tiny when I have these sort of tiny problems... Link to comment Share on other sites More sharing options...
PsaltyDS Posted September 8, 2007 Share Posted September 8, 2007 Well, I _did_ misunderstand then. I read your intention as splitting a large file into multiple files of 100MB each. Looking back at the posts, I'm not sure now how I got that impression... As for just removing something from the beginning of a file without writing a new file out... I don't think it can be done with normal file operation APIs. The file begins at a block boundry in the disk file tables. That has a disk address. Theoretically you could just change the starting disk address of the file in the file table. But that is NOT something you access with regular Windows file operations. If the part you want to remove doesn't happen to be an exact integer number of blocks long, then it has to be rewritten anyway because the file has to start on a disk address boundary. If the file system blocks are 64KB (a common size) then you would have to change the file by an integer multiple of 64KB blocks. You just don't have that kind of low-level access to the file system with easily available API calls. Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law Link to comment Share on other sites More sharing options...
Kram3r Posted September 8, 2007 Share Posted September 8, 2007 (edited) Use one of the usefull UDF's I have seenAPITailRWDescription: Functions that assist with reading binary files./ reading/ writing/ deleting to tail by line number fast/ writing to any linebased on "APIRW.au3" by Larry, modified Zedna; now by RandallcAuthor :Write tail etc by Randallc; original binary by Larry, Zedna [binary wites by EriFash?]Tip: don't read to all to one var, use a read/write cycle of one byte each cycle Edited September 8, 2007 by Kram3r To all the Devs: Thkx for AutoIt To all Mods: Thkx for the quality forum Link to comment Share on other sites More sharing options...
Siao Posted September 8, 2007 Share Posted September 8, 2007 (edited) Tip: don't read to all to one var, use a read/write cycle of one byte each cycleI'd say that would be very inefficient. Autoit's built in File IO functions aren't suited for working large files, that's for sure.For writing file, DllCall the WriteFile API, of course, I think there are no other options for the task OP wants to do. That APITailRW posted above has it UDF'ed.For reading large files, file mapping to the rescue... CreateFile, CreateFileMapping, MapViewOfFile, then write it to a new file, then unmap and close handles. Edited September 8, 2007 by Siao "be smart, drink your wine" Link to comment Share on other sites More sharing options...
Kram3r Posted September 8, 2007 Share Posted September 8, 2007 Siao can u be explicit on "file mapping to the rescue... CreateFile, CreateFileMapping, MapViewOfFile" where are that funcs ? UDFs ? To all the Devs: Thkx for AutoIt To all Mods: Thkx for the quality forum Link to comment Share on other sites More sharing options...
Siao Posted September 8, 2007 Share Posted September 8, 2007 (edited) At MSDN.DllCall'ing them shouldn't be too hard.Although I haven't tested if it would really pay off speedwise, because OP problem involves changing size of file, so you can't just FlushViewOfFile. So maybe it wouldn't be much faster than doing ReadFile/WriteFile in chunks (just not one byte chunks ). Although with file mapping you can do it without a loop, passing whole of MapViewOfFile to WriteFile and letting the system handle it. Edited September 8, 2007 by Siao "be smart, drink your wine" Link to comment Share on other sites More sharing options...
Kram3r Posted September 8, 2007 Share Posted September 8, 2007 (edited) Yes MapViewOfFile to WriteFile would be great, I confess that i don't know the existence of that Kernel32.dll func or possibility. ___________________ As for the above: Multi-Byte chunks would be more inteligent, if footswitch take in considerantion that in the last read he as to get off the extra bytes that he gets. Example: want 10 bytes and reads 4 byte chunks read 4 read 4 read 4 get rid of 2 Edited September 8, 2007 by Kram3r To all the Devs: Thkx for AutoIt To all Mods: Thkx for the quality forum Link to comment Share on other sites More sharing options...
footswitch Posted September 8, 2007 Author Share Posted September 8, 2007 (edited) Thank you PsaltyDS, Kram3r and Siao.Your replies have been educative, especially the one regarding how the filesystem works. It made sense somehow.And ptrex... I will take a look at the script you provided. Thanks for that one as well So I guess this is the end of our journey.Files can't be splitted without being copied.Well, actually, you can split a file but the only part of it that you don't have to copy is the first part.Since I want to remove 512 bytes from the start of a 4GB file, then I'll have to practically copy all of it.Cool. If all that was said is true, this is a nasty filesystem limitation... EDIT: I still believe that what I wanted to do is possible (although this can sound a little low level):- discard the first xxx bytes of the file;(this is where I start to sound stupid; please bear in mind I'm not that familiar with filesystems)- move (copy?) a portion of the file (until it reaches the next cluster) to the beginning of a free cluster, and set the beginning of the file to that new cluster;- create a pointer from the new cluster to the rest of the file.(I know this might not be exactly what happens in reality, but I think it's close )So the file would become fragmented, but in practical terms it would still be a single file, now without its first xxx bytes. Edited September 8, 2007 by footswitch Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now