Jump to content

Trim text in a file without storing the contents of anywhere


Recommended Posts

Hello, I have a simple task today, I am sure that I made a similar post a long time ago... I want trim (delete or remove or wipe) the contents of a file (text or binary) WITHOUT storing the contents of the file anywhere in the script. Here is a text file for the purpose of experimenting: (contents of text.txt)

1234567890

The task is simple, remove "456" from the contents of text.txt WITHOUT storing the contents anywhere! I wanted to post some code but it seems impossible to provide any relevant code this time...

 

Good luck with the challenge! TD :D

Edited by TheDcoder
removed "(whole)" from the question

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

What is the purpose of this "challenge"? Why should I use the slow file I/O only?

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Link to post
Share on other sites

@UEZ Autually it was my problem which I am facing right now...

Why should I use the slow file I/O only?

Because I process *big* binary files (archives to be precise), I need to trim some stuff in it :)

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

J1, helpful as always ;)

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2021-11-10 - Version 1.6.0.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (NEW 2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (NEW 2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to post
Share on other sites

Use FileRead in a loop and read the data in chunks by setting parameter "count" to e.g. 64K. Process the variable and write the result to a new file. When finished delete the original file and rename the new one.

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2021-11-10 - Version 1.6.0.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (NEW 2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (NEW 2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to post
Share on other sites

You can do this...

Local $sEndData = ""
Local $hFile = FileOpen("1.txt", 17)
FileSetPos($hFile, 3, 0)
FileSetPos($hFile, 3 + 3, 0)
$sEndData = FileRead($hFile)
FileSetPos($hFile, 3, 0)
FileSetEnd($hFile)
FileWrite($hFile, $sEndData)
FileClose($hFile)

Saludos

Link to post
Share on other sites

100MB isn't too big. You should be able to read and process the whole file in one cycle.

What have you tried so far?

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2021-11-10 - Version 1.6.0.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (NEW 2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (NEW 2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to post
Share on other sites

Because I process *big* binary files (archives to be precise), I need to trim some stuff in it :)

What kind of binary archives are you talking about?
Wouldn't changing the content/length destroy the internal structure of the archive?

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2021-11-10 - Version 1.6.0.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (NEW 2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (NEW 2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to post
Share on other sites

What have you tried so far?

Tried some crazy solutions like passing empty string, Null, Binary 0x000000 to trim the contents, currently I just finished my dinner...

Wouldn't changing the content/length destroy the internal structure of the archive?

I am writing additional information in the header, I tried it with a hex editor, It worked! :D

Use FileRead in a loop and read the data in chunks...

I will consider it using if its the most reliable :)

 

@Danyfirex Please wait, I am experimenting

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

danyfirex code in form of function

_FileEdit("file.txt", "456")


Func _FileEdit($sFilePath, $sText)
    $iPos = StringInStr(FileRead($sFilePath), $sText) -1
    $iLen = StringLen($sText)
    Local $sEndData = ""
    Local $hFile = FileOpen($sFilePath, 17)
    FileSetPos($hFile, $iPos, 0)
    FileSetPos($hFile, $iPos + $iLen, 0)
    $sEndData = FileRead($hFile)
    FileSetPos($hFile, $iPos, 0)
    FileSetEnd($hFile)
    FileWrite($hFile, $sEndData)
    FileClose($hFile)
EndFunc   ;==>_FileEdit

 

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to post
Share on other sites

Unless you're planning on hex editing the disk drive directly, your requirement is never going to be possible.

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Link to post
Share on other sites

When working with big files reading the file in chunks is the way to go. I wrote an AutoIt script at work to manipulate very large (15 to 20 GigaByte) XML files and it works like a charm. It's not difficult and relatively fast. The one case you need to consider that some forget, is where the section of the file that you are interested in falls across the boundary of 2 chunks. The way around this is to save some data from the end of the previous chunck, that is 1 byte shorter than what you are looking for and append the next chunk to this. This will ensure that you have the complete section of interest in one chunk of data. It's also worth experimenting with different chunk  sizees to find the optimal size for your particular set up.

"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Link to post
Share on other sites

Hello, I have a simple task today, I am sure that I made a similar post a long time ago... I want trim (delete or remove or wipe) the contents of a file (text or binary) WITHOUT storing the contents of the file anywhere in the script.

When you read data into a variable, that is stored in memory, not the script.

Good luck with the challenge! TD :D

If I'm not mistaken this is not a challenge as you're asking for help.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to post
Share on other sites

Quote (TheDcoder said): ...in the script.

When you read data into a variable, that is stored in memory, not the script.

I was referring to that, what is it called?.... uhhhh... >_<.... :idea: Ah! Synthetic Sugar!, "in the script" is synthetic sugar for "in the memory" :P

If I'm not mistaken this is not a challenge as you're asking for help.

Busted :lol:, Yep, I wanted some help :)

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By MrKm
      AutoIT-OCRSpace-UDF1.3.zip
      This tiny yet powerful UDF will help you to convert Images to text with the help of  OCRSpace API version 3.50 .
      Detect text from a local file.
      ; ========================================================= ; Example 2 : Gets text from an image from a local path reference ; : Searchable PDF is not requested by default. ; : Processes it using a basic OCR logic. ; ========================================================= $b_Create_Searchable_PDF = True ; Use a table logic for receipt OCR $b_Table = True ; Set your key here. $v_OCRSpaceAPIKey = "" $OCROptions = _OCRSpace_SetUpOCR($v_OCRSpaceAPIKey, 1, $b_Table, True, "eng", True, Default, Default, $b_Create_Searchable_PDF) $sText_Detected = _OCRSpace_ImageGetText($OCROptions, @scriptdir & "\receipt.jpg", 0, "SEARCHABLE_URL") ConsoleWrite( _ " Detected text : " & $sText_Detected & @CRLF & _ " Error Returned : " & @error & @CRLF & _ " PDF URL : " & Eval("SEARCHABLE_URL") & @CRLF)  
      Detect text from a URL reference.
      ; ========================================================= ; Example 1 : Gets text from an image using a url reference ; : Searchable PDF is not requested. ; : Processes it using a basic OCR logic. ; ========================================================= $v_OCRSpaceAPIKey = "" ; SetUp some preferences.. $OCROptions = _OCRSpace_SetUpOCR($v_OCRSpaceAPIKey, 1, False, True, "eng", True, Default, Default, False) ; Make the request.. $sText_Detected = _OCRSpace_ImageGetText($OCROptions, "https://i.imgur.com/vbYXwJm.png", 0) ConsoleWrite( _ " Detected text : " & $sText_Detected & @CRLF & _ " Error Returned : " & @error & @CRLF)    
      Detect text from a URL reference to an array
      #include "OCRSpaceUDF\_OCRSpace_UDF.au3" #include <array.au3> ; Set your key here. $v_OCRSpaceAPIKey = "" $OCROptions = _OCRSpace_SetUpOCR($v_OCRSpaceAPIKey, 1, $b_Table, True, "eng", True, Default, Default, False) ; Below, the return type is set to 1 to return an array containing the coordinates of the bounding boxes for each word detected, ; in the format : #WordDetected , #Left , #Top , 3Height, #Width $aText_Detected = _OCRSpace_ImageGetText($OCROptions, "https://i.imgur.com/Z1enogD.jpeg", 1) _ArrayDisplay($aText_Detected, "")  
       
       
       
       
      https://github.com/KabueMurage/AutoIT-OCRSpace-UDF
      Download Latest Version : 
       
    • By Chimp
      at this link https://codegolf.stackexchange.com/questions/37905/mixed-base-conversion there is an interesting "challenge",
      how would you solve it with AutoIt?
    • By lIlIIlIllIIIIlI
      $input = $CmdLine[1] $bytes = 1000000 ; 1000000 for 1 MB, 1000 for 1 KB $size = FileGetSize($input) $file = fileopen($input, 16) $max = ceiling($size / $bytes) for $i = 1 to $max $data = fileread($file, $bytes) $output = $input & '_' & $i & 'of' & $max filewrite($output, $data) next ^file split
      file path for a 20MB "video.mp4" is input, it will be output as "video.mp4_1of20", "video.mp4_2of20", etc. change $bytes to affect the split files size, this example made them 1MB (10^6 bytes)
       
      $input = $CmdLine[1] $name = stringtrimright($input, 1 + stringlen($input) - stringinstr($input, '_', 0, -1)) $split = stringsplit($input, 'of', 3) $max = $split[ubound($split) - 1] for $i = 1 to $max $in = $name & '_' & $i & 'of' & $max $file = fileopen($in, 16) $data = fileread($file) fileclose($file) filedelete($in) filewrite($name, $data) next ^ file join
      file path for any of the "video.mp4_Xof20" segments is input. the "video.mp4_Xof20" segments are read and written to "video.mp4" and then deleted, leaving the newly joined "video.mp4" in the end
       
      i made this today because i had a big file that i couldn't really open. i split it into 1000 chunks to make processing and stuff easier. i've also used it to split files to just under 25MB segments so i can attach and mail big files over gmail which is silly but it worked
    • By TheSaint
      I like and have been using TeraCopy, a third party program, for many years. Mostly it is a great program, but it does have some issues. On Windows XP for instance, Thumbs.db files could often hold up a copy or move process until the user manually responded to the error prompt. Some other issues I discuss below.
      PLEASE NOTE - I am not related to or affiliated in any way with the 3rd party TeraCopy program developers.
      ALSO NOTE - I myself have only tested TeraCopy Cure at this point, on Windows 7 (32 bit), and only with the free version of TeraCopy 2.27.
      This program, TeraCopy Cure, is related to another one of my TeraCopy assistant programs, TeraCopy Timer, but aims at being simpler and quicker to use ... if lacking its more advanced features.
      TeraCopy Cure is a frontend for TeraCopy and sets out to make up for its flaws and limitations.
      One of those flaws is queuing order, and the limitation relates to a same destination issue.
      You would think that queue order would be the same as add order, but that is not the case, and if you are doing a mix of COPY and MOVE then the COPY process could easily fail. It could fail with some source items, even if the  COPY process has already started before the same source MOVE process begins ... especially if the destination folder is on the source drive ... only the currently copying file is locked to that process.
      If you drag and drop another source for a same destination as an existing or impending COPY or MOVE process, then usually TeraCopy nicely adds it for you to that existing job. However, you might wish to avoid that, or it might occur during the testing phase of that active process, and then not be properly processed etc. But you are not given a choice and it just gets added.
       
           
      HOW TO USE
      See the right-click menu of the 'Batch List' (lowest field) and the right-click menu of the Tree field, for some useful options.
      (1) If desired, enable 'Auto Start'. NOTE - Even if enabled, this can be bypassed.
      (2) Set the destination folder, either by browsing on the tree or by dragging a folder to the Destination input field or label. Right-clicking on a folder in Explorer will also work, if enabled. The destination folder path will also show in the Tree if that right-click option is enabled.
      (3) Then drag & drop source file or folder onto one of three five areas - Folder (Drive) Tree field, or COPY or MOVE buttons ... this now also includes the source input and label.
      NOTE - If the Folder (Drive) Tree field is used, then you will additionally need to click either the COPY or MOVE button, to have that job added to the Batch List ... but this avoids 'Auto Start' if it is enabled. Drag & dropping to the buttons instead, saves on clicking, but starts the first job etc immediately if 'Auto Start' is enabled.
      (4) If needed and ready, click the START button to run the first job and those that follow.
      More information is included in program and in the NOTES etc sections below.
         
      WARNING - Depending on the amount of content on your destination drive, and folder level depth, and the speed of your PC, display of the full path in the Folder (Drive) Tree can take a while to show ... if you have that (right-click) option enabled ... it isn't by default. I found this feature quite tricky to get right, and I'm still not 100% sure it is now full-proof.
      TeraCopy Cure v1.5.zip  (source is included)
      TeraCopy Cure v1.6.zip  (source is included)
      NOTES
      UPDATES INFORMATION
      OLDER DOWNLOADS
       
    • By Zedna
      #include <WinAPI.au3> $text = FileReadLastChars("C:\Program Files\AutoIt3\Include\Array.au3", 1024) MsgBox(0, 'FileReadLastChars', $text) Func FileReadLastChars($sFile, $nChars)     Local $nBytes     $tBuffer = DLLStructCreate("char["&$nChars&"]")     $hFile = _WinAPI_CreateFile($sFile, 2, 2) ; open for read     _WinAPI_SetFilePointer($hFile, -1 * $nChars, 2) ; from end     _WinAPI_ReadFile($hFile, DLLStructGetPtr($tBuffer), $nChars, $nBytes)     _WinAPI_CloseHandle($hFile)     Return DLLStructGetData($tBuffer, 1) EndFunc ; included as standard UDF since AutoIt 3.2.13.6 version Func _WinAPI_SetFilePointer($hFile, $iPos, $iMethod = 0)     $aResult = DllCall( "kernel32.dll", "long", "SetFilePointer", "hwnd", $hFile, "long", $iPos, "long_ptr", 0, "long", $iMethod)     If @error Then Return SetError(1, 0, -1)     If $aResult[0] = -1 Then Return SetError(2, 0, -1) ; $INVALID_SET_FILE_POINTER = -1     Return $aResult[0] EndFunc ;==>_WinAPI_SetFilePointer Here is my topic about _WinAPI_SetFilePointer()
       
      EDIT: simpler version compatible with latest AutoIt
      $text = FileReadLastChars("C:\Program Files\AutoIt3\Include\Array.au3", 1024) MsgBox(0, 'FileReadLastChars', $text) Func FileReadLastChars($sFile , $nChars) $hFile = FileOpen($sFile, 0) ; open for read FileSetPos($hFile, -1 * $nChars, 2) ; from end $sRet = FileRead($hFile) FileClose($hFile) Return $sRet EndFunc  
×
×
  • Create New...