Text file delete function

[Champak]

I have a script that is looking at a text file. The script is pretty much done but there is one last thing I want to add in to clean things up. I have the GUI reading the current line that is being read in the text file as a variable; let's call that variable "X". I want to put a button on the GUI that will delete everything from the start of the text file down to that current line that is being displayed in the GUI which is "X", and then delete all of the blank space that is now there. I currently have it set up where after each line is read it is deleted, but that is taking FAR too long and slowing things up something terrible, so I decided to go this route. The text file is set up as follows, just a bunch of random numbers:

1431

52154

3421

54325456

344

435215

541561

54214

Any help please, I'm stuck.

Edited September 4, 2006 by Champak

[flyingboz]

I currently have it set up where after each line is read it is deleted, but that is taking FAR too long and slowing things up something terrible

one way, that's as fast as anything w/o using dllcall() to get true access to the location of the file pointer....read the entire file into memory -- write out only what you want to a temp file, then delete the original / move in your updated.... i.e.

$contents = file.original

while busy

;do stuff to contents

wend

write contents to file.updated

del file.original

move file.updated to file.original

[MHz]

I have the GUI reading the current line that is being read in the text file as a variable; let's call that variable "X".

A Gui does not read text files. Have you made a test script that reads the file to figure out a better method? if so, then help could be done with that to improve it. Other then showing some numbers sequence in a file, you actually show no attempt that you may have already tried. So what have you tried so consideration be be given?

[Champak]

flyingboz: Thanks, sounds like a good idea, but the principle still sounds like it would take too long.....coming from me who really doesn't know that much. I currently have it being open, string split, and then written back through each loop. I believe the more and more I look at it, with your way, probobly better than what I have now, that the reading through the file and looking for the number of lines to delete which would range from 300-5000 through compariason would take longer than to look for one line and delete that and everything before it with the push of a button. Or even skip the pushing of the button, it could be a function at the end of the entire loop function.

MHz: Sorry I mispoke. I didn't mean that the GUI itself was reading each line, I meant I had the script sending what was being read to a label where I would be able to see what was being read; that to say I know the last line being read in the file because it stays there until I restart or clear it.

This is what I got and adapted:

$ID = FileRead($Read)
$Open = FileOpen($Read, 2)
$Replace = StringRegExpReplace($ID, $IDLine, '')
$aArray = StringSplit(StringTrimRight($Replace, 0), @LF)
$AddLF = ''
For $iFile = 1 To UBound($aArray) - 1
    If $iFile < (UBound($aArray) - 1) And StringStripWS($aArray[$iFile], 7) <> '' Then
        $AddLF = $AddLF & $aArray[$iFile] & @LF
    ElseIf $iFile == (UBound($aArray) - 1) And StringStripWS($aArray[$iFile], 7) <> '' Then
        $AddLF = $AddLF & $aArray[$iFile]
    EndIf
Next
FileWrite($Read, $AddLF)
FileClose($Open)

Thanks for the help.

Edited September 4, 2006 by Champak

[MHz]

To try to improve on the code you show, I would try and use the StringRegExpReplace on each element rather then the whole file at once. Also try to limit conditions and calculations may help some more.

This may not make a big difference, but give a try:

$Open = FileOpen($Read, 2)
If $Open <> -1 Then
    $aArray = StringSplit(FileRead($Read), @LF)
    If Not @error Then
        $AddLF = ''
        $iUBound = UBound($aArray) - 1
        ; Remove whitespace
        For $iFile = 1 To $iUBound
            $aArray[$iFile] = StringRegExpReplace(StringStripWS($aArray[$iFile], 7), $IDLine, '')
        Next
        ; Loop through the array until 2 last element
        For $iFile = 1 To $iUBound - 1
            If $aArray[$iFile] = '' Then ContinueLoop
            $AddLF &= $aArray[$iFile] & @LF
        Next
        ; Process last element
        If $aArray[$iUBound] <> '' Then
            $AddLF &= $aArray[$iUBound]
        Else
            $AddLF = StringTrimRight($AddLF, 1)
        EndIf
        FileWrite($Read, $AddLF)
    EndIf
    FileClose($Open)
EndIf

Edit:

I see a bug with the Else, will look in repairing.

Edit:

Bug fixed.

Edited September 4, 2006 by MHz

[SmOke_N]

I'd be super careful of StringRegExpReplace()... if you are going to replace a specific string, I'd use StringReplace(), I had to change the way I do strings in EnCodeIt because of how buggy SREReplace is.

[Champak]

Thanks, but that code just wiped my entire text file....lucky thing I made a backup. What's wrong?

[SmOke_N]

Thanks, but that code just wiped my entire text file....lucky thing I made a backup. What's wrong?

He did FileOpen(FileName, 2<<<) ... I'm sure he was assuming that you didn't have the file yet... because that erases the file completely.

[randallc]

Hi,

Here's a simple looking script, using regExpReplace from my signature if you're worried about the inbuilt one;

Not sure how the speed would compare to what you need, though?

I interpreted that you wanted to delete ALL lines up to and including the string searched?

Is that correct?

Randall

;DeleteToLineFound.au3
#include<StringRegExpVBS.au3>
$ReadFile = @ScriptDir & "\Example.txt"
$WriteFile = @ScriptDir & "\ExampleWrite.txt"
FileDelete($WriteFile)
$i_Timer = TimerInit()
_DeleteToLineFound($ReadFile, "435215", $WriteFile)
ConsoleWrite(TimerDiff($i_Timer) & @LF)
RunWait("notepad " & $WriteFile)
Func _DeleteToLineFound($ReadFile, $IDLine, $WriteFile)
    $s_ReadID = FileRead($ReadFile)
    $Replace = _StringregexpReplace($s_ReadID, "^[\w\W\s]*?"&$IDLine&"[\w\W\s]*?\n", '')
    FileWrite($WriteFile, $Replace)
EndFunc   ;==>_DeleteToLineFound

[Champak]

Yeah I know the "2" would erase everything---I'm sure he knows its there, it would need to be there or everything would append at the end with a "1" or "0". And the file would have to exist, because that is the file everything is being read from. The "2" is there in the original one as well. There is something else in the code that is doing this. Also, when it is writing "things" back (using 1 or 2 on OpenFile) it is doing it incorrectly. Meaning, each "thing" isn't on its own line (sorry I don't know the terminology).

This does seem to be doing things a little faster (by using 1 and 2 at least), but it is still too slow for what I need to do, especially since it is doing this throughout a loop. It would be fine if it was at the end of the function. Is it not possible to do what my original idea was?

Thanks for the help.

[SmOke_N]

Yeah I know the "2" would erase everything---I'm sure he knows its there, it would need to be there or everything would append at the end with a "1" or "0".

Like I said it was only an observation!

As do I, but we aren't so sure on what level your comprehension is on..

[Champak]

randallc: I'm getting the following errors.

\test3.au3(11,88) : ERROR: _StringregexpReplace(): undefined function.
    $Replace = _StringregexpReplace($s_ReadID, "^[\w\W\s]*?"&$IDLine&"[\w\W\s]*?\n", '')

Smoke_N: By the bold lettering and exclamation, I hope you weren't taking offense to what I said. I wasn't stating it in a sarcastic manner, just simply replying specifically to what you were saying.

[SmOke_N]

randallc: I'm getting the following errors.

\test3.au3(11,88) : ERROR: _StringregexpReplace(): undefined function.
    $Replace = _StringregexpReplace($s_ReadID, "^[\w\W\s]*?"&$IDLine&"[\w\W\s]*?\n", '')

Smoke_N: By the bold lettering and exclamation, I hope you weren't taking offense to what I said. I wasn't stating it in a sarcastic manner, just simply replying specifically to what you were saying.

Offense? Nah, I've had a 12 pack already.. I didn't mean to bold the 2, I was too lazy to do anything other than copy and paste.

The answer was given to your "specific" question though. You have a habbit of asking a question, getting an answer, and saying something doesn't work and never showing your source... I think that's why I quit answering your threads a long time ago... of course it coule be the 12 pack talking.

[randallc]

Hi,

Did you put the file from my signature in the script directory?

regExpReplace from my signature

Best, Randall

[MHz]

Champak, my apologies. I put the fileread() below the fileopen() by mistake. Moving the code around I did not realize I created the problem.

This now has the fileread() before the fileopen().

$aArray = StringSplit(FileRead($Read), @LF)
If Not @error Then
    $Open = FileOpen($Read, 2)
    If $Open <> -1 Then
        $AddLF = ''
        $iUBound = UBound($aArray) - 1
        ; Remove whitespace
        For $iFile = 1 To $iUBound
            $aArray[$iFile] = StringRegExpReplace(StringStripWS($aArray[$iFile], 7), $IDLine, '')
        Next
        ; Loop through the array until 2 last element
        For $iFile = 1 To $iUBound - 1
            If $aArray[$iFile] = '' Then ContinueLoop
            $AddLF &= $aArray[$iFile] & @LF
        Next
        ; Process last element
        If $aArray[$iUBound] <> '' Then
            $AddLF &= $aArray[$iUBound]
        Else
            $AddLF = StringTrimRight($AddLF, 1)
        EndIf
        FileWrite($Read, $AddLF)
    EndIf
    FileClose($Open)
EndIf

[Champak]

Smoke_N: Yeah I do know I have a habit of that, but 98% of the time when I do that, it is because I did not incorporate whatever suggested fix/script into the whole file yet, I am seeing that is not working from the exact code that is given to me, so in that case I don't see the necessity in putting a full page worth of script in the forum when someone can just look one or two posts up and see the problem. Now if I test the code out by itself and it works, and then when I put it in it doesn't work...there I can see the need to put it in....but I like I said, I also do see the habit I have SOME of the times in that particular case. I shall try to fix my ways, and hopefully you can start supplying me with your wisdom once again.

randallc: I don't see a file named "regExpReplace", what I did was try "StringRegExpVBS.au3", but when I include that file, I get the following error:

C:\ADDITI~1\Misc\AUTOIT~1.1\beta\Include\StringRegExpVBS.au3(2,9) : WARNING: already included file: C:\ADDITI~1\Misc\AUTOIT~1.1\beta\Include\StringRegExpVBS.au3
#include<StringRegExpVBS.au3>
~~~~~~~~^
C:\ADDITI~1\Misc\AUTOIT~1.1\beta\Include\StringRegExpVBS.au3(2,9) : WARNING: already included file: C:\ADDITI~1\Misc\AUTOIT~1.1\beta\Include\StringRegEx!>AU3Check ended.rc:-1073741819
>Exit code: 0   Time: 6.398

And the script you supplied is copied verbatim.

MHz: It pretty much works great. It is much faster than what I had before, however:

1 - It is still putting everything on one line----pretty much word wrapping the entire list, instead of each set on its own line.

2 - The files are a minimum starting at 1MB, it is taking 10-15 seconds for that minimum file size for this function alone per loop. Multiply that by a file run of at least 5000 at a time, that's a run time of an hour, not including the rest of the script and any sleep time that is included. I believe it would be much faster randallc's way...if that will work, if you could help with that. I do appreciate the effort you put with this though, thanks.

[randallc]

Hi,

when I include that file,

- so I had already "included" it within that code; you did not need to add it ;

Did you simply put that file in the current "@scriptdir"? [no - it looks like you put it in your beta/ include directory instead or as well- unpredictable results]

maybe try;

#include"StringRegExpVBS.au3"

instead or remove it from one location or the other....

Otherwie, I am highly confused; it is certainly telling you that you are including it twice;; maybe post your exact script? and a 100Kb snippet zipped of one of your files?

Best, randall

Edited September 5, 2006 by randallc

[randallc]

btw,

Given the size of your files, I would suggest a different approach might be faster; lets get this one working first and we can compare

;================================

[The other;

1. Use dOS findstring to get the first line number with the string [fast, I think].

2. Use the FileWriteLine in my "TailRW.au3" [maybe modified] to write all the lines after that in one read/write with APIRead]

;================================

Best, randall

[SmOke_N]

I'm curious if this is the effect your going for?

_DeleteTxtInfo(@DesktopDir & '\textbeingread.txt', GUICtrlRead($Edit))

Func _DeleteTxtInfo($hFile, $sText)
    Local $aSplit = StringSplit(StringStripCR(FileRead($hFile)), @LF)
    Local $iAdd, $sHold
    For $xCC = 1 To $aSplit[0]
        If $aSplit[$xCC] == $sText Then
            $iAdd = ($xCC - 1)
            Do
                $iAdd += 1
                $sHold &= $aSplit[$iAdd] & @CRLF
            Until $iAdd = $aSplit[0]
            FileClose(FileOpen($hFile, 2))
            FileWrite($hFile, StringTrimRight($sHold, 2))
            Return 1
        EndIf
    Next
    Return SetError(1, 0, 0)
EndFunc

[Champak]

Smoke_N: You are beautiful, that is EXACTLY what I needed. I just took away the "- 1" from the "$xCC" to make sure the same number it is looking for is deleted. And it works as fast as I feel it should. Thank you.

randallc: Thanks for your efforts, but I'm going to go with this one for now.

I just tested it independently, I'm going to go ahead and put it in the full script and see how it does...I'll get back if it doesn't work middle of week. Thanks.