File Read Filtering

[Hadez]

I have a script that works good, but im stuck n one part. Currently, my script searching a folder for new .txt files. If a new .txt file is not there, then it loops and checks again 10 seconds later. If a .txt file is present, then it reads the .txt file. The .txt file contains a file name. The script reads this file name and then processes the file that the filename corresponds to. I would like the script to read the .txt file and if the file extension is a .jpeg,.gif or .png, then delete the .txt file and loop to the beginning to check the folder gain for new .txt files (This is because there is no need for me to process these) My current script is:

While 1
    Search()
WEnd

Func Search()
    $chkNewFileLocation = FileFindFirstFile("C:\toprocess\" & "\*.txt")
    If $chkNewFileLocation = -1 Then
        MsgBox(4096, "Error", "No files/directories matched the search pattern", 1)
        Sleep(10000)
        FileClose($chkNewFileLocation)
        Search()
    EndIf
    While 1
    ;script to process the file goes here but is too long to paste
    Wend
EndFunc

As you see, it processes every file. Anyone know how to tell the script to read the .txt file and dont process filenames with those filextensions mentioned above? This is becasue i only want to process folders, but not individual files.

Thanks for any help you can give.

[Zibit]

Why are you using FileClose ? You didnt even open the damn file.

and @error = 0 not -1 ( atleast i think )

[Hadez]

i dont know why i used fileclose but it doesnt do any harm, and the If $chkNewFileLocation = -1 works fine. The script works fine. i just need help with readin the .txt file if it exists and telling it to loop to the beginning if the file extension of the filename is .jpeg,.gif or .png and to continue processing it if it is anything else.

[PsaltyDS]

@Zibit: You might want to zip-it.

What Hadez closed was a search handle, which is quite appropriate. From the help file under FileFindFirstFile():

Return Value

Success: Returns a search "handle" for use with subsequent FileFindNextFile functions.

Failure: Returns -1 if error occurs. If the Folder is empty the @error is set to 1.

Remarks

When you have finished searching with the FileFind... functions you must call FileClose() to release the search handle.

[Hadez]

From having a look around, i think

StringInStr ( "string", "substring" [, casesense [, occurrence [, start [, count]]]] )

is what i may need.

So, it may look like:

$result = StringInStr ( $chkNewFileLocation, ".jpeg", 0, -1)
$result2 = StringInStr ( $chkNewFileLocation, ".gif", 0, -1)
$result3 = StringInStr ( $chkNewFileLocation, ".png", 0, -1)

if $result = 1 then filedelete("C:\toprocess\" & "\*.txt")
else 
if $result2 = 1 then filedelete("C:\toprocess\" & "\*.txt")
else 
if $result3 = 1 then filedelete("C:\toprocess\" & "\*.txt")
Endif 
search()

i think im on the right track, but am noob when it comes to this. can anyone shed some light as to how i would incorporate this into the main script in the first post? Thanks for any help.

Edited August 10, 2010 by Hadez

[omikron48]

Your use of recursion sucks. You're better off using a loop for this. The way you currently recurse in your function, you go a level deep for every time you fail to find a .txt file in the folder, not good.

Global $chkNewFileLocation
While 1
    ;gives you a list of all the .txt files in the C:\toprocess folder
    $chkNewFileLocation = FileFindFirstFile("C:\toprocess\*.txt")
    If @error Then
        MsgBox(4096, "Error", "No files/directories matched the search pattern", 1)
        Sleep(10000)
    Else
        Local $filename
        ;process files returned by search filter
        While 1
            $filename = FileFindNextFile($chkNewFileLocation)
            ;exit loop if no more .txt files to process
            If @error Then ExitLoop
            ;process $filename
        Wend
    EndIf
WEnd

Edited August 11, 2010 by omikron48

[Hadez]

Thanks for that, but i follow 'if it isnt broke, dont fix it', and since the script has been working fine for months, i think ill keep it the way it was, but ill take a note of the way u did the original script incase anything does go wrong with mine .

But, i am still looking for a way of reading the .txt file and go back to the beginning of the search loop if the filetypes are not ones i need. You got any ideas or pointers of how i would go about this?

[FlyinRiz]

Thanks for that, but i follow 'if it isnt broke, dont fix it', and since the script has been working fine for months, i think ill keep it the way it was, but ill take a note of the way u did the original script incase anything does go wrong with mine .

But, i am still looking for a way of reading the .txt file and go back to the beginning of the search loop if the filetypes are not ones i need. You got any ideas or pointers of how i would go about this?

While that might work for your current use, keep in mind the following AutoIt restriction:

Maximum depth of recursive function calls: 5100 levels

So once you reach 5100 failed txt finds, your script could fail. Just something to keep in mind. I doubt that you are actually letting your script run that long (~15.6 hours).

Another note that isn't really important, but could cause errors in future scripts (if precise timing is important) is that your script is actually pausing 11 seconds (1 second for MsgBox timeout, 10 seconds for sleep). Like I said, not a huge issue, just something to keep in mind if your magic number of 10 seconds is really important.

-Aaron

[Hadez]

Thanks for the reply mate. Nah, the timing isnt really important. I just chose 10 secs as it wasnt too short and wasnt too long. And thanks for the info on the 5100 limit. Luckily, there are new txt files every 10 mins or so. hats prob y i havnt come across the error

[FlyinRiz]

But, i am still looking for a way of reading the .txt file and go back to the beginning of the search loop if the filetypes are not ones i need. You got any ideas or pointers of how i would go about this?

Look in the helpfile under FileRead or FileReadLine (if the filename is always on the first line). That should get you started. From there you can use the String functions mentioned above.

-Aaron

[omikron48]

Can't really help you much since your opening post is kinda vague.

The .txt file contains a file name.

How is the content of the .txt file organized? How many file names are we talking about? Is it always just the one file name?

The script reads this file name and then processes the file that the filename corresponds to.

Which file name are we talking about here, exactly? Is it the file name of the .txt file in the folder or the file name stored inside the .txt file found in the folder? What kind of processing are we talking about? Which file is to be processed, the .txt file or the file name contained inside the .txt file?

I would like the script to read the .txt file and if the file extension is a .jpeg,.gif or .png, then delete the .txt file

I can't quite make sense of this one.

You are too cheap with words, resulting in your post being vague. You're talking about at least two different file names and fail to properly refer to each, and you're talking about an unknown process to boot. You aren't being charged per word for posting so a well worded statement even if it's long will be more effective in getting your point across, more so if what you are describing is complicated.

Also, blindly following "If it ain't broke, don't fix it" is stupid. That would mean you'd rather settle driving your working beat up car than have a chance at fixing it to make it into a better automobile. I wouldn't have minded if your script was done well. It didn't even need to be great, but your current script has bad implementation stamped on its forehead. It's better to correct bad practice early, so I pointed out your bad use of recursion. You can't improve if you don't listen.

Edited August 11, 2010 by omikron48

[Hadez]

I wasnt trying to be vague. I was trying to be as clear as i cud but obviously didnt succeed. I appreciate your input on recursion, but my method runs perfectly fine the way i did it, and like i said before, if anything goes wrong, then i will try your method out, but until then, whats the point of trying to fix something that isnt broke. This is a script that runs line by line, and as such, it currently performs its task good. Your comparison with that of a car is a little off, since a car is alot more complex and every detail can make a difference. With this script, if it works, then i dont find it necessary to mess around with it as it is a personal script and not for professional use. For my next script, however, i will follow your guide to recursion.

In case it is me not being clear with my problem, then i shall try to be alot more clearer

My script runs like this:

My script watches a folder for new txt files. (E.G. C:\Watchfolder) These text files are named after a file elsewhere on the system that needs to be processed later. (E.G. apples.txt) This filename changes depending on the name of the file that needs to be processed.

Inside these text files there is 1 line. This line contains the location of a file that i would need to process later (E.G. C:\Photos\Apples.rar)

If it finds a txt file, then it reads the file location and continues to process the file. The processing is long so i didnt find the need to include it above. But for this case, lets say the process is to just move the file. So, C:\Photos\Apples.rar is then moved to another folder. (E.G. C:\fileafterprocess)

Once that is done, apples.txt is then deleted and the script loops from the beginning to process the next txt file. If no txt file is there, then it sleeps for 10 seconds and then rechecks.

The problem i currently have is that it will process (move) whatever file that is written inside the txt file. If it says C:\Photos\Apples.rar inside the txt file, then i would like it process it, but if it says C:\Photos\Apples.jpeg, C:\Photos\Apples.png or C:\Photos\Apples.gif, then i would like it to not process the file and to delete Apples.txt from C:\Watchfolder, before looping to the beginning of the script to check C:\Watchfolder for any other txt files.

I hope this is explained clearer now.

Edited August 11, 2010 by Hadez

[FlyinRiz]

I wasnt trying to be vague. I was trying to be as clear as i cud but obviously didnt succeed. I appreciate your input on recursion, but my method runs perfectly fine the way i did it, and like i said before, if anything goes wrong, then i will try your method out, but until then, whats the point of trying to fix something that isnt broke. This is a script that runs line by line, and as such, it currently performs its task good. Your comparison with that of a car is a little off, since a car is alot more complex and every detail can make a difference. With this script, if it works, then i dont find it necessary to mess around with it as it is a personal script and not for professional use. For my next script, however, i will follow your guide to recursion.

In case it is me not being clear with my problem, then i shall try to be alot more clearer

My script runs like this:

My script watches a folder for new txt files. (E.G. C:\Watchfolder) These text files are named after a file elsewhere on the system that needs to be processed later. (E.G. apples.txt) This filename changes depending on the name of the file that needs to be processed.

Inside these text files there is 1 line. This line contains the location of a file that i would need to process later (E.G. C:\Photos\Apples.rar)

If it finds a txt file, then it reads the file location and continues to process the file. The processing is long so i didnt find the need to include it above. But for this case, lets say the process is to just move the file. So, C:\Photos\Apples.rar is then moved to another folder. (E.G. C:\fileafterprocess)

Once that is done, apples.txt is then deleted and the script loops from the beginning to process the next txt file. If no txt file is there, then it sleeps for 10 seconds and then rechecks.

The problem i currently have is that it will process (move) whatever file that is written inside the txt file. If it says C:\Photos\Apples.rar inside the txt file, then i would like it process it, but if it says C:\Photos\Apples.jpeg, C:\Photos\Apples.png or C:\Photos\Apples.gif, then i would like it to not process the file and to delete Apples.txt from C:\Watchfolder, before looping to the beginning of the script to check C:\Watchfolder for any other txt files.

I hope this is explained clearer now.

Try this...

Local $avArray[4]

$avArray[0] = ".jpg"
$avArray[1] = ".png"
$avArray[2] = ".gif"
$avArray[3] = ".bmp"
;ect...

$line = FileReadLine($filename)
For $i = 0 to UBound($avArray) - 1
    If StringInStr($line,$avArray[$i]) > 0 Then
        FileDelete($filename)
        Return ;Go back to While 1
    EndIf
Next
;rest of processing

Didn't test it...

-Aaron

[saywell]

I wasnt trying to be vague. I was trying to be as clear as i cud but obviously didnt succeed. I appreciate your input on recursion, but my method runs perfectly fine the way i did it, and like i said before, if anything goes wrong, then i will try your method out, but until then, whats the point of trying to fix something that isnt broke. This is a script that runs line by line, and as such, it currently performs its task good. Your comparison with that of a car is a little off, since a car is alot more complex and every detail can make a difference. With this script, if it works, then i dont find it necessary to mess around with it as it is a personal script and not for professional use. For my next script, however, i will follow your guide to recursion.

In case it is me not being clear with my problem, then i shall try to be alot more clearer

My script runs like this:

My script watches a folder for new txt files. (E.G. C:\Watchfolder) These text files are named after a file elsewhere on the system that needs to be processed later. (E.G. apples.txt) This filename changes depending on the name of the file that needs to be processed.

Inside these text files there is 1 line. This line contains the location of a file that i would need to process later (E.G. C:\Photos\Apples.rar)

If it finds a txt file, then it reads the file location and continues to process the file. The processing is long so i didnt find the need to include it above. But for this case, lets say the process is to just move the file. So, C:\Photos\Apples.rar is then moved to another folder. (E.G. C:\fileafterprocess)

Once that is done, apples.txt is then deleted and the script loops from the beginning to process the next txt file. If no txt file is there, then it sleeps for 10 seconds and then rechecks.

The problem i currently have is that it will process (move) whatever file that is written inside the txt file. If it says C:\Photos\Apples.rar inside the txt file, then i would like it process it, but if it says C:\Photos\Apples.jpeg, C:\Photos\Apples.png or C:\Photos\Apples.gif, then i would like it to not process the file and to delete Apples.txt from C:\Watchfolder, before looping to the beginning of the script to check C:\Watchfolder for any other txt files.

I hope this is explained clearer now.

So all you need to do is FileRead to get the contents as a string, then StringRegExp to see if it matches .jpeg , .png , etc. depending on result, you then either process and delete, or just delete.

William

[Spiff59]

All you'd need to do to eliminate your worst-sin-in-structured-programming use of recursion is to change the Search() call to a Return() like below. Your main While loop will still immediately kick off the Search() function.

The FileClose() in that location is unnecessary. The FileFindFirstFile() errored, so there is no search handle to close. You'd want the FileClose() after your FileFindNextFile(), whereever that is.

While 1
    Search()
WEnd

Func Search()
    $chkNewFileLocation = FileFindFirstFile("C:\toprocess\" & "\*.txt")
    If $chkNewFileLocation = -1 Then
        MsgBox(4096, "Error", "No files/directories matched the search pattern", 1)
        Sleep(10000)
        ; FileClose($chkNewFileLocation)
        Return ; Search()
    EndIf
    While 1
    ;script to process the file goes here but is too long to paste
    Wend
EndFunc

You could also use a Switch() statement to sort out the file types, something like:

Switch StringTrimLeft($file, StringInStr($file, ".", 0, -1) - 1)
    Case ".jpg", ".jpeg", ".bmp", ".png"
        ; do whatever invalid file types
    Case ".zip", ".rar"
        ; process archives
    Case ".txt"
        ; process text file
EndSwitch

I'm a little confused whether your requirement is to keep certain files types you want to process, or eliminate certain types that you don't want. This Switch() statement does a little of both which is probably unneeded.

Edited August 11, 2010 by Spiff59

[PsaltyDS]

While that might work for your current use, keep in mind the following AutoIt restriction:

Maximum depth of recursive function calls: 5100 levels

So once you reach 5100 failed txt finds, your script could fail. Just something to keep in mind. I doubt that you are actually letting your script run that long (~15.6 hours).

This is not really a problem with recursive search functions. Each time the function RETURNS the counter DECREMENTS. So it can only hit that limit if there are actually over five thousand elements in a single full path. In other words a single path like: "C:\Temp\Level_1\Level_2\Level_3\...\Level_5099\Level_5100\". If C:\Temp\ contains a thousand folders, which each contain an additional thousand subfolders, the max recursion level would only be 3.

There is a post floating around somewhere in which I think Melba23 demonstrates using a push/pop stack implemented as an array. It's a usable alternative if recursion still makes you queasy, but recursion works just fine.

[FlyinRiz]

This is not really a problem with recursive search functions. Each time the function RETURNS the counter DECREMENTS. So it can only hit that limit if there are actually over five thousand elements in a single full path. In other words a single path like: "C:\Temp\Level_1\Level_2\Level_3\...\Level_5099\Level_5100\". If C:\Temp\ contains a thousand folders, which each contain an additional thousand subfolders, the max recursion level would only be 3.

There is a post floating around somewhere in which I think Melba23 demonstrates using a push/pop stack implemented as an array. It's a usable alternative if recursion still makes you queasy, but recursion works just fine.

In the first example, the function never returns if the file isn't found (at least in theory), but they all return once a valid file is found, so it would only happen if the file wasn't found for over 15 hours like I had said above. Am I misunderstanding what you are saying?

[PsaltyDS]

In the first example, the function never returns if the file isn't found (at least in theory), but they all return once a valid file is found, so it would only happen if the file wasn't found for over 15 hours like I had said above. Am I misunderstanding what you are saying?

I was speaking generally about any (properly coded) recursive function. You can look at my _RegSearch() function in the Example Scripts forum as an example.

The function in the first post is not recursive (doesn't call itself). Which post # are you referring to when you say "the first example"?

(Yes it is, and yes it does. Sloppy reading on my part.)

Edited August 12, 2010 by PsaltyDS

[FlyinRiz]

I was speaking generally about any (properly coded) recursive function. You can look at my _RegSearch() function in the Example Scripts forum as an example.

The function in the first post is not recursive (doesn't call itself). Which post # are you referring to when you say "the first example"?

I was referring to the OP. I'm not saying that it is a properly coded recursive function, but it is recursive anyway because it's calling itself if no file is found.

[PsaltyDS]

I was referring to the OP. I'm not saying that it is a properly coded recursive function, but it is recursive anyway because it's calling itself if no file is found.

I see it now, and you're right. I missed where he had placed Search() in there.