Jump to content
Sign in to follow this  
Hadez

File Read Filtering

Recommended Posts

Hadez

I have a script that works good, but im stuck n one part. Currently, my script searching a folder for new .txt files. If a new .txt file is not there, then it loops and checks again 10 seconds later. If a .txt file is present, then it reads the .txt file. The .txt file contains a file name. The script reads this file name and then processes the file that the filename corresponds to. I would like the script to read the .txt file and if the file extension is a .jpeg,.gif or .png, then delete the .txt file and loop to the beginning to check the folder gain for new .txt files (This is because there is no need for me to process these) My current script is:

While 1
    Search()
WEnd

Func Search()
    $chkNewFileLocation = FileFindFirstFile("C:\toprocess\" & "\*.txt")
    If $chkNewFileLocation = -1 Then
        MsgBox(4096, "Error", "No files/directories matched the search pattern", 1)
        Sleep(10000)
        FileClose($chkNewFileLocation)
        Search()
    EndIf
    While 1
    ;script to process the file goes here but is too long to paste
    Wend
EndFunc

As you see, it processes every file. Anyone know how to tell the script to read the .txt file and dont process filenames with those filextensions mentioned above? This is becasue i only want to process folders, but not individual files.

Thanks for any help you can give.

Share this post


Link to post
Share on other sites
Hadez

i dont know why i used fileclose but it doesnt do any harm, and the If $chkNewFileLocation = -1 works fine. The script works fine. i just need help with readin the .txt file if it exists and telling it to loop to the beginning if the file extension of the filename is .jpeg,.gif or .png and to continue processing it if it is anything else.

Share this post


Link to post
Share on other sites
PsaltyDS

@Zibit: You might want to zip-it. :blink:

What Hadez closed was a search handle, which is quite appropriate. From the help file under FileFindFirstFile():

Return Value

Success: Returns a search "handle" for use with subsequent FileFindNextFile functions.

Failure: Returns -1 if error occurs. If the Folder is empty the @error is set to 1.

Remarks

When you have finished searching with the FileFind... functions you must call FileClose() to release the search handle.

;)


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
Hadez

From having a look around, i think

StringInStr ( "string", "substring" [, casesense [, occurrence [, start [, count]]]] )

is what i may need.

So, it may look like:

$result = StringInStr ( $chkNewFileLocation, ".jpeg", 0, -1)
$result2 = StringInStr ( $chkNewFileLocation, ".gif", 0, -1)
$result3 = StringInStr ( $chkNewFileLocation, ".png", 0, -1)

if $result = 1 then filedelete("C:\toprocess\" & "\*.txt")
else 
if $result2 = 1 then filedelete("C:\toprocess\" & "\*.txt")
else 
if $result3 = 1 then filedelete("C:\toprocess\" & "\*.txt")
Endif 
search()

i think im on the right track, but am noob when it comes to this. can anyone shed some light as to how i would incorporate this into the main script in the first post? Thanks for any help.

Edited by Hadez

Share this post


Link to post
Share on other sites
omikron48

Your use of recursion sucks. You're better off using a loop for this. The way you currently recurse in your function, you go a level deep for every time you fail to find a .txt file in the folder, not good.

Global $chkNewFileLocation
While 1
    ;gives you a list of all the .txt files in the C:\toprocess folder
    $chkNewFileLocation = FileFindFirstFile("C:\toprocess\*.txt")
    If @error Then
        MsgBox(4096, "Error", "No files/directories matched the search pattern", 1)
        Sleep(10000)
    Else
        Local $filename
        ;process files returned by search filter
        While 1
            $filename = FileFindNextFile($chkNewFileLocation)
            ;exit loop if no more .txt files to process
            If @error Then ExitLoop
            ;process $filename
        Wend
    EndIf
WEnd
Edited by omikron48

Share this post


Link to post
Share on other sites
Hadez

Thanks for that, but i follow 'if it isnt broke, dont fix it', and since the script has been working fine for months, i think ill keep it the way it was, but ill take a note of the way u did the original script incase anything does go wrong with mine :blink:.

But, i am still looking for a way of reading the .txt file and go back to the beginning of the search loop if the filetypes are not ones i need. You got any ideas or pointers of how i would go about this?

Share this post


Link to post
Share on other sites
FlyinRiz

Thanks for that, but i follow 'if it isnt broke, dont fix it', and since the script has been working fine for months, i think ill keep it the way it was, but ill take a note of the way u did the original script incase anything does go wrong with mine :blink:.

But, i am still looking for a way of reading the .txt file and go back to the beginning of the search loop if the filetypes are not ones i need. You got any ideas or pointers of how i would go about this?

While that might work for your current use, keep in mind the following AutoIt restriction:

Maximum depth of recursive function calls: 5100 levels

So once you reach 5100 failed txt finds, your script could fail. Just something to keep in mind. I doubt that you are actually letting your script run that long (~15.6 hours).

Another note that isn't really important, but could cause errors in future scripts (if precise timing is important) is that your script is actually pausing 11 seconds (1 second for MsgBox timeout, 10 seconds for sleep). Like I said, not a huge issue, just something to keep in mind if your magic number of 10 seconds is really important.

-Aaron

Share this post


Link to post
Share on other sites
Hadez

Thanks for the reply mate. Nah, the timing isnt really important. I just chose 10 secs as it wasnt too short and wasnt too long. And thanks for the info on the 5100 limit. Luckily, there are new txt files every 10 mins or so. hats prob y i havnt come across the error :blink:

Share this post


Link to post
Share on other sites
FlyinRiz

But, i am still looking for a way of reading the .txt file and go back to the beginning of the search loop if the filetypes are not ones i need. You got any ideas or pointers of how i would go about this?

Look in the helpfile under FileRead or FileReadLine (if the filename is always on the first line). That should get you started. From there you can use the String functions mentioned above.

-Aaron

Share this post


Link to post
Share on other sites
omikron48

Can't really help you much since your opening post is kinda vague.

The .txt file contains a file name.

How is the content of the .txt file organized? How many file names are we talking about? Is it always just the one file name?

The script reads this file name and then processes the file that the filename corresponds to.

Which file name are we talking about here, exactly? Is it the file name of the .txt file in the folder or the file name stored inside the .txt file found in the folder? What kind of processing are we talking about? Which file is to be processed, the .txt file or the file name contained inside the .txt file?

I would like the script to read the .txt file and if the file extension is a .jpeg,.gif or .png, then delete the .txt file

I can't quite make sense of this one.

You are too cheap with words, resulting in your post being vague. You're talking about at least two different file names and fail to properly refer to each, and you're talking about an unknown process to boot. You aren't being charged per word for posting so a well worded statement even if it's long will be more effective in getting your point across, more so if what you are describing is complicated.

Also, blindly following "If it ain't broke, don't fix it" is stupid. That would mean you'd rather settle driving your working beat up car than have a chance at fixing it to make it into a better automobile. I wouldn't have minded if your script was done well. It didn't even need to be great, but your current script has bad implementation stamped on its forehead. It's better to correct bad practice early, so I pointed out your bad use of recursion. You can't improve if you don't listen.

Edited by omikron48

Share this post


Link to post
Share on other sites
Hadez

I wasnt trying to be vague. I was trying to be as clear as i cud but obviously didnt succeed. I appreciate your input on recursion, but my method runs perfectly fine the way i did it, and like i said before, if anything goes wrong, then i will try your method out, but until then, whats the point of trying to fix something that isnt broke. This is a script that runs line by line, and as such, it currently performs its task good. Your comparison with that of a car is a little off, since a car is alot more complex and every detail can make a difference. With this script, if it works, then i dont find it necessary to mess around with it as it is a personal script and not for professional use. For my next script, however, i will follow your guide to recursion.

In case it is me not being clear with my problem, then i shall try to be alot more clearer

My script runs like this:

My script watches a folder for new txt files. (E.G. C:\Watchfolder) These text files are named after a file elsewhere on the system that needs to be processed later. (E.G. apples.txt) This filename changes depending on the name of the file that needs to be processed.

Inside these text files there is 1 line. This line contains the location of a file that i would need to process later (E.G. C:\Photos\Apples.rar)

If it finds a txt file, then it reads the file location and continues to process the file. The processing is long so i didnt find the need to include it above. But for this case, lets say the process is to just move the file. So, C:\Photos\Apples.rar is then moved to another folder. (E.G. C:\fileafterprocess)

Once that is done, apples.txt is then deleted and the script loops from the beginning to process the next txt file. If no txt file is there, then it sleeps for 10 seconds and then rechecks.

The problem i currently have is that it will process (move) whatever file that is written inside the txt file. If it says C:\Photos\Apples.rar inside the txt file, then i would like it process it, but if it says C:\Photos\Apples.jpeg, C:\Photos\Apples.png or C:\Photos\Apples.gif, then i would like it to not process the file and to delete Apples.txt from C:\Watchfolder, before looping to the beginning of the script to check C:\Watchfolder for any other txt files.

I hope this is explained clearer now.

Edited by Hadez

Share this post


Link to post
Share on other sites
FlyinRiz

I wasnt trying to be vague. I was trying to be as clear as i cud but obviously didnt succeed. I appreciate your input on recursion, but my method runs perfectly fine the way i did it, and like i said before, if anything goes wrong, then i will try your method out, but until then, whats the point of trying to fix something that isnt broke. This is a script that runs line by line, and as such, it currently performs its task good. Your comparison with that of a car is a little off, since a car is alot more complex and every detail can make a difference. With this script, if it works, then i dont find it necessary to mess around with it as it is a personal script and not for professional use. For my next script, however, i will follow your guide to recursion.

In case it is me not being clear with my problem, then i shall try to be alot more clearer

My script runs like this:

My script watches a folder for new txt files. (E.G. C:\Watchfolder) These text files are named after a file elsewhere on the system that needs to be processed later. (E.G. apples.txt) This filename changes depending on the name of the file that needs to be processed.

Inside these text files there is 1 line. This line contains the location of a file that i would need to process later (E.G. C:\Photos\Apples.rar)

If it finds a txt file, then it reads the file location and continues to process the file. The processing is long so i didnt find the need to include it above. But for this case, lets say the process is to just move the file. So, C:\Photos\Apples.rar is then moved to another folder. (E.G. C:\fileafterprocess)

Once that is done, apples.txt is then deleted and the script loops from the beginning to process the next txt file. If no txt file is there, then it sleeps for 10 seconds and then rechecks.

The problem i currently have is that it will process (move) whatever file that is written inside the txt file. If it says C:\Photos\Apples.rar inside the txt file, then i would like it process it, but if it says C:\Photos\Apples.jpeg, C:\Photos\Apples.png or C:\Photos\Apples.gif, then i would like it to not process the file and to delete Apples.txt from C:\Watchfolder, before looping to the beginning of the script to check C:\Watchfolder for any other txt files.

I hope this is explained clearer now.

Try this...

Local $avArray[4]

$avArray[0] = ".jpg"
$avArray[1] = ".png"
$avArray[2] = ".gif"
$avArray[3] = ".bmp"
;ect...

$line = FileReadLine($filename)
For $i = 0 to UBound($avArray) - 1
    If StringInStr($line,$avArray[$i]) > 0 Then
        FileDelete($filename)
        Return ;Go back to While 1
    EndIf
Next
;rest of processing

Didn't test it...

-Aaron

Share this post


Link to post
Share on other sites
saywell

I wasnt trying to be vague. I was trying to be as clear as i cud but obviously didnt succeed. I appreciate your input on recursion, but my method runs perfectly fine the way i did it, and like i said before, if anything goes wrong, then i will try your method out, but until then, whats the point of trying to fix something that isnt broke. This is a script that runs line by line, and as such, it currently performs its task good. Your comparison with that of a car is a little off, since a car is alot more complex and every detail can make a difference. With this script, if it works, then i dont find it necessary to mess around with it as it is a personal script and not for professional use. For my next script, however, i will follow your guide to recursion.

In case it is me not being clear with my problem, then i shall try to be alot more clearer

My script runs like this:

My script watches a folder for new txt files. (E.G. C:\Watchfolder) These text files are named after a file elsewhere on the system that needs to be processed later. (E.G. apples.txt) This filename changes depending on the name of the file that needs to be processed.

Inside these text files there is 1 line. This line contains the location of a file that i would need to process later (E.G. C:\Photos\Apples.rar)

If it finds a txt file, then it reads the file location and continues to process the file. The processing is long so i didnt find the need to include it above. But for this case, lets say the process is to just move the file. So, C:\Photos\Apples.rar is then moved to another folder. (E.G. C:\fileafterprocess)

Once that is done, apples.txt is then deleted and the script loops from the beginning to process the next txt file. If no txt file is there, then it sleeps for 10 seconds and then rechecks.

The problem i currently have is that it will process (move) whatever file that is written inside the txt file. If it says C:\Photos\Apples.rar inside the txt file, then i would like it process it, but if it says C:\Photos\Apples.jpeg, C:\Photos\Apples.png or C:\Photos\Apples.gif, then i would like it to not process the file and to delete Apples.txt from C:\Watchfolder, before looping to the beginning of the script to check C:\Watchfolder for any other txt files.

I hope this is explained clearer now.

So all you need to do is FileRead to get the contents as a string, then StringRegExp to see if it matches .jpeg , .png , etc. depending on result, you then either process and delete, or just delete.

William

Share this post


Link to post
Share on other sites
Spiff59

All you'd need to do to eliminate your worst-sin-in-structured-programming use of recursion is to change the Search() call to a Return() like below. Your main While loop will still immediately kick off the Search() function.

The FileClose() in that location is unnecessary. The FileFindFirstFile() errored, so there is no search handle to close. You'd want the FileClose() after your FileFindNextFile(), whereever that is.

While 1
    Search()
WEnd

Func Search()
    $chkNewFileLocation = FileFindFirstFile("C:\toprocess\" & "\*.txt")
    If $chkNewFileLocation = -1 Then
        MsgBox(4096, "Error", "No files/directories matched the search pattern", 1)
        Sleep(10000)
        ; FileClose($chkNewFileLocation)
        Return ; Search()
    EndIf
    While 1
    ;script to process the file goes here but is too long to paste
    Wend
EndFunc

You could also use a Switch() statement to sort out the file types, something like:

Switch StringTrimLeft($file, StringInStr($file, ".", 0, -1) - 1)
    Case ".jpg", ".jpeg", ".bmp", ".png"
        ; do whatever invalid file types
    Case ".zip", ".rar"
        ; process archives
    Case ".txt"
        ; process text file
EndSwitch

I'm a little confused whether your requirement is to keep certain files types you want to process, or eliminate certain types that you don't want. This Switch() statement does a little of both which is probably unneeded.

Edited by Spiff59

Share this post


Link to post
Share on other sites
PsaltyDS

While that might work for your current use, keep in mind the following AutoIt restriction:

Maximum depth of recursive function calls: 5100 levels

So once you reach 5100 failed txt finds, your script could fail. Just something to keep in mind. I doubt that you are actually letting your script run that long (~15.6 hours).

This is not really a problem with recursive search functions. Each time the function RETURNS the counter DECREMENTS. So it can only hit that limit if there are actually over five thousand elements in a single full path. In other words a single path like: "C:\Temp\Level_1\Level_2\Level_3\...\Level_5099\Level_5100\". If C:\Temp\ contains a thousand folders, which each contain an additional thousand subfolders, the max recursion level would only be 3.

There is a post floating around somewhere in which I think Melba23 demonstrates using a push/pop stack implemented as an array. It's a usable alternative if recursion still makes you queasy, but recursion works just fine.

:blink:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
FlyinRiz

This is not really a problem with recursive search functions. Each time the function RETURNS the counter DECREMENTS. So it can only hit that limit if there are actually over five thousand elements in a single full path. In other words a single path like: "C:\Temp\Level_1\Level_2\Level_3\...\Level_5099\Level_5100\". If C:\Temp\ contains a thousand folders, which each contain an additional thousand subfolders, the max recursion level would only be 3.

There is a post floating around somewhere in which I think Melba23 demonstrates using a push/pop stack implemented as an array. It's a usable alternative if recursion still makes you queasy, but recursion works just fine.

:blink:

In the first example, the function never returns if the file isn't found (at least in theory), but they all return once a valid file is found, so it would only happen if the file wasn't found for over 15 hours like I had said above. Am I misunderstanding what you are saying?

Share this post


Link to post
Share on other sites
PsaltyDS

In the first example, the function never returns if the file isn't found (at least in theory), but they all return once a valid file is found, so it would only happen if the file wasn't found for over 15 hours like I had said above. Am I misunderstanding what you are saying?

I was speaking generally about any (properly coded) recursive function. You can look at my _RegSearch() function in the Example Scripts forum as an example.

The function in the first post is not recursive (doesn't call itself). Which post # are you referring to when you say "the first example"?

(Yes it is, and yes it does. Sloppy reading on my part.)

;)

Edited by PsaltyDS

Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
FlyinRiz

I was speaking generally about any (properly coded) recursive function. You can look at my _RegSearch() function in the Example Scripts forum as an example.

The function in the first post is not recursive (doesn't call itself). Which post # are you referring to when you say "the first example"?

;)

I was referring to the OP. I'm not saying that it is a properly coded recursive function, but it is recursive anyway because it's calling itself if no file is found.

Share this post


Link to post
Share on other sites
PsaltyDS

I was referring to the OP. I'm not saying that it is a properly coded recursive function, but it is recursive anyway because it's calling itself if no file is found.

I see it now, and you're right. I missed where he had placed Search() in there.

;)


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×