[SOLVED] Improve searching file speed: is this scenario possible?

[michaelslamet]

Greetings to all fellow members,

My application need to search location of a text file (based on user input) on a network drive/folder that contain thousands of files in hundreads of folder (and the number of files and folder are growing each day).

Everytime the user input a new value, the application start searching again from the beginning. The searching is slow because of the numbers of files and folders also because it's a network (windows sharing/smb) drive.

Currently i'm using _RecFileListToArray

An idea come to me: what if this proccess only needed to run one time, only at the first search. On the 2nd or next search, the application will search on the ARRAY which is much faster because on the first search it already put all the file location in the array?

Assuming if after first search the are new text files come to that folder, we can ingore it.

Or on the first search we list all the text files complete with the path and save it in a text file, so on the next search we can read from that text file.

Is this idea possible? How to accomplish this?

A example code will be very appreciated.

Thanks!

Edited January 7, 2013 by michaelslamet

[BrewManNH]

You should already have the code for this, you searched the file shares once, don't search them again if you need to do it again. What you should do is get a list of all files first, then search the array for the file you want, rather than using the function to search for just a file or files that are asked for. It's basically the same as how Windows indexing works.

[AZJIO]

[KaFu]

Make it a two step process (like I do with SMF), first search the files and add them to a SQLite database, second let the user search those results.

[michaelslamet]

AZJIO, thanks a lot for pointing _FileSearch UDF.

I'm using it now on my application, it's about 15% faster than current UDF.

[michaelslamet]

KaFu, thanks for your advice.

Now I'm make it in 2 process:

1. Read the files list and put it on array

2. let the user search on the array

I dont need SQLite for now, the files is still not too many and can be handled using array