Sign in to follow this  
Followers 0
michaelslamet

[SOLVED] Improve searching file speed: is this scenario possible?

6 posts in this topic

#1 ·  Posted (edited)

Greetings to all fellow members,

My application need to search location of a text file (based on user input) on a network drive/folder that contain thousands of files in hundreads of folder (and the number of files and folder are growing each day).

Everytime the user input a new value, the application start searching again from the beginning. The searching is slow because of the numbers of files and folders also because it's a network (windows sharing/smb) drive.

Currently i'm using _RecFileListToArray

An idea come to me: what if this proccess only needed to run one time, only at the first search. On the 2nd or next search, the application will search on the ARRAY which is much faster because on the first search it already put all the file location in the array?

Assuming if after first search the are new text files come to that folder, we can ingore it.

Or on the first search we list all the text files complete with the path and save it in a text file, so on the next search we can read from that text file.

Is this idea possible? How to accomplish this?

A example code will be very appreciated.

Thanks! :)

Edited by michaelslamet

Share this post


Link to post
Share on other sites



You should already have the code for this, you searched the file shares once, don't search them again if you need to do it again. What you should do is get a list of all files first, then search the array for the file you want, rather than using the function to search for just a file or files that are asked for. It's basically the same as how Windows indexing works.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

Make it a two step process (like I do with SMF), first search the files and add them to a SQLite database, second let the user search those results.

Share this post


Link to post
Share on other sites

AZJIO, thanks a lot for pointing _FileSearch UDF.

I'm using it now on my application, it's about 15% faster than current UDF.

:)

Share this post


Link to post
Share on other sites

KaFu, thanks for your advice.

Now I'm make it in 2 process:

1. Read the files list and put it on array

2. let the user search on the array

I dont need SQLite for now, the files is still not too many and can be handled using array :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0