[solved] Arrays, search text inside files

[goldenix]

I have 5 txt files with 100 000 lines in each, inside a folder (Its possible that I will add more files lather). They are my blacklists.

As a sample, lets say I have 100 strings to compare. So i need to search all files 100(or more) times

Im looking for the fastest way to get results.

Right now I have a code that uses filereadline() & if stringinstring()

I wonder, if it will be faster If Ill use _Filereadtoarray() (to read each file in separate array). I do this only once.

And now ill use _arraysearch() to search each array 100 or more times.

is there any faster way in autoitscript? Or maybe I should try other language like visual basic or something, but if its only 3 seconds or so, faster then id like to stick with autoitscript.

Edit: umm or maybe I should put all the lines in a separate databases & use autoitscript SQL UDFS to search the database?

Edited September 14, 2009 by goldenix

[jvanegmond]

Don't use FileReadLine. From help file:

From a performance standpoint it is a bad idea to read line by line specifying "line" parameter whose value is incrementing by one. This forces AutoIt to reread the file from the beginning until it reach the specified line.

Just use FileRead and split it into lines, if you must.

Do you have to use StringInStr or can you do an absolute match?

[goldenix]

Do you have to use StringInStr or can you do an absolute match?

What is an absolute match?

I got tons of files & 100 strings, so I want to see if any of the files, contains any of the 100 strings.

No code exists yet, im in a stage of planning. cuz the code will be long. & I dont want screw up in the beginning.

Edit, actually its more like 300-400 strings & tons of txt files

Edited September 14, 2009 by goldenix

[jvanegmond]

Say the files contain:

lol
wut
apple

Do you want to find "apple" or just "ap" or "ap*"?

Prototype app:

; Initialize the array of 100 strings and fill it
; Find files

; Loop through files
   ; Turn file into array of strings delimited by whatever you are delimiting it by
   ; Loop through file lines
      ; Loop through 100 strings
         ; if fileline matches stringline then
             ;Whatever you want to do on match
         ; end
      ; end
   ; end
; end

[goldenix]

Say the files contain:

lol
wut
apple

Do you want to find "apple" or just "ap" or "ap*"?

Hire are more details as ample:

I drag & drop 2 Files into Gui:

[desudesu] Yukimoto Hitotsu - loving your sister from under her skirt.rar
[Kazuhiko Chakura] Little Girls Kiss [2009-01-10].rar

As a result I will get 9 strings & will search each txt file 9 times:

Yukimoto Hitotsu
loving your
your sister
sister from
from under
under her
her skirt

Kazuhiko Chakura
Little Girls Kiss

Txt Files contain:

lol
wut
apple
skirt

Final resut no match = Item is not blacklisted.

Edited September 14, 2009 by goldenix

[jvanegmond]

Ok, then the app prototype I posted above would work very well.