Finding unknown strings in files

[niubbone]

Hi community,

I'm almost sure you can do this, and i think it may be a bit hard, so it's just an idea more than a real project. I was wondering if it's possible to open any kind of file in txt format (also by renaming them .txt) and check into them full lenght deep for the largest recursive string.

The main idea behind this is:

1)Do some exercise with strings.

2)While doing point 1, creating a very rough compression algorithm to compress files.

So the idea behind this is this:

For example i have a txt file like this:

abcdedf34rf4rg45gt4kkkefglo6ssas-èàpoªבᇖ㱠݊abcderhukty45tttttttr4tttrtrr4fqqf657kkksdfgvrtg56h67i8ok,poòèàè+ùuylktttttttr4tttfvtyh87i795tr653

And so on. I colored some recursive strings you can find in the file. The idea is to open any file, scan the given file for the largest recursive string, then convert it to a shorter one and substitute it to the original one.

Example: tttttttr4ttt would become a random charachters set(maybe done with Chr(Random(1,500,1) or just decided by me) like 12XY. So all the tttttttr4ttt would be replaced with 12XY and the file would be smaller. Decompressing would be the opposite. All the corrispondences (12XY would be stored in the file or in a ini file and deleted after decompressing is done. Is this possible/worth a try?

[jchd]

You are confusing "recursion" and "repetition".

Anyway, try by "simply" follow your agenda and imagine a general algorithm able to determine the string with will be the first best candidate for compression (most repeated and largest).

Hint: in your example string, if abcde is repeated 5 times or more, it becomes a better choice than tttttttr4ttt which is only there twice.

That itself would be a very good exercise. Then find out how you could go through the whole file by making such substitutions one after the other. Then estimate the complexity of your algorithm. Then study a classical scheme (eg. ZLW) and find avenues of improvement. You'll learn a lot.

[UEZ]

Here are some examples about compression:

http://www.autoitscript.com/forum/index.php?showtopic=112273

http://www.autoitscript.com/forum/index.php?showtopic=77988

http://www.autoitscript.com/forum/index.php?showtopic=87441

http://www.autoitscript.com/forum/index.php?showtopic=113794

BR,

UEZ

[niubbone]

Thanks. On my way to learning.