Jump to content
topten

Need Expert Advise about files and DB

Recommended Posts

topten

I have multi-processes and several scripts , where I need to open files for reading/writing encrypt info, save it in files, read it, update and when the amount of process becomes too large the files become irresponsive (I am talking about 50-70 processes in minute)

I need an expert advise, which way should I organize my data: Should I continue working with files or should I do something with data bases or structure the info in arrays? Which way is the best to accomplish that?

Share this post


Link to post
Share on other sites
jchd

This is way too vague to guess anything.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
topten

Sure, I should have given more info

The main problem I am facing

$FileOP4 = FileOpen ($pathfile & "idcounter.dat",0)
   $idcounter = FileRead ($FileOP4)
   FileClose ($FileOP4)

I am getting the value of $idcounter for while untill I have more scripts reading and writing into this file. And at some point I even can't read this file, getting zero value. Which is the best way to deal with this?

Edited by topten

Share this post


Link to post
Share on other sites
jchd

Then your problem is contention created by multiple asynchronous concurrent read/write accesses. Then yes, a database is precisely suited to handle this smartly.

If all your processes run on a single machine, then SQLite can greatly help. If you're accessing data remotely over a LAN, then a client-server engine will be necessary (MS SQL, MySQL, Postgress, ...) but they need administration and maintainance, contrary to the simple SQLite engine.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
TheSaint

Do you need everything to be read so quickly?

If not, then some sleeps would help.

If so, then using a database would be better.


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×