What is the best way to store resources, settings, strings etc.?

[Guest]

I'm developing a program that should be as small and not extensive as possible and also fully portable, however, it contains many resource files, settings etc.. So I've had to make an external file that would store all these data. I was thinking about a dll file, but as far as I know it can store only icons, I took under consideration also WinAPI resources, but I excluded this way, because I want to be able to manage my resources even without recompiling the source. Finally I ended up with SQLite database. I use external database file secured with AES and sqlite.dll included in binary in my source. The process is then like this: the program loads dll library to memory, decrypts the database file to a temporary location, loads it to memory, deletes it from the temporary location and loads all resource files saved as binary in the database to controls in GUI or makes temporary files from them. It can also read and write settings, strings for UI and anything else saved in the database. When the program exits it flushes the databse from memory to disk, encrypts it and replaces the old one. The cons of this solution are: I have to load the database to the memory and keep it there until the program exits, the database is unsecured for about one or two seconds everytime when the program loads/unloads the database to/from memory.

So I'm asking, are there any better solutions for storing these kinds of data and that would suit my conditions or at least a way to load only a part of the database?

[orbs]

settings: portable apps use ini files, located in the same folder or a subfolder of the executable. no need to extract (except for initialization or for revert to defaults), no need to retract, where's the problem with that?

icons: use the wrapper directive to include icons directly in your script, no need to extract or retract:

#AutoIt3Wrapper_Res_Icon_Add=iconfile.ico

others (for example, many files of unchanging data, like graphics, sound, etc.): compress all to a single archive, use FileInstall to include the archive in your exe, also use FileInstall to include the archiver program (i use 7z.exe & 7z,dll for this, works flawlessly).

and you don't have to retract them back to your script when you leave the pc, just delete them from whatever temp folder you extract them to.

[Guest]

Thanks for the reply, but in my case FileInstall isn't  a good option, because I want to have graphics, sounds etc. replaceable even without the need of recompiling as I've written above. Ini file would be then just an additional file. Ideally, end user sees only two files, exe and resource file, and any data from the resource file are never exposed decrypted. I hope there is a time-tested solution somewhere.

[Mat]

Well you don't have to recompile to change resources. It's a little tricky but there are plenty of examples around the forum.

[Guest]

Well you don't have to recompile to change resources. It's a little tricky but there are plenty of examples around the forum.

 

I've searched and haven't found, if you know where is it, post link please.

[orbs]

Thanks for the reply, but in my case FileInstall isn't  a good option, because I want to have graphics, sounds etc. replaceable even without the need of recompiling as I've written above. Ini file would be then just an additional file. Ideally, end user sees only two files, exe and resource file, and any data from the resource file are never exposed decrypted. I hope there is a time-tested solution somewhere.

 

data never exposed decrypted - impossible. when data is used by your app, it must be loaded to memory in its native form, i.e. not encrypted. utilities exist to capture the data in memory.

resources replaceable - by what? by user-created resources? i.e. will a user generate a graphic, and then your app will embed it instead of the graphic that it's shipped with?

replaceable by any of several predefined options embedded in your app? that's FileInstall and .ini perfect match.

i think you should be a bit less abstract in your description of the situation, so we can suggest less abstract solutions.

[UEZ]

You might have a closer look to File to Base64 String Code Generator. There are some examples how to use it.

Br,

UEZ

[Guest]

data never exposed decrypted - impossible. when data is used by your app, it must be loaded to memory in its native form, i.e. not encrypted. utilities exist to capture the data in memory.

well, never heard about that and also don't know how it could work as the utility can't know exact addresses of objects in memory, but maybe I still don't know much about the handling of objects in the memory.

resources replaceable - by what? by user-created resources? i.e. will a user generate a graphic, and then your app will embed it instead of the graphic that it's shipped with?

replaceable by any of several predefined options embedded in your app? that's FileInstall and .ini perfect match.

i think you should be a bit less abstract in your description of the situation, so we can suggest less abstract solutions.

not by users, but by me, I think its simpler to reskin UI just by replacing a few string in the database than by rewriting some parts of the source code.

FileInstall seems to have good compression/storing capabilities, but is unable to process variables as source files so its useless for projects that handle more than just a few resource files. And as for my abstraction, I'm searching for a universal solution I could use in any further project I make.

@UEZ

I use >BinaryToAu3Compressor so far as it can combine up to 3 compression methods.

[TheSaint]

I get that you want your program to be portable, but you must expect any user to know that if they relocate the program somewhere, then they may need to relocate it's parent folder too, along with all settings and additional files that may have been added since install. The registry can also be used instead of an .ini file, but isn't portable to another device/PC.

In the above regard, it is quite feasible to use FileInstall, etc as has been suggested. Anything else is a bit unrealistic, and no-one I know would expect something to work that way.

Short of recompiling to add new user elements, the only other way, would be to use a third party app to embed/attach those elements ... much like some do with something hidden in a Jpeg for example or an mp3 etc.

Edited July 5, 2013 by TheSaint

[orbs]

so your problem is that FileInstall extracts more files than actually needed, i.e. it extracts all optional files for specific usage, but then your app is using only one of those files.

i can relate to that, i encountered the same issue myself. i chose, for simplicity sake, to go with that flow. i also found that this is more useful, when you want to use your app "Settings" dialog to change any of the options. this way, you have all options readily available for your app to choose from. if you extract just what you need, you need many more extractions to perform. all in all, i found that this method is best. it also requires far less coding.

"And as for my abstraction, I'm searching for a universal solution I could use in any further project I make." - noble cause, but to no avail. no two projects are similar, you would have to adjust anyway.

[ActualAkshay]

If app's gonna be portable, use external files (ini, txt preferably)
If app's not gonna be portable, I suggest Windows Registry

[orbs]

If app's gonna be portable, use external files (ini, txt preferably)

If app's not gonna be portable, I suggest Windows Registry

 

why use registry, even for non-portable apps? what advantages it has over ini?

and if you choose registry, but later want to make your app portable... what then   ? many examples to this scenario exist: PuTTY, WinMerge, ...

guys, please, keep up with the times and make your apps portable. "Setup" is so 90's

[UEZ]

@UEZ

I use >BinaryToAu3Compressor so far as it can combine up to 3 compression methods.

 

Only 2 compression methods are available - Base64 is not a compression method. Further 2nd one needs additional DLL to extract. Anyhow, you can use what ever you want.

I hope you have understood how to use embedded stuff without extracting it to the hd first and load it again...

Br,

UEZ

Edited July 5, 2013 by UEZ

[guinness]

orbs,

The registry has it's usage and some would argue with you that .ini files are "so 90's".

[orbs]

orbs,

The registry has it's usage and some would argue with you that .ini files are "so 90's".

 

i asked "why use registry, even for non-portable apps? what advantages it has over ini?" - that was not a rhetorical question, i'd really like to know.

i did not say that .ini are so 90's - i said that about installed apps vs portable apps (where portability is possible, anyway). as a developer, why would you want to bother to develop an installation package, when you can just extract and run your app.

(sky3s, sorry for the discussion getting off-topic)

[guinness]

Create a topic in Chat orbs.

[TheSaint]

why use registry, even for non-portable apps? what advantages it has over ini?

One quick answer for you.

With reg entries, you can move your executable anywhere on the current PC and it can retrieve stored values. With ini files you always have to include them with the executable when relocating. You can even run two instances of a program, each stored in a different location, and they can share the same basic settings maybe.

Edited July 7, 2013 by TheSaint

[Guest]

Well, it seems that there is no universal solution and everyone has to figure out what fits his needs best.

However, thanks for your responses.