Sign in to follow this  
Followers 0
header127

What is the best way to store resources, settings, strings etc.?

18 posts in this topic

I'm developing a program that should be as small and not extensive as possible and also fully portable, however, it contains many resource files, settings etc.. So I've had to make an external file that would store all these data. I was thinking about a dll file, but as far as I know it can store only icons, I took under consideration also WinAPI resources, but I excluded this way, because I want to be able to manage my resources even without recompiling the source. Finally I ended up with SQLite database. I use external database file secured with AES and sqlite.dll included in binary in my source. The process is then like this: the program loads dll library to memory, decrypts the database file to a temporary location, loads it to memory, deletes it from the temporary location and loads all resource files saved as binary in the database to controls in GUI or makes temporary files from them. It can also read and write settings, strings for UI and anything else saved in the database. When the program exits it flushes the databse from memory to disk, encrypts it and replaces the old one. The cons of this solution are: I have to load the database to the memory and keep it there until the program exits, the database is unsecured for about one or two seconds everytime when the program loads/unloads the database to/from memory.

So I'm asking, are there any better solutions for storing these kinds of data and that would suit my conditions or at least a way to load only a part of the database?


An honorory member of The League Against Messy Signatures

Share this post


Link to post
Share on other sites



settings: portable apps use ini files, located in the same folder or a subfolder of the executable. no need to extract (except for initialization or for revert to defaults), no need to retract, where's the problem with that?

icons: use the wrapper directive to include icons directly in your script, no need to extract or retract:

#AutoIt3Wrapper_Res_Icon_Add=iconfile.ico

others (for example, many files of unchanging data, like graphics, sound, etc.): compress all to a single archive, use FileInstall to include the archive in your exe, also use FileInstall to include the archiver program (i use 7z.exe & 7z,dll for this, works flawlessly).

and you don't have to retract them back to your script when you leave the pc, just delete them from whatever temp folder you extract them to.

Share this post


Link to post
Share on other sites

Thanks for the reply, but in my case FileInstall isn't  a good option, because I want to have graphics, sounds etc. replaceable even without the need of recompiling as I've written above. Ini file would be then just an additional file. Ideally, end user sees only two files, exe and resource file, and any data from the resource file are never exposed decrypted. I hope there is a time-tested solution somewhere.


An honorory member of The League Against Messy Signatures

Share this post


Link to post
Share on other sites

Well you don't have to recompile to change resources. It's a little tricky but there are plenty of examples around the forum.

Share this post


Link to post
Share on other sites

Well you don't have to recompile to change resources. It's a little tricky but there are plenty of examples around the forum.

 

I've searched and haven't found, if you know where is it, post link please.


An honorory member of The League Against Messy Signatures

Share this post


Link to post
Share on other sites

Thanks for the reply, but in my case FileInstall isn't  a good option, because I want to have graphics, sounds etc. replaceable even without the need of recompiling as I've written above. Ini file would be then just an additional file. Ideally, end user sees only two files, exe and resource file, and any data from the resource file are never exposed decrypted. I hope there is a time-tested solution somewhere.

 

data never exposed decrypted - impossible. when data is used by your app, it must be loaded to memory in its native form, i.e. not encrypted. utilities exist to capture the data in memory.

resources replaceable - by what? by user-created resources? i.e. will a user generate a graphic, and then your app will embed it instead of the graphic that it's shipped with?

replaceable by any of several predefined options embedded in your app? that's FileInstall and .ini perfect match.

i think you should be a bit less abstract in your description of the situation, so we can suggest less abstract solutions.

Share this post


Link to post
Share on other sites

You might have a closer look to File to Base64 String Code Generator. There are some examples how to use it.

Br,

UEZ


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

data never exposed decrypted - impossible. when data is used by your app, it must be loaded to memory in its native form, i.e. not encrypted. utilities exist to capture the data in memory.

well, never heard about that and also don't know how it could work as the utility can't know exact addresses of objects in memory, but maybe I still don't know much about the handling of objects in the memory.

resources replaceable - by what? by user-created resources? i.e. will a user generate a graphic, and then your app will embed it instead of the graphic that it's shipped with?

replaceable by any of several predefined options embedded in your app? that's FileInstall and .ini perfect match.

i think you should be a bit less abstract in your description of the situation, so we can suggest less abstract solutions.

not by users, but by me, I think its simpler to reskin UI just by replacing a few string in the database than by rewriting some parts of the source code.

FileInstall seems to have good compression/storing capabilities, but is unable to process variables as source files so its useless for projects that handle more than just a few resource files. And as for my abstraction, I'm searching for a universal solution I could use in any further project I make.

@UEZ

I use >BinaryToAu3Compressor so far as it can combine up to 3 compression methods.


An honorory member of The League Against Messy Signatures

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

I get that you want your program to be portable, but you must expect any user to know that if they relocate the program somewhere, then they may need to relocate it's parent folder too, along with all settings and additional files that may have been added since install. The registry can also be used instead of an .ini file, but isn't portable to another device/PC.

In the above regard, it is quite feasible to use FileInstall, etc as has been suggested. Anything else is a bit unrealistic, and no-one I know would expect something to work that way.

Short of recompiling to add new user elements, the only other way, would be to use a third party app to embed/attach those elements ... much like some do with something hidden in a Jpeg for example or an mp3 etc.

Edited by TheSaint

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites

so your problem is that FileInstall extracts more files than actually needed, i.e. it extracts all optional files for specific usage, but then your app is using only one of those files.

i can relate to that, i encountered the same issue myself. i chose, for simplicity sake, to go with that flow. i also found that this is more useful, when you want to use your app "Settings" dialog to change any of the options. this way, you have all options readily available for your app to choose from. if you extract just what you need, you need many more extractions to perform. all in all, i found that this method is best. it also requires far less coding.

"And as for my abstraction, I'm searching for a universal solution I could use in any further project I make." - noble cause, but to no avail. no two projects are similar, you would have to adjust anyway.

Share this post


Link to post
Share on other sites

If app's gonna be portable, use external files (ini, txt preferably)
If app's not gonna be portable, I suggest Windows Registry :)

Share this post


Link to post
Share on other sites

If app's gonna be portable, use external files (ini, txt preferably)

If app's not gonna be portable, I suggest Windows Registry :)

 

why use registry, even for non-portable apps? what advantages it has over ini?

and if you choose registry, but later want to make your app portable... what then  :mad2: ? many examples to this scenario exist: PuTTY, WinMerge, ...

guys, please, keep up with the times and make your apps portable. "Setup" is so 90's ;)

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

@UEZ

I use >BinaryToAu3Compressor so far as it can combine up to 3 compression methods.

 

Only 2 compression methods are available - Base64 is not a compression method. Further 2nd one needs additional DLL to extract. Anyhow, you can use what ever you want.

I hope you have understood how to use embedded stuff without extracting it to the hd first and load it again...

Br,

UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

orbs,

The registry has it's usage and some would argue with you that .ini files are "so 90's".


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

orbs,

The registry has it's usage and some would argue with you that .ini files are "so 90's".

 

i asked "why use registry, even for non-portable apps? what advantages it has over ini?" - that was not a rhetorical question, i'd really like to know.

i did not say that .ini are so 90's - i said that about installed apps vs portable apps (where portability is possible, anyway). as a developer, why would you want to bother to develop an installation package, when you can just extract and run your app.

(sky3s, sorry for the discussion getting off-topic)

Share this post


Link to post
Share on other sites

Create a topic in Chat orbs.


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

#17 ·  Posted (edited)

why use registry, even for non-portable apps? what advantages it has over ini?

One quick answer for you.

With reg entries, you can move your executable anywhere on the current PC and it can retrieve stored values. With ini files you always have to include them with the executable when relocating. You can even run two instances of a program, each stored in a different location, and they can share the same basic settings maybe.

Edited by TheSaint

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites

Well, it seems that there is no universal solution and everyone has to figure out what fits his needs best.

However, thanks for your responses.


An honorory member of The League Against Messy Signatures

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0