TheSaint

Virtual INI UDF & ASCII Checker

4 posts in this topic

#1 ·  Posted (edited)

I had a need, and so I developed.

Before I did that though, I did a quick search of the forum, and found a few things, but none seemed to suit what I really wanted ... though to be honest, I did not investigate them deeply or search extensively. In any case, I am not very good at modifying the code of others, nor do I enjoy doing so, so in the end I decided it was best to start from scratch ... re-inventing the wheel perhaps, but a wheel I feel a certain degree of comfort with. I also thought about Maps, but don't know enough.

_IniString Functions (was Ini_InMem)

Ini in memory

Settings INI - Read/Write once to the disk

INI File Processing Functions

... sure to be others.

As with all my stuff, feel free to use (no guarantees) or modify ... just give me credit where due.

I always go for the quick & simple approach, as those who know me here, are well aware, so without a doubt, my code could be improved - sped up, RegExp used, etc. So feel free to do that, and convert into a proper UDF, if you want, following best practices etc, as I am sure others will appreciate it.

Quote

Some background.
I have been a long time user and fan of INI files. I realize, they have their issues, and in many cases you are no doubt better off using XML, SQL or other (be assured though, they all have their own issues). But if your needs are simple, and you want quick coding and processing, then INI files are hard to beat.

A few years ago, while trying to perfect the INI elements of a project I was working on, I discovered that quotes (double & single) get stripped if a value is encapsulated in them. That seemed stupid to me, and I wondered if AutoIt was responsible or just some quirk etc of Windows (Microsoft), and why. I don't recall ever getting a satisfactory explanation at the time, after starting a topic about it (in MVP Chat I think), though it was made clear it was a Windows issue. Later, I discovered via Wikipedia, that quotes were used to encapsulate 'escape characters' and then stripped after doing their job. EDIT - By the way, I think it pretty lame of Microsoft to use both Single Quotes (apostrophe) and Double Quotes to do that. One of either would surely have been sufficient, leaving the other as a replacement possibility for the other. I do understand that INI files were initially conceived as settings files, but really that issue should have been fixed at a later date, especially as they themselves started using INI files for other things (i.e. cdplayer.ini), and where text encapsulated by quotes could become quite likely.

INI file (Wikipedia)

Recently, I have come upon a need to extract data from an approximately 50 Mb downloaded Index file, just extracting the few elements I need, and storing them in an INI file. It is quite a time consuming process, especially on my underpowered Netbook, which is where I am using the program I built. I have been considering ways to speed up things. Two good sounding methods, out of several possibles, have come to mind - 1) RAM Disk and 2) Virtual INI processing. That last being a better option, when sharing code (program) with others.

Another method, which I am currently using, that did give me some measurable benefit, was to download the almost 50 Mb Index file in Stages (i.e. 30 x 1.4 Mb roughly), extracting data from each in turn. I have also considered extracting to 10 (or 11) separate INI files, rather than the single large INI file (4~5 Mb), as things noticeably slow up as the INI file increases in size ... obviously due to writing time.

I can't really speed up the extraction process, but that last method may speed up the writing stage.  However, it would require significant program changes, and perhaps not gain me much benefit. To complicate things, there are too many Section names (Index ID's) for the IniReadSectionNames command, so I have had to split them off into their own (read/write) line-by-line index file, and create code to deal with duplicates. When read, Index ID's and Titles populate a Listbox control in a user GUI.

So at this point in time, it seems best to use the Virtual INI approach to gain a significant reduction in time taken. For my own specific use, with my slow Netbook, I may also use a RAM Disk.

Due to the aforementioned INI issues, and lack of available specific data, probably because it is always  program concentric, I have created an ASCII Checker program, which I guess you can say, is AutoIt  concentric. Because others may also find it handy, I am providing it here. NOTE - As yet, I am only aware that Escape characters exist, but have not investigated or catered for them in any way.

ASCII Checker.zip

ASCchecker.png

The Virtual_INI_UDF is still a work in progress, though I have done most of the functions now, with the working but incomplete _Ini_Test function, still requiring changes etc from the information gleaned by the ASCII Checker program.

Here's something to play with meanwhile.

Updated files (24th February 2017)

Virtual_INI_UDF.au3  NEW

Example.au3 (23rd February 2017)

All pretty basic, but managed to load my 4 Mb INI file (seemingly ok, but took a few seconds) ... though I have only worked with much smaller testing ones so far.

P.S. While I have done a fair amount of testing, it has been pretty basic, and mostly limited to the examples provided. Testing has definitely not been extensive.

Spoiler

Previous Downloads

17-02-2017 - UDF (12) + Example (9)
21-02-2017 - UDF (10) + Example (5)
23-02-2017 - UDF (6)

 

Edited by TheSaint
Updates
1 person likes this

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Well I finally got some time free to do a bit more work on this project. Files at first post, were removed and updated. INI files no longer required, as they get created as they are needed during the order of Examples.

I have rewritten some of it, and finished a few more functions, added one or two, plus pretty much rewritten all the Examples.

I've also been a good boy, and added in lots of short comments.

Current status is as follows.

FUNCTIONS
_Ini_AddUpdate($ini_virtual, $section, $key, $value = "")
_Ini_Close($ini_virtual)
_Ini_Create($section, $key, $value = "")
_Ini_Inform($ini_virtual, $show = "")
_Ini_Open($ini_fle)
_Ini_Read($ini_virtual, $section, $key = "", $report = "")
_Ini_Remove($ini_virtual, $section, $key = "")
_Ini_Rename($ini_virtual, $section, $new, $key = "")
_Ini_Save($ini_virtual, $ini_fle)
_Ini_Show($ini_virtual, $section = "")
_Ini_WriteSection($ini_virtual, $data, $section, $replace = 0)

NOT YET FULLY IMPLEMENTED
_Ini_Test($ini_virtual, $section, $key, $value, $new, $job)

NOT YET IMPLEMENTED
_Ini_CopySection($ini_virtual, $section, $newsect)
_Ini_Sections($ini_virtual)
_Ini_Log (maybe)

 

Edited by TheSaint
1 person likes this

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Files at first post, were updated.

I have rewritten some of it again, and finished two more functions (one a new one), plus rewritten some aspects of most Examples.

Basically, I rewrote some code, because I felt some aspects should not be solely reliant on square bracket detection, but also include Chr(13) and Chr(10) in the mix, so as not to leave any room for ambiguity.

Current status is as follows.

FUNCTIONS
_Ini_AddUpdate($ini_virtual, $section, $key, $value = "")
_Ini_Close($ini_virtual)
_Ini_Create($section, $key, $value = "")
_Ini_Inform($ini_virtual, $show = "")
_Ini_KeyNames($ini_virtual, $section)  NEW
_Ini_Open($ini_fle)
_Ini_Read($ini_virtual, $section, $key = "", $report = "")
_Ini_Remove($ini_virtual, $section, $key = "")
_Ini_Rename($ini_virtual, $section, $new, $key = "")
_Ini_Save($ini_virtual, $ini_fle)
_Ini_SectionNames($ini_virtual) RENAMED & COMPLETED
_Ini_Show($ini_virtual, $section = "", $title = "") CHANGED - TITLE parameter added
_Ini_WriteSection($ini_virtual, $data, $section, $replace = 0)

NOT YET FULLY COMPLETED (but working)
_Ini_Test($ini_virtual, $section, $key, $value, $new, $job) FURTHER IMPLEMENTATIONS

NOT YET IMPLEMENTED
_Ini_CopySection($ini_virtual, $section, $newsect) (maybe)
_Ini_Log (maybe)

NOTES - It has occurred to me, that being virtual, I could add a switch (flag), that allows for a virtual INI that doesn't conform entirely to the standard file based one. For example, quotes could be allowed ... remembering that any such virtual INI written to file, should really only ever be used in a virtual environment (read & written to that way). I haven't thought too deeply about the benefits etc yet, but I believe you could think of it, as almost another type of database (if extremely similar to INI files). To reduce confusion or problems, another file type could be specified (i.e. .vini or .virt etc).

Edited by TheSaint

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

While incorporating this into the program, that essentially inspired me to create the UDF, I decided I really should give most of the parameter variables a more unique name, so I have added 'virtual_' to them (i.e. 'virtual_section, virtual_key, etc'.

So the UDF file has been updated at first post. Example remains unchanged.

As for the benefit I was hoping for with my program, I actually got the opposite in the area that really mattered most. I did gain a 36 second benefit writing to a small INI file (creating virtually then dumping to file), that was around 100 Kb's or so, but as the virtual INI grew in size, the benefit was lost, and before long was worse than using regular INI functions. Especially as the eventual INI file was gonna be around 4~5 Mb's. Most disappointing.

So unless someone else has a great way to speed things up (beyond the limit of my coding experience), then this project, while useful perhaps for some smaller applications, is dead and buried as far as I am concerned. I suspect there are ways to speed things up, but I just don't have that knowledge or skill.

I was hoping to reduce a two hour process on my Netbook (using regular INI functions), to something a bit quicker, but at the two and a half hour mark (with the virtual functions), and not quite a third done, it was clear it was going to extend out to 6 hours plus. And it was obvious that my Netbook had been working too hard, and was running slow and quite hot, even causing the fan to play up a little later, requiring a shutdown and a rest.

So I guess it is Plan B now ... RAM Disk. Or just live with things as they are. Certainly once I upgrade the Netbook to a more powerful PC, the times will be much shorter. The current maximum 2 Gb of RAM is not much.

Edited by TheSaint

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • TheDcoder
      By TheDcoder
      Hello, I recently opened a bug report without reading the Helpfile... My bad . After @Melba23's gentle reminder, I was curious about why it was like that.
      It is about SetError's behaviour. This is the example from the bug report:
      Example() If @error Then ConsoleWrite("Error" & @CRLF) Else ConsoleWrite("No Error" & @CRLF) EndIf Func Example() SetError(1) Sleep(1000) EndFunc What I tried to do is set Example's (my user defined function's) @error value to 1... but the value set by SetError is cleared after calling a function, I wonder why? Why should calling to an external function effect my function's @error which is set when my function returns.
      Setting the error of a UDF in advance by using SetError makes sense... but I cannot find a reason why calling a function should clear it? Please note that I am not talking about @error, I am talking about the @error set by my function when it ends/returns!
      I hope someone can enlighten me, thanks for the answers in advance!
      P.S I tried to explain my best but my English is not very good and I didn't feel like I did a good job explaining today, so please pardon any mistakes that I have made
    • pboom
      By pboom
      I am looking for a way to retrieve filtered messages from the ‘system debug channel.' also known as  ‘kernel-mode debug output.'

      AutoIt must do the capture in real time. The following AutoIt UDF almost does what is required but it only captures application level, or Win32 debug output.

      https://www.autoitscript.com/forum/topic/82889-capture-debug-information-udf/#comment-593268

      The utility DebugView by Sysinternals captures the information as required by turning on Capture Kernal and in my case using the Filter include:

      *Incoming connection*

      The use of DebugView to do this is covered in the following tech note;

      https://www.tacticalsoftware.com/support/tech-notes/logging-com-port-activity.htm

      https://technet.microsoft.com/en-us/sysinternals/debugview.aspx

      However to make to make the information from DebugView available to my AutoIt script required DebugView capture to a text file and then my AutoIt script monitor that file for changes. The use of DebugView to capture the system debug channel could be made to work, but it was less than reliable and difficult to get started. The startup wasn’t something that could be easily automated not even with AutoIt.

      If you understood what I am talking about and made it this far, I think an explanation of the application is in order. Lots of details here sorry trying to answer questions in advance.

      I support a large installation of General Electric MUSE application. MUSE is a Windows-based medical application that processes and archives ECGs (electrocardiograms) taken on dedicated hardware (ECG Carts). Several methods exist on the cart to get the ECG from the Carts to the MUSE system; they range from floppies (on old obsolete hardware), memory cards, RS232 serial ports, and hardwired network connections.

      In our installation, we choose not to use the vendor-supplied network solution due to a variety of reasons I won’t get into here.  Instead, we have designed our own connection solution.

      We use a wireless serial server mounted on the ECG carts connecting to a server running a Serial/IP COM Port Redirector. The ECG cart and MUSE application think they are talking to each other via an RS232 port and as far as they are concerned, they are. However, this RS232 cable happens to run through our province (think State) wide Health Care WAN.  The hardware and software used can be seen on these two sites;

      http://www.bb-elec.com/Products/Wireless-Cellular/AirborneM2M-802-11-a-b-g-n-Dual-Band-Wireless/AirborneM2M-Industrial-Dual-Band-Wi-Fi-Router-Brid.aspx

      https://www.tacticalsoftware.com/virtual-serial-port-redirector/serial-ip.htm

      This setup works well we have over 130 ECG carts connecting using this setup. However, the end users are not technical, and there is a lot that can go wrong with wireless connections. So we do get complaints, often after the fact, that the ECG cart would not connect. A log of what ECG carts connected and when would be very helpful.

      The Serial redirector software can be configured to log all activity to the Kernal-mode Debug output. The serial redirector software itself being kernel level software. For configuration of the Wireless modules, we have custom written software (written in AutoIt) that amongst other things can display relevant configuration information for a Wireless module given it’s IP address.

      By extracting messages like the ones below from the Kernal-Mode Debug channel;

      COM56 : ½ Incoming connection from 10.158.188.172:51562

      COM18 : ½ Incoming connection from 10.158.188.200:50896

      COM19 : ½ Incoming connection from 10.158.188.180:59074

      COM68 : ½ Incoming connection from 142.239.15.82:34322

      We can have the module configuration program retrieve the configuration. The retrieved configuration contains more information such as the module ID number and wireless signal strength. This information is then logged to a file which is later loaded into a database. We can then query the database for connections made by a particular module within a specified time frame. The results of these Queries help us determine if the module was connected or is having problems connecting. Problems are usually indicated by poor signal strength and frequent re-connecting.

      So what I am looking for is a way for our module configuration program (written in AutoIt) to retrieve filtered Kernal-Level debug messages directly without using the DebugView application.

      The Forum post listed at the first of this message includes the source code for the DLL. So if you are versed in these matters and Visual Studio this may be an easy task. I looked at what needed to be done but, I was way over my head. If you look up the price of the serial IP redirector software, you can see that there is some money in our project for such things however, I do have a spending limit for purchases such as this.

       
    • jollypk
      By jollypk
      hI,
      Is there any UDF for Zeromq ?
      I need to communicate with a remote and local socket. Any suggestion or guidance will be very appreciated.
       
      Thanks
    • FrancescoDiMuro
      By FrancescoDiMuro
      Good morning everyone
      I am working on a little script, which takes some data from a SQLite DB and should create a sort of report, inserting rows in a Word Document... I arrived at the point of:
      _Word_DocTableWrite() and, I don't know how to set the range parameter? What does that specify? 
      Thanks a lot for the help
      EDIT:
      Managed to write a table in the Word document, but now I get an error when I save the document with _Word_DocSaveAs(), with error 2.
      Which are possible causes? Thanks a lot, again
      EDIT 2:
      ... And, how can I set a border to the table? Maybe, with a sort of auto-formatting for text ( larger is the text, larger is the height/width of the table's cell ).
      Thanks  
      EDIT 3 ( bug ):
      Including the parameter $WdSaveChanges in the function _Word_DocSaveAs(), a save dialog box appears, and it should not do it, as it's written in the MSDN documentation:
      wdSaveChanges -1 Save pending changes automatically without prompting the user. Thanks again for everyone will answer to me  
    • SorryButImaNewbie
      By SorryButImaNewbie
      Hello,
      I try to add a new UDF to my AutoIT library, and it seems I fail. I started to work following the insturctions found on the wiki . 
      I made an extra folder just next to my AutoIT folder, called ExtraUDF, and copied the file CUIAutomation2.au3 in it (I try to use this UDF for automating a program) then I run sciteconfig, set this folder up as User Include Folder, applied the changes, then run the calltip manager and got a msg when I tried to parse it. I attach a few pictures for better present the problem.
      I think I miss something here, should i put anything else on the ExtraUDF folder? I dont think that the CallTip manager added the udf to scite, If I try to run it i get a msg saying that I Scite cant open the include file and naturally after that, It can't use the functions called from it.
      Thank you for your help and insight as always! Please ask, if anything is fuzzy about the problem