Sign in to follow this  
Followers 0
martin42

regular expressions questions?

11 posts in this topic

Hi All,

I have to ask afew questions with regards to Autoit and regular expressions. First Which flavour of regular expression engine does the latest version ofr Autoit have. I found an old post from 6 years ago and that tells me that autoit uses the PCRE regular expression engine. If this is the case, does the latest version of Autoit have the latest version of the PCRE engine, which is Version 8.30 04-February-2012 or at least version eight. Second question: If Autoit indeed does have the PCRE engine and I want the (\;) word boundary tokens to except unicode characters how do I get Autoit to compile in UTF8?, as this is what the docs state you must do if you want the (\:) word boundary tokens to except unicode. I have to say at this point I have not tried to compile an Autoit script containing regular expressions, however I have compiled other autoit scripts and the only option that seems to be available is whether you want to compile for a 64 bit operating system or 32. Lastly, are there any examples of Autoit being used with regular expressions other than the short article in the help docs.

Warm regards.

Martin Webster.

Share this post


Link to post
Share on other sites



Hi,

Maybe you didnt read the whole article, there are links to some internet pages:

http://en.wikipedia.org/wiki/Regular_expression

http://www.codeproject.com/dotnet/regextutorial.asp

There is also an unfinished wiki article with a handful of examples: http://www.autoitscript.com/wiki/Tutorial_RegularExpressions

Other examples you can find in the forum with a search for +StringRegExp +regular


Teamspeak 3 User Viewer - Quick and functional TS3 Query script, which shows online users.Cached Screenshot Deleter - Deletes older Fraps Screenshots if they exceed a specified limit.Unresolved Topics:Intercept and modify dragdrop text behaviour in scite

Share this post


Link to post
Share on other sites

Current PCRE implementation in AutoIt does NOT include the PCRE_UCP option, that means only characters 0x00 .. 0x7F have their properties recognized using pattern metacharacters.

I witness more and more use of AutoIt outside english speaking parts of the world and I'll do my best to persuade the devs to include this build option in some future release. It's true that it's about 180kb increment in size but this will have to be put in balance with universal use and highly simplified code and increased speed that recent PCRE releases allow (JIT compiler and UTF-16 native support).

I've been working on an external PCRE lib and companion UDF for handling more of Unicode in PCRE but this work is stalled for now due to personal issues. Anyway, it doesn't make real sense to have a PCRE lib included in AutoIt executables and still be forced to use another external DLL just for handling Unicode properly, but as per now I don't have any solution to this issue.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

I've been working on an external PCRE lib and companion UDF for handling more of Unicode in PCRE but this work is stalled for now due to personal issues. Anyway, it doesn't make real sense to have a PCRE lib included in AutoIt executables and still be forced to use another external DLL just for handling Unicode properly, but as per now I don't have any solution to this issue.

How would you pass the results to AutoIt? As IDispatch-Objects or with dedicated function calls and DLLStructs?

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

I was using classical C-style Dll stuff. My intent is to have the PCRE regexp engine interfaced for both AutoIt and SQLite in the same DLL (using different API, obviously).


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Hi and thanks for the answers so far,

I think UTF8 support should be implemented in the next mayjor release of Autoit, as the docs on regular expressions programming say ascii is a lonely old world now a days. I am not an Autoit developer, but an extra optional flag could be added to StringRegExp so if the Autoit programmer wanted to compile the regular expression in UTF8 to support unicode within meta tokens we would have that choice. I do believe python handles this in a similar way, and I am not mixing and matching programming languages for my project having to learn two or three programming languages just to get what I want. what is the exact version of the PCRE engine Autoit is using?.

Warm regards.

Martin Webster.

Share this post


Link to post
Share on other sites

There is no need to add any parameter to the StringRegExp* functions.

You can allow PCRE UCP (Unicode Character Properties) option by issuing (*UCP) at the head of the pattern, provided the library is compiled with the PCRE_UCP build option, obviously.

Note that AutoIt PCRE is currently (and as always was AFAIK) compiled with UTF-8 support so that you never have to issue the (*UTF8) option in the pattern.

A quick search in the detailed changelog found:

3.3.7.11 (4th July, 2011)

(Beta) AutoIt:

- Changed: PCRE updated to 8.12.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Hi,

this was my original point. there is no option to compile with UTF8 in Autoit, or if there is I have not come across this option. The only options I am aware of is to compile for 64 bit os or 32 bit os.

Warm regards.

Martin Webster.

Share this post


Link to post
Share on other sites

There isn't an option to compile in ANSI, as it's all Unicode now.

From the help file:

Unicode Support

From version 3.2.4.0 AutoIt is supplied as a Unicode program. The Unicode versions will allow our international friends to finally use AutoIt with extended characters and scripts!


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

I was talking about the build of PCRE included in AutoIt, not AutoIt core.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

When AutoIt will have PCRE built with PCRE_UCP, those (Perlish) notations won't work. Refer to the PCRE documentation to have a complete view of what syntax will be accepted.

As previously mentionned I fully agree with the need for AutoIt to support UCP.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0