10 replies to this topic

[martin42]

Hi All,
I have to ask afew questions with regards to Autoit and regular expressions. First Which flavour of regular expression engine does the latest version ofr Autoit have. I found an old post from 6 years ago and that tells me that autoit uses the PCRE regular expression engine. If this is the case, does the latest version of Autoit have the latest version of the PCRE engine, which is Version 8.30 04-February-2012 or at least version eight. Second question: If Autoit indeed does have the PCRE engine and I want the (\ word boundary tokens to except unicode characters how do I get Autoit to compile in UTF8?, as this is what the docs state you must do if you want the (\ word boundary tokens to except unicode. I have to say at this point I have not tried to compile an Autoit script containing regular expressions, however I have compiled other autoit scripts and the only option that seems to be available is whether you want to compile for a 64 bit operating system or 32. Lastly, are there any examples of Autoit being used with regular expressions other than the short article in the help docs.
Warm regards.
Martin Webster.

[qsek]

Hi,
Maybe you didnt read the whole article, there are links to some internet pages:
http://en.wikipedia.org/wiki/Regular_expression
http://www.codeproject.com/dotnet/regextutorial.asp

There is also an unfinished wiki article with a handful of examples: http://www.autoitscript.com/wiki/Tutorial_RegularExpressions
Other examples you can find in the forum with a search for +StringRegExp +regular

[jchd]

Current PCRE implementation in AutoIt does NOT include the PCRE_UCP option, that means only characters 0x00 .. 0x7F have their properties recognized using pattern metacharacters.

I witness more and more use of AutoIt outside english speaking parts of the world and I'll do my best to persuade the devs to include this build option in some future release. It's true that it's about 180kb increment in size but this will have to be put in balance with universal use and highly simplified code and increased speed that recent PCRE releases allow (JIT compiler and UTF-16 native support).

I've been working on an external PCRE lib and companion UDF for handling more of Unicode in PCRE but this work is stalled for now due to personal issues. Anyway, it doesn't make real sense to have a PCRE lib included in AutoIt executables and still be forced to use another external DLL just for handling Unicode properly, but as per now I don't have any solution to this issue.

[ProgAndy]

I've been working on an external PCRE lib and companion UDF for handling more of Unicode in PCRE but this work is stalled for now due to personal issues. Anyway, it doesn't make real sense to have a PCRE lib included in AutoIt executables and still be forced to use another external DLL just for handling Unicode properly, but as per now I don't have any solution to this issue.

How would you pass the results to AutoIt? As IDispatch-Objects or with dedicated function calls and DLLStructs?

[jchd]

I was using classical C-style Dll stuff. My intent is to have the PCRE regexp engine interfaced for both AutoIt and SQLite in the same DLL (using different API, obviously).

[martin42]

Hi and thanks for the answers so far,
I think UTF8 support should be implemented in the next mayjor release of Autoit, as the docs on regular expressions programming say ascii is a lonely old world now a days. I am not an Autoit developer, but an extra optional flag could be added to StringRegExp so if the Autoit programmer wanted to compile the regular expression in UTF8 to support unicode within meta tokens we would have that choice. I do believe python handles this in a similar way, and I am not mixing and matching programming languages for my project having to learn two or three programming languages just to get what I want. what is the exact version of the PCRE engine Autoit is using?.
Warm regards.
Martin Webster.

[jchd]

There is no need to add any parameter to the StringRegExp* functions.
You can allow PCRE UCP (Unicode Character Properties) option by issuing (*UCP) at the head of the pattern, provided the library is compiled with the PCRE_UCP build option, obviously.

Note that AutoIt PCRE is currently (and as always was AFAIK) compiled with UTF-8 support so that you never have to issue the (*UTF8) option in the pattern.

A quick search in the detailed changelog found:
3.3.7.11 (4th July, 2011)
(Beta) AutoIt:
- Changed: PCRE updated to 8.12.

[martin42]

Hi,
this was my original point. there is no option to compile with UTF8 in Autoit, or if there is I have not come across this option. The only options I am aware of is to compile for 64 bit os or 32 bit os.
Warm regards.
Martin Webster.

[BrewManNH]

There isn't an option to compile in ANSI, as it's all Unicode now.

From the help file:

Unicode Support
From version 3.2.4.0 AutoIt is supplied as a Unicode program. The Unicode versions will allow our international friends to finally use AutoIt with extended characters and scripts!

[jchd]

I was talking about the build of PCRE included in AutoIt, not AutoIt core.

[jchd]

When AutoIt will have PCRE built with PCRE_UCP, those (Perlish) notations won't work. Refer to the PCRE documentation to have a complete view of what syntax will be accepted.

As previously mentionned I fully agree with the need for AutoIt to support UCP.