StringRegExp(): optional capture

[c.haslam]

This regexp

'(?xU) (?!Func)  (Local)? \  (\$\w+) \s* (\=) \s* (\w+) \( ([^\(]*) \)'

matches

Local $gNowSecs = NowSecs($a,45)

with flag=3, and StringRegExp returns an array: [ "Local","$gNowSecs","=","NowSecs","$a,45"]

but it does not match

$gNowSecs = NowSecs($a,45)

How can I get StringRegExp() to handle both strings? In the latter case, I would prefer that it return ["$gNowSecs","=","NowSecs","$a,45"]

BTW I think the help for StringRegExp() is excellent. I have consulted  pcrepattern man page .

[jchd]

The number of capturing groups in the pattern dictates how many results you're going to get. That means that you can't expect this to work without giving you an empty capture first when Local isn't there. Also the whitespace following optional Local is not itself optional and this is why your pattern fails.

Local $s = "$gNowSecs = NowSecs($a,45)"
Local $a = StringRegExp($s, '(?xU) (?!Func)  (Local\ )? (\$\w+) \s* (\=) \s* (\w+) \( ([^\(]*) \)', 3)
_ArrayDisplay($a)

Observe that the first capture is now an empty string.

Thank you for compliments!

Edited August 13, 2015 by jchd

[c.haslam]

Thank you for solving my problem.

And I have found that this works for Local, Global and none:

(?xU) (?!Func)  ((?|Local|Global))? \s* (\$\w+) \s* (\=) \s* (\w+) \( ([^\(]*) \)

I find it interesting that the the match made by an alternative group, which is always non-capturing , can be captured by using plain ( ... )

[jchd]

Of course. Think of groups (capturing or not) as block constructs of a programming language (which PCRE is).

[c.haslam]

I am now trying to get my regexp to handle PopulateLvlFnamAr('nat',$gNatLvlFnamAr)

The proposed regexp is

(?xU) (?!Func)  ((?|Local|Global))? \s* (((\$\w+) \s*)? (?:\=))? \s* (\w+) \( ([^\(]*) \)

For PopulateLvlFnamAr('nat',$gNatLvlFnamAr) it produces

["","","","","PopulateLvlFnamAr","'nat',$gNatLvlFnamAr"]

and for Local $gNowSecs = NowSecs(12,34) it produces

["Local","$gNowSecs =","$gNowSecs ","$gNowSecs","NowSecs","12,34"]

Ideally, I would like the elements of the vector to be [ Local/Global, var, function, arguments] where Local/Global, var and arguments can be empty.

What am I not understanding?

How fo I get from where I am now to where I want to go?

[mikell]

May I suggest

#Include <Array.au3>

$str1 = "  Local $gNowSecs = NowSecs(12,34)"
$res1 = StringRegExp($str1, '(?x) \s*((?|Local|Global))? \s* (\$\w+)? \s*=?\s* (\w+) \(? ([^\)]*) ', 3)

$str2 = " PopulateLvlFnamAr('nat',$gNatLvlFnamAr)"
$res2 = StringRegExp($str2, '(?x) \s*((?|Local|Global))? \s* (\$\w+)? \s*=?\s* (\w+) \(? ([^\)]*) ', 3)

 _ArrayDisplay($res1)
 _ArrayDisplay($res2)

 

Edited August 13, 2015 by mikell

[jchd]

For parsing AutoIt code in general you'll need a grammar parser. Not as simple as it looks first, but you can still start with regexps. I played a bit with this but never took the time/pain to finish it:

Local $REdef_Spacing =              '(?(DEFINE) (?<spacing> (?: \ \t )* ) )'

Local $REdef_Directive =            '(?(DEFINE) (?<directive> ^ (?&spacing) # .* ) )'

Local $REdef_Blkcomment =           '(?(DEFINE) (?m) (?<blkcomment> (?: ^ (?&spacing) #c (?: s | omments-start ) .*? ^ (?&spacing) #c (?: e | omments-end ) .*? \R (?-m) ) )'
Local $REdef_Linecomment =          '(?(DEFINE) (?<linecomment> (?&spacing) ; .*? $ ) )'

Local $REdef_String =               '(?(DEFINE) (?<string> (?: " [^"]*? " | '' [^'']*? '' )+ ) )'
Local $REdef_Integer =              '(?(DEFINE) (?<integer> [-+]? \d+ ) )'
Local $REdef_Real =                 '(?(DEFINE) (?<real> [-+]? \d* \. \d+ (?: E [-+]? \d*) ) )'
Local $REdef_Boolean =              '(?(DEFINE) (?<boolean> \b (?: True | False ) \b ) )'
Local $REdef_Hexvalue =             '(?(DEFINE) (?<hexvalue> 0x [[:xdigit:]]+ ) )'
Local $REdef_Macro =                '(?(DEFINE) (?<macro> @ \w+ ) )'
Local $REdef_Keyword =              '(?(DEFINE) (?<keyword> \b (?: Null | Default ) \b ) )'
Local $REdef_Fctname =              '(?(DEFINE) (?<fctname> \b \w+ \b ) )'
Local $REdef_Literal =              '(?(DEFINE) (?<literal> (?&spacing) (?: (?&string) | (?&hexvalue) | (?&real) | (?&integer) | (?&macro) | (?&boolean) | (?&keyword) | (?&fctname) ) (?&spacing) ) )'

Local $REdef_Varname =              '(?(DEFINE) (?<varname> \$ \w+ ) )'
Local $REdef_Operator =             '(?(DEFINE) (?<operator> (?&spacing) (?: \bAND\b | \bNOT\b | \bOR\b | <= | < | == | = | >= | > | & | + | - | \* | / | \^ ) (?&spacing) ) )'
Local $REdef_Expression =           '(?(DEFINE) (?<expression>  ) )'
Local $REdef_Rvalue =               '(?(DEFINE) (?<rvalue> (?&expression) (?: (?&operator) (?&expression) )* ) )'
Local $REdef_AssignOper =           '(?(DEFINE) (?<assignoper> += | -= | *= | /= |&= | = ) )'
Local $REdef_Lvalue =               '(?(DEFINE) (?<lvalue> (?&spacing) (?&varname) (?: (?&spacing) (?: \[ (?&rvalue) \] | \. [^\ \t\-+\*/\^\[\$.\s<=>&]+ ) )* (?&spacing) ) )'
Local $REdef_Ternary =              '(?(DEFINE) (?<ternary> (?&rvalue) \? (?&rvalue) : (?&rvalue) ) )'
...

This definitely incomplete and most probably wrong in many places. Still much work towards a pattern for "AutoIt statement". At least this gives you the idea, but a dedicated grammar parser would be more adequate.

[c.haslam]

mikell,

The problem that I have with your pattern is that when I apply it to lines of a script, lines that do not contain calls to functions also match it.

jchd,

Thank you for your thoughts. My objective is to create a tracing utility that offers more than the existing one: one that will report values passed in parameters to calls to user-written functions. I have got a bit further. This is where I am now:

Local $regexp = '(?xJ) (?!Func)                                     '& _
    '(?|                                                            '& _
        '(Local) \s+ (\$\w+) \s+ = \s+ (\w+)                    |   '& _
        '(Global) \s+ (\$\w+) \s+ = \s+ (\w+)                   |   '& _
        '(If)  \s+ (\w+)                                        |   '& _
        '(ElseIf)  \s+ (\w+)                                    |   '& _
        '(Until)  \s* (?!\() \s* (?!\() \s* (?!Not) \s+ (\w+)   |   '& _
        '(While)  \s+ (\w+)                                     |   '& _
        '\s* (\$\w*) \s+ = \s+ (\w+)                            |   '& _
        ' (\w+)                                                     '& _
    ')                                                              '& _
    '\(                                                             '& _
    '([^\)]*)                                                       '& _
    '\)                                                             '

But I think that you are correct: even for my subset of the language, I need to write a grammar parser. The start that you have made will be a help.

[jchd]

I'm not so sure it could help you: it may reveal more of a pitfall than anything.

I was testing nesting (?(DEFINE) ...) behavior with PCRE and had the idea to write this. So you understand it isn't any close to something well tought out. Also you know the slightiest mistake with regexp can put the whole thing down, so look at it like warm nitroglycerin.