StringRegExp and enumerating data inbetween a particular function.

[guinness]

Below is a working example of what I'm trying to achieve.

As you can see it will find all the strings in between the function labelled _LanguageString(), but where this fails is when there is a closing bracket ")" inside the function itself, due to this being interpreted as the function's closing bracket. So my question is how can I improve the SRE to disregard the bracket in the function's string? Any advice would be greatly appreciated.

#include <Array.au3>

Example()

Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE", "An example string"))' & @CRLF & @CRLF & 'Local $sVar = _LanguageString("MENU_HELP", "Help file")' & @CRLF & _
    "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)    "

    ; Find data in between _LanguageString( and ). The function _LanguageString is included in a separate AutoIt.
    Local $aReturn = StringRegExp($sData, '(?s)(?i)_LanguageString\((.*?)\)', 3)
    If Not @error Then
    _ArrayDisplay($aReturn) ; Display the strings returned.
    EndIf
EndFunc   ;==>Example

[Robjong]

Hi,

this might not be the best solution but I think it will fit your needs

#include <Array.au3>

Example()

Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE", "An example string"))' & @CRLF & @CRLF & 'Local $sVar = _LanguageString($MENU_HELP, "Help file")' & @CRLF & _
    "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)    "
    ; Find data in between _LanguageString( and ). The function _LanguageString is included in a separate AutoIt.
    Local $aReturn = StringRegExp($sData, '(?s)(?i)_LanguageString\s*\(\s*((?:[''"].*?[''"]|\$\w+),\s*(?:[''"].*?[''"]|\$\w+))\)', 3) ; will allow the params to be literal strings and variables.
;~   Local $aReturn = StringRegExp($sData, '(?s)(?i)_LanguageString\s*\(\s*((?:[''"].*?[''"]|\$\w+)(?:,\s*(?:[''"].*?[''"]|\$\w+))?)\)', 3) ; if the second param of _LanguageString is optional use this one.
    _ArrayDisplay($aReturn) ; Display the strings returned.
EndFunc   ;==>Example

[guinness]

Thanks Robjong, it's perfect!

Edit: In the initial example I wrote $MENU_HELP when it should've been "MENU_HELP" so this is what led me to believe it was wrong, but it was fault for writing the example incorrect. Cheers Robjong.

Edited February 1, 2012 by guinness

[Beege]

Wow! Very nice indeed Robjong. I couldnt get around the single/double quotes

Edited February 1, 2012 by Beege

[Beege]

Got mine to work. Only after looking at seeing how Robjong handled the quotes.

#include <Array.au3>
Example()
Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE", "An example string"))' & @CRLF & @CRLF & _
     'Local $sVar = _LanguageString("MENU_HELP", "Help file")' & @CRLF & _
     "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)"
    ; Find data in between _LanguageString( and ). The function _LanguageString is included in a separate AutoIt.
    Local $aReturn = StringRegExp($sData, '(?s)(?i)_LanguageString((.*?["|'']))', 3)
    If Not @error Then
    _ArrayDisplay($aReturn) ; Display the strings returned.
    EndIf
EndFunc   ;==>Example

Edited February 1, 2012 by Beege

[guinness]

Beege, that's amazing too

[Beege]

Thanks

[Robjong]

Nice one Beege.

However there is a small error in your pattern, the | int the character set should not be there, it is only used as a conditional in capturing groups, it would now also match |) as an ending.

The pattern could be something like this:

'(?is)_LanguageString((.*?["'']))'

Edit: added everything below...

The pattern Beege provided (the one above here) is nice and short but it has a small drawback compared to the pattern I posted, you should consider this when writing the final pattern.

If the string happens to include a quote followed by a closing parentheses it will 'break' the match, the pattern I provided only has this for the second param of _LanguageString,

if you will never encounter a quote followed by a closing parentheses in the first parameter of _LanguageString you should use Beege's pattern.

Also, I wrote mine that way because I suspected guinness would wan't to have both params separately at some point.

Example:

#include <Array.au3>

Example()

Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE with a single quote (''foo'')", "An example string"))' & @CRLF & @CRLF & 'Local $sVar = _LanguageString($MENU_HELP, "Help file")' & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (Having a quote followed by a closing parentheses in this string will break the match'') so you won''t see this part'), 0, 5, 100, 20) "
    ; Find data in between _LanguageString( and ). The function _LanguageString is included in a separate AutoIt.
;~   Local $aReturn = StringRegExp($sData, '(?is)_LanguageStrings*(s*((?:[''"].*?[''"]|$w+)s*,s*(?:[''"].*?[''"]|$w+)s*))', 3) ; last match is broken
    Local $aReturn = StringRegExp($sData, '(?is)_LanguageString((.*?["'']))', 3); first match is broken
    _ArrayDisplay($aReturn) ; Display the strings returned.
EndFunc   ;==>Example

Edited February 1, 2012 by Robjong

[Beege]

Ahh yes your right. I was thinking or operator, but thats missing the whole point of the brackets.

[guinness]

Gotcha! Of the top of my head I don't believe I have quotes inside brackets, but I'll do some more testing later on today.

Also, I wrote mine that way because I suspected guinness would wan't to have both params separately at some point.

As of now both are mandatory but this could change in the future.

[Robjong]

Hi again,

I tried some more patterns and this is what i came up with.

It matches the first closing parentheses with an unescaped quote with the help of nagative lookbehind. this avoids most broken matches due to ') or ") in the strings and unescaped the quotes would cause a syntax error and are therefor unlikely to appear that way.

The pattern is a bit long but this is make sure signel and double quotes don't get mixed up, it allows literal string, variables and a single combination of the two.

I think this will work in most cases but it has not been thoroughly tested yet, so that's up to you if you like.

Pattern:

'(?is)_LanguageStrings*(s*((?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")")?s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")")?)s*)'

Example:

#include <Array.au3>

Example()

Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE with a single quote (""foo"")", "An example string"))' & @CRLF & @CRLF & 'Local $sVar = _LanguageString($MENU_HELP, "Help file")' & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString($foo & 'MENU_LICENSE', 'Please see readme (""License.txt"")'), 0, 5, 100, 20) " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (Having a quote followed by a brace closing parentheses in this string will break the match'') so you won''t see this part, or you will now'), 0, 5, 100, 20)  " & @CRLF
    Local $aReturn = StringRegExp($sData, '(?is)_LanguageStrings*(s*((?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")")?s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")")?)s*)', 3)
    If Not @error Then  _ArrayDisplay($aReturn) ; Display the strings returned.
EndFunc   ;==>Example

Edit: corrected pattern, moved the last questionmark in the pattern 1 pos to the left.

Edited February 1, 2012 by Robjong

[guinness]

This appears to be a lot better than your initial post from the tests I've conducted. Though one last thing if for example I have the following '_LanguageString("TIP_1", "It appears you don't have administrator privileges.", 0)' and I simply duplicate this part ,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"))?s* it fails, but this is no biggy!

[Beege]

Robjong, Did you like contribute in the creation of regexp or something? cause that shit is crazy!

[Robjong]

Hey guinness,

try this one, it allows for the params to be digits (just digits, no floats or anything and not in combination with strings or vars) and has an optional 3rd param.

'(?is)_LanguageStrings*(s*((?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?(?:s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?)?)s*)'

I also made a minor correction to the last pattern I posted.

Edit: If you have any more requirements just tell me now so we can get it all in there , do you need access to the separate params for example?

Another edit: @Beege I wish

Yet another edit: Fixed syntax error in the pattern

Edited February 1, 2012 by Robjong

[guinness]

Hey guinness, try this one, it allows for the params to be digits (just digits, no floats or anything and not in combination with strings or vars) and has an optional 3rd param.

Hey, you're quick for me Robjong. I tried it and there was an error with the syntax so I added an additional ' where the error occurred without any luck. Sorry about that. You've both helped me more than I could imagine.

[Robjong]

...error in the syntax? checking...

Edit: I updated my previous post with the fixed pattern and here is a working example...

#include <Array.au3>

Example()

Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE with a single quote (""foo"")", "An example string"))' & @CRLF & @CRLF & 'Local $sVar = _LanguageString( $MENU_HELP ,"Help file", 1 )' & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString($foo & 'MENU_LICENSE', 'Please see readme (""License.txt"")'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (Having a quote followed by a brace closing parentheses in this string will break the match'') so you won''t see this part, or you will now'), 0, 5, 100, 20)    " & @CRLF
;~     Local $aReturn = StringRegExp($sData, '(?isx)_LanguageStrings*(s*( (?:$w+(?:s*&)?s*)? (?: ''.*?(?<![^'']'')'' | ".*?(?<![^"]")" | d)?  s*,s*  (?:$w+(?:s*&)?s*)?  (?: ''.*?(?<![^'']'')'' | ".*?(?<![^"]")" | d )? (?: s*,s*  (?:$w+(?:s*&)?s*)?  (?: ''.*?(?<![^'']'')'' | ".*?(?<![^"]")" | d )?  )?  ) s*)', 3)
    Local $aReturn = StringRegExp($sData, '(?is)_LanguageStrings*(s*((?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?(?:s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?)?)s*)', 3)
    If Not @error Then _ArrayDisplay($aReturn) ; Display the strings returned.
EndFunc   ;==>Example

Edited February 1, 2012 by Robjong

[guinness]

That's it! Thanks ever so much for your help and putting up with my constant changing the goal posts, you've managed to 'hit the nail on the head' with the last example. Lets hope someone else can benefit from this SRE in the future.

[Robjong]

I'm glad I was able to help. The principle is quite simple but the pattern a little less.

If anyone needs this I suggest checking out the pattern that is commented out in my previous post as it's a little bit easier to read.

[jchd]

Just a hint for everyone having to deal/build non trivial patterns: make yourself the favor of using the PCRE_EXTENDED option (?x). That will allow you to freely use whitespaces to give your pattern superior readability. It of course need that when you need significant whitespace, you escape them.

I'll have my own look at the OP problem later.

[Robjong]

Yes jchd that's well put, and don't forget you can also use # (Cross hatch) to comment lines in the patter, they are trailing comments and behave like the semi colon in AutoIt, this also means you have to escape any # characters you need in your pattern with a (backslash) like this # or it will cause an invalid pattern error.

Also here's is another version of the pattern which is a little shorter and requires atleast 1 parameter but also allows for as many as you like.

#include <Array.au3>

Example()

Func Example()
    ; An example of an AutoIt file.
    Local $sData = 'MsgBox(4096, "", _LanguageString("MENU_EXAMPLE with a single quote (""foo"")", "An example string"))' & @CRLF & @CRLF & 'Local $sVar = _LanguageString( $MENU_HELP ,"Help file", 1 )' & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString($foo & 'MENU_LICENSE', 'Please see readme (""License.txt"")'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (License.txt)'), 0, 5, 100, 20)    " & @CRLF & _
            "Local $iControl = GUICtrlCreateLabel(_LanguageString('MENU_LICENSE', 'Please see readme (Having a quote followed by a brace closing parentheses in this string will break the match'') so you won''t see this part, or you will now'), 0, 5, 100, 20)    " & @CRLF
;~     Local $aReturn = StringRegExp($sData, '(?isx)_LanguageStrings*(s*( (?:$w+(?:s*&)?s*)? (?: ''.*?(?<![^'']'')'' | ".*?(?<![^"]")" | d)?  (?: s*,s*  (?:$w+(?:s*&)?s*)?  (?: ''.*?(?<![^'']'')'' | ".*?(?<![^"]")" | d )?  )*  ) s*)', 3)
    Local $aReturn = StringRegExp($sData, '(?isx)_LanguageStrings*(s*((?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?(?:s*,s*(?:$w+(?:s*&)?s*)?(?:''.*?(?<![^'']'')''|".*?(?<![^"]")"|d)?)*)s*)', 3)
    If Not @error Then _ArrayDisplay($aReturn) ; Display the strings returned.
EndFunc   ;==>Example

Edit: It still has some problems with empty strings ""

Edit: grammer and explenation of comments

Edited February 1, 2012 by Robjong