[Solved] StringRegExp, weird case.

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[GEOSoft]

Does this do what you expect?

$aSRE = "word1,word2,,,word3,"
$sPattern = "([\w\s]+?)(?:,|$)"

Edit:

Tip: Don't use \1,\2,\3 &etc in a replace unless you have to. Use $1, $2, $3 instead.

Edited February 8, 2010 by GEOSoft

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[Richard Robertson]

Best I can come up with that would produce all matches would be this, but it uses a different set of groups and would require some extra processing.

"^([^,]*),|,([^,]*),|,([^,]*)$|^([^,]*)$"

That should pick up zero width groups as well, but is a little more complicated...

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[Malkey]

This example shows a method of changing the ",,,"part of a source string into a ",[0:],[0:],"part of the return string from a modified version of the RE_Debug function at Post #1.

It might give you some ideas.

MAIN()

Func MAIN()
    Local $sPattern = '[^,]+|(?:,{2,})'
    Local $sSource = 'word1,,word2,,,word3,word4'
    ConsoleWrite('1: = ' & RE_Debug($sSource, $sPattern) & @CRLF)
EndFunc ;==>MAIN

;; Support function.
Func RE_Debug($sSource, $sPattern)
    $sSource = StringRegExpReplace($sSource, $sPattern, _
            '[0:\0][1:\1][2:\2][3:\3][4:\4][5:\5][6:\6][7:\7][8:\8][9:\9]')
    If @error Then Return SetError(@error, @extended, '')
    $sSource = StringRegExpReplace($sSource, '\[[1-9]:\]', '\1')
    If @error Then Return SetError(@error, @extended, '')
    $sSource = Execute('"' & StringRegExpReplace($sSource, _
            "\[(\d+):(,{2,})\]", '" & __Rep($1,"$2") & "') & '"')
    If @error Then Return SetError(@error, @extended, '')
    Return $sSource
EndFunc ;==>RE_Debug

;Internal function called from RE_Debug()
Func __Rep($No, $RE)
    Local $sRet = "", $iNum
    StringReplace($RE, ",", ",")
    $iNum = @extended - 1
    For $i = 1 To $iNum
        $sRet &= ",[" & $No & ":]"
    Next
    Return $sRet & ","
EndFunc ;==>__Rep

[Richard Robertson]

Interesting.

Although its not doing what I'm after. Its for example completely skipping word 2 in the string "word1,word2,,,word3".

It shows a (to me) unexpected result. (So I'm not done yet with your suggestion)

It also shows a result that shows that I need to recall that RE_Debug() function for its own debug session.

Thanks.

I have no idea how it could skip word2. Did it get word1 and word3 and the blanks? Edited February 8, 2010 by Richard Robertson

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[jchd]

3) I Think its a bug. So I'll focus on finding some proof so it might be fixed.

Reread my post #10 using the pcretest utility. Pretty close to a proof and using the very same engine (on the paper). I didn't try with the latest pcretest version, this one is a bit outdated.

[Richard Robertson]

I haven't got an easy way to test regular expressions right now. Firebug is being retarded and keeps dying every time I switch tabs. Getting on my nerves.

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[Bowmore]

I was initially confused as to what the problem you were having, but I think I understand now. 

I'm coming round to the point of view that there possibly is some sort of bug here, possibly with the way the PCRE dll has been incorperated into AutoIt.

I would expect these 2 bits of code to give the same result.

#include <array.au3>
    $sPattern = '[^,]+'
    $sSource = 'word1,word2,word3'
    $aArray = StringRegExp($sSource,$sPattern,3)
    _ArrayDisplay($aArray)

#include <array.au3>
    $sPattern = '[^,]*'
    $sSource = 'word1,word2,word3'
    $aArray = StringRegExp($sSource,$sPattern,3)
    _ArrayDisplay($aArray)

Edited February 8, 2010 by Bowmore

[jchd]

I would expect these 2 bits of code to give the same result.

Exactly, and that's surprising since the pcre test utility doesn't show this unexpected behavior. As I said, it's posibly due to build options for the PCRE module that AutoIt is using.

[weaponx]

Code:

expandcollapse popup

$result = _StringSplitRegExp('data1,data2,,data3,,,data4', ',')
For $X = 1 to $result[0]
    ConsoleWrite('[' & $X & ']: ' & $result[$X] & @CRLF)
Next

#cs ----------------------------------------------------------------------------

 AutoIt Version: 3.2.10.0
 Author: WeaponX

 Script Function:
    Split string on regular expression

 Parameters:
    String = String to be split
    Pattern = Pattern to split on
    IncludeMatch = True / False - Indicates whether or not to include the match in the return (back-reference)
    Count = Number of splits to perform

#ce ----------------------------------------------------------------------------
Func _StringSplitRegExp($sString, $sPattern, $sIncludeMatch = false, $iCount = 0)

    ;All matches will be replaced with this string
    Local $sReservedPattern = Chr(0)
    Local $sReservedPattern = "#"
    Local $sReplacePattern = $sReservedPattern

    ;Modify the reserve pattern to include back-reference
    If $sIncludeMatch Then $sReplacePattern = "$0" & $sReplacePattern

    ;Replace all occurences of the search pattern with a replace string
    $sTemp = StringRegExpReplace($sString, $sPattern, $sReplacePattern, $iCount)

    ;Consolewrite($sTemp & @CRLF)

    ;Strip trailing character if it matches the reserved pattern
    If StringRight($sTemp, 1) = $sReservedPattern Then $sTemp = StringTrimRight($sTemp, 1)

    ;Split string using entire reserved string
    $aResult = StringSplit($sTemp, $sReservedPattern, 1)

    Return $aResult
EndFunc

Output:

[1]: data1
[2]: data2
[3]: 
[4]: data3
[5]: 
[6]: 
[7]: data4

[jchd]

Warm thanks Sniper, but the point now boils down to the "is it an AutoIt bug" question rather than "how to get around".

Oops, I'm putting words in the OP's mouth about his thread!

[MvGulik]

whatever Edited February 7, 2011 by MvGulik

[Mison]

I don't think it's a bug. It might has something to do with what kind of regex flavor we are using. Go to here, and try this example:

String = word1,word2,word3

Pattern = [^,]*

Use different dialect and it will return different results depending on the dialect.

[jchd]

Silly thing: my composed message went in the clouds...

Yes, it's likely a distinct set of build options. Look at PCRE sources and .configure and you'll see that there are various options that condition the flavor or mix of flavors that will be built.

The issue discussed here may not qualify as bug, but rather as side-effect of the build options that the AutoIt team decided to use. These options are likely a bit specific to AutoIt.

They aren't being building Perl but AutoIt. Regexps are fundamental in Perl and it can be safe to assume that Perl users will learn a lot about regexps. Also regexps efficiency is paramount in Perl. This isn't the same with AutoIt, where most users are kept out of using regexp by their ignorance of how to use them and little programming experience. Regexp can always be replaced by other string manipulation functions and that's what you see in most forum posts. Don't see anything pejorative here, that's just facts.

Depending on the engine used, it can be very easy to write an innocent-looking regexp that will backtrack as hell and grab 100% of one CPU for years when trying to match on some innocent string. If that occurs in a Perl program, there's always someone around to explain how to fix it. In the AutoIt community, things are very different and the focus is rather to avoid build options that can cause problems in laymen hands.