_StringReverse - improved approach?

[BrewManNH]

Func _StringReverse($s_String)

    Local $i_len = StringLen($s_String)
    If $i_len < 1 Then Return SetError(1, 0, "")

    Local $t_chars = DllStructCreate("wchar[" & $i_len + 1 & "]")
    DllStructSetData($t_chars, 1, $s_String)

    Local $a_rev = DllCall("MSVCR100.DLL", "ptr:cdecl", "_wcsrev", "struct*", $t_chars) ;<<<<<<<<<<<<
    If @error Or $a_rev[0] = 0 Then Return SetError(2, 0, "")

    Return DllStructGetData($t_chars, 1)
EndFunc ;==>_StringReverse

This works to reverses the string without using msvcrt.dll. Not sure which versions of Windows it would work on because of the dll used, but someone with more information than me can tweak as needed.

Edited February 3, 2013 by BrewManNH

[guinness]

I think adding new features to AutoIt is fine, just don't break my code which often relys on accurate split second timing. I don't think it's an improvement at all.

Well I haven't done so far, in fact I've improved the syntax and speed in quite a few functions (_ReplaceStringInFile being one.) It's the whole reason I created this thread, because I knew speed would affect some if not all people who use that function.

[czardas]

@JCHD Would it work for UTF8?

@guinness I think strings have been used as an alternative to arrays specifically for speed benefits. My most important projects are music based, and there are an enormous number of permutations and interpretations to cruch through. Not having to store reverse patterns can be very handy indeed.

When playing back sound, there is potentially a very small time window in which to send additional commands. There's no room for inaccuracy. Speed is topmost priority if additional processing takes place during playback. This is critical to functionality. I implore you to not sacrifice speed benefits for the sake of current opinion.

Edited February 3, 2013 by czardas

[jchd]

Why UTF-8? Of course if you happen to have a buffer with UTF-8 inside (fairly uncommon in AutoIt!) you can still perform codepoint-by-codepoint reading by decoding successive bytes and write to the output buffer accordingly. But honestly I don't see a point in creating special UTF-8 support since it's of very seldom use (within AutoIt).

[czardas]

I will have to read up on it. Perhaps I was misinformed. I thought UTF-8 was more universal generally. It was someone who works with linux who told me to use UTF-8, which is why I asked.

Edit

After looking around, I found all the information I need on wiki, but I'm grateful for all the advice I get here.

Edited February 3, 2013 by czardas

[jchd]

Remember that we're talking about reversing an AutoIt string variable. AutoIt strings use UCS-2 encoding, a subset of UTF-16 limited to one 16-bit codeword per codepoint (Unicode plane 0). So whatever file you read, if you give the correct encoding as parameter to FileOpen, you end up with UTF-16-like strings in your variables, arrays, ...

The problem with string reversal in the general case is more subtle than simply reversing codepoints. It has to do with two very distinct things: Unicode string normalization and complex graphemes that are unavoidable in a given script (writen human language).

Unicode string may have any of 4 normalization forms, which are how codepoints are to be interpreted as characters (things the user sees). See the link for a quick explanation. Non well-formed Unicode string mix up more than one normalization form, making things unmanageable. In our Windows environments, almost all Unicode string are in C-form, that is with characters recomposed from their canonical equivalents.

Now not all possible "characters" (things that make up the script from the user's point of view) are being assigned a codepoint in Unicode. For instance an uppercase A with both a cedilla, a circumflex accent and a double below line (that combination doesn't exist as a single glyph point) will be ̧̳̂A represented by the sequence U+0327 U+0302 U+0333 U+0041.

Even if the resulting display looks weird here, it _should_ be correct if both the used font and the rendering engine were conformal. This combination might be absolutely meaningful and mandatory to represent some user character in some uncommon script. Reversing that sequence blindly makes no more sense. For something that you may be more familiar with, the traditional spanish makes a distinction between ch as two separate characters and ch as a single char unit. Reversing the latter to hc would be a gross error.

The same issue happens in a large number of common languages, not only in Navajo or some obscure primitive african script. That's why Unicode string reversal is a hard matter.

Comparing string for equality or order is even harder.

English-only concerned people may find those problems too exotic for them, but let's remind that if we what want AutoIt to gain more users worldwide we need to address many more delicate aspect than we currently do. Even counting together all latin scripts, we're only a small minority!

Edited February 3, 2013 by jchd

[czardas]

Thanks for taking the time to explain all this JCHD. I agree that it would be good if more unicode support were included in AutoIt, but I still think a separate set of functions, or perhaps an option could be set to allow for this. I believe this would be preferable to creating sluggish string functions. And as I indicated I rely on the speed of such functions quite heavily already. I believe I'm not on my own in this respect.

Edited February 3, 2013 by czardas

[jchd]

Since there _is_ fully compliant Unicode support built into every Windows system in a very efficient way, I don't see why we shouldn't make use of it.

For instance, every Explorer windows sort contents (folders, links, files) in Unicode order compliant with UCA for your locale setting and in pseudo-real time.

The only catch is how to access that set of functionnalities effectively.

[czardas]

If it can be done without compromising current projects. I'm all for it being integrated. Otherwise I would prefer full unicode string function support as an additional Opt() parameter if that is possible.

Edited February 3, 2013 by czardas

[SmOke_N]

Hmm, I'll have to take the topic and the quote that was referred to more than once a bit personal.

I could babble on how, at the time, I had little knowledge on dll/api calls, but why bother.

A year or two after I wrote those, I asked Valik to implement the ability of a char array scenario with the ability to revert to string as well.

He came up with StringToASCIIArray() and StringFromASCIIArray() that included an encoding method.

Granted, not really what I was asking for 100%, because, IMHO, it took too long... when I thought we could just play with the ptr instead, it did leave some options open.

If you wanted a totally AutoIt available option, since DllStruct* anything is pretty slow as well, you could play with speed variants using something like the below pseudo code:

Func ___StringReverse($s_strin, $i_start = Default, $i_end = Default, $i_encoding = Default)

    Local $i_len = StringLen($s_strin)
    If $i_len = 0 Then Return SetError(1, 0, "")

    If $i_start = Default Then $i_start = 0
    If $i_end = Default Then $i_end = $i_len
    If $i_encoding = Default Then $i_encoding = 0

    Local $a_data = StringToASCIIArray($s_strin, $i_start, $i_end, $i_encoding)
    Local $i_ub = UBound($a_data)
    Local $a_retdata[$i_ub]
    $i_ub -= 1

    For $i = 0 To $i_ub
        $a_retdata[$i] = $a_data[$i_ub - $i]
    Next

    Local $s_addto
    If $i_start > 0 Then $s_addto = StringLeft($s_strin, $i_start)
    Local $s_ret = $s_addto & StringFromASCIIArray($a_retdata, $i_start, $i_end, $i_encoding)
    If $i_end < $i_len Then $s_ret &= StringRight($s_strin, ($i_len - $i_end))

    Return $s_ret
EndFunc

All in all, the quote by trancexx on the msvcrt.dll is correct, and almost quoted from here: http://msdn.microsoft.com/en-us/library/abx4dbyh%28v=vs.80%29.aspx , but I didn't know/see that at the time I wrote them... All I saw was a significant increase from a normal loop.

Edited February 3, 2013 by SmOke_N

[guinness]

Thanks for the code, also I hope you don't think that this topic wasn't aimed at you personally because it wasn't my intention. You know me, I just want the best for AutoIt.

[czardas]

That is also painfully much slower than the current function.

[Malkey]

How does this "StringReverse function" example shape up?

Func __StringReverse($sString, $iStart = Default, $iEnd = Default)
    Local $sRet, $aString
    $aString = StringSplit($sString, "")
    If $iStart = Default Then $iStart = 0
    If $iEnd = Default Then $iEnd = $aString[0] - 1
    For $i = $iStart To $iEnd
        $sRet &= $aString[$aString[0] - $i]
    Next
    Return $sRet
EndFunc   ;==>__StringRever

[czardas]

How does this "StringReverse function" example shape up?

expandcollapse popup

test()

Func test()
    Local $string = "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789"
    $iTimer = TimerInit()
    For $iCount = 1 To 1000
        $sRev = __StringReverse($string)
    Next
    ConsoleWrite(TimerDiff($iTimer) & @LF)

    $iTimer = TimerInit()
    For $iCount = 1 To 1000
        $sRev = ___StringReverse($string) ; Original
    Next
    ConsoleWrite(TimerDiff($iTimer) & @LF)

    $iTimer = TimerInit()
    For $iCount = 1 To 1000
        $sRev = ____StringReverse($string)
    Next
    ConsoleWrite(TimerDiff($iTimer) & @LF)

    $iTimer = TimerInit()
    For $iCount = 1 To 1000
        $sRev = _____StringReverse($string)
    Next
    ConsoleWrite(TimerDiff($iTimer) & @LF)
EndFunc

Func __StringReverse($s_String)
    Local $i_len = StringLen($s_String)
    If $i_len < 1 Then Return SetError(1, 0, "")
    Local $sRet = ""
    For $i = $i_len To 1 Step -1
        $sRet &= StringMid($s_String, $i, 1)
    Next
    Return $sRet
EndFunc

Func ___StringReverse($s_String) ; Original

    Local $i_len = StringLen($s_String)
    If $i_len < 1 Then Return SetError(1, 0, "")

    Local $t_chars = DllStructCreate("char[" & $i_len + 1 & "]")
    DllStructSetData($t_chars, 1, $s_String)

    Local $a_rev = DllCall("msvcrt.dll", "ptr:cdecl", "_strrev", "ptr", DllStructGetPtr($t_chars))
    If @error Or $a_rev[0] = 0 Then Return SetError(2, 0, "")

    Return DllStructGetData($t_chars, 1)
EndFunc   ;==>_StringReverse

Func ____StringReverse($s_strin, $i_start = Default, $i_end = Default, $i_encoding = Default)

    Local $i_len = StringLen($s_strin)
    If $i_len = 0 Then Return SetError(1, 0, "")

    If $i_start = Default Then $i_start = 0
    If $i_end = Default Then $i_end = $i_len
    If $i_encoding = Default Then $i_encoding = 0

    Local $a_data = StringToASCIIArray($s_strin, $i_start, $i_end, $i_encoding)
    Local $i_ub = UBound($a_data)
    Local $a_retdata[$i_ub]
    $i_ub -= 1

    For $i = 0 To $i_ub
        $a_retdata[$i] = $a_data[$i_ub - $i]
    Next

    Local $s_addto
    If $i_start > 0 Then $s_addto = StringLeft($s_strin, $i_start)
    Local $s_ret = $s_addto & StringFromASCIIArray($a_retdata, $i_start, $i_end, $i_encoding)
    If $i_end < $i_len Then $s_ret &= StringRight($s_strin, ($i_len - $i_end))

    Return $s_ret
EndFunc

Func _____StringReverse($sString, $iStart = Default, $iEnd = Default)
    Local $sRet, $aString
    $aString = StringSplit($sString, "")
    If $iStart = Default Then $iStart = 0
    If $iEnd = Default Then $iEnd = $aString[0] - 1
    For $i = $iStart To $iEnd
        $sRet &= $aString[$aString[0] - $i]
    Next
    Return $sRet
EndFunc   ;==>__StringReverse

Try it on 1000 characters instead of 100. I get:

1893.04031657655

54.5924132815763

3207.44871205698

2866.47576717153

Edited February 3, 2013 by czardas

[guinness]

My version in post #21 - StringReverseChrArray doesn't enumerate through the entire array.

[czardas]

Using an array limits string length to approx 16,000,000 characters.

[BrewManNH]

Is using "MSVCR100.DLL" just as bad as using "msvcrt.dll"? If not, then the code I posted above should work in its place in the function, if it is just as "bad" is there a dll that exposes the functions that do string reversing that isn't "bad" to use?

[czardas]

Sorry I didn't include your function in the above test BrewManNH. That was an oversight on my part. From preliminary tests your version seems significantly faster than the original , where all other versions I tested seem to require too much of a concession to really be practical. I haven't fully tested the function, only a speed test.

201.21384142398

48.0041457783042 - Original version

278.529533781528

242.376894270082

25.0358634966176 - BrewManNH version

Edited February 3, 2013 by czardas

[SmOke_N]

That is also painfully much slower than the current function.

That was the point I was trying to make. Ideally, it should be very close in speed to the DLL calls out there if it's a simple char swap, however, the way it (StringToASCII/StringFromASCII) was written ( no idea ) doesn't seem to be the way I originally requested.

@BrewManNH... I'm not 100%, but I was under the impression that the Microsoft Visual C++ Redistributable ( I believe 2010 for msvcr100.dll ), would have to be on the machine, or the dll added as a dependency ( unsure also if it itself requires any dependencies ).

The ability to work with the string ptr directly without building a struct or repeatedly calling getdata/setdata would solve most of the issues you've all talked about, and work similar to what I provided earlier.

Edited February 3, 2013 by SmOke_N

[czardas]

I hope you guys can come up with a suitable alternative, or at least test to see if a more flexible function is needed when a string contains unicode. A simple switch statement to run the most appropriate code ought to be enough, I think. I don't want to be seen to be kicking up a fuss about this, but sluggishness really will affect many projects, not only my own.