How to find how many @LFs in a given text

[UEZ]

Ops, after editing the post I've deleted the important line in the function:
 

Return DllCall("user32.dll", "uint", "CallWindowProcW", "ptr", DllStructGetPtr($tCodeBuffer), "str", $g_sString, "int", 0, "int", 0, "int", 0)[0] 

 
>Please check again.

Edited April 8, 2015 by UEZ

[JohnOne]

AMD cpu.

1000000 lines have been created.
Melba23                 3.8662 secs 1000000
Mikell                  1.31874 secs    1000000
Mikell 2                0.93202 secs    1000000
Malkey                  0.63843 secs    1000000
jchd                    0.5851 secs 1000000
Tekk                    0.25879 secs    1000000
UEZ                     0.27756 secs    1000000

[Gianni]

Ops, after editing the post I've deleted the important line in the function:

 

Return DllCall("user32.dll", "uint", "CallWindowProcW", "ptr", DllStructGetPtr($tCodeBuffer), "str", $g_sString, "int", 0, "int", 0, "int", 0)[0] 

 

>Please check again.

 

... I thought maybe missing a piece ...

now it's ok, more or less same speed as Tekk

Melba23                 4.43371 secs    1000000

Mikell                  0.93724 secs    1000000

Mikell 2                0.59906 secs    1000000

Malkey                  0.35922 secs    1000000

jchd                    0.38844 secs    1000000

Tekk                    0.16141 secs    1000000

UEZ                     0.16097 secs    1000000

p.s.

a little suggestion (if allowed)

as already posted >here for assembly code by Tekk , also your function could be used for the counting of any other char instead of only @LF with this little modification:

Local $sMyString = "the quick brown fox jumps over the lazy dog"
Local $sMyChar = "o"

ConsoleWrite('there are ' & StringCountChar($sMyString, $sMyChar) & ' occurrences of the char "' & $sMyChar & '"' & @CRLF)


Func StringCountChar(ByRef $g_sString, $sMyChar)
    Local $tCodeBuffer = DllStructCreate("byte ASM[22]")
    $tCodeBuffer.ASM = "0x" & _
            "8B742404" & _ ;  mov esi, dword[esp+4] -> get start address (pointer) of the string
            "31DB" & _ ;      xor ebx, ebx -> set ebx = 0
            "AC" & _ ;        lodsb ebx -> load char from [esi] to al register as byte and increment esi by one
            "3C00" & _ ;      cmp al, 0 -> is char = 0
            "7409" & _ ;      jz _end -> if yes then jump to xchange and exit
            "3C" & Hex(Asc($sMyChar), 2) & _ ;  cmp al, NNh -> if not compare it with char ; <-------------------------- modified here
            "75F7" & _ ;      jne lodsb -> if not char load next byte from string
            "83C301" & _ ;    add ebx, 1 -> if yes increase ebx by one -> ebx += 1 (add is faster then inc)
            "EBF2" & _ ;      jmp lodsb -> load next byte from string
            "93" & _ ;        xchg eax, ebx -> result is in ebx but return value is always eax -> swap eax with ebx
            "C3" ;            return eax
    Return DllCall("user32.dll", "uint", "CallWindowProcW", "ptr", DllStructGetPtr($tCodeBuffer), "str", $g_sString, "int", 0, "int", 0, "int", 0)[0]
EndFunc   ;==>StringCountChar

Edited April 8, 2015 by Chimp

[UEZ]

Thanks Chimp but the comment has to be amended, too.

[Gianni]

Thanks Chimp but the comment has to be amended, too.

 

done

[iamtheky]

Since yall are learning this much faster than I, how much modification to the current efforts to replace every instance of that character with another?  I was thinking that since you had already everything up to evaluating the character, that it would be simple enough to either return that character or a replacement, but I was suuuuper wrong.

[Tekk]

Since yall are learning this much faster than I, how much modification to the current efforts to replace every instance of that character with another?  I was thinking that since you had already everything up to evaluating the character, that it would be simple enough to either return that character or a replacement, but I was suuuuper wrong.

ConsoleWrite(StringreplaceAll("deer", "e", "o") & @CRLF)

Func StringreplaceAll($sString, $sProbe, $sPatch)
    Local $tProc = DllStructCreate("BYTE ASM[20]") ;~ AutoIt allocates memory with execute rights?
                                                   ;~ I have NX bit enabled, I assumed issues if not explicitly giving execute rights
    $tProc.ASM = "0x" _
        & "8B742404"  _                         ;~     mov   esi, dword[esp+4] -> get start address (pointer) of the string
        & "AC"        _                         ;~ @@: lodsb                   -> load char from [esi] to al & increment esi by one
        & "3C00"      _                         ;~     cmp   al, 0             -> is char = 0
        & "740A"      _                         ;~     jz    @f                -> if yes then jump to return
        & "3C"        & Hex(Asc($sProbe), 2)  _ ;~     cmp   al, NNh           -> if not then compare with $sProbe
        & "75F7"      _                         ;~     jne   @b                -> if not equal then load next byte from string
        & "C646FF"    & Hex(Asc($sPatch), 2)  _ ;~     mov   byte[esi-1], NNh  -> if equal then replace with $sPatch
        & "EBF1"      _                         ;~     jmp   @b                -> load next byte from string
        & "C3"                                  ;~ @@: ret                     -> return

;~  Technically shouldn't we preserve esi?

    Return DllCallAddress("NONE:cdecl", DllStructGetPtr($tProc), "STR", $sString)[1]
EndFunc

[iamtheky]

If I have done this correctly it is noticeably outracing some character replacement funcs as well.  I am super interested in an  _ASM_StringReplace Function that takes the same parameters as the internal stringreplace. Thanks all for making this the least intimidating intro to ASM ever.

expandcollapse popup

#include <array.au3>

Local $sString, $hTimer, $aCompetitors[3][2] = [ _
        ["StringReplace   ", 0], _
        ["SRER    ", 0], _
        ["ASM  ", 0]]

For $i = 1 To 100000
    $sString &= "long a, is pline " & $i & @CRLF
Next




$hTimer = TimerInit()

$a = StringReplace($sString, "i", "a")
Stopwatch(0, $hTimer, $aCompetitors)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[0][0]) & @TAB & $aCompetitors[0][1] & " secs" &  @CRLF)



$b = StringRegExpReplace($sString, "i", "a")
Stopwatch(1, $hTimer, $aCompetitors)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[1][0]) & @TAB & $aCompetitors[1][1] & " secs" & @CRLF)



$c = StringreplaceAll($sString, "i" , "a")
Stopwatch(2, $hTimer, $aCompetitors)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[2][0]) & @TAB & $aCompetitors[2][1] & " secs" & @TAB & @CRLF)


_ArrayDisplay($aCompetitors)

Func Stopwatch($iCompetitor, ByRef $hTimer, ByRef $aCompetitors)
    $aCompetitors[$iCompetitor][1] = Round(TimerDiff($hTimer) / 1000, 5)
    $hTimer = TimerInit()
EndFunc   ;==>Stopwatch



Func StringreplaceAll($sString, $sProbe, $sPatch)
    Local $tProc = DllStructCreate("BYTE ASM[20]") ;~ AutoIt allocates memory with execute rights?
                                                   ;~ I have NX bit enabled, I assumed issues if not explicitly giving execute rights
    $tProc.ASM = "0x" _
        & "8B742404"  _                         ;~     mov   esi, dword[esp+4] -> get start address (pointer) of the string
        & "AC"        _                         ;~ @@: lodsb                   -> load char from [esi] to al & increment esi by one
        & "3C00"      _                         ;~     cmp   al, 0             -> is char = 0
        & "740A"      _                         ;~     jz    @f                -> if yes then jump to return
        & "3C"        & Hex(Asc($sProbe), 2)  _ ;~     cmp   al, NNh           -> if not then compare with $sProbe
        & "75F7"      _                         ;~     jne   @b                -> if not equal then load next byte from string
        & "C646FF"    & Hex(Asc($sPatch), 2)  _ ;~     mov   byte[esi-1], NNh  -> if equal then replace with $sPatch
        & "EBF1"      _                         ;~     jmp   @b                -> load next byte from string
        & "C3"                                  ;~ @@: ret                     -> return

;~  Technically shouldn't we preserve esi?

    Return DllCallAddress("NONE:cdecl", DllStructGetPtr($tProc), "STR", $sString)[1]
EndFunc

Edited April 9, 2015 by boththose

[jchd]

I don't want to be a killjoy in your playground guys, but ASM versions posted so far --fast as they are-- are incorrect.

You're all thinking 8-bit ANSI but the truth is elsewhere:

expandcollapse popup

#include <array.au3>

Local $sString, $hTimer, $aCompetitors[3][3] = [ _
        ["StringReplace   ", 0], _
        ["SRER    ", 0], _
        ["ASM  ", 0]]

For $i = 1 To 1
    $sString &= "I forgot that AutoIt uses UCS2 charset ● J'ai oublié qu'AutoIt utilise le jeu de caractères UCS2 ● 我忘了AutoIt的使用UCS2字符集 ● Ich habe vergessen, daß AutoIt verwendet UCS2 Zeichensatz ● Ξέχασα ότι AutoIt χρησιμοποιεί UCS2 σύνολο χαρακτήρων ● Я забув, що AutoIt використовує UCS2 набір символів " & $i & @CRLF
Next

$hTimer = TimerInit()

$a = StringReplace($sString, "●", "―")
Stopwatch(0, $hTimer, $aCompetitors, $a)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[0][0]) & @TAB & $aCompetitors[0][1] & " secs" &  @CRLF)

$b = StringRegExpReplace($sString, "●", "―")
Stopwatch(1, $hTimer, $aCompetitors, $b)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[1][0]) & @TAB & $aCompetitors[1][1] & " secs" & @CRLF)

$c = StringreplaceAll($sString, "●" , "―")
Stopwatch(2, $hTimer, $aCompetitors, $c)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[2][0]) & @TAB & $aCompetitors[2][1] & " secs" & @TAB & @CRLF)

_ArrayDisplay($aCompetitors)

Func Stopwatch($iCompetitor, ByRef $hTimer, ByRef $aCompetitors, ByRef $s)
    $aCompetitors[$iCompetitor][1] = Round(TimerDiff($hTimer) / 1000, 5)
    $aCompetitors[$iCompetitor][2] = $s
    $hTimer = TimerInit()
EndFunc   ;==>Stopwatch

Func StringreplaceAll($sString, $sProbe, $sPatch)
    Local $tProc = DllStructCreate("BYTE ASM[20]") ;~ AutoIt allocates memory with execute rights?
                                                   ;~ I have NX bit enabled, I assumed issues if not explicitly giving execute rights
    $tProc.ASM = "0x" _
        & "8B742404"  _                         ;~     mov   esi, dword[esp+4] -> get start address (pointer) of the string
        & "AC"        _                         ;~ @@: lodsb                   -> load char from [esi] to al & increment esi by one
        & "3C00"      _                         ;~     cmp   al, 0             -> is char = 0
        & "740A"      _                         ;~     jz    @f                -> if yes then jump to return
        & "3C"        & Hex(Asc($sProbe), 2)  _ ;~     cmp   al, NNh           -> if not then compare with $sProbe
        & "75F7"      _                         ;~     jne   @b                -> if not equal then load next byte from string
        & "C646FF"    & Hex(Asc($sPatch), 2)  _ ;~     mov   byte[esi-1], NNh  -> if equal then replace with $sPatch
        & "EBF1"      _                         ;~     jmp   @b                -> load next byte from string
        & "C3"                                  ;~ @@: ret                     -> return

;~  Technically shouldn't we preserve esi?

    Return DllCallAddress("NONE:cdecl", DllStructGetPtr($tProc), "STR", $sString)[1]
EndFunc

[UEZ]

Fyi, this is only for byte based search / replace! You cannot search / replace for 2 or more characters! Further wchars are not supported!

@jchd: for me the ASM code is just a play ground to learn asm not a real alternative to the internal functions!

Edited April 9, 2015 by UEZ

[jchd]

 

You cannot search / replace for 2 or more characters!

Even 1 fails

[jchd]

Wasn't a sarcasm directed to you, nor anyone else. Of course one can asm-inline much of AutoIt but indeed that doesn't make any sense at all. Also the asm version is only faster on pretty large strings (and I agree that it was the topic at hand).

[UEZ]

I know but sometimes it is good to think outside the box and to broaden the horizon...

Currently I'm trying to convert a GDI+ script to ASM just for learning purposes.


Further I heard that BigNum UDF will be converted to ASM 

Edited April 9, 2015 by UEZ

[UEZ]

This should work also for wchars:

expandcollapse popup

#include <array.au3>

Local $sString, $hTimer, $aCompetitors[3][3] = [ _
        ["StringReplace   ", 0], _
        ["SRER    ", 0], _
        ["ASM  ", 0]]

For $i = 1 To 1
    $sString &= "I forgot that AutoIt uses UCS2 charset ● J'ai oublié qu'AutoIt utilise le jeu de caractères UCS2 ● 我忘了AutoIt的使用UCS2字符集 ● Ich habe vergessen, daß AutoIt verwendet UCS2 Zeichensatz ● Ξέχασα ότι AutoIt χρησιμοποιεί UCS2 σύνολο χαρακτήρων ● Я забув, що AutoIt використовує UCS2 набір символів " & $i & @CRLF
Next

$hTimer = TimerInit()

$a = StringReplace($sString, "●", "―")
Stopwatch(0, $hTimer, $aCompetitors, $a)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[0][0]) & @TAB & $aCompetitors[0][1] & " secs" &  @CRLF)

$b = StringRegExpReplace($sString, "●", "―")
Stopwatch(1, $hTimer, $aCompetitors, $b)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[1][0]) & @TAB & $aCompetitors[1][1] & " secs" & @CRLF)

$c = StringreplaceAll($sString, "●" , "―")
Stopwatch(2, $hTimer, $aCompetitors, $c)
ConsoleWrite(StringFormat("%-20s", $aCompetitors[2][0]) & @TAB & $aCompetitors[2][1] & " secs" & @TAB & @CRLF)

_ArrayDisplay($aCompetitors)

Func Stopwatch($iCompetitor, ByRef $hTimer, ByRef $aCompetitors, ByRef $s)
    $aCompetitors[$iCompetitor][1] = Round(TimerDiff($hTimer) / 1000, 5)
    $aCompetitors[$iCompetitor][2] = $s
    $hTimer = TimerInit()
EndFunc   ;==>Stopwatch

Func StringreplaceAll($sString, $sProbe, $sPatch)
    Local $tCodeBuffer = DllStructCreate("byte ASM[39]"), $tASM_Parameter = DllStructCreate("word search;word replace")
    $tASM_Parameter.search = AscW($sProbe)
    $tASM_Parameter.replace = AscW($sPatch)
    $tCodeBuffer.ASM = "0x8B7424048B7C240831DB31C9668B1F668B4F0266AD6683F800740B6639D875F366894EFEEBEDC3"
    Return DllCall("user32.dll", "none", "CallWindowProcW", "ptr", DllStructGetPtr($tCodeBuffer), "wstr", $sString, "ptr", DllStructGetPtr($tASM_Parameter), "int", 0, "int", 0)[2]
#cs
    _("use32") ;32Bit!
    _("mov esi, dword[esp+4]") ;get start address of the string
    _("mov edi, dword[esp+8]") ;pointer to the parameter struct
    _("xor ebx, ebx") ;ebx = 0
    _("xor ecx, ecx") ;ecx = 0
    _("mov bx, [edi]") ;copy search wstring to bx register
    _("mov cx, [edi + 2]");copy replace wstring to cx register
    _("_loop:")
        _("lodsw") ;load char as wide char (word) and increase the pointer to next wchar
        _("cmp ax, 0") ;end of string reached
        _("jz _end") ;if yes then jump to end mark
        _("cmp ax, " & Swap(Hex(AscW($sProbe),4))) ;compare wchar from string with search wchar
        _("jne _loop") ;if not equal than get next wchar from string
        _("mov word[esi - 2], " & Swap(Hex(AscW($sPatch),4))) ;if equal replace wchar (esi - 2 is because lodsw increases pointer)
        _("jmp _loop") ;get next wchar from string
    _("_end:")
    _("ret") ;return eax
#ce
EndFunc

Edit: removed the swap function by adding paramter struct Edited April 9, 2015 by UEZ

[JohnOne]

In that test, ASM only starts coming into its own with very large strings, but SRE beats it hands down.

Results with loop set to 10000

StringReplace           0.44877 secs
SRER                    0.00469 secs
ASM                     0.01941 secs

[UEZ]

The execution of the asm is not asm only. There is also the swap function which takes also some milliseconds.

But you are right, ASM beats the rest only for large strings.

Edit: changed the ASM code a little bit to remove the swap function!

Edited April 9, 2015 by UEZ

[jchd]

UEZ,

Yeah, thanks for AutoIt users using for instance some asian character set as well as those who need to process generic Unicode text.

Given that the speed gain is at best negligible for most common (non-exceptional) string lengths and that, in general, asm proves prone to oversight (this demonstrates the point) I don't advocate for a routine use of such trick.

Routine use typically needs to process moderately-sized strings in (possibly) bulk volume, but very large strings are exceptional and probably processed in only few instances. The overall gain is questionable.

Saying so I'm not trying to dismiss the value of people or code examplified here. What I'm saying is a paraphrase of the pitfalls often associated with the well-know "premature optimization" paradigm.

My position is that I strongly believe that once a given development platform is selected on solid grouds for 99% of the application needs and only when a peephole optimization is actually important for a given use case, one should seek sidetracks like these.

Disclaimer: I've myself spent considerable time in my life with low- to very-low-level stuff when needed, hence I do know the value to look thru a "microscope" when needed and what a 1450-pages core dump produced by a line printer means (even from the times where memory were actually memory).

[iamtheky]

I get to devote most of my time to event logs that are limited to ANSI.  and I needed baby steps, sorry.  I dragged it aside to see how large the difference was between returning a count and returning an altered string. My next hurdles would be strings of characters or inserting characters based off the other characters, rather than a pressing need to expand beyond ANSI.

and in my example it is 5x faster, and 100,000 lines would be on the very small side of our logs.

*And thanks for yall comments in the assembly portions, they greatly sped up the gathering of materials of which I understand slightly more than jackballs.

[UEZ]

@jchd: I agree with you that I wouldn't use ASM for the most of the scenarios which Autoit speed is more or less comparable.

Also it makes no sense here to discuss any ASM issues.

It was just an instructive exercise for me regarding ASM coding practises.

Edited April 9, 2015 by UEZ

[iamtheky]

is

UBound(StringRegExp($sString, @LF, 3))

functionally the same as

$char = @LF
UBound(stringregexp($sString , '[?^' & $char & ']' , 3))

and if ive not overlooked something glaring,  when is the latter more useful (if ever).  In this case it is twice as slow.

Edited April 13, 2015 by boththose