_GUICtrlRichEdit_SetCharBkColor performance help please

[Melba23]

orbs,

As promised, a "proof of concept" script for you:

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <GuiRichEdit.au3>
#include <String.au3>
#include <Color.au3>
#include <Array.au3>

Global Const $nGUI_MaxW = 500, $nGUI_MaxH = 400
Global $nGUI_W = 720, $nGUI_H = 500, $nRichEditOffset = 10, $nToolbarOffset = 35

Global $aHiLites[] = ["22,27,1", "54,88,2", "121,150,3", "250,251,2", "373,472,1"]
Global $aColours[3] = [0xFFCCCC, 0xCCFFCC, 0xCCCCFF]

Global $hGui = GUICreate('_GUICtrlRichEdit_SetCharBkColor performance test', $nGUI_W, $nGUI_H)
GUISetBkColor(0xCCDDEE)

Global $sContentSource = _StringRepeat("12345678901234567890" & @CRLF, 25)

Global $hRichEdit = _GUICtrlRichEdit_Create($hGui, $sContentSource, $nRichEditOffset, $nRichEditOffset + $nToolbarOffset, $nGUI_W - $nRichEditOffset * 2, $nGUI_H - $nRichEditOffset * 2 - $nToolbarOffset - 24, BitOR($ES_MULTILINE, $ES_READONLY, $WS_HSCROLL, $WS_VSCROLL))
_GUICtrlRichEdit_SetFont($hRichEdit, Default, 'Courier New')

$nBegin = TimerInit()

; Make sure there is no selection
_GUICtrlRichEdit_Deselect($hRichEdit)
; Read the RTF stream
$sContent = _GUICtrlRichEdit_StreamToVar($hRichEdit)

; Migh need to remove any current highlighting here - a simple regex should do the trick

; Define the highlight colour table
$sColDef = "{\colortbl ;"
For $i = 0 To UBound($aColours) - 1
    $sColDef &= "\red" & _ColorGetRed($aColours[$i]) & "\green" & _ColorGetGreen($aColours[$i]) & "\blue" & _ColorGetBlue($aColours[$i]) & ";"
Next
$sColDef &= "}"
; And add it as line 2
$sContent = StringReplace($sContent, ";}}", ";}}" & @CRLF & $sColDef)

; Split the stream on "}}"
Global $aSplit_1 = StringSplit($sContent, "}}" & @CRLF, $STR_ENTIRESPLIT)
;_ArrayDisplay($aSplit_1, "1", Default, 8)

; Now split text part on "/par"
Global $aSplit_2 = StringSplit($aSplit_1[2], "\par" & @CRLF, $STR_ENTIRESPLIT)
; Add column for length
_ArrayColInsert($aSplit_2, 1)
For $i = 2 To $aSplit_2[0][0] - 2
    $aSplit_2[$i][1] = $aSplit_2[$i - 1][1] + StringLen($aSplit_2[$i][0])
Next
;_ArrayDisplay($aSplit_2, "2", Default, 8)

For $i = 0 To UBound($aHiLites) - 1
    ; Parse each element to get highlight coords and colour
    $aData = StringSplit($aHiLites[$i], ",")
    ;_ArrayDisplay($aData, "Highlight", Default, 8)

    ; Find start index of the highlight string
    For $j = 2 To $aSplit_2[0][0] - 2
        If $aSplit_2[$j][1] > $aData[1] Then
            $iLen = StringLen($aSplit_2[$j][0])
            $iBreak = $iLen - ($aSplit_2[$j][1] - $aData[1])

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ; Insert colour string
            $aSplit_2[$j][0] = StringMid($aSplit_2[$j][0], 1, $iBreak) & "\highlight" & $aData[3] & " " & StringMid($aSplit_2[$j][0], $iBreak + 1)

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ExitLoop
        EndIf
    Next

    ; Find end index
    For $k = $j - 1 To $aSplit_2[0][0] - 2
        If $aSplit_2[$k][1] > $aData[2] Then
            $iLen = StringLen($aSplit_2[$k][0])
            $iBreak = $iLen - ($aSplit_2[$k][1] - $aData[2])

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ; Insert normal string
            $aSplit_2[$k][0] = StringMid($aSplit_2[$k][0], 1, $iBreak) & "\highlight0 " & StringMid($aSplit_2[$k][0], $iBreak + 1)

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ExitLoop
        EndIf
    Next

Next

; Recreate stream
$sContent = $aSplit_1[1] & "}}" & @CRLF
For $i = 1 To $aSplit_2[0][0] - 1
    $sContent &= $aSplit_2[$i][0] & "\par" & @CRLF
Next
$sContent &= "}"

; Now replace the current text with the highlighted version
_GUICtrlRichEdit_StreamFromVar($hRichEdit, $sContent)

ConsoleWrite(TimerDiff($nBegin) & @CRLF)

GUISetState()

While 1
    Switch GUIGetMsg()
        Case $GUI_EVENT_CLOSE
            _GUICtrlRichEdit_Destroy($hRichEdit)
            Exit
    EndSwitch
WEnd

I hope it is self-explanatory - if not you will have to wait until tomorrow.

M23

Edit:

A better version that runs from the bottom up:

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <GuiRichEdit.au3>
#include <String.au3>
#include <Color.au3>
#include <Array.au3>

Global Const $nGUI_MaxW = 500, $nGUI_MaxH = 400
Global $nGUI_W = 720, $nGUI_H = 500, $nRichEditOffset = 10, $nToolbarOffset = 35

Global $aHiLites[] = ["22,27,1", "54,88,2", "121,150,3", "250,251,2", "251,252,3", "373,472,1"]
Global $aColours[3] = [0xFFCCCC, 0xCCFFCC, 0xCCCCFF]

Global $hGui = GUICreate('_GUICtrlRichEdit_SetCharBkColor performance test', $nGUI_W, $nGUI_H)
GUISetBkColor(0xCCDDEE)

Global $sContentSource = _StringRepeat("12345678901234567890" & @CRLF, 25)

Global $hRichEdit = _GUICtrlRichEdit_Create($hGui, $sContentSource, $nRichEditOffset, $nRichEditOffset + $nToolbarOffset, $nGUI_W - $nRichEditOffset * 2, $nGUI_H - $nRichEditOffset * 2 - $nToolbarOffset - 24, BitOR($ES_MULTILINE, $ES_READONLY, $WS_HSCROLL, $WS_VSCROLL))
_GUICtrlRichEdit_SetFont($hRichEdit, Default, 'Courier New')

$nBegin = TimerInit()

; Make sure there is no selection
_GUICtrlRichEdit_Deselect($hRichEdit)
; Read the RTF stream
$sContent = _GUICtrlRichEdit_StreamToVar($hRichEdit)

; Might need to remove any current highlighting here - a sinple regex should do the trick

; Define the highlight colour table
$sColDef = "{\colortbl ;"
For $i = 0 To UBound($aColours) - 1
    $sColDef &= "\red" & _ColorGetRed($aColours[$i]) & "\green" & _ColorGetGreen($aColours[$i]) & "\blue" & _ColorGetBlue($aColours[$i]) & ";"
Next
$sColDef &= "}"
; And add it as line 2
$sContent = StringReplace($sContent, ";}}", ";}}" & @CRLF & $sColDef)

; Split the stream on "}}"
Global $aSplit_1 = StringSplit($sContent, "}}" & @CRLF, $STR_ENTIRESPLIT)
;_ArrayDisplay($aSplit_1, "1", Default, 8)

; Now split text part on "/par"
Global $aSplit_2 = StringSplit($aSplit_1[2], "\par" & @CRLF, $STR_ENTIRESPLIT)
; Add column for length
_ArrayColInsert($aSplit_2, 1)
For $i = 2 To $aSplit_2[0][0] - 2
    $aSplit_2[$i][1] = $aSplit_2[$i - 1][1] + StringLen($aSplit_2[$i][0])
Next
;_ArrayDisplay($aSplit_2, "2", Default, 8)

For $i = UBound($aHiLites) - 1 To 0 Step -1
    ; Parse each element to get highlight coords and colour
    $aData = StringSplit($aHiLites[$i], ",")
    ;_ArrayDisplay($aData, "Highlight", Default, 8)

    ; Find end index of the highlight string
    For $j = $aSplit_2[0][0] - 2 To 2 Step -1
        ; Found line to amend
        If $aSplit_2[$j][1] < $aData[2] Then
            ; Calculate insert point
            $iBreak = $aData[2] - $aSplit_2[$j][1]

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ; Insert default colour string
            $aSplit_2[$j][0] = StringMid($aSplit_2[$j][0], 1, $iBreak) & "\highlight0 " & StringMid($aSplit_2[$j][0], $iBreak + 1)

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ExitLoop
        EndIf
    Next

    ; Find start index of the highlight string
    For $j = $j + 1 To 2 Step -1
        If $aSplit_2[$j][1] < $aData[1] Then
            $iBreak = $aData[1] - $aSplit_2[$j][1]

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ; Insert required colour string
            $aSplit_2[$j][0] = StringMid($aSplit_2[$j][0], 1, $iBreak) & "\highlight" & $aData[3] & " " & StringMid($aSplit_2[$j][0], $iBreak + 1)

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ExitLoop
        EndIf
    Next

Next

; Recreate stream
$sContent = $aSplit_1[1] & "}}" & @CRLF
For $i = 1 To $aSplit_2[0][0] - 1
    $sContent &= $aSplit_2[$i][0] & "\par" & @CRLF
Next
$sContent &= "}"

; Now replace the current text with the highlighted version
_GUICtrlRichEdit_StreamFromVar($hRichEdit, $sContent)

ConsoleWrite(TimerDiff($nBegin) & @CRLF)

GUISetState()

While 1
    Switch GUIGetMsg()
        Case $GUI_EVENT_CLOSE
            _GUICtrlRichEdit_Destroy($hRichEdit)
            Exit
    EndSwitch
WEnd

That way the colouring inserts do not affect the break point count if there are several on the same line.

Second Edit:

Still some problems with the first line but working on it.

Edited February 16, 2015 by Melba23

[orbs]

this is great, but - it misses any highlight on the first line.

my RTF stream header looks like this:

{\rtf1\fbidis\ansi\ansicpg1255\deff0\deflang1037{\fonttbl{\f0\fnil\fcharset0 MS Shell Dlg;}{\f1\fnil\fcharset177 MS Shell Dlg;}}
{\*\generator Msftedit 5.41.21.2510;}\viewkind4\uc1\pard\ltrpar\lang1033\f0\fs17 1234567890\par
1234567890\par
1234567890\par
...

so you see, the first line is actually part of the header as split by }} - and is ignored by the loops that look for the break points.

i amended it so:

1) calculate the actual first line length (as mentioned in earlier post)

2) after par split, remove from the first line the left-side part, which is the header part (using the length calculated) and remember that part to be later rejoined to the stream

3) the loops scan includes first line

4) rejoin the header part to the stream

here's the amended script:

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <GuiRichEdit.au3>
#include <String.au3>
#include <Color.au3>
#include <Array.au3>

Global Const $nGUI_MaxW = 500, $nGUI_MaxH = 400
Global $nGUI_W = 720, $nGUI_H = 500, $nRichEditOffset = 10, $nToolbarOffset = 35

; >>>>>>>>>>>>>>>>>>>> changed the first substring to be in the first line
Global $aHiLites[6] = ["2,7,1", "54,88,2", "121,150,3", "250,251,2", "251,252,3", "373,472,1"]
Global $aColours[3] = [0xFFCCCC, 0xCCFFCC, 0xCCCCFF]

Global $hGui = GUICreate('_GUICtrlRichEdit_SetCharBkColor performance test', $nGUI_W, $nGUI_H)
GUISetBkColor(0xCCDDEE)

; >>>>>>>>>>>>>>>>>>>> changed the length of the line from 20 to 10 for easier visibility of the result
Global $sContentSource = _StringRepeat("1234567890" & @CRLF, 50)

; >>>>>>>>>>>>>>>>>>>> remember the first line - will be used later
Global $sContentSourceLine1=StringLeft($sContentSource,StringInStr($sContentSource,@CRLF)-1)

Global $hRichEdit = _GUICtrlRichEdit_Create($hGui, $sContentSource, $nRichEditOffset, $nRichEditOffset + $nToolbarOffset, $nGUI_W - $nRichEditOffset * 2, $nGUI_H - $nRichEditOffset * 2 - $nToolbarOffset - 24, BitOR($ES_MULTILINE, $ES_READONLY, $WS_HSCROLL, $WS_VSCROLL))
_GUICtrlRichEdit_SetFont($hRichEdit, Default, 'Courier New')

$nBegin = TimerInit()

; Make sure there is no selection
_GUICtrlRichEdit_Deselect($hRichEdit)
; Read the RTF stream
$sContent = _GUICtrlRichEdit_StreamToVar($hRichEdit)

ConsoleWrite($sContent&@LF)

; Might need to remove any current highlighting here - a sinple regex should do the trick

; Define the highlight colour table
$sColDef = "{\colortbl ;"
For $i = 0 To UBound($aColours) - 1
    $sColDef &= "\red" & _ColorGetRed($aColours[$i]) & "\green" & _ColorGetGreen($aColours[$i]) & "\blue" & _ColorGetBlue($aColours[$i]) & ";"
Next
$sColDef &= "}"
; And add it as line 2
$sContent = StringReplace($sContent, ";}}", ";}}" & @CRLF & $sColDef, 1)

; Split the stream on "}}"
Global $aSplit_1 = StringSplit($sContent, "}}" & @CRLF, $STR_ENTIRESPLIT)
;_ArrayDisplay($aSplit_1, "1", Default, 8)

; Now split text part on "/par"
Global $aSplit_2 = StringSplit($aSplit_1[2], "\par" & @CRLF, $STR_ENTIRESPLIT)

; >>>>>>>>>>>>>>>>>>>> remember first line prefix (part of RTF header) - will be rejoined later to the stream
Global $sLine1Prefix=StringTrimRight($aSplit_2[1],StringLen($sContentSourceLine1))
; >>>>>>>>>>>>>>>>>>>> but for now, remove it from the first line
$aSplit_2[1]=StringRight($aSplit_2[1],StringLen($sContentSourceLine1))

; Add column for length
_ArrayColInsert($aSplit_2, 1)
For $i = 2 To $aSplit_2[0][0] - 2
    $aSplit_2[$i][1] = $aSplit_2[$i - 1][1] + StringLen($aSplit_2[$i][0])
Next

; >>>>>>>>>>>>>>>>>>>> just for consistence
$aSplit_2[1][1]=0

;_ArrayDisplay($aSplit_2, "2", Default, 8)

For $i = UBound($aHiLites) - 1 To 0 Step -1
    ; Parse each element to get highlight coords and colour
    $aData = StringSplit($aHiLites[$i], ",")
    ;_ArrayDisplay($aData, "Highlight", Default, 8)

    ; Find end index of the highlight string
    ; >>>>>>>>>>>>>>>>>>>> backscanning until line #1, not only until line #2
    For $j = $aSplit_2[0][0] - 2 To 1 Step -1
        ; Found line to amend
        If $aSplit_2[$j][1] < $aData[2] Then
            ; Calculate insert point
            $iBreak = $aData[2] - $aSplit_2[$j][1]

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ; Insert default colour string
            $aSplit_2[$j][0] = StringMid($aSplit_2[$j][0], 1, $iBreak) & "\highlight0 " & StringMid($aSplit_2[$j][0], $iBreak + 1)

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ExitLoop
        EndIf
    Next

    ; Find start index of the highlight string
    ; >>>>>>>>>>>>>>>>>>>> backscanning until line #1, not only until line #2
    For $j = $j + 1 To 1 Step -1
        If $aSplit_2[$j][1] < $aData[1] Then
            $iBreak = $aData[1] - $aSplit_2[$j][1]

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ; Insert required colour string
            $aSplit_2[$j][0] = StringMid($aSplit_2[$j][0], 1, $iBreak) & "\highlight" & $aData[3] & " " & StringMid($aSplit_2[$j][0], $iBreak + 1)

            ;_ArrayDisplay($aSplit_2, "2", Default, 8)

            ExitLoop
        EndIf
    Next

Next

;_ArrayDisplay($aSplit_2, "2", Default, 8)

; Recreate stream
; >>>>>>>>>>>>>>>>>>>> and rejoing the part of the header that was the prefix of first line
$sContent = $aSplit_1[1] & "}}" & @CRLF & $sLine1Prefix
For $i = 1 To $aSplit_2[0][0] - 1
    $sContent &= $aSplit_2[$i][0] & "\par" & @CRLF
Next
$sContent &= "}"

;ConsoleWrite($sContent & @LF)

; Now replace the current text with the highlighted version
_GUICtrlRichEdit_StreamFromVar($hRichEdit, $sContent)

ConsoleWrite(TimerDiff($nBegin) & @CRLF)

GUISetState()

While 1
    Switch GUIGetMsg()
        Case $GUI_EVENT_CLOSE
            _GUICtrlRichEdit_Destroy($hRichEdit)
            Exit
    EndSwitch
WEnd

for my modifications listed above look for this indicator:

; >>>>>>>>>>>>>>>>>>>> [my comment]

so finally it seems to work - i'm now testing it to apply in my real script! 

 

EDIT: i also changed the line length to 10 characters - it's easier on my eyes, to examine the final result.

EDIT #2: when inserting the colors string, it should happen only once - i added the flag 1.

Edited February 16, 2015 by orbs

[Melba23]

orbs,

Yes, the first line was not being separated from the remaining header and as all the lines were the same it took me a while to realise.

Your fix seems to work - well done. Delighted I could point the way.

M23

[orbs]

ok, so the technique to manipulate the RTF stream is finally implemented in this project:

'?do=embed' frameborder='0' data-embedContent>>

it is somewhat different than discussed, though. it goes like this:

1) on the original text - find a character of ASCII code in range 33..127, that is not a backslash or curly brackets. that does not already exist in the text

2) insert this 'marker' in all positions where the "highlight" needs to be inserted into the stream

3) put the text in the RichEdit and read it back as stream

4) in the stream, replace the markers with their respective highlights

5) feed the modified stream back to the RichEdit

this method makes the "split stream to lines" and associated issues unnecessary, and avoids the calculations of the highlight positions in the stream. these calculations needed to compensate for escaped characters as well as unicode and extended ASCII, tabs, etc. which is by far more complicated than the method i eventually took.

still major thanks to Melba23 , for introducing me to the concept of manipulating the RTF stream.

[Melba23]

orbs,

Delighted I could help - I learnt a lot too.

In the _ArrayDisplay UDF I used one of the "user-defined" Unicode characters to act as a delimiter so that the user could have the standard delimiter ("|") within the elements - you might like to think about doing the same thing here so that you can skip step 1 of the above.

M23

[grisina]

AMD Athlon II 620 QuadCore

 

2.5 seconds on my turtle...