qwert

Is _GUICtrlRichEdit_StreamToVar limited to 2048 chars?

24 posts in this topic

#1 ·  Posted

My testing shows that 2048 is the maximum number of characters that can be read from a RichEdit to a variable.  This is the statement I'm using:

$line = _GUICtrlRichEdit_StreamToVar($hRichEdit, False, False)

Everything works, except that $line only receives the first 2048 characters. I haven't seen anything that mentions this.

Is this a known limit?  If so, is it adjustable?

Thanks in advance for any help.

Share this post


Link to post
Share on other sites



#2 ·  Posted

Digging deeper, I found that the UDF uses _SendMessage to stream from the control.

MSDN, in turn, offers this regarding the EM_STREAMOUT:

Quote

When you send an EM_STREAMOUT message, the rich edit control makes repeated calls to the EditStreamCallback function specified by the pfnCallback member of the EDITSTREAM structure. Each time it calls the callback function, the control passes a buffer containing a portion of the contents of the control. This process continues until the control has passed all its contents to the callback function, or until an error occurs.

"A portion" explains the behavior ... but without a hint of how to make the UDF "continue the process".

Suggestions?  Has anyone else experienced this?

Share this post


Link to post
Share on other sites

#3 ·  Posted

Makes sense to me, that it seems to be reading max length each time, and then combine it into a whole string at the end?

Like:

Message = Some string
Size = 10,308

It then splits that into 6 chunks ( 2048, 2048, 2048, 2048, 2048, 68 )
Then combine. This is what I gather from your explanation. It would make sense... but for some reason it's only pulling the first chunk, and not the remainder of the of the chunks.

Share this post


Link to post
Share on other sites

#4 ·  Posted

Another small step forward:  Link to codeproject post

Note, in particular:

Quote

"Example callback functions are frequently found on the Internet, however, for my own use these were unsuitable as they did not handle large amounts of text."

Unfortunately (for me), I can't make sense of his, what appear to be, clearcut solutions, in an unfamiliar language.

I'll continue my search.

 

Share this post


Link to post
Share on other sites

#5 ·  Posted

After much testing, I haven't been able to crack the code regarding how StreamToVar is designed to work with large RichEdit content. It doesn't seem designed to cycle, as the msdn statement I cited describes.

I've attached a comparison of the two "stream to" functions.  StreamToFile works fully.  StreamToVar truncates the content.  If anyone can point out why that might be, I'll be glad to try other settings.

But, for now, I've fashioned a workaround that works 100% of the time (... although I'm never comfortable with these kinds of thing):

Local $num = _GUICtrlRichEdit_GetTextLength($hRichEdit)
If $num > 2000 Then
    _GUICtrlRichEdit_StreamToFile($hRichEdit, "C:\Temp\RichContent.txt", False, False)
    $line = FileRead("C:\Temp\RichContent.txt")
Else
    $line = _GUICtrlRichEdit_StreamToVar($hRichEdit, False, False)
EndIf

The simple act of directing the content to a file causes the entire RichEdit to be extracted (as text). 

Stream Compare.PNG

Share this post


Link to post
Share on other sites

#6 ·  Posted

please make two diffrent au3 file with and without change, zip them, and upload here.

I will check it this weekend.

Thanks, for this findings, and your commitment.

Regards,

mLipok

 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#7 ·  Posted

Thanks for your response ... only, I don't quite understand what you need.

The above capture is comparing ToFile with ToVar.  I didn't make any change to either.

Just let me know.

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

More details needed because this works for me ...
 

Spoiler

 

#include <GUIConstantsEx.au3>
#include <GuiRichEdit.au3>
#include <WindowsConstants.au3>

$hGui = GUICreate("Example (" & StringTrimRight(@ScriptName, 4) & ")", 320, 350, -1, -1)
$hRichEdit = _GUICtrlRichEdit_Create($hGui, "", 10, 10, 300, 220, _
            BitOR($ES_MULTILINE, $WS_VSCROLL, $ES_AUTOVSCROLL))
$idBtnNext = GUICtrlCreateButton("StreamToVar", 220, 310, 90, 30)
GUISetState()

$text = "AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying runtimes required! AutoIt was initially designed for PC =roll out= situations to reliably automate and configure thousands of PCs. Over time it has become a powerful language that supports complex expressions, user functions, loops and everything else that veteran scripters would expect. Features: AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying runtimes required! AutoIt was initially designed for PC =roll out= situations to reliably automate and configure thousands of PCs. Over time it has become a powerful language that supports complex expressions, user functions, loops and everything else that veteran scripters would expect. Features: AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying runtimes required! AutoIt was initially designed for PC =roll out= situations to reliably automate and configure thousands of PCs. Over time it has become a powerful language that supports complex expressions, user functions, loops and everything else that veteran scripters would expect. Features:"

_GUICtrlRichEdit_SetText($hRichEdit, $text)
msgbox(0,"text", "text length is " & stringlen($text) )


While 1
        $iMsg = GUIGetMsg()
        Select
            Case $iMsg = $GUI_EVENT_CLOSE
                _GUICtrlRichEdit_Destroy($hRichEdit) 
                Exit
            Case $iMsg = $idBtnNext
                 $var = _GUICtrlRichEdit_StreamToVar($hRichEdit)
                 msgbox(0,"var", "var length is " & stringlen($var) )
        EndSelect
WEnd


 

Sorry. it doesn't work indeed with flag $bRtf = false
 

 

Edited by mikell

Share this post


Link to post
Share on other sites

#9 ·  Posted

4 hours ago, qwert said:

Thanks for your response ... only, I don't quite understand what you need.

The above capture is comparing ToFile with ToVar.  I didn't make any change to either.

Just let me know.

I read this on my smartphone thsi is qutie posible i misunderstod something.

I will it in the evening.


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

That's really strange. So if you initially try to read the RichEdit to a variable, it gets truncated. But when you first write that same RichEdit to another file and then read that file, it reads the entire thing? If that's the case, it sounds like some weird edge case with the streaming is occurring. I'll take a look at it for a bit too.

Edited by anthonyjr2

UHJvZmVzc2lvbmFsIENvbXB1dGVyZXI=

Share this post


Link to post
Share on other sites

#11 ·  Posted

Alright, I did some digging. It seems that the problem lies within the calls StreamToVar makes itself. We can trace this back to

Func __GCR_StreamToFileCallback($hFile, $pBuf, $iBuflen, $pQbytes)
    Local $tQbytes = DllStructCreate("long", $pQbytes)
    DllStructSetData($tQbytes, 1, 0)
    Local $tBuf = DllStructCreate("char[" & $iBuflen & "]", $pBuf)
    Local $s = DllStructGetData($tBuf, 1)
    FileWrite($hFile, $s)
    DllStructSetData($tQbytes, 1, StringLen($s))
    Return 0
EndFunc   ;==>__GCR_StreamToFileCallback

There is some memory issue here, either in one of its calls or one of the DLLs.

If you go and make a slight change to the GUIRichEdit.au3 UDF and throw in a random error message box like so:

$__g_pGRC_sStreamVar = ""
    _SendMessage($hWnd, $EM_STREAMOUT, $wParam, $tEditStream, 0, "wparam", "struct*")
    Local $iError = DllStructGetData($tEditStream, "dwError")
    If $iError <> 0 Then
        SetError(700, $iError, "")
        MsgBox(0, "", $iError)
    EndIF
    Return $__g_pGRC_sStreamVar

We can see that $iError is set to "2147680368" which according to the internet is related to the constant " STG_E_MEDIUMFULL." If we go to the MSDN page for this constant, https://msdn.microsoft.com/en-us/library/ee489596.aspx we see that it states that there in insufficient disk space to complete the operation. What does this mean exactly? According to what I've researched it seems to be an issue with EM_LIMITTEXT in the RichEdit control. You can manually change this if you are using something like C++, but I can't think of any way to fix it from AutoIt because we are just directly calling the DLLs.

The only solution I see here is maybe coming at it from the other direction and filtering input somehow. Maybe trying to convert the RichText to normal text another way and then putting that into a variable?


UHJvZmVzc2lvbmFsIENvbXB1dGVyZXI=

Share this post


Link to post
Share on other sites

#12 ·  Posted

2 minutes ago, anthonyjr2 said:

According to what I've researched it seems to be an issue with EM_LIMITTEXT in the RichEdit control.

Interesting that you mention this, because this happens to be the only RichEdit that I've ever defined where I set a limit (3600 characters).  

_GUICtrlRichEdit_SetRealLimitOnText($hRichEdit, 3600)

While this is apparently apart from the stream calls, maybe it comes into play, internally. 

 

Share this post


Link to post
Share on other sites

#13 ·  Posted

It still happens even if we don't set a limit, such as in the example posted slightly above. I'm looking into it more right now, EditStreamCallback is a real pain to debug because I can't seem to find much information on why it is throwing STG_E_MEDIUMFULL. I've traced the calls all the way up and see no reason why it should error out and discard the buffer after 2048 characters.


UHJvZmVzc2lvbmFsIENvbXB1dGVyZXI=

Share this post


Link to post
Share on other sites

#14 ·  Posted

Quote

It still happens even if we don't set a limit

Yes, I just confirmed that, as well.  Regarding the setting of "False - write only text", this may explain why, generally speaking, I don't recall ever having a problem loading and saving rich text content.  But in this case, I have to extract only the text.

 

Share this post


Link to post
Share on other sites

#15 ·  Posted

Suggestion - in GuiRichEdit.au3

Func __GCR_StreamToVarCallback($iCookie, $pBuf, $iBuflen, $pQbytes)
    #forceref $iCookie
    Local $tQbytes = DllStructCreate("long", $pQbytes)
    DllStructSetData($tQbytes, 1, 0)
    Local $tBuf = DllStructCreate("char[" & $iBuflen & "]", $pBuf)
    Local $s = DllStructGetData($tBuf, 1)
    $__g_pGRC_sStreamVar &= $s
    DllStructSetData($tQbytes, 1, StringLen($s))    ;<<<<<<<<<<<<<< add this line
    Return 0
EndFunc   ;==>__GCR_StreamToVarCallback

 

1 person likes this

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

@mikell Nice solution, you beat me to it by a few minutes, haha. I was just about to post something similar :D

For an explanation, it seems we were overwriting our buffer in the callback, which made EM_STREAMOUT throw an error. Annoyingly, Microsoft has not documented the error codes for any of these functions so it makes debugging stuff like this quite annoying.

Edited by anthonyjr2

UHJvZmVzc2lvbmFsIENvbXB1dGVyZXI=

Share this post


Link to post
Share on other sites

#17 ·  Posted

mikell you already have the size. $iBuflen

 

Saludos

Share this post


Link to post
Share on other sites

#18 ·  Posted

Confirmation!

That works! And with my limited understanding, I can almost follow why, now that I see it. But I never would have thought to try such a thing.

Thanks to each of you.

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

qwert here is the information

pcb [in]

Type: LONG*

Pointer to a variable that the callback function sets to the number of bytes actually read or written.

 

The control calls the callback function repeatedly, transferring a portion of the data with each call. The control continues to call the callback function until one of the following conditions occurs:

The callback function returns a nonzero value.

The callback function returns zero in the * pcb parameter.

An error occurs that prevents the rich edit control from transferring data into or out of itself. Examples are out-of-memory situations, failure of a system function, or an invalid character in the read buffer.

For a stream-in operation, the RTF code contains data specifying the end of an RTF block.

For a stream-in operation on a single-line edit control, the callback reads in an end-of-paragraph character (CR, LF, VT, LS, or PS).

Saludos

 

Edited by Danyfirex

Share this post


Link to post
Share on other sites

#20 ·  Posted

59 minutes ago, Danyfirex said:

mikell you already have the size. $iBuflen

Right. this line

DllStructSetData($tQbytes, 1, 0)

should be like this

DllStructSetData($tQbytes, 1, $iBuflen)

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now