How to pass concatenated strings using Const ByRef - warning ugly

[JockoDundee]

I have a log function (not quite as simple as what is shown below), and to cut down on the added overhead that this can bring, I have attempted to pass the log messages in as Const ByRef.  ByRef because the strings can often be quite long - error strings from running other programs etc., so passing as a ptr should be more efficient.  Const because god forbid I modify a passed variable accidentally and it changes in the program.

Sometimes I pass in a literal string “Error - bad filehandle”, sometimes it might be already in a variable, some times it is a combination that I concatenate on the fly, $sdatetime & “Error: IP “ &  $sIPaddress & “ not found” etc.  Unfortunately, this last case gives an error: function expecting a variable.

Although, I understand why the function expects a variable, I would note that in the case of a literal there is no explicit variable at all.

Furthermore, just by pre-concatenating the statement with a “” literal “fixes” the problem.

To wit:

LogIt("sometimes literal strings") ; WORKS

$sVar1="sometimes variables"
LogIt($sVar1) ; WORKS

LogIt($sVar1 & "sometimes both concatenated")  ; ERROR 

LogIt("" & $sVar1 & "leading empty string")  ; WORKS BUT UGLY 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
Func LogIt(Const ByRef $sLogLine)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

ConsoleWrite($slogline &@CRLF)

EndFunc

Question, is there a better way than the ugly “” workaround, that still retains all the benefits outlined?

[mLipok]

ByRef is needed when you want to change passed variable back, or when you passing realy long, big data as parameters.

So why you need Const ByRef  in your scenario ?

Edited November 13, 2020 by mLipok

[JockoDundee]

  On 11/13/2020 at 11:29 PM, mLipok said:

ByRef is needed when you want to change passed variable back, or when you passing realy long, big data as parameters.

Expand  

Agreed.  Sometimes I’m passing a line of text, sometimes (when I am passing thru diagnostic output from another program), it can be upwards of 500 chars.  It’s not megabytes, but it’s still something.  
Why allocate/deallocate 500 bytes if you don’t have to?  And it doesn’t require anything but the keywords.  If it wasn’t for the “” things I’d have no qualms.

I definitely see a reduction in performance when logging is on, so it’s one thing I’m trying...  Probably not going to help much, it’s most likely the disk access/writing that takes the lions share.

[mLipok]

If you use this in this way only for this one function, and as I assume only to to SciTE console via ConsoleWrite() function, then the problem is SciTE.

Take a look what is going on with SciTE when the console pane is filled with 2 MB of logs.

And you are not able to fix this with any other solutions like to buy faster CPU for example the newest Intel i7

The ByRef in regards of speed speed it only makes sense when you pass real big data or minior data but very often.
For example when I get Binardata from MS SQL which contains PDF files, then I pass them as ByRef as they usually have about 100KB and when I process a few thousands of such PDF then it can be very stressfull for computer to pass them as normaln variable, in comparition with the case when I pass them as ByRef.

 

 

[JockoDundee]

  On 11/14/2020 at 12:08 AM, mLipok said:

And you are not able to fix this with any other solutions like to buy faster CPU for example the newest Intel i7

Expand  

 But mLipok, in general, why not pass any read only variable no matter the length as Const ByRef?

What’s the downside?

The upside is slightly better performance (if you do it everywhere), and it is a check on your code because it won’t let you modify variables that you have deemed immutable.  It is also a form of self-commenting, making for easier debugging because you don’t have to trace/break a variable.

The only downside I know of is this concatenation kludge, which is only an aesthetic, and typing 11 characters.

 

[Gianni]

since "Const" is a keyword to declare a (new) constant, I am afraid that using it in that way, that is, in a function parameter, could override the purpose of ByRef, as I suppose that a new  local constant that contains the entire passed string is created inside the function  and thus forcing the transfer of all bytes from the caller to the function. I used "I suppose" because I'm not sure about this and maybe I'm wrong (?)

[JockoDundee]

  On 11/14/2020 at 8:40 AM, Chimp said:

since "Const" is a keyword to declare a (new) constant, I am afraid that using it in that way, that is, in a function parameter, could override the purpose of ByRef, as I suppose that a new  local constant that contains the entire passed string is created inside the function  and thus forcing the transfer of all bytes from the caller to the function. I used "I suppose" because I'm not sure about this and maybe I'm wrong (?)

Expand  

Found this here.

Btw, one  thing I just found out that also sucks a bit is that apparently Const ByRef arguments can’t have default values...

[Gianni]

.... well, clearly my guess was wrong ... 👎

[Earthshine]

Why are you even doing this at all? Use a professional log tool like log4a.au3

[Nine]

I believe it is a good candidate for bug tracker

[Jos]

  On 11/14/2020 at 1:43 PM, Nine said:

I believe it is a good candidate for bug tracker

Expand  

Why?  At best all formats need to get an error when the parameter isn't a variable as the helpfile clearly states that:

  Quote

Note that not only a named variable can be passed for a ByRef parameter - unnamed temporary variables, such as function return values, may be passed as ByRef parameters as well. However, a literal cannot be passed to a ByRef parameter.

Expand  

Jos

[Nine]

But look at example provided by jocko above.  You can with Const ByRef, it is clear that it is a valid option. 

[Jos]

  On 11/14/2020 at 2:00 PM, Nine said:

You can with Const ByRef, it is clear that it is a valid option. 

Expand  

You mean it is working, which doesn't perse means it is valid.
I do agree that the Helpfile states something different from what is shown by the example script in this thread thought. 

Jos

 

[Nine]

If it is not valid then Tutorial should be modified accordingly and Au3Check should provide some kind of warnings.  But I still think it would lame to remove it when it is (or seems to be) working correctly in many situations except for a specific one.

[Jos]

  On 11/14/2020 at 2:22 PM, Nine said:

I still think it would lame to remove it when it is (or seems to be) working correctly in many situations except for a specific one.

Expand  

Not sure "lame" would be a valid argument and lets simply look at what one could expect from a Byref keyword. 
I honestly would say in this above case to simply get rid of the "Const Byref" keywords as it is clear that the UDF should support a mix of Literal strings/Variables/Formula's, and only use Byref when you use a Variable as parameter....

...  but it really isn't my call as I don't do any development in AutoIt3 it self and merely look at it that way as that is how it works in other languages.   

Jos 

[JockoDundee]

  On 11/14/2020 at 1:34 PM, Earthshine said:

Why are you even doing this at all? Use a professional log tool like log4a.au3

Expand  

Because I wanted a low-overhead way of doing things, one that minimizes the “observer effect”.

log4a seems like a robust package and is low-overhead from a economy of code perspective, but it clearly was not designed for performance. Consider the following code snippet from log4a:

expandcollapse popup

Func _log4a_Fatal($sMessage, $bOverride = False)
    _log4a_Message($sMessage, $LOG4A_LEVEL_FATAL, $bOverride)
EndFunc   ;==>_log4a_LogFatal

Func _log4a_Message($sMessage, $eLevel = $LOG4A_LEVEL_INFO, $bOverride = False)
    If Not $__LOG4A_ENABLED And Not $bOverride Then Return
    If $eLevel < $LOG4A_LEVEL_TRACE Or $eLevel > $LOG4A_LEVEL_FATAL Then Return SetError(1)
    If ($eLevel < $__LOG4A_LEVEL_MIN Or $eLevel > $__LOG4A_LEVEL_MAX) And Not $bOverride Then Return SetError(2)

    Local $bConsole = False, $bFile = False, $iMethod = $__LOG4A_OUTPUT
    If @Compiled Then $iMethod = $__LOG4A_COMPILED_OUTPUT
    Local $sLine = __log4a_FormatMessage($sMessage, $__aLog4aLevels[$eLevel])

    Switch $iMethod
        Case $LOG4A_OUTPUT_CONSOLE
            $bConsole = True
        Case $LOG4A_OUTPUT_FILE
            $bFile = True
        Case $LOG4A_OUTPUT_BOTH
            $bConsole = True
            $bFile = True
    EndSwitch

    If $eLevel >= $LOG4A_LEVEL_ERROR Then
        If $__LOG4A_WRITE_ERRSTREAM Then
            ConsoleWriteError($sLine)
        ElseIf $bConsole Then
            ConsoleWrite($sLine)
        EndIf
    ElseIf $bConsole Then
        ConsoleWrite($sLine)
    EndIf

    If $bFile Then
        FileWrite($__LOG4A_LOG_FILE, $sLine)
    EndIf
EndFunc   ;==>_log4a_Message


#endregion Message Functions

#region Internal Functions

Func __log4a_FormatMessage($sMessage, $sLevel)
    Local $sFormatted = $__LOG4A_FORMAT

    $sFormatted = StringReplace($sFormatted, "${date}", _
        StringFormat("%02d\\%02d\\%04d %02d:%02d:%02d", @MON, @MDAY, @YEAR, @HOUR, @MIN, @SEC))
    $sFormatted = StringReplace($sFormatted, "${host}", @ComputerName)
    $sFormatted = StringReplace($sFormatted, "${level}", $sLevel)
    $sFormatted = StringReplace($sFormatted, "${message}", $sMessage)
    $sFormatted = StringReplace($sFormatted, "${newline}", @CRLF)
    $sFormatted = StringReplace($sFormatted, "${shortdate}", _
        StringFormat("%02d\\%02d\\%04d", @MON, @MDAY, @YEAR))
    $sFormatted &= @CRLF

    Return $sFormatted
EndFunc   ;==>__log4a_FormatMessage

It would appear that on a typical call, the whole error string gets passed by value at least 3(!) times, maybe a lot more depending how StringReplace is actually implemented (since it is an internal function i have no way of knowing).

Worse than that though, the FileWrite doesn’t use file handles, definitely much slower when writing a lot of info.

I’m not knocking the package, it’s well written, but for my needs it’s not great.

[Earthshine]

it's really quick for everything i need it for. searching for controls and whatnot. but whatever floats your boat. i was using it on my UIAutomation stuff to see what's happening and it's aces for me. and when i make custom little software package patches.

Edited November 15, 2020 by Earthshine

[JockoDundee]

  On 11/15/2020 at 2:47 AM, Earthshine said:

it's really quick for everything i need it for. searching for controls and whatnot. but whatever floats your boat. i was using it on my UIAutomation stuff to see what's happening and it's aces for me. and when i make custom little software package patches.

Expand  

No doubt it’s great for you.  Not knocking it.

The app I’m working on now interfaces a chess engine to a GUI to play one-minute chess.

If I run with too much logging on it slows the engine to the point where it makes different moves, which makes it hard to replicate errors, so I really have to be bare bones.

 

Edited November 15, 2020 by JockoDundee