JSON UDF in pure AutoIt

[Yaerox]

2 minutes ago, Musashi said:

As far as I know, this UDF works with datatype Maps and therefore requires AutoIt version 3.3.16.1 or higher.

Thank you Sir!

[TheDcoder]

Or you can also use an older Beta version if needed.

[seangriffin]

Any reason for having $s_String as "ByRef" in _JSON_Parse ?

I have a use case for minimal script, and wish to embed the JSON string directly in the _JSON_Parse such as

$search = _JSON_Parse('{"name":"Sean"}')
$s_Type = $search.name

I took the "ByRef" keyword out of _JSON_Parse and seems to work?

[AspirinJunkie]

That had performance reasons. With a classic ByRef, large strings have to be duplicated when they are transferred. As the function also works recursively, this would take place several times and would have a corresponding impact on performance.

Important: “Would have”. In fact, AutoIt has quite obviously built in a kind of COW mechanism. This means that a copy is not created automatically, but only when the content is actually changed in the function. However, I do not know this with certainty but only deduce it from the performance behavior.

The important point is: ByRef has no (more?) influence on the performance of the _JSON_Parse() function.

So the recommendation would actually be to remove ByRef to enable direct string inputs.
Which I have just done myself and adapted the repository accordingly.

Edited April 1 by AspirinJunkie

[Nine]

Once you are confortable with your script, you can remove au3check (or compile, or run from Explorer) then you can use direct string even with ByRef.

#AutoIt3Wrapper_Run_Au3Check=n

Func Test(ByRef $sString)
  MsgBox(0, "Test", $sString)
EndFunc

Test("This is a test")

Meanwhile, you can fool au3check by having a wrapper of the original (included) function.  Since au3check is a single pass process, it won't detect if you use it like this :

Func Test(ByRef $sString)
  MsgBox(0, "Test", $sString)
EndFunc

TestEx("This is a test")

Func TestEx(ByRef $sString)
  Test($sString)
EndFunc

That way you still get the benefit of ByRef all the time...

Edited April 1 by Nine

[Mison]

Hi everyone. When running the example script, I got this error:
 

"C:\Users\xxx\Downloads\JSON.au3" (80) : ==> Variable subscript badly formatted.:
Local $o_Current[]
Local $o_Current[^ ERROR

Just wanted to let you know and thank you for the UDF.

[argumentum]

6 minutes ago, Mison said:

When running the example script, I got this error

update your AutoIt version so that maps are implemented

[argumentum]

...just had a problem with the SciTE AI assistant post and is because your current version on Git, were __JSON_FormatString() is broken. Thanks

[AspirinJunkie]

Well, I read that there should be problems and that the function should be broken, but so far I have not been able to discover anything concrete about what it is actually about.

I couldn't find an example string where i could check whether it is correctly encoded in JSON. Therefore, it is not yet clear to me exactly what the problem should be.

When I look at the supposed fix, it differs from __JSON_FormatString() in the handling of @CRLF. The fact that it only turns @CRLF into \n was a deliberate design decision at the time. For good platform compatibility, \n has basically established itself as the standard separator. This is not a bad thing, as basically every reasonable JSON decoder converts the line break sequence suitable for the respective system. It also makes for more compact and readable JSON strings.

I would be somewhat surprised if an application explicitly insisted on \r\n for its JSON input. But yes - this approach of mine is of course open to discussion - there are of course also arguments against it. At least this choice was not made by accident.
On the other hand, that doesn't seem to be the problem either, since (as I interpret the statements) an earlier version obviously doesn't seem to have the problem. But even then, the behavior regarding @CRLF was the same.

I therefore suspect that it's more about the not so long ago introduced variant for small strings with the StringRegExpReplace(). Since I am very unhappy with this ugly cascade of StringReplace I have been looking for (more performant) alternatives and have found them at least for small strings by handling the replacements in a single StringRegExpReplace call (yes I know - the pattern comes directly from the RegEx hell...). It may well be that there are string constellations where this variant produces incorrect results. But: I have not yet seen a concrete example. Therefore, I need input data to see if there is a bug and if so, how it can be fixed.

On the subject of the version numbers in the file: Yes, I was too careless because I also have a pure git view rather than manual numbers

[argumentum]

it used to be ByRef but now works by return
$sPrompt = __JSON_FormatString($sPrompt)

If you can not replicate what looks obvious...

ok, about this:

Func __JSON_FormatString_maybe(ByRef $sString)
    $sString = StringLen($sString) < 50 ? _
    StringTrimRight(StringRegExpReplace($sString & '\\\b\f\n\r\t\"', '(?s)(?|\\(?=.*(\\\\))|[\b](?=.*(\\b))|\f(?=.*(\\f))|\r\n(?=.*(\\n))|\n(?=.*(\\n))|\r(?=.*(\\r))|\t(?=.*(\\t))|"(?=.*(\\")))', '\1'), 15) : _
    StringReplace( _
        StringReplace( _
            StringReplace( _
                StringReplace( _
                    StringReplace( _
                        StringReplace( _
                            StringReplace( _
                                StringReplace($sString, '\', '\\', 0, 1) _
                            , Chr(8), "\b", 0, 1) _
                        , Chr(12), "\f", 0, 1) _
                    , @CRLF, "\n", 0, 1) _
                , @LF, "\n", 0, 1) _
            , @CR, "\r", 0, 1) _
        , @TAB, "\t", 0, 1) _
    , '"', '\"', 0, 1)
    Return $sString ; to have it both ways
EndFunc

 

[AspirinJunkie]

It may be obvious to you, but it still isn't to me. You have now written the function again but still haven't explained which input values cause it to fail. I have fed several test strings to it and all came back as expected. That's why I'm asking again: For which specific data does the function not return the desired result?

About the byref/return story: In old versions, the function was designed to process the variable directly (a "procedure").
However, since you cannot include the function call in string definitions, this was later changed to a return the target value (a "function").
This has simplified the code that uses this function.
The way in which the function must be used has of course changed: Before you had to make a single call and got the variable changed, now you have to use the return value instead. The ByRef was left in (for performance reasons at the time) to avoid additional copies. We had already covered the fact that these considerations are not fulfilled in reality in the discussion on _JSON_Parse(). This function is therefore also a candidate where the ByRef can be removed if the opportunity arises. But until then, this is simply irrelevant for the functionality of the function and has no influence. ByRef and Return are not incompatible.

Edit: Ok, a little more thought: Could it be that it's not about the function being faulty, but that you just want a feature extension so that you can process variables directly and at the same time get them back as return values?

Well of course you can discuss that but this function is not part of the user interface (it begins with 2 underscores) but an internal helper function for a specific purpose.
So far it has fulfilled this purpose, so I have no compulsion to change it accordingly.
If someone absolutely needs a function that processes a variable accordingly in-place, then this is a three-liner for him.

But that would just be a feature request. In fact, the function was said to be “broken”. But I can't see that so far.

 

Edited May 13 by AspirinJunkie

[argumentum]

8 minutes ago, AspirinJunkie said:

You have now written the function again but still haven't explained which input values cause it to fail.

Not the input. In "SciTE AI assistant" code, he coded it as:

...
    If $Seed = "" Or $Seed = -1 Or $Seed = Default Then $Seed = 0
    __JSON_FormatString($sPrompt)
    Local $sRequest = '{'
...

with the expectation of ByRef.
The code I posted works with both so that the old expectation of ByRef will work, and as a function returning the value will work too.

The request is not for me particularly. Either you change your code or he does his, for his project/code to work as expected.
He is using an older version. You have a newer version. What I presented keeps it backwards compatible to not brake code.

Do as you see fit. To me is all good either way.

[AspirinJunkie]

Ok, so let's just say that the function is not “broken” after all, does what it is supposed to do and does it correctly without any known errors?

It has always been the responsibility of the application code to use their respective dependencies correctly. Especially as non-public interfaces are used here, which were never intended to be used by the user. The _JSON_Generate() function is intended to format a string in a JSON-compliant manner as a user. Its behavior has not changed over time.

A change would have no effect on the described functionality of the UDF, which is why there is no need for action from my point of view.

Edited May 13 by AspirinJunkie

[SOLVE-SMART]

Thank you both (@argumentum and @AspirinJunkie) for the clarification 👌 .
I hope you folks continue to have a nice week, best regards, Sven.

[WildByDesign]

I've taken some more time to learn about JSON format in the last day or so. But I do have a question.

How can I create an object in an array without overwriting what may already be there?

For example, I am trying to add a theme to settings.json file of Windows Terminal.

By default, the theme array is empty as follows:

"themes": []

I have figured out how to add exactly what I want to that json array with the following:

_JSON_addChangeDelete($vJSON, "profiles.defaults.opacity", 0)
_JSON_addChangeDelete($vJSON, "theme", "Default Theme Test")
_JSON_addChangeDelete($vJSON, "themes[0].name", "Default Theme Test")
_JSON_addChangeDelete($vJSON, "themes[0].tab.background", "#00000020")
_JSON_addChangeDelete($vJSON, "themes[0].tab.unfocusedBackground", "#00000000")
_JSON_addChangeDelete($vJSON, "themes[0].tabRow.background", "#00000000")
_JSON_addChangeDelete($vJSON, "themes[0].tabRow.unfocusedBackground", "#00000000")
_JSON_addChangeDelete($vJSON, "themes[0].window.applicationTheme", "dark")
_JSON_addChangeDelete($vJSON, "themes[0].window.useMica", true)

However, the code that I came up with assumes that I am adding to object 0 of the themes array. The problem is that it is very much possible that there could already be a bunch of different themes in that array when I add it.

So let's assume there is already themes[0] through themes[6] in a users' settings.json file.

How can I adjust my code so that my theme simply gets added to the end of that theme array without overwriting a theme that a user already has?

Thank you

[TheDcoder]

1 hour ago, WildByDesign said:

How can I adjust my code so that my theme simply gets added to the end of that theme array without overwriting a theme that a user already has?

You'll have to parse the array first and then manually insert the correct index into your commands. If you want to do it in a "cleaner" way, you can just modify the Array in AutoIt and insert the whole thing back with your changes.

[SOLVE-SMART]

Here a example how you can do it with jq directly instead of using the JSON.au3 UDF.

Spoiler

 The input file demo.json as example with testCases array.

{
    "id": "62570c9c8a",
    "isAuthRequest": false,
    "request": {
        "method": "GET",
        "url": "https://example.com",
        "endpoint": "/placeholder",
        "headers": []
    },
    "testCases": [
        {
            "name": "first try",
            "paramsData": "1",
            "postBodyData": ""
        }
    ]
}

The necessary jq command to append (+=) to the array.
(Run this in your terminal which has to have access to jq on your system.)

jq --indent 4 '
    .testCases += [
        {
            "name":        "second try",
            "paramsData":  "2",
            "postBodyData": ""
        }
    ]
' demo.json > result.json

The created result.json will look like this:

{
    "id": "62570c9c8a",
    "isAuthRequest": false,
    "request": {
        "method": "GET",
        "url": "https://example.com",
        "endpoint": "/placeholder",
        "headers": []
    },
    "testCases": [
        {
            "name": "first try",
            "paramsData": "1",
            "postBodyData": ""
        },
        {
            "name": "second try",
            "paramsData": "2",
            "postBodyData": ""
        }
    ]
}

 

Don't get me wrong, I like the JSON library (UDF) from @AspirinJunkie - he already knows that.
I simply try to give you another perspective how you can handle a JSON file with a widespread (AutoIt independent) library.
In case you want to stick with AutoIt, no problem, use the JQ adaption of @TheXman who creates this lovely json-processor.

Best regards
Sven

 

Edited 6 hours ago by SOLVE-SMART

[WildByDesign]

1 hour ago, TheDcoder said:

You'll have to parse the array first and then manually insert the correct index into your commands. If you want to do it in a "cleaner" way, you can just modify the Array in AutoIt and insert the whole thing back with your changes.

Thank you. I am a complete rookie with this UDF. I think that your "cleaner" way sounds like my best option. I do have the JSON loaded with the _JSON_Parse() function. Does that make the json array available in AutoIt already in some way or do I need to do something else to access it as an array?

1 hour ago, SOLVE-SMART said:

Here a example how you can do it with jq directly instead of using the JSON.au3 UDF.

Thank you for your suggestion and example as well, Sven. I really appreciate it. I was not familar with jq before but I can understand the power and beauty of it because I am familiar with sed, awk and grep.

I can see myself using jq for my own purposes. But not so much for this project because it will be a single binary on other users' systems and, in those scenarios, I try my best not to drop any additional binaries on disk whenever possible.

[argumentum]

3 hours ago, WildByDesign said:

I've taken some more time to learn about JSON format in the last day or so. But I do have a question.

..and a question unrelated to the thread should go in a help area instead of taking one over.

11 minutes ago, WildByDesign said:

Does that make the json array available in AutoIt already in some way or do I need to do something else to access it as an array?

Ditto.