Jump to content
jchd

VarDump with JSON support

Recommended Posts

Here's a new version of my Dump.au3 file (dirty). I've recently added support to dissect (dump) as a treeview-like view any string in the variable conforming to a strict JSON grammar, whereever said string is: as a standalone string variable or in an array or a map. AFAICT the only relaxed constraint not implemented here is that numbers in parsed JSON may have leading zeroes, although leading zeroes are forbidden in pure JSON specifications. If required it wouldn't be hard to add this constraint.

@c.haslam (among others) may find it useful. Use freely.

Dump.au3

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Scite output:

Method 1 : 0, 0,
Method 2 : 8, 0, [0][0]
Method 4 : 16, 0, [0][0]
Method 8 : 32, 0, [0][0]
Method 15 : 56, 0, [0][0]
"D:\Scripts\JSON\Dump.au3" (491) : ==> Variable subscript badly formatted.:
Local $m[], $n[]
Local $m[^ ERROR

Share this post


Link to post
Share on other sites

As shown in the # headers, you need to run under the beta to benefit of support for Map datatype.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By UEZ
      Hi,
      here a little tool to create ISO files from default ISO 9660 (2048 bytes/sector) CD or DVD format (no audio cd and BD support yet!)

      Source is too huge for code box -> Look here to have a look to the source code
      Additional credits to:
      Ward for MD5 checksum / MemoryDLL routines
      Harald Vistnes for cd2iso used in v1 and v2
      Yashied for WinAPIEx.au3
      wolf9228 to play wave from memory
      AutoItObject Team
      Download (purely written with AutoIt): ISO Creator v1.16 build 2015-07-13 beta.7z (1110 downloads previously)
      Thanks to smashly for pointing me to right direction
      The development of v1 and v2 is discontinued!
      Download v1: ISO Creator v1.0.0 build 2011-08-03 beta v1.7z (195 downloads previously)
      Download v2 (everything is called directly from memory): ISO Creator v1.0.0 build 2011-08-03 beta v2.7z (117 downloads previously)
      Thanks to smartee for the DLL version (experimental) of cd2iso!
      v1 is using cd2iso.exe to create the ISO
      v2 is using cd2iso.dll which was created by smartee.
      For compiled v1, v2 and pure AutoIt versions only (x86) visit (ISO Creator Exe only): 4shared.com or MediaFire
      You can call ISO Creator.exe also with command line parameters: ISO Creator.exe -s [source cd/dvd drive] -d [filename] (-md5) (-aem) (-exit)
      -s and -d are mendatory if called from command line!
      Tested on Win7 x64.
      If you find any bug please report here!
      Many thanks to smartee and smashly for their efforts on this project!
      Br,
      UEZ
      Change Log:
       
       
    • By Ward
      Introduction
      JSON (Javascript Object Notation) is a popular data-interchange format and supported by a lot of script languages. On AutoIt, there is already a >JSON UDF written by Gabriel Boehme. It is good but too slow, and not supports unicode and control characters very well. So I write a new one (and of course, fast one as usual).

      I use a machine code version of JSON parser called "jsmn". jsmn not only supports standard JSON, but also accepts some non-strict JSON string. See below for example.
       
      Important Update!!
      I rename the library from jsmn.au3 to json.au3. All function names are changed, too.
       
      Decoding Function
      Json_Decode($Json) $Json can be a standard or non-standard JSON string. For example, it accepts:
      { server: example.com port: 80 message: "this looks like a config file" } The most JSON data type will be decoded into corresponding AutoIt variable, including 1D array, string, number, true, false, and null. JSON object will be decoded into "Windows Scripting Dictionary Object" retuned from ObjCreate("Scripting.Dictionary"). AutoIt build-in functions like IsArray, IsBool, etc. can be used to check the returned data type. But for Object and Null, Json_IsObject() and Json_IsNull() should be used.

      If the input JSON string is invalid, @Error will be set to $JSMN_ERROR_INVAL. And if the input JSON string is not finish (maybe read from stream?), @Error will be set to $JSMN_ERROR_PART.

      Encoding Function
      Json_Encode($Data, $Option = 0, $Indent = "\t", $ArraySep = ",\r\n", $ObjectSep = ",\r\n", $ColonSep = ": ") $Data can be a string, number, bool, keyword(default or null), 1D arrry, or "Scripting.Dictionary" COM object. Ptr will be converted to number, Binary will be converted to string in UTF8 encoding. Other unsupported types like 2D array, dllstruct or object will be encoded into null.

      $Option is bitmask consisting following constant:
      $JSON_UNESCAPED_ASCII ; Don't escape ascii charcters between chr(1) ~ chr(0x1f) $JSON_UNESCAPED_UNICODE ; Encode multibyte Unicode characters literally $JSON_UNESCAPED_SLASHES ; Don't escape / $JSON_HEX_TAG ; All < and > are converted to \u003C and \u003E $JSON_HEX_AMP ; All &amp;amp;amp;amp;s are converted to \u0026 $JSON_HEX_APOS ; All ' are converted to \u0027 $JSON_HEX_QUOT ; All " are converted to \u0022 $JSON_PRETTY_PRINT ; Use whitespace in returned data to format it $JSON_STRICT_PRINT ; Make sure returned JSON string is RFC4627 compliant $JSON_UNQUOTED_STRING ; Output unquoted string if possible (conflicting with $JSMN_STRICT_PRINT) Most encoding option have the same means like PHP's json_enocde() function. When $JSON_PRETTY_PRINT is set, output format can be change by other 4 parameters ($Indent, $ArraySep, $ObjectSep, and $ColonSep). Because these 4 output format parameters will be checked inside Jsmn_Encode() function, returned string will be always accepted by Jsmn_Decode(). $JSON_UNQUOTED_STRING can be used to output unquoted string that also accetped by Jsmn_Decode().

      $JSON_STRICT_PRINT is used to check output format setting and avoid non-standard JSON output. So this option is conflicting with $JSON_UNQUOTED_STRING.

      Get and Put Functions
      Json_Put(ByRef $Var, $Notation, $Data, $CheckExists = False) Json_Get(ByRef $Var, $Notation) These functions helps user to access object or array more easily. Both dot notation and square bracket notation can be supported. Json_Put() by default will create non-exists objects and arrays. For example:
      Local $Obj Json_Put($Obj, ".foo", "foo") Json_Put($Obj, ".bar[0]", "bar") Json_Put($Obj, ".test[1].foo.bar[2].foo.bar", "Test") Local $Test = Json_Get($Obj, '["test"][1]["foo"]["bar"][2]["foo"]["bar"]') ; "Test" Object Help Functions
      Json_ObjCreate() Json_ObjPut(ByRef $Object, $Key, $Value) Json_ObjGet(ByRef $Object, $Key) Json_ObjDelete(ByRef $Object, $Key) Json_ObjExists(ByRef $Object, $Key) Json_ObjGetCount(ByRef $Object) Json_ObjGetKeys(ByRef $Object) Json_ObjClear(ByRef $Object) These functions are just warps of "Scripting.Dictionary" COM object. You can use these functions if you are not already familiar with it.
       
      == Update 2013/05/19 ==
       
      * Add Jsmn_Encode() option "$JSMN_UNESCAPED_ASCII". Now the default output of Json_Encode() is exactly the same as PHP's json_encode() function (for example, chr(1) will be encoded into u0001).
      $JSON_UNESCAPED_ASCII ; Don't escape ascii charcters between chr(1) ~ chr(0x1f)  
      == Update 2015/01/08 ==
      * Rename the library from jsmn.au3 to json.au3. All function names are changed, too.
      * Add Json_Put() and Json_Get()
      * Add Null support
      * Using BinaryCall.au3 to loading the machine code.
       
      == Update 2018/01/13==   (Jos)
      * Add JsonDump() to list all Json Keys and their values to easily figure out what they are.
      == Update 2018/10/01==  (Jos)
      * Fixed JsonDump() some fields and values were not showing as discussed here  - tnx @TheXman .
      == Update 2018/10/01b==   (Jos)
      * Added Json_ObjGetItems, Tidied source and fixed au3check warnings - tnx @TheXman .
      == Update 2018/10/28==   (Jos)
      * Added declaration for $value to avoid au3check warning - tnx @DerPensionist
      == Update 2018/12/16==   (Jos)
      * Added another declaration for $value to avoid au3check warning  and updated the version at the top - tnx @maniootek
      == Update 2018/12/29==   (Jos)
      * Changed Json_ObjGet() and Json_ObjExists() to allow for multilevel object in string.
      == Update 2019/01/17==   (Jos)
      * Added support for DOT notation in JSON functions.
      _Json(2019.01.17).zip   reverted to this version as the last version has some regression issues that is being worked on.
      == Update 2019/07/15==   (Jos)
      *  Added support for reading keys with a dot inside when using a dot as separator (updated)
       
    • By uncommon
      So I have been reading on how to use OAuth2 Service accounts(https://developers.google.com/identity/protocols/OAuth2ServiceAccount) and ran into an issue with autoit I can not solve. To keep this short to communicate to the Google API I need to use a JSON Web signature containing
      {Base64url encoded header}.{Base64url encoded claim set}.

      The Header and claim set work fine but I do not know how to convert them to a byte array using autoit. I have looked around on the forums and website but have not found anything that seems to work. Here is my code.
      #include-once #include <Array.au3> #include <Constants.au3> #include <Debug.au3> #include <File.au3> #include "Json.au3" #include <Process.au3> #include <ProgressConstants.au3> #include <String.au3> #include <WindowsConstants.au3> #include "WinHttp.au3" #include <UnixTime.au3> #include <StringConstants.au3> Global $UnixTime = _TimeGetStamp() $sJWTheader = '{"alg":"RS256","typ":"JWT"}' $sJWTclaimset = '{"iss":"Removed@forprivacy.com","scope":"https://www.googleapis.com/auth/spreadsheets","aud":"https://www.googleapis.com/oauth2/v4/token","exp":' & $UnixTime + 3600 & ',"iat":' & $UnixTime & '}' Local $taiData = DllStructCreate("BYTE[256]") Local $sText = _base64($sJWTheader)&'.'&_base64($sJWTclaimset) DllStructSetData($taiData, 1, StringToBinary($sText, 4)) Local $bvResult = DllStructGetData($taiData, 1) $sJWTSigature = $bvResult $sJWT = _base64($sJWTheader)&'.'&_base64($sJWTclaimset)&'.'&_base64($sJWTSigature) $POSTHeader = "Content-Type: application/x-www-form-urlencoded" $hOpen = _WinHttpOpen() $hConnect = _WinHttpConnect($hOpen, "https://www.googleapis.com/") $sRead = _WinHttpSimpleSSLRequest($hConnect, "POST", "oauth2/v4/token", Default, "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=" & $sJWT) _WinHttpCloseHandle($hConnect) _WinHttpCloseHandle($hOpen) ConsoleWrite($sRead & @LF) The whole bytes thing is beyond with what I know about auotit. I can't tell what I need to fix.
       
       
    • By Miliardsto
      You can see my script to POST json and the problem where occurs backslashes before quotations. Second problem is the apperance , there is no new lines for every new value.
      LOOK in attached images!
       
      You can see rest api on the screen and with id = 0 its made my postman and the rest by autoit (with backslashes)
       
      Func jsonSave() ; Make login request to server JSON POST Local $data, $request Local $obj Local $adress = 'http://35.195.249.40:3004/msg' Json_Put($Obj, ".msgName", "Point HPs") Json_Put($Obj, ".msg", "Not pointed correctly") Json_Put($Obj, ".username", "karolek890") Local $Json = Json_Encode($Obj,$JSON_UNESCAPED_SLASHES) ; WITHOUT preetier becouse it makes clashes r n and t only works in autoit $request = _httpRequestBot($adress, 'POST', $Json) ; only get login if correct then success If @error Then Exit MsgBox(0, "Error", "Open method returned @error = " & @error & " and @extended = " & @extended) Else ConsoleWrite("Data to json server posted!") EndIf EndFunc  
      Im using this JSON UDF
       
       
       


    • By TheXman
      This UDF brings the power and flexibility of jq to AutoIt scripts.  jq is an open-source, powerful, and flexible command-line based JSON processor.  As it says on their website, jq is like 'sed' for JSON.  jq can be used for the simplest of tasks like retrieving JSON objects and values, to very advanced JSON processing using its numerous built-in functions and conditional processing.  Its built-in functions can handle math, selection, conditional processing, mapping, object and array manipulation, flattening, reduction, grouping, and much more.  You can even create your own jq functions.  You can learn more about jq and even play with it in real-time, using jq's online jq playground, all on their website.
      Here and some helpful links to get you more familiar with jq, what can be done with it, its built-in functions, and its syntax.
      jq Website: https://stedolan.github.io/jq/ jq Manual: https://stedolan.github.io/jq/manual/ jqWiki (FAQ, Cookbook, Advanced Topics) https://stedolan.github.io/jq/manual/ jq Online Testing Playground: https://jqplay.org/ jq is a single 32 or 64 bit executable that has no other dependencies.  Just like using the SQLite UDF, the only requirement to use this UDF is that the jq executable reside in a location in which the UDF can execute it.  The latest win32 & win64 versions have been included in the UDF download.  You can always get newer versions from the jq website.
      I don't consider this a replacement for some of the other JSON UDFs like the one based on JSMN.  If speed and simple JSON parsing are you primary goals, then other UDFs may be a better choice.  However, if you like having the power to do just about anything related to JSON processing/manipulation, using a single UDF, then jq may be worth checking out.  It can be used by novices and experts alike.  Below, is a brief explanation of how jq works, how to use the jq UDF, and a few examples of how to do some simple tasks.  If you want to learn more about jq and what it can do, I would highly suggest checking out the jq website and some of the other resources listed above.
      jq at a high level
      Like 'sed', jq reads JSON in, either through STDIN or one or more files, processes it thru one or more "filters", and outputs the results.  You can, optionally, supply "options" that affect how it reads the input, where it gets its "filters", and how it writes its output.  It looks a little like this:
      JSON ---> jq processor (using supplied filters and options) ---> Output
      So in jq lingo, you basically use "Filters" to tell jq what you want it to do.  So in the UDF file, that is why the main functions ( _jqExec() and _jqExecFile() ) refer to filters and options.  Please make note that jq works with relatively strict JSON.  This means that all JSON read must be conform to the standard.  Luckily, jq is pretty good at identifying where a format error exists in non standard JSON.
      The jq UDF
      There are 2 main funtions in the UDF file, _jqExec and jqExecFile.  With these 2 functions, you can pretty much do anything that jq can do.  The only difference between to two functions is whether the JSON is supplied by a string or a file.  The 2 primary functions simply call the jq executable with the supplied information, after properly formatting the parameters.  There are additional functions in the UDF to easily pretty-print your json, compact-print your json, dump the json data with its associated paths, and see if specific JSON keys exist, but they all just execute the _jqExec or _jqExecFile function with the proper filter.  There are also a couple of extra functions to display what version of the UDF and jq executable you are currently using.  There are also a couple of functions to enable and disable logging of jq information for debugging purposes.  Most of the jq UDF file functions return an @error if unsuccessful.  Some also include @extended info.  Please see the actual function headers for more information on their usage and return values.
      The 2 primary functions below just format your jq request and pass it on the jq executable.  The functions will also properly escape double quotes (") that are used in the filter.  For most simple tasks, you just need to supply the JSON source and a filter.
      _jqExec($sJson, $sFilter, $sOptions = Default, $sWorkingDir = Default) Or _jqExecFile($sJsonFile, $sFilter, $sOptions = Default, $sWorkingDir = Default) Using jq in your script
      As stated earlier, the jq executable must reside somewhere where the script can locate and execute it.  The _jqInit() function always has to be executed before any jq processing occurs.  _jqInit() merely locates the executable or uses the supplied path.  It also clears any previous debug log.  The jq UDF folder contains a jq example script that has several examples to how to do some of the most common JSON processing tasks.  Here are a few examples to get you started:

      How to pretty-print some JSON
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"fruits":[{"Apple":{"color":"Red","season":"Fall"}}, {"Banana":{"color":"Yellow","season":"Summer"}}]}' $sCmdOutput = _jqPrettyPrintJson($sJson) ConsoleWrite(@CRLF & "Pretty-Print JSON" & @CRLF & $sCmdOutput & @CRLF) How to compact-print some JSON
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{ "fruits" : [{"Apple" : {"color":"Red","season":"Fall"}}, {"Banana":{"color":"Yellow","season":"Summer"}}]}' $sCmdOutput = _jqCompactPrintJson($sJson) ConsoleWrite(@CRLF & "Compact-Print JSON" & @CRLF & $sCmdOutput & @CRLF) Dump JSON data (paths and values)
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{ "fruits" : [{"Apple" : {"color":"Red","season":"Fall"}}, {"Banana":{"color":"Yellow","season":"Summer"}}]}' $sCmdOutput = _jqDump($sJson) ConsoleWrite(@CRLF & "Dump JSON paths and values" & @CRLF & $sCmdOutput & @CRLF) How to GET JSON values
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple" : {"color":"Red","season":"Fall"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = '.Banana.color' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Get color of banana" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) or
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple" : {"color":"Red","season":"Fall"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = 'getpath(["Banana", "color"])' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Get color of banana" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF)  
      Check for the existence of a key
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple":{"color":"Red","season":"Fall"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = '.Banana | has("color")' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Check for existence of color key within Banana object" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) Count of how many Items in an object
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple":{"color":"Red"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = '.Banana | length' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("How many items in the Banana object" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) How to PUT/Create/Modify JSON
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sInput = "" $sFilter = 'setpath(["Apple","color"];"Red") | setpath(["Banana","color"];"Yellow") | setpath(["Banana","season"];"Summer")' $sOptions = '-n' ;required if no input supplied $sCmdOutput = _jqExec($sInput, $sFilter, $sOptions) ConsoleWrite("Update/Create JSON" & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & @CRLF & $sCmdOutput & @CRLF) List all of the fruits (top-level keys)
      #include "jq.au3" _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple":{"color":"Red"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = 'keys | .[]' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("List all top-level keys (fruits)" & @CRLF) ConsoleWrite("Input : " & $sJson & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & @CRLF & $sCmdOutput & @CRLF) The examples above, and the ones in the example file, merely scratch the surface of what jq can do.  It may look intimidating at first but it really isn't that bad once you start playing with it.
      If you have any suggestions on how to make the UDF better, or find a bug, please let me know.  I only started using it about a week ago.  Nevertheless, if you have any questions regarding the UDF, or how to perform a certain task using jq, I'll try my best to answer them.  Since jq has been around for a while now, there's also several jq-related questions and answers on StackOverflow.
      If you work with JSON, I hope you find this UDF useful.
       
      Download:
      jq_udf_v1.1.1.zip
×
×
  • Create New...