Sign in to follow this  
Followers 0

JSON UDF Library (fully RFC4627-compliant)

17 posts in this topic

#1 ·  Posted (edited)

I started this JSON library back in late 2007, because I needed a way to pass structured data between Javascript HTAs and AutoIt3 scripts. I've recently worked to get it in good enough shape to share with the AutoIt community, and while it still needs a bit more polish, the core functionality should be solid.

Three files are included in the attached archive:

  • JSON.au3 – the UDF Library itself. It includes a (very large) comment section at the top, which should provide sufficient documentation on how the various data types are encoded and decoded, as well as the various Javascript-based decoding extensions allowed (single-quoted strings, comments, etc.).
  • JSON_Translate.au3 provides example functions for the powerful "translator" functionality. Translator functions can be used for both encoding and decoding. When encoding, it allows you to recursively convert your data from its normal format into the required data structures used by the JSON UDF library. When decoding, it allows you to convert the data in the opposite direction – from the JSON UDF library's data structures to the data structures your program normally uses. To use this functionality, you'll need to write your own translator functions, but this allows you to meet the custom needs of your own programs without having to hack JSON.au3. The examples provided in this file can probably be improved, but show one way of translating AutoIt-specific data types (such as binary and hwnd) back and forth.
  • testJSON.au3 provides some quick examples of regular and translator-enhanced usage, which also help verify that encoding and decoding are working properly. These tests may not be comprehensive.

The area needing the most work is error-handling. JSON data structures can be deeply nested, and it's tricky to make sure errors "bubble up" properly in AutoIt. (Plus, I need a flowchart to keep track of all my setError() numbers!)

Thanks to Douglas Crockford of JSON.org, whose 2005 Javascript code for JSON encoding and decoding provided an excellent starting point for this library.

Edit: updated to 0.9.1 on 2009-10-19

  • substantially updated opening comments
  • encoding: implemented logic to return "warning" @error flags for unsupported variable types & JSON objects with non-unique key strings
  • other minor fixes & tweaks

JSON.7z

Edited by Gabriel13
3 people like this

[url='http://www.autoitscript.com/forum/index.php?showtopic=104150']JSON UDF Library (fully RFC4627-compliant)[/url][url='http://www.autoitscript.com/forum/index.php?showtopic=104325']Keep Windows Clear of Top Taskbar (for those who like their taskbar on top)[/url]

Share this post


Link to post
Share on other sites



#2 ·  Posted

Thank You for sharing. Just what I was looking for.

;-)

/t

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

You're welcome! I'd appreciate any feedback you have on it.

In particular, the use of 2D arrays to represent JSON objects, how well does that work for everyone? I believe they should be compatible with the 2D array handling routines from the standard Array.au3 UDF library.

I'll be working on adding comments for each function next – hopefully, that'll help me get my error codes organized.

Slightly edited for clarity.

Edited by Gabriel13

[url='http://www.autoitscript.com/forum/index.php?showtopic=104150']JSON UDF Library (fully RFC4627-compliant)[/url][url='http://www.autoitscript.com/forum/index.php?showtopic=104325']Keep Windows Clear of Top Taskbar (for those who like their taskbar on top)[/url]

Share this post


Link to post
Share on other sites

#4 ·  Posted

You're welcome! I'd appreciate any feedback you have on it.

In particular, the use of 2D arrays to represent JSON objects, how well does that work for everyone? I believe they should be compatible with the 2D array handling routines from the standard Array.au3 UDF library.

I'll be working on adding comments for each function next – hopefully, that'll help me get my error codes organized.

Slightly edited for clarity.

Thanks, a project I'm working spits out JSON formatted files, hopefully I'll be able to use your UDf to make working with these files easier. I like easier!

I tip my hat to you. ;)

Share this post


Link to post
Share on other sites

#5 ·  Posted

Great job :D

Share this post


Link to post
Share on other sites

#7 ·  Posted

is it possible to make this http://code.google.com/p/as3corelib/source/browse/trunk/src/com/adobe/serialization/json/?r=83 with outoit?

specially the JSONDecoder.as

thanks and best regards

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

It seems like this is the only good JSON AutoIt UDF I can find, but how do I cycle through the values?

Edited by 1RV34

MsgBox(0x40040, "", "Hello Forum!")

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

This version fixes problems with undeclared variables and variables not being used amongst other issues encountered e.g. encoding of the au3 files.

Note: I haven't tested, just fixed the errors.

JSON.zip

Edited by guinness

UDF List:

 

_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

 

 

Updated: 02/04/2015

Share this post


Link to post
Share on other sites

#10 ·  Posted

Thanks a bunch

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

Thanks for this great work i have been looking for a proper JSON library for 2 days before i found this one!

To show my thanks heres a little helper function addition, its purpose is to enable to query the json object in an easier way like "key1.key2", i hope this helps someone =)

#include <Array.au3>
#include "JSON.au3"
#include "JSON_Translate.au3" ; examples of translator functions, includes JSON_pack and JSON_unpack
Func _JSONGet($json, $path, $seperator = ".")
Local $seperatorPos,$current,$next,$l

$seperatorPos = StringInStr($path, $seperator)
If $seperatorPos > 0 Then
$current = StringLeft($path, $seperatorPos - 1)
$next = StringTrimLeft($path, $seperatorPos + StringLen($seperator) - 1)
Else
$current = $path
$next = ""
EndIf

If _JSONIsObject($json) Then
$l = UBound($json, 1)
For $i = 0 To $l - 1
If $json[$i][0] == $current Then
If $next == "" Then
return $json[$i][1]
Else
return _JSONGet($json[$i][1], $next, $seperator)
EndIf
EndIf
Next
ElseIf IsArray($json) And UBound($json, 0) == 1 And UBound($json, 1) > $current Then
If $next == "" Then
return $json[$current]
Else
return _JSONGet($json[$current], $next, $seperator)
EndIf
EndIf

return $_JSONNull
EndFunc

;create an json object to test
Local $json = _JSONDecode('{"test":{"x":[11,22,{"y":55}]}}')

;query this object
Local $result = _JSONGet($json, "test.x.2")
;query the previous result
$result = _JSONGet($result, "y")
;output the result
ConsoleWrite(_JSONIsNull($result) & @CRLF)
ConsoleWrite($result & @CRLF & @CRLF)
_ArrayDisplay($result)

;query this object using another seperator
$result = _JSONGet($json, "test->x->2->y", "->")
;output the result
ConsoleWrite(_JSONIsNull($result) & @CRLF)
ConsoleWrite($result & @CRLF & @CRLF)

;query this object
$result = _JSONGet($json, "test.x.2")
;output the result
ConsoleWrite(_JSONIsNull($result) & @CRLF & @CRLF)
_ArrayDisplay($result)

;query this object
$result = _JSONGet($json, "test.non.existent.key")
;output the result
ConsoleWrite(_JSONIsNull($result) & @CRLF)
ConsoleWrite($result & @CRLF & @CRLF)
Edited by nobody0

Share this post


Link to post
Share on other sites

#12 ·  Posted

great job! i have used in my project,thank you for sharing it! :ILA2:


Thanksgiving...

Share this post


Link to post
Share on other sites

#13 ·  Posted

I agree, and I recently used this in a project as well. nobody0's helper function was also a lifesaver and should be added to the main .au3 file.

One question:

I noticed that @error was 2 after a call to _JSONDecode(). This seems to be coming from the final line of __JSONSkipWhitespace(). Can someone explain the reasoning behind @error being set in this routine?

Thx!

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

Thanks for the effort, Gabriel!

I have found that the JSON UDF appears to choke on the following JSON returned from Google Books API, whereas other JSON parsers don't have issues.

Specifically, the JSON array for the "items" key is not parsed.

{
"kind": "books#volumes",
"totalItems": 1,
"items": [
{
"kind": "books#volume",
"id": "1sCDyGWDgQsC",
"etag": "ND3n2hIHVlg",
"selfLink": "https://www.googleapis.com/books/v1/volumes/1sCDyGWDgQsC",
"volumeInfo": {
"title": "Excel 2007 Advanced Report Development",
"authors": [
     "Timothy Zapawa"
],
"publisher": "Wiley",
"publishedDate": "2007-03-26",
"description": "Create powerful, innovative reporting solutions with Excel 2007! With this new book, which is significantly updated from the bestselling Excel 2003 version, author Timothy Zapawa provides you with in-depth coverage of Excel 2007’s enhanced reporting capabilities. Discover what you can do with PivotTable and Spreadsheet reports, enhanced conditional formatting and filtering functions, improved data access and management tools, and expanded OLAP features. With notes, tips, warnings, real-world examples, and how-to videos in every chapter, you'll soon be an expert on Excel report development!",
"industryIdentifiers": [
     {
     "type": "ISBN_10",
     "identifier": "0470046449"
     },
     {
     "type": "ISBN_13",
     "identifier": "9780470046449"
     }
],
"pageCount": 576,
"printType": "BOOK",
"categories": [
     "Computers"
],
"averageRating": 4.0,
"ratingsCount": 1,
"contentVersion": "preview-1.0.0",
"imageLinks": {
     "smallThumbnail": "http://bks4.books.google.com/books?id=1sCDyGWDgQsC&printsec=frontcover&img=1&zoom=5&source=gbs_api",
     "thumbnail": "http://bks4.books.google.com/books?id=1sCDyGWDgQsC&printsec=frontcover&img=1&zoom=1&source=gbs_api"
},
"language": "en",
"previewLink": "http://books.google.com/books?id=1sCDyGWDgQsC&dq=isbn:9780470046449&hl=&cd=1&source=gbs_api",
"infoLink": "http://books.google.com/books?id=1sCDyGWDgQsC&dq=isbn:9780470046449&hl=&source=gbs_api",
"canonicalVolumeLink": "http://books.google.com/books/about/Excel_2007_Advanced_Report_Development.html?hl=&id=1sCDyGWDgQsC"
},
"saleInfo": {
"country": "US",
"saleability": "NOT_FOR_SALE",
"isEbook": false
},
"accessInfo": {
"country": "US",
"viewability": "NO_PAGES",
"embeddable": false,
"publicDomain": false,
"textToSpeechPermission": "ALLOWED_FOR_ACCESSIBILITY",
"epub": {
     "isAvailable": false
},
"pdf": {
     "isAvailable": false
},
"webReaderLink": "http://books.google.com/books/reader?id=1sCDyGWDgQsC&hl=&printsec=frontcover&output=reader&source=gbs_api",
"accessViewStatus": "NONE"
},
"searchInfo": {
"textSnippet": "In this book, Timothy Zapawa has provided you with in-depth coverage of Excel 2007&#39;s enhanced reporting capabilities."
}
}
]
}
Edited by MrTWist

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

To show my thanks heres a little helper function addition, its purpose is to enable to query the json object in an easier way like "key1.key2", i hope this helps someone =)

Thanks nobody0 - this helped me a lot. Here is a complementary _JSONSet() function I wrote to also set JSON entries in the dotsyntax.

#include
#include "JSON.au3"
#include "JSON_Translate.au3" ; examples of translator functions, includes JSON_pack and JSON_unpack

;Expands upon _JSONGet from nobody0
;http://www.autoitscript.com/forum/topic/104150-json-udf-library-fully-rfc4627-compliant/#entry1030327
Func _JSONSet($writeValue, ByRef $json, $path, $seperator = ".")
    Local $seperatorPos, $current, $next, $l
    $seperatorPos = StringInStr($path, $seperator)

    If $seperatorPos > 0 Then
        $current = StringLeft($path, $seperatorPos - 1)
        $next = StringTrimLeft($path, $seperatorPos + StringLen($seperator) - 1)
    Else
        $current = $path
        $next = ""
    EndIf

    If _JSONIsObject($json) Then
        $l = UBound($json, 1)
        Local $matchFound = False
        For $i = 0 To $l - 1
            If $json[$i][0] == $current Then
                $matchFound = True
                If $next == "" Then
                    $json[$i][1] = $writeValue
                    Return
                Else
                    _JSONSet($writeValue, $json[$i][1], $next, $seperator)
                    Return
                EndIf
            EndIf
        Next
        If Not $matchFound Then
            ReDim $json[UBound($json)+1][2]
            If $next == "" Then
                $json[UBound($json)-1][0] = $current
                $json[UBound($json)-1][1] = $writeValue
            Else
                $json[UBound($json)-1][0] = $current
                Local $newjsonobject[1][2]
                $newjsonobject[0][0] =""
                $newjsonobject[0][1] ='JSONObject'
                $json[UBound($json)-1][1] = $newjsonobject
                _JSONSet($writeValue, $json[UBound($json)-1][1], $next, $seperator)
                Return
            EndIf
        EndIf
    ElseIf IsArray($json) And UBound($json, 0) == 1 Then
        If UBound($json, 1) > $current Then
            If $next == "" Then
                $json[$current]
                Return
            Else
                _JSONSet($writeValue, $json[$current], $next, $seperator)
                Return
            EndIf
        Else
            ReDim $json[$current+1]
            Local $newjsonobject[1][2]
            $newjsonobject[0][0] =""
            $newjsonobject[0][1] ='JSONObject'
            $json[$current] = $newjsonobject
            If $next == "" Then
                $json[$current]
                Return
            Else
                _JSONSet($writeValue, $json[$current], $next, $seperator)
                Return
            EndIf
        EndIf
    Else
        return $_JSONNull
    EndIf
EndFunc
Edited by robertcollier4

Share this post


Link to post
Share on other sites

#16 ·  Posted

Thanks for sharing this! I will be using this to build a simple jquery grid to display on a website. This UDF will constantly run in a while loop with sleep timers comparing flat file modified date with current systems date and query a SQL server to build the Jason files if the date difference between file date and system date is 1

Share this post


Link to post
Share on other sites

#17 ·  Posted

Quick question guys. With the functions given by nobody0 and robertcollier4 . If I want to loop through a JSON file, will I need to build a loop outside of the function examples provided? In my file I have roughly 1000 sections of data. Within each section is roughly 50 items.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0