The JSON Situation in Xojo

My open source app, Beacon, makes extensive use of JSON. It's what the document format is built from. So the topic of JSON is of great interest to me, especially when it comes to LOTS of JSON. Beacon files can get pretty big. I have one here that is 10MB, which is a lot for a block of text. So needless to say, I'm interested in speed.

I recently ran some tests to get a sense of how to squeeze the most performance from Xojo. For my case, I can't use third party plugins because this is an open source project and I want users to be able to just grab the code and go. I parsed that 10MB file on Mac and Windows, then generated the JSON back again in both compact and "pretty" format. And I tested the classic JSONItem, the new Xojo.Data methods, and Kem Tekinay's JSONItem_MTC class using Xojo 2018r3. Both platforms are running 64-bit code with optimization level set to aggressive. Mac hardware is a 3.1GHz 2017 MacBook Pro and Windows hardware is a 3.7GHz Ryzen 2700X.

The results are... not fantastic.

Parsing

On both platforms, Xojo.Data.ParseJSON is the clear winner in performance alone, but of course gives you back new framework objects. In the classic framework, news isn't great either. While JSONItem_MTC is faster on Mac, it's much slower on Windows in this case. For parsing, the classic JSONItem is probably the better choice. You'll lose speed on Mac parsing, but Windows users will be a little happier. Interestingly, the Xojo.Data.ParseJSON method is much slower on Windows, but still creams the classic framework options.

Compact Generation

There is a very clear winner here, and it's the new framework again. So why not just use the new framework? We'll get to that at the end. In the classic framework, JSONItem_MTC has a massive advantage over JSONItem. So remember when I said "classic JSONItem is probably the better choice?" To hell with that unless you're parsing only.

Pretty Generation

The new framework has no option to pretty-print JSON, so this was done with a post-processing routine. Obviously, it is very expensive.

In the classic framework side of things, pretty-printing doesn't cost much performance with JSONItem, but since it's so slow to begin with, that's not saying much. JSONItem_MTC is the clear winner overall though. While it's slightly slower than the new framework on Mac, it's slightly faster than the new framework on Windows.

So Why Not Use the New Framework?

The big issue with the new framework options is the use of the Text type. The Text type is absurdly slow on Windows. I previously ran a test using For Each Char As Text In SomeVariable.Characters to iterate over every character in a 3MB block of text. On Mac, the test completed very quickly. Sorry, I don't recall exact numbers. On Windows, I waited 15 minutes before giving up on the test. For Beacon, I had to put out an emergency update last week because doing using Text.ReplaceAll on even a moderate chunk of text - 100KB for example - was causing hangups on Windows.

Text is just too slow to use on Windows. That makes things very awkward with the new framework JSON routines.

The good news is some exchanges work fine. For example:

Dim Dict As Xojo.Core.Dictionary = Xojo.Data.ParseJSON("{""key"": ""value""}")
Dim StringValue As String = Dict.Value("key")
Dict.Value("key") = StringValue
Dim JSON As Text = Xojo.Data.GenerateJSON(Dict)

This code works perfectly fine. Which is a bit confusing because Auto does not automatically convert between Text and String. If key in this dictionary is a Text, why I can pull it out as a String?

Conversely, this code does not work:

Dim StringValue As String = "One"
Dim AutoValue As Auto = StringValue
Dim TextValue As Text = AutoValue

It will fire a TypeMismatchException because the value in AutoValue is a String not a Text.

So using the new framework JSON routines with strings is counterintuitive. And you need your own pretty-print routine, if your app needs that. But the classic framework options are slow in comparison.

Conclusion

The JSON situation with Xojo really exemplifies Xojo's problems as a tool right now. The new framework, now deprecated, does JSON way faster but leaves you with data types that are way slower on a major platform. The situation with JSON is a mess and something needs to be done. I'd be happy if JSONItem got behind-the-scenes tuning to bring it on par with Xojo.Data. Or if we got a replacement for Xojo.Data that returned classic objects and could pretty-print.

For Beacon, I'll be using the new framework routines, converting to String when I need speed, and outputting compressed non-pretty JSON unless the user really wants a format compatible with version control. Some do.