Investigate micro-benchmark JSON.dump performance

[byroot]

Shared by @fractaledmind: https://gist.github.com/fractaledmind/b309ba91fa9f0fe52e87a7399f162f56
Refactored to re-use the generator: https://gist.github.com/byroot/a79fc8008c5c866d90defe4c9644d4e1

• Even re-using generator there is still a 1.2x difference, so need to look into that.
• Would be good to better document how to re-use generator state
• Perhaps even offer first class API to re-use like I did with MessagePack::Factory::Pool? Implement Factory#pool msgpack/msgpack-ruby#266
• JSON.dump_default_options makes it hard to automatically re-use a State, but there might be some creative solution.

[byroot]

JSON: https://share.firefox.dev/4f2La69
Oj: https://share.firefox.dev/3YHEgxi

No very obvious difference, but we seem to spend a bit more time in generate_json_string vs oj_dump_cstr, that probably explain part of the difference.

[byroot]

Alright, I figured it out.

If I change:

test_data = [1, "string", { a: 1, b: 2 }, [3, 4, 5]]

Which generate just 36 bytes of JSON, into:

test_data = [1, "string", { a: 1, b: 2 }, [3, 4, 5]] * 150

Which generate 5400 bytes of JSON,:

$ ruby --yjit -Ilib:ext /tmp/foo.rb 
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
                  Oj     6.156k i/100ms
          JSON reuse     6.342k i/100ms
Calculating -------------------------------------
                  Oj     59.775k (± 1.0%) i/s   (16.73 μs/i) -    301.644k in   5.046892s
          JSON reuse     63.700k (± 0.9%) i/s   (15.70 μs/i) -    323.442k in   5.078008s

Comparison:
                  Oj:    59774.7 i/s
          JSON reuse:    63700.3 i/s - 1.07x  faster

The reason is that Oj allocate an initial buffer of 4kiB on the stack, and only if that buffer is full, it spills it on the heap.

That's for sure a good way to speedup small documents, but I'm a bit worried about allocating this much memory on the stack, as it can easily lead in stack overflow. But we could still do it with a more reasonable buffer (e.g. 1kIB), that would be enough to close the gap on most micro-benchmarks, and could be useful in the real world.

Similarly, as a solution for the "reuse" trick, we could try to elude the State allocation until we need to call a to_json method.

[fractaledmind]

This is such useful analysis, as it makes clear that in my actual use case (which was harder to write a benchmark for) I can and should simply use JSON and don't need the extra dependency.

[eregon]

The reason is that Oj allocate an initial buffer of 4kiB on the stack, and only if that buffer is full, it spills it on the heap.

Doesn't one need to, in the end, have the bytes owned by the Ruby String?
So if using a stack buffer it would need a copy at the end and wouldn't be better than rb_str_buf_new(capa) (which does zero copy if the capa is enough), right?

[byroot]

Yes, that's what I meant when I posted

if you got some realistic benchmarks

I kinda meant to exclude ridiculously small payloads, but all things considered, this can actually matter in real world workloads.

[byroot]

Doesn't one need to, in the end, have the bytes owned by the Ruby String?

Yes, but both JSON and Oj copy from their internal buffer into the final String.

wouldn't be better than rb_str_buf_new(capa) (which does zero copy if the capa is enough), right?

On paper yes, I even explored replacing the buffer with an RString with @etiennebarrie, we must have a branch somewhere.

But using an RString as a buffer can be quite slow because accessing it's capacity etc is much slower. You also have to be super careful because if you don't continiously update the RString->len and GC trigger, the GC might thing the string is small enough to be re-embeded and move it.

So it's doable, but very finicky. Ultimately it would be easier if there was a proper API to create a String by adopting an externally allocated buffer.

[byroot]

I even explored replacing the buffer with an RString with @etiennebarrie, we must have a branch somewhere.

Found it: master...etiennebarrie:json:use-ruby-strings

[byroot]

I did a clean implementation of it, and as suspected we spend even more time in allocations / memset etc: casperisfine@7dc4534

So I'm afraid, stack allocation is the way to go.

[casperisfine]

I did a quick proof of concept of stack allocation: casperisfine@b8db61b

$ ruby --yjit -Ilib:ext /tmp/j.rb 
Ruby version: 3.3.4
Oj version: 3.16.6
JSON version: 2.8.0.alpha1
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
                  Oj   521.306k i/100ms
          JSON reuse   556.121k i/100ms
Calculating -------------------------------------
                  Oj      5.718M (± 1.7%) i/s  (174.90 ns/i) -     28.672M in   5.016178s
          JSON reuse      5.824M (± 1.6%) i/s  (171.70 ns/i) -     29.474M in   5.062220s

Comparison:
                  Oj:  5717510.6 i/s
          JSON reuse:  5824042.5 i/s - same-ish: difference falls within error