bpo-37587: _json: use _PyUnicodeWriter when scanning string

[methane]

$ ./python -m pyperf compare_to before.json patched.json
short-ascii: Mean +- std dev: [before] 23.4 ms +- 0.3 ms -> [patched] 23.8 ms +- 0.1 ms: 1.02x slower (+2%)
long-no-ascii: Mean +- std dev: [before] 47.1 us +- 0.2 us -> [patched] 14.1 us +- 0.2 us: 3.34x faster (-70%)

https://bugs.python.org/issue37587

[serhiy-storchaka]

What about mostly-ASCII strings with rare non-ASCII inclusions?

[serhiy-storchaka]

Also what about short ASCII-only strings?

[serhiy-storchaka]

_PyUnicodeWriter is a deep implementation detail. _json.scanstring is also an implementation detail. The end user should not be aware of them. Just describe the effect of the optimization. It is worth to add also a What's New entry.

[methane]

Also what about short ASCII-only strings?

In the worst case, this pull request make it more than 2x slower.

$ ./python -m pyperf timeit -o patched.json -s 'import json; x=json.dumps(["a"]*1000000)' -- 'json.loads(x)'

$ ./python -m pyperf compare_to master.json patched.json
Mean +- std dev: [master] 24.6 ms +- 0.0 ms -> [patched] 61.0 ms +- 0.2 ms: 2.48x slower (+148%)

[methane]

$ ./python -m pyperf compare_to before.json patched.json
short-ascii: Mean +- std dev: [before] 23.4 ms +- 0.3 ms -> [patched] 23.8 ms +- 0.1 ms: 1.02x slower (+2%)
long-no-ascii: Mean +- std dev: [before] 47.1 us +- 0.2 us -> [patched] 14.1 us +- 0.2 us: 3.34x faster (-70%)

The difference between 53.5 us and 47.1 us is came from compilers: 53.5 us on clang-7 and 47.1 us on gcc 8.3.0.