* This adds SIMDJSON_DEVELOPMENT_CHECKS to the DOM API to help users
in the scenario of issue 1914.
* More documentation and warnings.
* Updating following comments by Tyson
Related to #1904
Users of the simdjson library will see json2msgpack as an available
example of how to recursively process json with the ondemand parser,
and checking for trailing tokens in a document is one part of json validation.
These checks shouldn't affect benchmark results performance.
The benchmark is run on the 631KB twitter.json file.
Make it less likely to accidentally introduce tabs, trailing whitespace,
carriage returns, non-utf8 in files, or files without trailing newlines.
https://editorconfig.org/ has plugins for various editors/IDEs and is
enabled by default in some IDEs.
* Fixing issue 1898 Preserve sign for number with underflowing exponent (#1900)
Before this commit, simdjson parsed "-1e-999" and "-0e-999" and "-1e-342"
as 0.0.
After this commit, those JSON strings get parsed as -0.0.
(https://en.wikipedia.org/wiki/Signed_zero)
The old behavior was inconsistent with the way simdjson parsed "-0.0" as -0.0.
Co-authored-by: Daniel Lemire <daniel@lemire.me>
Co-authored-by: Tyson Andre <tysonandre775@hotmail.com>
Load 2 bytes and compare the 2 bytes against `"\u"`
Compilers with optimizations turned on will turn this into a 16-bit load
then 16-bit compare on supported platforms
(with smaller compiled code size).
Make it obvious to the compiler that it's reading two
consecutive bytes of the same pointer
Add parse_surrogate_pairs to show the difference exists.
See discussion in #1896
Closes#1894
Reject low surrogates outside of the range U+DC00—U+DFFF
Related to https://unicodebook.readthedocs.io/unicode_encodings.html#utf-16-surrogate-pairs
A surrogate pair should consist of a high surrogate and low surrogate.
They're used to represent 0x010000-0x10FFFF in the JSON spec because
the JavaScript specification originally only supported `\uXXXX`.
Previously, simdjson would accept some combinations of valid high
surrogates and invalid low surrogates due to a bug in the check.
(e.g. `\uD888\u1234` was accepted)
U+D800—U+DBFF (1,024 code points): high surrogates
U+DC00—U+DFFF (1,024 code points): low surrogates
Dirk Stolle