* This adds SIMDJSON_DEVELOPMENT_CHECKS to the DOM API to help users
in the scenario of issue 1914.
* More documentation and warnings.
* Updating following comments by Tyson
Related to #1904
Users of the simdjson library will see json2msgpack as an available
example of how to recursively process json with the ondemand parser,
and checking for trailing tokens in a document is one part of json validation.
These checks shouldn't affect benchmark results performance.
The benchmark is run on the 631KB twitter.json file.
Make it less likely to accidentally introduce tabs, trailing whitespace,
carriage returns, non-utf8 in files, or files without trailing newlines.
https://editorconfig.org/ has plugins for various editors/IDEs and is
enabled by default in some IDEs.
* Fixing issue 1898 Preserve sign for number with underflowing exponent (#1900)
Before this commit, simdjson parsed "-1e-999" and "-0e-999" and "-1e-342"
as 0.0.
After this commit, those JSON strings get parsed as -0.0.
(https://en.wikipedia.org/wiki/Signed_zero)
The old behavior was inconsistent with the way simdjson parsed "-0.0" as -0.0.
Co-authored-by: Daniel Lemire <daniel@lemire.me>
Co-authored-by: Tyson Andre <tysonandre775@hotmail.com>
Load 2 bytes and compare the 2 bytes against `"\u"`
Compilers with optimizations turned on will turn this into a 16-bit load
then 16-bit compare on supported platforms
(with smaller compiled code size).
Make it obvious to the compiler that it's reading two
consecutive bytes of the same pointer
Add parse_surrogate_pairs to show the difference exists.
See discussion in #1896
Closes#1894
Reject low surrogates outside of the range U+DC00—U+DFFF
Related to https://unicodebook.readthedocs.io/unicode_encodings.html#utf-16-surrogate-pairs
A surrogate pair should consist of a high surrogate and low surrogate.
They're used to represent 0x010000-0x10FFFF in the JSON spec because
the JavaScript specification originally only supported `\uXXXX`.
Previously, simdjson would accept some combinations of valid high
surrogates and invalid low surrogates due to a bug in the check.
(e.g. `\uD888\u1234` was accepted)
U+D800—U+DBFF (1,024 code points): high surrogates
U+DC00—U+DFFF (1,024 code points): low surrogates
* build: add pkg-config support
The CMake build script now generates a simple pkg-config files that can
be easily used by non-CMake users.
The file is generated from a template file that gets filled in at
configure time.
As CMake doesn't have anything similar to Meson's pkg-config generator
the file is quite static, i.e. new simdjson public defines/dependencies
won't be picked up automatically.
This approach also suffers from one minor issue, mentioned in
[jtojnar/cmake-snips][]; in short, it doesn't work well when users
specify CMAKE_INSTALL_INCLUDEDIR and similar as absolute paths. It's not
a big deal, and it will easily fixable once you'll require CMake >=3.20.
Fixes#1763
[jtojnar/cmake-snips]: https://github.com/jtojnar/cmake-snips#concatenating-paths-when-building-pkg-config-files
* build: handle absolute paths in .pc generation
As mentioned in the previous commit message, correct concatenation of
paths is only available in CMake >=3.20, so handling absolute paths in
pkg-config file generation requires using jtojnar's JoinPaths module.
* ci: add debian job
This new jobs compiles simdjson on Debian Testing, a semi-rolling
release, so that new compilers are always tested.
This job also tests the pkg-config file introduced in commit
1096c3b299
Daniel Lemire