Commit Graph

3253 Commits

Author SHA1 Message Date
Daniel Lemire 263e58d9d7 guarding against zero memory case 2026-06-07 14:28:13 -04:00
Daniel Lemire ed345772cd we don't need this 2026-06-06 15:35:36 -04:00
Daniel Lemire 92123faa7d making perfect hash the default 2026-06-06 14:47:09 -04:00
Daniel Lemire 12518a278a minor fixes 2026-06-06 14:36:32 -04:00
Daniel Lemire dc9a8e2890 adding a new method... 2026-06-06 00:51:09 -04:00
Daniel Lemire 12062bc5bc simplifying. 2026-06-05 19:24:19 -04:00
Daniel Lemire 6e0abb7f9c lower level. 2026-06-03 00:32:41 -04:00
Daniel Lemire 686c9bff54 experimental 2026-06-02 23:39:47 -04:00
Daniel Lemire e1654be9b3 removing useless comment. 2026-06-01 20:57:12 -04:00
Daniel Lemire de1b38c459 better documentation. 2026-05-31 18:22:12 -04:00
Daniel Lemire 3d08f995d0 minor optimization. 2026-05-31 15:07:01 -04:00
Daniel Lemire 135159012f update 2026-05-30 21:38:40 -04:00
Daniel Lemire 936afe9aa1 missing file 2026-05-30 19:39:14 -04:00
Daniel Lemire 1cc03882de minor extension. 2026-05-30 19:19:30 -04:00
Daniel Lemire 9cd389ae84 extra documentation. 2026-05-29 22:05:07 -04:00
Daniel Lemire 64d5ee550f maybe the final update. 2026-05-29 21:37:02 -04:00
Daniel Lemire 18cdecaf8e update 2026-05-29 20:08:39 -04:00
Daniel Lemire c33e4441a0 safety fix for yyjson benchmark 2026-05-29 18:12:27 -04:00
Daniel Lemire 732ca8cf70 new file 2026-05-29 17:47:47 -04:00
Daniel Lemire 2143ebe538 update 2026-05-29 17:47:10 -04:00
Daniel Lemire df770af450 better 2026-05-29 17:46:03 -04:00
Daniel Lemire 1985584de7 inlining. 2026-05-29 17:46:03 -04:00
Daniel Lemire 6a74cd41d6 proto 2026-05-29 17:46:03 -04:00
Daniel Lemire dcaad94b53 init 2026-05-29 17:46:03 -04:00
Daniel Lemire 98fde69253 adding clang 20 to our CI coverage. (#2739) 2026-05-29 17:43:48 -04:00
Daniel Lemire dc485f845d let us use lower-case words for layout_mode (#2740) 2026-05-29 17:43:33 -04:00
Daniel Lemire 79bbba3e3e saving. 2026-05-27 10:01:05 -04:00
吴杨帆 1a29a5e53a docs: clarify raw_json() consumption and rewind option (#2734)
* docs: clarify raw_json() consumption and rewind option

Document that raw_json() consumes objects/arrays and point readers to
reset() or document::rewind() when they need to parse again.

Related to #1943

* ci: retrigger workflow
2026-05-27 08:12:00 -04:00
Daniel Lemire 57561f64f4 adding warning that this is experimental. 2026-05-26 15:19:32 -04:00
Daniel Lemire 00563f9133 documenting annotation. 2026-05-26 13:34:37 -04:00
Makkar 5db72ab9d3 add: C++26 annotations support for rename and skip (#2730) 2026-05-26 12:58:46 -04:00
Daniel Lemire d3d99b8fc5 we omit ondemand_cacheline under riscv64 2026-05-25 11:19:05 -04:00
Daniel Lemire 168ef58075 simplify the tests for memory mapping under Windows. (#2729) 2026-05-20 14:37:52 -04:00
Daniel Lemire fb8c575121 more verbose error messages 2026-05-20 14:37:05 -04:00
Daniel Lemire 7732480b25 we need to require rvv for the risc64 kernel 2026-05-18 17:24:56 -04:00
wankun 1ae93a70d4 Bug fix for get_string() crash when parse incomplete json string (in option disabled my default) (#2720)
* get_string() crash when parse incomplete json string

* get_string() crash when parse incomplete json string

* Bug fix for RISC-V fail

* Update tests/ondemand/CMakeLists.txt

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Remove unnecessary changes

---------

Co-authored-by: Daniel Lemire <daniel@lemire.me>
2026-05-18 17:18:58 -04:00
吴杨帆 396ca141fd perf(builder): add LSX fast_needs_escaping for LoongArch (#2727)
Use LSX intrinsics for the string-builder escape scan on LoongArch,
matching the existing find_next_json_quotable_character LSX path.

Fixes #2603
2026-05-18 17:13:43 -04:00
Daniel Lemire 8e2e9b8e02 adding silent flag to amalgamation. (#2721) 2026-05-18 11:20:02 -04:00
吴杨帆 599895e54d docs: use HTTPS for YouTube embed links in README (#2724) 2026-05-18 11:19:46 -04:00
吴杨帆 77a53d19a2 docs: use HTTPS for Boost license link in README (#2725) 2026-05-18 11:16:31 -04:00
吴杨帆 8c7cc2f963 docs: use HTTPS for arXiv link in README (#2726) 2026-05-18 11:16:11 -04:00
xaldarof 43766fc7ef add simdjson-dart binding to docs (#2719)
* Add simdjson-dart binding description

Fix typo in simdjson-dart description.

* Update simdjson-dart description in README
2026-05-12 11:42:16 -04:00
Aylin Dmello c2f25ab455 Add section to basics.md documenting the SIMDJSON_ENABLE_NAN_INF flag (#2718)
Co-authored-by: Aylin Dmello <admello@mathworks.com>
2026-05-11 17:11:47 -04:00
Daniel Lemire ec49efa5da avoid unnecessary pointer member (#2715) 2026-05-08 00:26:57 -04:00
Daniel Lemire a16e5128fe pendantic guard 2026-05-07 23:13:40 -04:00
Alecto Irene Perez bd42858383 fix -Wunneeded-internal-declaration warning (#2714) 2026-05-07 21:46:52 -04:00
jmestwa-coder 7d30ff2574 JSON Pointer array index overflow handling (#2713) 2026-05-07 19:07:39 -04:00
Francisco Geiman Thiesen 42c047a927 builder: position-as-local writer for reflection serializer (#2708)
The dominant cost in the reflection serializer was the strict-aliasing
reload pattern: every char* write through `string_builder::buffer.get()`
forced the compiler to assume `string_builder::position` and `::capacity`
may have been clobbered, so it reloaded both members from memory after
each byte. perf annotate showed `ldr [position]` and `ldr [capacity]`
taking 15-20% of CITM serialization time.

This change refactors the reflection atom path to use an internal
`writer` struct (buffer pointer + position + capacity + back-ref to
string_builder). The writer lives as a stack-local in the top-level
append() entry point, threaded through the inlined atom() chain via
`writer&`. After SROA, the three fields are register-resident across
the entire serialization. The `pos` is never round-tripped through
memory between writes — it's a local size_t.

This mirrors the pattern Glaze uses (passing `B&& b, auto&& ix` through
every helper). The simdjson-specific bit is keeping the public
string_builder API intact: only the internal atom() family is
refactored to take `writer&`. The top-level append(string_builder&, T)
constructs a writer on the stack, threads it through atom(), then
syncs the local position back at the end.

For string fields specifically, the escape path is also inlined
through the writer — the existing `write_string_escaped(input, out)`
helper already takes a destination pointer and returns bytes-written,
so it composes cleanly with `w.ptr + w.pos`. Without this, sync/
reload around each string write was a real cost on Twitter (-7%).

Performance
===========

TRUE A/B in single docker invocation, 7 alternating rounds, byte-
identical output, all static_reflection_comprehensive_tests pass.
Build: clang-p2996 21.0.0git, -O3 -DNDEBUG, aarch64 Linux.

CITM Catalog (496 682 bytes):
  baseline (master): 4356 MB/s
  patched:           6776 MB/s   +55.5%
  Glaze:             4860 MB/s   simdjson now 39.4% AHEAD of Glaze

Twitter (81 927 bytes):
  baseline (master): 6437 MB/s
  patched:           6452 MB/s   +0.2% (break-even)
  Twitter is string-heavy; the writer's gain over the existing
  member-based path is small here, but it no longer regresses.

Default initial capacity is unchanged at 1024.
2026-05-07 13:28:03 -04:00
Alecto Irene Perez 376ef7e8e9 Add support for writing NaN/Infinity if SIMDJSON_ENABLE_NAN_INF is enabled (#2711)
* Add tests for dumping out NaN/Infinity to the json builder

* If SIMDJSON_ENABLE_NAN_INF is set, write out NaN and Infinity

* Add tests for dumping out NaN/Infinity when serializing DOM

* If SIMDJSON_ENABLE_NAN_INF, use 'NaN' and 'Infinity' for nan/inf

* Fix bug where 'Inf' as a root atom + spaces of padding parses incorrectly

* Update FracturedJson printers so tables containing NaN/Infinity are aligned
2026-05-07 13:11:03 -04:00
Daniel Lemire 0a23ebbfef adding gcc16 (#2712) 2026-05-07 13:10:42 -04:00