simdjson was designed for buffer reuse, so showing both variants:
- Buffer reuse: realistic production performance (~12-13% faster)
- Fresh allocation: fair comparison with other libraries
- Update yyjson serialization to match C++ CitmCatalog struct (only events + performances)
- Add yyjson include and benchmark function to CITM serialization
- Link yyjson to CITM serialization benchmark in CMakeLists.txt
- Output size matches other libraries: 496,682 bytes
The CITM benchmark was using simdjson::to_json_string() while Twitter
used simdjson::builder::to_json(). This caused misleading results
(532 MB/s vs 2780 MB/s). Now both use the same API for consistency.
Memory Allocation Fairness:
- Add "fair" benchmark variants that allocate fresh buffers each iteration
- Add "reuse" variants showing optimized API potential with buffer reuse
- Fair variants match allocation behavior of competing libraries
Rust/serde CITM Fix:
- Rewrite Rust CitmCatalog struct to match C++ exactly
- Now only serializes events + performances (matching C++ behavior)
- Output size now matches: 496,682 bytes for all libraries
Documentation:
- Add comprehensive benchmark_fairness.md report
- Document reflect-cpp output size discrepancy (optional field handling)
- Document Rust FFI overhead (negligible for this data size)
- Include reproducibility instructions
Results after fixes:
- Twitter: All libraries produce identical 81,927 byte output
- CITM: simdjson/nlohmann/Rust all produce 496,682 bytes
- CITM reflect-cpp: 476,270 bytes (documented optional handling difference)
- Fix typo: SIMDJSON_USER_RUST -> SIMDJSON_USE_RUST in CMakeLists.txt
- Update unified_benchmark.sh to use correct SIMDJSON_USE_RUST flag
- Add yyjson to Twitter serialization benchmark
- Fix CITM catalog field names to match JSON keys (audienceSubCategoryId, seatCategoryId)
- Fix CMake target names for yyjson and rapidjson in CITM benchmark
If you provide a custom serializer for range types it is currently never used due to the requires clause for string_builder::append with ranges is overly broad
* Adding compile time json path
* using string_view
* Adding support for compile-time json pointer as well.
* Removing unnecessary comment
* Tests now working, still will re-review.
* Adding documentation on the compile-time json path/pointer parsing feature.
* Adding benchmark showing the significant performance advantage of using compiled paths whenever you have them a priori.
* going for JSONPath (correct wording).
* minor update (mostly doc)
---------
Co-authored-by: Daniel Lemire <daniel@lemire.me>
Francisco Geiman Thiesen