31 Commits

Author SHA1 Message Date
Daniel Lemire 73e69a5e84 Get reflection working without warnings under GCC 16 (#2658)
* wip gcc16

* progress

* minor gcc16 fixes

* final step

* Update include/simdjson/padded_string-inl.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update include/simdjson/padded_string.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* renaming test function

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-03 15:32:42 -04:00
Daniel Lemire c806e955c4 C++26 static reflection (#2282)
* Initial work on JSON builder

* moving the files back to ondemand for now.

* tweak

* more later

* update

* minor edits

* dropping vs arm (missing support)

* adding tests. we still specialized write_string_escaped

* tweaking

* fix typo

* tweaking the approach

* minor fix

* missing store

* another missing store

* Attempt at fixing failing serialization tests. (#2292)

* Fixing appeand_float typo (#2294)

* applying a couple of fixes

* updating single header

* fix for pre C++17 if constexpr

* Fixing unused argument problem and updating the singleheader file

* various pedantic fixes

* Sketch of builder

* reordering.

* simplify

* Adding draft of static reflection based deserialization

* Updating simdjson singleheader

* patching the automated deserialization.

* automated

* Adding support for smart pointers of user defined types.

* Adding specialization for smart pointers for basic types. I think it is highly likely that this can be done in a more generic way.

* Referncing a later version of rapidjson that fixed the issue related with assignment attempt of a const variable for GenericStringRef class.

* guarding the tests

* adding documentation for string_builder

* saving

* rename to 'append'

* saving

* non-functional benchmarks (#2342)

* non-functional benchmarks

* Fix typo

* various fixes

* tweaking

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>

* tuning

* various minor fixes

* minor tweak

* minor simplification

* updating amal

* adding a cast

* update

* fancy casting

* removing dead code

* Pushing latest changes. CITM benchmark is still not working.

* Still not working, but now I am getting only 10 errors.

* add static reflection benchmark to 'large random' benchmark and allows (#2349)

deserialization (with static reflection) from objects and arrays.

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* Removing std::map from CitmCatalog definition, since that is not currently supported.

* Added free to rust bench, segfault is still happening..

* The syntax changed: ^E became ^^E. (#2350)

* The syntax changed: ^E became ^^E.

* guarding

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* Adding support for string_view_keyed_map types.

* Adding concepts as a conditional include.

* updating single-header

* Adding concepts to ondemand deps

* rust benchmark is finally working

* Fixing small typo in docs.

* adding docker config and instructions so that our users can test the static reflection (#2358)

* adding docker config and instructions so that our users can test the
static reflection

* completing the instructions

* pruning white spaces

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* minor optimizations on the JSON builder branch

* avoiding undef behaviour

* saving

* somewhat nicer builder

* make it possible to run just one benchmark

* adding linux perf

* fixing minor issue

* updating swar

* Adding real world compilation benchmark (#2379)

* Adding compilation benchmark for json parsing with and without reflection

* Moving it to the benchmark folder, also reducing a bit the number of iterations.

* Removing script from root folder.

* Reducing number of iterations

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Making the script more customizable and also test whether the compiler being used supports reflection before actually running the benchmark

---------

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Using define_static_string from  https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3491r2.html (#2389)

* Applying changes needed after latest reflection paper updates.

* Working, but no template for yet.

* Updating single-header to incldue the use of define_static_string.

* copying over master

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>
2025-07-14 15:43:52 -04:00
Daniel Lemire 714f0ba222 This deletes most of our data files making the repository much smaller (#1582)
* This deletes most of our data files making the repository much smaller.

* Removing dead code.

* Various minor fixes.
2021-06-04 09:24:03 -04:00
D. Stolle be9d5d4e31 adjust GitHub links to current repository URL (#1553)
Switch links (mostly in comments) from old repository URL
<https://github.com/lemire/simdjson/> to the current URL
<https://github.com/simdjson/simdjson/>.
2021-04-26 09:08:14 -04:00
John Keiser 1b4d3bcbb6 Add sajson benchmarks 2021-01-11 15:12:12 -08:00
Daniel Lemire 109bb505d8 Adding a new test file. (#922)
* Adding a new test file.

* Renaming.
2020-08-18 10:42:45 -04:00
myd7349 332dd76867 Fix Ninja build
When using Ninja as generator, it failed to build:
$ cmake .. -G Ninja
$ ninja
> ninja: error: '/jsonexamples/generated/miss-templates/*.txt', needed by 'jsonexamples/generated/utf-8.json', missing and no known rule to make it
2020-05-21 12:29:39 +08:00
John Keiser 1e8a54af0b Fix benchfeatures.rb on Ruby 1.9 2020-05-05 11:52:07 -07:00
John Keiser aa53d8708e Generate test data when tests are built 2020-05-05 11:29:41 -07:00
John Keiser 8c600ca553 Make benchfeatures work again 2020-05-05 09:39:29 -07:00
John Keiser d3e44b1108 Add amalgamation support to cmake 2020-04-20 19:50:51 -07:00
Paul Dreik 75545ff70d ref qualify parser methods to avoid use of dangling objects (#703)
To avoid using data belonging to a temporary, the parse functions are ref qualified to get a compile error if used on an rvalue. See https://github.com/simdjson/simdjson/issues/696

Compilation tests are also added, to make sure bad usage fails to compile.

Reviewed by jkeiser.
2020-04-15 09:57:52 +02:00
Daniel Lemire ab78482ee7 The testing files should not be all over the place. The "small" directory should be only for small files (duh). (#650) 2020-03-30 16:24:03 -04:00
Daniel Lemire 32afcd2e48 Better documentation for issue 70 (#638) 2020-03-27 19:44:01 -04:00
Jeremie Piotte bdc2b07339 Streams of JSON documents + Large files (>4GB) (#350) (#364)
* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* type

* minor fixes and cleaning.

* minor fixes and cleaning.

* removing warnings

* removing some copies

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* merged main into branch

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* Fix for https://github.com/lemire/simdjson/issues/345

* Follow up test and fix for https://github.com/lemire/simdjson/issues/345 (#347)

* Final (?) fix for https://github.com/lemire/simdjson/issues/345

* Verbose basictest

* Being more forgiving of powers of ten.

* Let us zero the tail end.

* add basic fuzzers (#348)

* add basic fuzzing using libFuzzer

* let cmake respect cflags, otherwise the fuzzer flags go unnoticed

also, integrates badly with oss-fuzz

* add new fuzzer for minification, simplify the old one

* add fuzzer for the dump example

* clang format

* adding Paul Dreik

* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* type

* minor fixes and cleaning.

* Fixing issue 351 (#352)

* Fixing issues 351 and 353

* minor fixes and cleaning.

* removing warnings

* removing some copies

* Fix ARM compile errors on g++ 7.4 (#354)

* Fix ARM compilation errors

* Update singleheader

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* fix integer overflow in subnormal_power10 (#355)

detected by oss-fuzz

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18714

* Adding new test file, following https://github.com/lemire/simdjson/pull/355

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* merged main into branch

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* merging main

* rough prototype working.  Needs more test and fine tuning.

* prototype working on large files.

* prototype working on large files.

* Adding benchmarks

* jsonstream API adjustment

* minor fixes and cleaning.

* minor fixes and cleaning.

* removing warnings

* removing some copies

* runtime dispatch error fix

* makefile linking src/jsonstream.cpp

* fixing arm stage 1 headers

* fixing stage 2 headers

* fixing stage 1 arm header

* making jsonstream portable

* cleaning imports

* including <algorithms> for windows compiler

* cleaning benchmark imports

* adding jsonstream to amalgamation

* bug fix where JsonStream would bug on rare cases.

* Addind a JsonStream Demo to Amalgamation

* rough prototype working.  Needs more test and fine tuning.

* minor fixes and cleaning.

* adding jsonstream to amalgamation

* merged main into branch

* Addind a JsonStream Demo to Amalgamation

* merging main

* merging main

* make file fix
2019-11-08 17:39:45 -05:00
saka1 58697f6f3b Remove unnecessary x permissions from JSON files (#290) 2019-08-18 20:48:48 -04:00
ioioioio bcabdfc1ae Json pointer (#220)
* json pointer support

* Addition of tests for the json pointer

* Adding a new tool for the JSON Pointer support, and some documentation.
2019-07-26 18:38:10 -04:00
Daniel Lemire 463ef9b08f Adding reduced demo. 2019-06-05 21:41:13 -04:00
Daniel Lemire e6c9d7092f Adding copies of data files with credit to @plokhotnyuk so that we can track issue https://github.com/lemire/simdjson/issues/70 2019-02-25 14:02:03 -05:00
Daniel Lemire 85d8c9dbf0 Minor code cleaning 2019-01-18 15:21:54 -05:00
Daniel Lemire 82cd8eae5d Moving the small files to the small directory. 2018-12-12 22:45:27 -05:00
Daniel Lemire f08920f19a Adding more files. 2018-12-11 17:50:45 -05:00
Daniel Lemire e4d4158e3f Added dependencies. 2018-11-20 16:43:22 -05:00
Daniel Lemire 006eb76bcd Adding a special case. 2018-11-16 21:49:37 -05:00
Daniel Lemire e517414080 We include character-encoding validation. 2018-10-17 19:22:09 -04:00
Daniel Lemire 577d6792f4 Integrating sajson. 2018-09-28 00:00:52 -04:00
Daniel Lemire d29e0b4dcf Adding a "numbers" file. 2018-05-02 21:12:40 -04:00
Daniel Lemire 3bc81cf9cf Adding another example. 2018-04-30 17:35:11 -04:00
Daniel Lemire b55e8c01a3 Making my adversarial example a bit harder. 2018-04-06 11:22:49 -04:00
Daniel Lemire 0e5b0f776f Adding an adversarial example. 2018-04-05 21:49:30 -04:00
Daniel Lemire bc1331283a Private research repo. 2018-03-23 00:05:32 -04:00