654 Commits

Author SHA1 Message Date
Daniel Lemire 3d1ab87ecb Merge branch 'master' into release_candidate_4_0_0 2025-07-31 10:24:54 -04:00
evbse 6029af75b3 Improve fallback implementation (#2393) 2025-07-15 14:39:09 -04:00
Daniel Lemire b5e27af4da release candidate 4.0.0 2025-07-14 15:55:00 -04:00
Daniel Lemire c806e955c4 C++26 static reflection (#2282)
* Initial work on JSON builder

* moving the files back to ondemand for now.

* tweak

* more later

* update

* minor edits

* dropping vs arm (missing support)

* adding tests. we still specialized write_string_escaped

* tweaking

* fix typo

* tweaking the approach

* minor fix

* missing store

* another missing store

* Attempt at fixing failing serialization tests. (#2292)

* Fixing appeand_float typo (#2294)

* applying a couple of fixes

* updating single header

* fix for pre C++17 if constexpr

* Fixing unused argument problem and updating the singleheader file

* various pedantic fixes

* Sketch of builder

* reordering.

* simplify

* Adding draft of static reflection based deserialization

* Updating simdjson singleheader

* patching the automated deserialization.

* automated

* Adding support for smart pointers of user defined types.

* Adding specialization for smart pointers for basic types. I think it is highly likely that this can be done in a more generic way.

* Referncing a later version of rapidjson that fixed the issue related with assignment attempt of a const variable for GenericStringRef class.

* guarding the tests

* adding documentation for string_builder

* saving

* rename to 'append'

* saving

* non-functional benchmarks (#2342)

* non-functional benchmarks

* Fix typo

* various fixes

* tweaking

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>

* tuning

* various minor fixes

* minor tweak

* minor simplification

* updating amal

* adding a cast

* update

* fancy casting

* removing dead code

* Pushing latest changes. CITM benchmark is still not working.

* Still not working, but now I am getting only 10 errors.

* add static reflection benchmark to 'large random' benchmark and allows (#2349)

deserialization (with static reflection) from objects and arrays.

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* Removing std::map from CitmCatalog definition, since that is not currently supported.

* Added free to rust bench, segfault is still happening..

* The syntax changed: ^E became ^^E. (#2350)

* The syntax changed: ^E became ^^E.

* guarding

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* Adding support for string_view_keyed_map types.

* Adding concepts as a conditional include.

* updating single-header

* Adding concepts to ondemand deps

* rust benchmark is finally working

* Fixing small typo in docs.

* adding docker config and instructions so that our users can test the static reflection (#2358)

* adding docker config and instructions so that our users can test the
static reflection

* completing the instructions

* pruning white spaces

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* minor optimizations on the JSON builder branch

* avoiding undef behaviour

* saving

* somewhat nicer builder

* make it possible to run just one benchmark

* adding linux perf

* fixing minor issue

* updating swar

* Adding real world compilation benchmark (#2379)

* Adding compilation benchmark for json parsing with and without reflection

* Moving it to the benchmark folder, also reducing a bit the number of iterations.

* Removing script from root folder.

* Reducing number of iterations

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Making the script more customizable and also test whether the compiler being used supports reflection before actually running the benchmark

---------

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Using define_static_string from  https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3491r2.html (#2389)

* Applying changes needed after latest reflection paper updates.

* Working, but no template for yet.

* Updating single-header to incldue the use of define_static_string.

* copying over master

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>
2025-07-14 15:43:52 -04:00
Daniel Lemire 4cdc4f18ef documenting fatal errors 2025-03-13 13:26:50 -04:00
zhanglistar f615112093 Disabling memory sanitizer with parse_string (#2310) 2024-12-09 10:20:33 -05:00
Daniel Lemire a05a56856d fix: use On-Demand throughout. (#2222) 2024-07-29 15:54:21 -04:00
Daniel Lemire 4180e05730 [no-ci] fix 'null_ptr' written as 'null_nullptrptr' in the comments 2024-07-11 08:25:27 -04:00
Yuriy Chernyshov c80dda7c58 Fix building simdjson against libc++ with _LIBCPP_REMOVE_TRANSITIVE_INCLUDES defined (#2184)
```
src/implementation.cpp:193:20: error: no template named 'is_trivially_destructible' in namespace 'std'; did you mean 'is_trivially_move_constructible'?
static_assert(std::is_trivially_destructible<detect_best_supported_implementation_on_first_use>::value, "detect_best_supported_implementation_on_first_use should be trivially destructible");
              ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
                   is_trivially_move_constructible
```
2024-05-23 16:54:23 -04:00
Daniel Lemire c888075f8d makes implementations trivially destructible (#2171) 2024-04-23 23:49:20 -04:00
Javier Blazquez 40b414d184 add Windows ARM64EC build and CI support (#2168)
* add Windows ARM64EC support

* add ARM64EC to vs17-arm-ci workflow
2024-04-09 14:56:58 -04:00
Jinyang He 4c98e51c53 Add LoongArch LSX and LASX support (#2159)
* Add LoongArch SX support

* Add LoongArch ASX support
2024-04-04 12:09:36 -04:00
Daniel Lemire 37a9aaad48 [no ci] tweaking the wording 2024-02-26 14:05:12 -05:00
Serge Aleynikov 78325888ea Add functionality to detect big integers (#2139) 2024-02-26 14:00:33 -05:00
Daniel Lemire e510ed2774 Release candidate (3.7.0) (#2136)
* Release candidate (3.7.0)

* some extra comments.
2024-02-23 16:50:55 -05:00
Daniel Lemire 6430fbd39a Better documentation and support for the case where you have a single implementation. (#2118) 2024-02-06 13:20:23 -05:00
Daniel Lemire 4d2fc31b27 removes simdjson.cpp include from the amalgated demo (#2119)
Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2024-02-05 19:34:50 -05:00
Niles Salter 9b0435d864 [utf8-validator] eliminate unnecessary comparison from must_be_2_3_continuation (#2113)
* [utf8-validator] eliminate unnecessary comparison from must_be_2_3_continuation

* Fix comment in
2024-01-28 12:42:59 -05:00
Daniel Lemire 97196fa969 Fixing warnings issued by clang (Node issue 50930) (#2090)
* Fixing warnings issued by clang (Node issue 50930)

* removing space

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-11-30 18:06:50 -05:00
Daniel Lemire 80d26298a0 Redesigning visit_primitive so that it is optimized for strings and (#2060)
numbers.

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-09-20 10:09:56 -04:00
John Keiser 8dabd02c3a Default to step = 4 2023-08-29 15:48:52 -04:00
Daniel Lemire c9692005fd This allows us to pass SIMDJSON_STRUCTURAL_INDEXER_STEP as a command-line option. (#2055) 2023-08-25 13:47:20 -04:00
Daniel Lemire b2e20e04c9 Standard compatibility fixes (#2053)
* Standard compatibility fixes

* missing commit

* Should work.

* Fix.

* Fix.

* Should work now.

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-08-25 10:18:02 -04:00
John Keiser adc9d18efd Change step for structural_indexes to 2 2023-08-23 19:02:05 -07:00
John Keiser ac78c625df simdjson_inline -> simdjson_really_inline 2023-08-02 08:28:05 -07:00
John Keiser 390a66c6e8 Make next_structurals inline 2023-08-02 08:28:05 -07:00
John Keiser 084f662f32 Don't use C++20 designated initializers 2023-08-02 08:28:05 -07:00
John Keiser c878ff2500 Fix compiler warnings 2023-08-02 08:28:05 -07:00
John Keiser 06afe9ecb3 New escape algorithm with significantly less data dependency 2023-08-02 08:28:05 -07:00
John Keiser b383c717d7 Have amalgamate.py verify it's actually amalgamating all the files 2023-07-20 15:11:52 -07:00
John Keiser ae9ba2bed2 Rename SIMDJSON_AMALGAMATED to SIMDJSON_CONDITIONAL_INCLUDE 2023-07-20 11:43:19 -07:00
John Keiser 4f4e81668e Change all include paths to <> instead of "" to avoid relative path includes 2023-07-20 10:48:25 -07:00
John Keiser bb54946b78 Disallow including headers from implementation files 2023-07-20 10:01:22 -07:00
John Keiser e3cac71afe Enforce that dependencies.h and generic/*.h include the same dependencies. 2023-07-17 16:07:42 -07:00
John Keiser ab09e96de9 Only use amalgamated.h in generic/ 2023-07-16 19:18:58 -07:00
John Keiser 3c446c551b Move AMALGAMATED up to include implementation.cpp 2023-07-15 17:04:14 -07:00
John Keiser 28db262b47 Make SIMDJSON_AMALGAMATED checking much stricter 2023-07-15 16:46:15 -07:00
John Keiser bf076bfb5f Begin amalgamation early in simdjson.cpp 2023-07-15 15:55:38 -07:00
John Keiser 7bf1ecff9e Mark amalgamation early in 2023-07-15 15:53:09 -07:00
John Keiser e25a40facf Check if amalgamation rules are not being followed while compiling 2023-07-15 15:47:14 -07:00
John Keiser a442842f0d Use SIMDJSON_AMALGAMATED (which works in-editor and out) instead of editor-specific tweaks 2023-07-15 14:53:02 -07:00
John Keiser 9a0527c380 Only make editor stuff work in editor 2023-07-14 20:05:05 -07:00
John Keiser b160cf0610 Make simdjson.cpp compile on VSCode 2023-07-08 11:53:58 -07:00
John Keiser 553ee69b7c Make implementation .h/.cpp compile 2023-07-08 11:42:57 -07:00
Daniel Lemire cfd337bb50 This PR adds an 'at_end()' method. (#1978)
* This PR adds an 'at_end()' method.

* Adding 1111 }

* Tweaking test.
2023-03-30 10:47:13 -04:00
Daniel Lemire 37e87f6f99 Adding support for wobbly strings to On Demand and lossy trancoding from escaped (with replacement) (#1947)
* Adding support for wobbly strings.

* Minor tweak.

* Adopt substitution code point.

* Lossy

* Patching
2023-02-28 22:56:44 -05:00
wanweiqiangintel cd13b02bf0 add UTF-8 validation flag (#1957) 2023-02-28 09:47:08 -05:00
Daniel Lemire f151300a76 Disabling fallback kernel on systems where it is not needed (#1930)
* Fixing issue 1772

* More cleaning.

* Allow disabling other kernels.

* Minor tweaks.
2023-01-03 09:52:41 -05:00
Daniel Lemire e94f027310 Serialize 0 as 0.0 (#1921)
* Serialize 0 as 0.0

* Extending the '.0'.
2022-11-23 10:31:42 -05:00
Tyson Andre 00564bf7d5 Micro-optimization for parsing surrogate pairs (#1897)
Load 2 bytes and compare the 2 bytes against `"\u"`
Compilers with optimizations turned on will turn this into a 16-bit load
then 16-bit compare on supported platforms
(with smaller compiled code size).

Make it obvious to the compiler that it's reading two
consecutive bytes of the same pointer

Add parse_surrogate_pairs to show the difference exists.
See discussion in #1896
2022-10-02 12:10:53 -04:00