Commit Graph

63 Commits

Author SHA1 Message Date
Daniel Lemire 8aae14931d release candidate 4.0.2 (#2441)
* release candidate 4.0.2

* more fixes

* fixing typos

* adding macro check
2025-09-15 09:17:43 -06:00
Daniel Lemire bd86bb0cfb adding UTF-8 test 2025-09-05 14:56:02 -04:00
Daniel Lemire 1ebf115b2b improving the documentation of raw json access (#2416)
* improving the documentation of raw json access

* minor fix

* documentation update

* guarding for exceptions

* fixing exception issue

* fix test

* update.

* saving comments

* more technical fixes

* correcting path in ci test

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2025-08-15 10:26:20 -04:00
Joseph Olabisi fb9a689dff Adding DOM support for json path with wildcard (#2346)
* wip brute-force wildcard for json_path

* wip - bruteforce surface wildcard with result

* partially handle keys with wildcard

* wip - nested paths/pointers on wildcard results

* wip

* wip - handling child properties of wildcard result

* done - handling child properties of wildcard result

* fix key

* add support for wildcard for arrays

* handle array INCORRECT_TYPE

* rename at_path_new to at_path_with_wildcard

* add benchmark

* fix benchmark

* use memcmp

* minor improvements

* refactor to tail recursion

* nit

* approximately 30% improvement in runtime

* nit

* corrected logic

* nit

* cleanup

* add some initial tests

* cleanup 2

* modified:   CMakeLists.txt

* cleanup

* cleanup

* cleanup

* cleanup

* restore examples/quickstart/CMakeLists.txt

* cleanup

* restore quickstart.cpp

* revert array-inl.h

* cleanup array-inl.h

* revert object-inl.h

* cleanup object-inl.h

* cleanup

* nit

* nit

* add test

* address some feedbacks

* address additional feedbacks (copilot)

* final changes based on feedback - reduce string allocations

* fix bug in object-inl.h

* fix logic for wildcards inside arrays

* add test for wildcard in nested array

* refactor and create util for getting key and json path

* minor error handling

* refactor process_json_path_of_child_element from recursion to loop in order to prevent stack overflow

* fix bug with array, add boundary check to get_next_key_and_json_path function

* add more tests

* fix boundary check and unnecessary string allocation

* minor changes

* fix

* fix jsonpathutil

* remove unneccessary string allocation

* some minor fixes

* documentation

* removing printout

* various fixes

* reorg of the CI test file

---------

Co-authored-by: Daniel Lemire <daniel@lemire.me>
2025-08-13 21:16:32 -04:00
Francisco Geiman Thiesen 3e00a431a0 Merge pull request #2406 from simdjson/fix-convert-ci-failures
Introducing simplified api with from/to adapters (thanks to @the-moisrex for driving this) and fixing all the CI errors.
2025-08-08 10:42:32 -07:00
Daniel Lemire 4c0f86db44 minor fixes 2025-08-07 23:59:30 -04:00
Daniel Lemire b7c51156b9 Sped up serialization fix (#2409)
* clang format

* added `chars()` method

* implemented vector with small buffer instead of `std::vector`

* added missing <utility> header

* minor fixes

---------

Co-authored-by: Pavel Novikov <dev-ape@yandex.ru>
Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2025-08-07 16:04:20 -04:00
Daniel Lemire c806e955c4 C++26 static reflection (#2282)
* Initial work on JSON builder

* moving the files back to ondemand for now.

* tweak

* more later

* update

* minor edits

* dropping vs arm (missing support)

* adding tests. we still specialized write_string_escaped

* tweaking

* fix typo

* tweaking the approach

* minor fix

* missing store

* another missing store

* Attempt at fixing failing serialization tests. (#2292)

* Fixing appeand_float typo (#2294)

* applying a couple of fixes

* updating single header

* fix for pre C++17 if constexpr

* Fixing unused argument problem and updating the singleheader file

* various pedantic fixes

* Sketch of builder

* reordering.

* simplify

* Adding draft of static reflection based deserialization

* Updating simdjson singleheader

* patching the automated deserialization.

* automated

* Adding support for smart pointers of user defined types.

* Adding specialization for smart pointers for basic types. I think it is highly likely that this can be done in a more generic way.

* Referncing a later version of rapidjson that fixed the issue related with assignment attempt of a const variable for GenericStringRef class.

* guarding the tests

* adding documentation for string_builder

* saving

* rename to 'append'

* saving

* non-functional benchmarks (#2342)

* non-functional benchmarks

* Fix typo

* various fixes

* tweaking

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>

* tuning

* various minor fixes

* minor tweak

* minor simplification

* updating amal

* adding a cast

* update

* fancy casting

* removing dead code

* Pushing latest changes. CITM benchmark is still not working.

* Still not working, but now I am getting only 10 errors.

* add static reflection benchmark to 'large random' benchmark and allows (#2349)

deserialization (with static reflection) from objects and arrays.

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* Removing std::map from CitmCatalog definition, since that is not currently supported.

* Added free to rust bench, segfault is still happening..

* The syntax changed: ^E became ^^E. (#2350)

* The syntax changed: ^E became ^^E.

* guarding

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* Adding support for string_view_keyed_map types.

* Adding concepts as a conditional include.

* updating single-header

* Adding concepts to ondemand deps

* rust benchmark is finally working

* Fixing small typo in docs.

* adding docker config and instructions so that our users can test the static reflection (#2358)

* adding docker config and instructions so that our users can test the
static reflection

* completing the instructions

* pruning white spaces

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>

* minor optimizations on the JSON builder branch

* avoiding undef behaviour

* saving

* somewhat nicer builder

* make it possible to run just one benchmark

* adding linux perf

* fixing minor issue

* updating swar

* Adding real world compilation benchmark (#2379)

* Adding compilation benchmark for json parsing with and without reflection

* Moving it to the benchmark folder, also reducing a bit the number of iterations.

* Removing script from root folder.

* Reducing number of iterations

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Update benchmark/benchmark_reflection_usage_compilation.sh

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Making the script more customizable and also test whether the compiler being used supports reflection before actually running the benchmark

---------

Co-authored-by: Daniel Lemire <daniel@lemire.me>

* Using define_static_string from  https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3491r2.html (#2389)

* Applying changes needed after latest reflection paper updates.

* Working, but no template for yet.

* Updating single-header to incldue the use of define_static_string.

* copying over master

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>
2025-07-14 15:43:52 -04:00
Daniel Lemire 4acaf2ea1c improving DOM ranges test and silencing a warning (#2385)
* improving DOM ranges test and silencing a warning

* moving test_main to macros (even though it is not a macro)

* moving test_main
2025-07-04 15:46:50 -04:00
Cuda Chen 7d86b04247 Add document and tests of std::ranges support of DOM API (#2381)
* Add document and tests of std::ranges support of DOM API

* Alter lambda

* Fix require of range
2025-06-24 15:13:44 -04:00
Daniel Lemire a260c967ed adding test for issue 2375 (#2376)
* adding test for issue 2375

* update

* removing space
2025-06-02 09:24:40 -04:00
Daniel Lemire 16e390d81a fix for issue 2373 (#2374)
* fix for issue 2373

* bumping google bench

* removing json11 since dropbox retired it

* removing json11 as a dependency
2025-05-31 00:56:19 -04:00
Daniel Lemire 818c0491a1 Treat -0 as -0.0 when SIMDJSON_MINUS_ZERO_AS_FLOAT is set (#2364)
* In the DOM API, treat -0 as -0.0

* documenting...

* adding it to On-Demand
2025-04-18 13:56:02 -04:00
Daniel Lemire 6cd8a6d776 adding pad function (#2283) 2024-10-29 21:26:01 -04:00
Daniel Lemire c066b5421b Jsonpath (#2272)
* Add support for JSONPath with '$' prefix and integrate into dom::at_path (#2266)

- Updated `json_path_to_pointer_conversion` to support JSONPath starting with the '$' prefix, while maintaining compatibility with the existing implementation.
- Moved `json_path_to_pointer_conversion` to a separate header file for better modularity and to support JSONPath queries in `dom` mode.
- Implemented `at_path` functionality in `dom` mode to enable querying JSON using JSONPath.
- Added unit tests to validate the new JSONPath support in `dom` mode and ensure compatibility with both standard and existing JSONPath formats.

* two minor fixes

* more tests and documentation

* damn compiler warnings

* more documentation fixes

---------

Co-authored-by: Zhengguo Yang <yangzhgg@gmail.com>
2024-10-08 10:37:59 -04:00
didarpin 58173a6a1f Added the functionality to convert dom::object and dom::array to dom::element. (#2221)
Co-authored-by: didarpin <didarpin@163.com>
2024-07-25 22:26:20 -04:00
Daniel Lemire 5090247c34 adding test (#2214) 2024-07-13 11:40:13 -04:00
Daniel Lemire ac719827ff fix: solve issue 2181 (#2182) 2024-05-11 20:44:38 -04:00
pnck e2f879751c fix: issue #2154 (#2178) 2024-05-10 00:33:09 -04:00
Daniel Lemire 8519e24f12 issue 2170 (#2172) 2024-04-24 01:41:40 -04:00
Daniel Lemire 0c08894a54 adopting string literals (#2128)
* adopting string literals

* String suffixes are C++14
2024-02-15 10:40:11 -05:00
Daniel Lemire 32d072a208 fixing issue2120 (#2121) 2024-02-08 11:34:36 -05:00
Bo Anderson 33610b59eb Allow some iterators to be used in standard C++ iterator functions (#2106) 2024-02-05 17:52:01 -05:00
Bo Anderson 02bee7d1b9 Add load_into_document to simdjson::dom::parser (#2103) 2024-01-08 20:30:26 -05:00
Daniel Lemire 8b2af0473a skip UTF-8 BOM if present. (#2079) 2023-10-30 12:51:02 -04:00
Momtchil Momtchev e11ad58aad add a total order for elements (#2072) 2023-10-09 15:41:57 -04:00
Daniel Lemire b2e20e04c9 Standard compatibility fixes (#2053)
* Standard compatibility fixes

* missing commit

* Should work.

* Fix.

* Fix.

* Should work now.

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-08-25 10:18:02 -04:00
John Keiser 06afe9ecb3 New escape algorithm with significantly less data dependency 2023-08-02 08:28:05 -07:00
Daniel Lemire 65f9822542 Adding tests for issue 2017 (#2018) 2023-06-07 09:57:28 -04:00
Daniel Lemire 85661a2901 Some systems release without NDEBUG set, let us not assert in these cases. (#2014) 2023-06-01 16:18:36 -04:00
Daniel Lemire 7db3f97ba8 Adding CXX 20 to CI (#2005)
* Adding CXX 20 to CI

* side-stepping new CXX 20 guard.

* Going another way

* Saving.

* Explicit.

* Saving...

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-05-26 09:23:02 -04:00
Daniel Lemire 2a68a32012 Checking OS support for AVX-512 (#1998)
* Adding support for AVX-512 on macOS.

* Fix.

* Fix.

* Minor fix

* Setting the variable to zero.

* Fixing include

* Checking if the OS supports AVX-512

* Tweaking.

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-05-12 19:25:02 -04:00
Daniel Lemire 21a2ccaeb8 Another fun one 2023-05-05 22:12:59 -04:00
Daniel Lemire 3c29f40f2f Trying to verify issue 1979 (#1980) 2023-04-04 11:03:37 -04:00
Daniel Lemire 75c6c950e5 Ok. 2023-03-13 21:26:32 -04:00
Daniel Lemire 338224849d Disabling memory sanitizer with one function. Might help with issue 1965 (#1966)
* Might help with issue 1965

* Fix macro

* Adding test.

* Stupid compiler

* Silly compiler

* Stupid compilers

* Unnecessary fixes

* Update developer-options.cmake
2023-03-13 12:34:47 -04:00
Daniel Lemire ede9d57f22 Adding tests with _GLIBCXX_ASSERTIONS (#1969)
* Allowing users to specify -D_GLIBCXX_ASSERTIONS

* Adding CI tests for GLIBCXX_ASSERTIONS

* Adding curly brackets.
2023-03-13 12:32:43 -04:00
Daniel Lemire 33dbd44098 Better way to put -Og in debug mode (#1967)
* Better way to put -Og in debug mode

* Silly compiler

* Stupid compiler.

* Stupid compiler.

* Silly compiler
2023-03-13 10:11:33 -04:00
Daniel Lemire e0dcf8adc9 Silly compiler 2023-03-12 20:57:45 -04:00
Daniel Lemire 17a1a8e187 Adding -Og to debug builds. (#1964)
* Adding -Og to debug builds.

* Stupid compiler.

* bad, bad, bad compiler
2023-03-09 11:14:42 -05:00
Daniel Lemire 25e7f1765a Various tweaks. 2023-03-09 10:42:23 -05:00
Daniel Lemire b9234be80f complete number_in_string for documents, stringify the version macro and improving trailing content errors (#1934)
* This completes the '*_in_string' support within documents.

* Minor updates.

* Additional fixes.
2023-01-21 15:29:08 -05:00
Daniel Lemire f151300a76 Disabling fallback kernel on systems where it is not needed (#1930)
* Fixing issue 1772

* More cleaning.

* Allow disabling other kernels.

* Minor tweaks.
2023-01-03 09:52:41 -05:00
Daniel Lemire db08d78ed0 Documents better the type method and makes is_null return an error condition in some instances (#1909) 2022-10-06 11:47:40 -04:00
Daniel Lemire b92cbbe280 Fixing issue 1898 (#1899)
* Fixing issue 1898 Preserve sign for number with underflowing exponent (#1900)


Before this commit, simdjson parsed "-1e-999" and "-0e-999" and "-1e-342"
as 0.0.
After this commit, those JSON strings get parsed as -0.0.
(https://en.wikipedia.org/wiki/Signed_zero)

The old behavior was inconsistent with the way simdjson parsed "-0.0" as -0.0.

Co-authored-by: Daniel Lemire <daniel@lemire.me>

Co-authored-by: Tyson Andre <tysonandre775@hotmail.com>
2022-10-02 16:25:39 -04:00
Daniel Lemire 16b3816455 This fixes an error caused by overeager gcc static analyser (#1891) 2022-09-19 10:20:48 -04:00
Daniel Lemire 3fde8a4eac New release candidate (#1856)
* Patch for possible AVX-512 overflow.

* Updating the test for new padding.

* Preparing new version.

* replace binary integer literals with hex literals for C++11 compatibility (#1855)

Binary integer literals are a C++14 feature, so those are not supported
in C++11 and should be replaced by hexadecimal literals instead.

Fixes #1854.

Co-authored-by: Dirk Stolle <striezel-dev@web.de>
2022-07-04 21:54:19 -04:00
Daniel Lemire c6f9c93c33 Use static variables to enforce initialization order. (#1773) (#1777)
Fixes https://github.com/simdjson/simdjson/issues/1771

Co-authored-by: Hao Chen <chenhao.yalier@gmail.com>
2022-01-03 15:54:57 -05:00
N. Kolotov 7e1893db42 Added SIMDJSON_ prefix to STRINGIFY and to NO_SANITIZE_UNDEFINED macros. (#1766) 2021-12-21 09:53:06 -05:00
Daniel Lemire 6d308a08c5 Fixing issue 1736 (#1737)
* Fixing issue 1736

* Updating google benchmark.

* Minor trimming.

* Using the variable (to silence a warning).

* Adding assignment operator.
2021-10-20 12:15:35 -04:00