Commit Graph

55 Commits

Author SHA1 Message Date
Daniel Lemire c066b5421b Jsonpath (#2272)
* Add support for JSONPath with '$' prefix and integrate into dom::at_path (#2266)

- Updated `json_path_to_pointer_conversion` to support JSONPath starting with the '$' prefix, while maintaining compatibility with the existing implementation.
- Moved `json_path_to_pointer_conversion` to a separate header file for better modularity and to support JSONPath queries in `dom` mode.
- Implemented `at_path` functionality in `dom` mode to enable querying JSON using JSONPath.
- Added unit tests to validate the new JSONPath support in `dom` mode and ensure compatibility with both standard and existing JSONPath formats.

* two minor fixes

* more tests and documentation

* damn compiler warnings

* more documentation fixes

---------

Co-authored-by: Zhengguo Yang <yangzhgg@gmail.com>
2024-10-08 10:37:59 -04:00
Daniel Lemire acdbbab916 adding an example of value capture with std::string_view (#2216)
* adding an example of value capture with std::string_view

* minor fix

* minor fix
2024-07-16 15:51:43 -04:00
Daniel Lemire 3e94eea939 adding another example (#2206)
Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2024-06-27 13:22:44 -04:00
Daniel Lemire 2028bdaeac This documents the big-int feature, and adds a few tests. (#2142)
* This documents the big-int feature, and adds a few tests.

* moving check_if_integer

* trimming the example.

* More trimming.

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2024-02-26 20:39:20 -05:00
Serge Aleynikov 78325888ea Add functionality to detect big integers (#2139) 2024-02-26 14:00:33 -05:00
Daniel Lemire aef7111c05 Documenting Unicode support in JSONPath and JSON Pointer (#2132)
* Documenting our support of JSON Path and JSON Pointer vs. Unicode characters

* using more standard terminology (nitpicking)

* specifying UTF-8 encoding

* allow char8_t when compiling as C++20

* casting

* minor doc corrections
2024-02-15 20:28:38 -05:00
Daniel Lemire 0c08894a54 adopting string literals (#2128)
* adopting string literals

* String suffixes are C++14
2024-02-15 10:40:11 -05:00
Daniel Lemire 2fe3a1467d This PR adds a convenience method (is_string) as well as a new method (#2131)
key_raw_json_token() to get the raw key token.
2024-02-15 10:39:30 -05:00
Francisco Geiman Thiesen 2b532489cc Adding on-demand support for a subset of json path. (#2127)
* First implementation of the conversion logic.

- Not yet tested
- Not yet compiled
- Not yet used in ondemand array/object/etc...

* Adding at_path to ondemand arrays

* Adding at_path to array, document and value.

* Adding at_path to ondemand object as well.

Pending:
- Building
- Adding tests for each of the on-demand classes usage of at_path
- Properly documenting the subset of json path that is currently supported.

* Adding a simple json_path test to on_demand

* Still trying to compile the ondemand_readme_examples test with the at_path() call

* Fixing linking issues with array::at_path

* Fixing issues with the path -> pointer conversion and removing comments

* Adding a clone of json_pointer to test json_path extensively

* Adding ondemand_json_path_tests (all tests passing)

* Adding documentation for at_path()

* Removing some newlines

* Removing simdjson_result from the return type of the string conversion function. Now json_path_to_pointer_conversion returns a std::string.

* Fixing typo

* Making string concatenation explicit to avoid ubuntu gcc12 issue with -O3 flag.

* Addressing latest reviews

* Addressing latest reviews
2024-02-15 09:47:25 -05:00
Daniel Lemire 5eab555053 documenting the support for custom types (#2098)
* documenting the support for custom types

* fixes

* getting around -Weffc++

* calling get_raw_json_string()

* documenting the support for custom types

* fixes

* getting around -Weffc++

* calling get_raw_json_string()
2024-02-06 13:20:35 -05:00
Daniel Lemire baa7d961ce Allowing users to write directly to std::optional<std::string> (#2078)
* Allowing users to write directly to std::optional<std::string>

* better fallback

* do not force the cast to std::string

* missing header

* removing abort
2023-10-27 20:56:12 -04:00
Daniel Lemire 503da5ec83 raw_json() on values 2023-10-27 16:22:22 -04:00
Daniel Lemire 6db330c1a2 Automating string reallocation when it is possible (#2076)
* Automating string reallocation

* Typo
2023-10-25 19:34:52 -04:00
Daniel Lemire 993ac4b87c Adding the ability to write directly to an std::string. It is mostly syntaxic sugar (#2075) 2023-10-25 19:32:12 -04:00
Daniel Lemire ff77ac801e Additional documentation based on issue 2063 (#2065)
* Additional documentation based on issue 2063

* Update basics.md
2023-09-11 10:01:35 -04:00
Daniel Lemire 2934892c0b Improving documentation. (#2020)
* Improving documentation.

* Fix typo

* Moving the definition.

* [skip ci] typo
2023-06-20 13:51:53 -04:00
Daniel Lemire 7db3f97ba8 Adding CXX 20 to CI (#2005)
* Adding CXX 20 to CI

* side-stepping new CXX 20 guard.

* Going another way

* Saving.

* Explicit.

* Saving...

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2023-05-26 09:23:02 -04:00
Daniel Lemire 6d92563280 Fixing issue 1943 (#2006)
* Fixing issue 1943

* Tweaking.
2023-05-23 11:15:22 -04:00
Daniel Lemire 0998233d58 More documentation. (#2007) 2023-05-23 11:15:07 -04:00
Daniel Lemire b5cf53232c More documentation. 2023-04-06 10:52:18 -04:00
Daniel Lemire cfd337bb50 This PR adds an 'at_end()' method. (#1978)
* This PR adds an 'at_end()' method.

* Adding 1111 }

* Tweaking test.
2023-03-30 10:47:13 -04:00
Daniel Lemire 59025bc8b1 Avoiding the issue where doc.get_value().get_object() differs from doc.get_object() in how errors are reported. (#1975)
* Avoiding the issue where doc.get_value().get_object() differs from doc.get_object() in how errors are reported.

* Minor tweaks
2023-03-24 12:44:18 -04:00
Daniel Lemire 3f3f6cde7c Bad compiler 2023-03-13 13:33:20 -04:00
Daniel Lemire 33dbd44098 Better way to put -Og in debug mode (#1967)
* Better way to put -Og in debug mode

* Silly compiler

* Stupid compiler.

* Stupid compiler.

* Silly compiler
2023-03-13 10:11:33 -04:00
Daniel Lemire 4a587cb898 Adding guards. 2023-03-06 16:04:42 -05:00
Daniel Lemire d65acbd47b Tuning documentation. 2023-03-02 09:39:35 -05:00
Daniel Lemire 12a1b6db53 This adds some examples to the documentation. (#1959)
* This adds some examples to the documentation.

* Fix
2023-02-28 22:59:42 -05:00
Daniel Lemire db08d78ed0 Documents better the type method and makes is_null return an error condition in some instances (#1909) 2022-10-06 11:47:40 -04:00
Daniel Lemire 1a195623a5 Improve string performance in ondemand by making the string processing runtime dispatched. (#1849)
* This should improve string performance in ondemand by making the string processing runtime dispatched.
2022-06-24 09:57:16 -04:00
Daniel Lemire e8f370b085 Basic AVX-512 implementation (icelake or better) (#1813)
* Add cascadelake implementation, which use AVX512 Intrinsics to optimize performance(#1811)

Co-authored-by: mellonyou <fangzheng.zhang@intel.com>
Co-authored-by: wanweiqiangintel <weiqiang.wan@intel.com>
2022-05-25 11:14:02 -04:00
Daniel Lemire 8c1bfe782b Adds 'current_depth()'. (#1824) 2022-05-17 18:06:42 -04:00
Daniel Lemire af4ff7cc33 Adding fast "get_number_type()" function, bypassing "get_number()" (#1713)
* Adding fast "get_number_type" function, bypassing "get_number"

* Minor tweak.

* Adding missing get_number_type().
2021-09-07 14:34:40 -04:00
Nicolas Boyer c9179ad81d Add count_fields method for objects (#1712)
* Implement count_elements for object

* Add count_elements() for simdjson_result

* Add count_elements for documents(arrays,objects).

* Add tests for objects.

* Add tests for documents array. Typos.

* Renaming to count_fields() for objects.

* Update doc.

* Apply patch
2021-09-02 16:18:48 -04:00
Daniel Lemire cebe3fb299 Tweaking current_location(). (#1707)
* Tweaking current_location().

* Well.
2021-08-28 20:19:30 -04:00
Nicolas Boyer ed7343f7f2 Provide current location in JSON input (#1695)
* Setup.

* Add current_location().

* Make return simdjson_result and fix cast issues.

* Whitespace.

* Add broken JSON tests. Add null parser check.

* Remove unused variables.

* Alive fix.

* Fix merge issues.

* Simplification for out of bounds.

* More tests.

* Move pointer back for unrecoverable errors.

* Add new error OUT_OF_BOUNDS

* Remove unnecessary include and fix OUT_OF_BOUNDS.

* Add more tests. Fix unrecoverable errors.

* Fix tests.

* Modify one test.

* Update doc.

* Typos.

* Add read_me tests.

* Update doc.

* Add current_location for simdjson_result and document_reference

* Typos.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-08-27 13:41:59 -04:00
Daniel Lemire b935ce2e06 Allowing casts instead of get_double, get_uint64 and get_int64 (#1705) 2021-08-27 10:25:17 -04:00
Daniel Lemire 4afe7565b4 ondemand dynamically-typed numbers (#1704)
* Building up a number type.

* Implemented is_integer and is_negative.

* Implemented get_number in value_iterator.

* Final prototype.

* [no ci] typo
2021-08-26 12:16:44 -04:00
Daniel Lemire 6bed34ad61 This exposes 'reset' for object and array instances. (#1696)
* This exposes 'rewind' for object and array instances.

* Putting really_inline back to count_elements()

* Update array.h

* Adding empty array rewind.

* Adds "is_empty" method to arrays.

* More fragmentation.

* Tweaking implementation.

* Fixing issue with get_value() on document instances.

* Changing the name of the new rewind functions to reset.
2021-08-21 10:23:59 -04:00
Daniel Lemire de4deb8c4e Makes it possible to cast a document to a value. (#1690)
* Makes it possible to cast a document to a value.
2021-08-11 20:02:30 -04:00
Daniel Lemire 19902abaf8 Guarding first/second access. (#1688)
* Guarding first/second access.

* Correcting our own usage.

* Adding more documentation.
2021-08-06 20:25:05 -04:00
Daniel Lemire 06643fc9f5 Additional tests and document tuning (#1684)
* Additional example.

* Adds more tests.

* Actually using the variable.
2021-08-02 16:35:02 -04:00
Daniel Lemire cc98358453 Adding error handing examples to the documentation (#1679)
* Adding error handing examples.

* Guarding the exception-throwing test.
2021-07-31 14:31:48 -04:00
Daniel Lemire d83e69d977 Fix an issue with truncated-byte function. (#1674) 2021-07-30 13:12:42 -04:00
Daniel Lemire c6ef2105ab Minor tweak. 2021-07-27 10:56:05 -04:00
Daniel Lemire eb93b98d6a verify and fix issue 1668 (#1673)
* Adding test.

* Verifies and fix issue 1668. This commit updates the previous behavior of the
On Demand stream support by return a value type (document_reference) instead
of a reference to a document. This allows us to bridge with the usually simdjson
error system, with its simdjson_result types.

* Minor reformat.

* Adds a test with initial tests passing.

* Adding an example.
2021-07-27 08:51:07 -04:00
Nicolas Boyer 5c590b8434 Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire 2dac3705d2 renames 'to_string' to 'to_json_string' and makes it ridiculously fast (#1642)
* Changing the name of the function to 'to_json_string' from 'to_string' to avoid confusion.

* Moving to a fast string_view model

* Making it exception-safe.

* Tweaking.

* Workaround for exceptions.

* more robust to_json_string (#1651)

* WIP.

* Fuzzing timeout  (bug fix) (#1650)

* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

* WIP.

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

Co-authored-by: Paul Dreik <github@pauldreik.se>

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-19 10:24:36 -04:00
Nicolas Boyer eb849662c0 Update basic.md to document JSON pointer for On Demand. (#1618)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-06-26 11:38:17 -04:00
Daniel Lemire 16e8db1f17 Adding 'count_elements' method. (#1577)
* Adding 'count_elements' method.

* Actually reporting errors.

* removing white space.

* Removing white space again.

* Adding an extra example.

* Prettier.

* Making the functionality more error-proof.

* Avoiding exceptions.

* Various fixes including extending count_elements to value types.

* Various fixes.

* Minor fixes.

* Correcting comment.

* Trimming white spaces.
2021-06-06 17:56:00 -04:00
Daniel Lemire 19c3b1315a Rewind functionality. (#1539)
* Rewind functionality.


* Keeping just the document rewind.
2021-06-04 09:22:33 -04:00