Commit Graph

76 Commits

Author SHA1 Message Date
Daniel Lemire 3ef3078e51 documentation for issue 2259 (clarifying) (#2260) 2024-09-21 09:10:35 -04:00
Daniel Lemire d506af0a79 fix: add tests related to issue 2227 (#2229)
* fix: add tests related to issue 2227

* avoiding name clash

* pedantic fix

* deprecate rvalue get on document

* selectively deprecating
2024-08-07 20:15:06 -04:00
Daniel Lemire a33fdc2770 Unconfusing clang (#2147)
* In some cases, clang might try to cast an ondemand::document to an
ondemand::document, instead of calling the move constructor. So we
can disable the template cast.

* In some cases, clang might get confused when constructing an ondemand
document in a constructor: instead of calling the move constructor, it
somehow ends up trying to cast a document to a document. We can easily
disallow this behavior with std::enable_if.

* making compatible with C++11

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2024-03-10 13:20:39 -04:00
Daniel Lemire 87f5f7a250 added some documentation in the code related to custom types (#2146) 2024-03-09 18:45:05 -05:00
Daniel Lemire 2028bdaeac This documents the big-int feature, and adds a few tests. (#2142)
* This documents the big-int feature, and adds a few tests.

* moving check_if_integer

* trimming the example.

* More trimming.

---------

Co-authored-by: Daniel Lemire <dlemire@lemire.me>
2024-02-26 20:39:20 -05:00
Daniel Lemire 37a9aaad48 [no ci] tweaking the wording 2024-02-26 14:05:12 -05:00
Serge Aleynikov 78325888ea Add functionality to detect big integers (#2139) 2024-02-26 14:00:33 -05:00
Daniel Lemire aef7111c05 Documenting Unicode support in JSONPath and JSON Pointer (#2132)
* Documenting our support of JSON Path and JSON Pointer vs. Unicode characters

* using more standard terminology (nitpicking)

* specifying UTF-8 encoding

* allow char8_t when compiling as C++20

* casting

* minor doc corrections
2024-02-15 20:28:38 -05:00
Daniel Lemire 2fe3a1467d This PR adds a convenience method (is_string) as well as a new method (#2131)
key_raw_json_token() to get the raw key token.
2024-02-15 10:39:30 -05:00
Francisco Geiman Thiesen 2b532489cc Adding on-demand support for a subset of json path. (#2127)
* First implementation of the conversion logic.

- Not yet tested
- Not yet compiled
- Not yet used in ondemand array/object/etc...

* Adding at_path to ondemand arrays

* Adding at_path to array, document and value.

* Adding at_path to ondemand object as well.

Pending:
- Building
- Adding tests for each of the on-demand classes usage of at_path
- Properly documenting the subset of json path that is currently supported.

* Adding a simple json_path test to on_demand

* Still trying to compile the ondemand_readme_examples test with the at_path() call

* Fixing linking issues with array::at_path

* Fixing issues with the path -> pointer conversion and removing comments

* Adding a clone of json_pointer to test json_path extensively

* Adding ondemand_json_path_tests (all tests passing)

* Adding documentation for at_path()

* Removing some newlines

* Removing simdjson_result from the return type of the string conversion function. Now json_path_to_pointer_conversion returns a std::string.

* Fixing typo

* Making string concatenation explicit to avoid ubuntu gcc12 issue with -O3 flag.

* Addressing latest reviews

* Addressing latest reviews
2024-02-15 09:47:25 -05:00
Daniel Lemire 5eab555053 documenting the support for custom types (#2098)
* documenting the support for custom types

* fixes

* getting around -Weffc++

* calling get_raw_json_string()

* documenting the support for custom types

* fixes

* getting around -Weffc++

* calling get_raw_json_string()
2024-02-06 13:20:35 -05:00
Daniel Lemire 3bb2f0a186 Return a better error when a user tries to cast a document that has already been accessed to a value (#2085)
* Fix.

* adding a test, cleaning some comments, adding documentation

* Correcting the JSON input.

* adding another test
2023-11-26 14:13:05 -05:00
Daniel Lemire baa7d961ce Allowing users to write directly to std::optional<std::string> (#2078)
* Allowing users to write directly to std::optional<std::string>

* better fallback

* do not force the cast to std::string

* missing header

* removing abort
2023-10-27 20:56:12 -04:00
Daniel Lemire 993ac4b87c Adding the ability to write directly to an std::string. It is mostly syntaxic sugar (#2075) 2023-10-25 19:32:12 -04:00
Daniel Lemire 13405afd4b This provides verbose static_assert messages (#2074)
* This provides verbose static_assert messages

* minor fix
2023-10-20 15:42:18 -04:00
John Keiser ae9ba2bed2 Rename SIMDJSON_AMALGAMATED to SIMDJSON_CONDITIONAL_INCLUDE 2023-07-20 11:43:19 -07:00
John Keiser a442842f0d Use SIMDJSON_AMALGAMATED (which works in-editor and out) instead of editor-specific tweaks 2023-07-15 14:53:02 -07:00
John Keiser 9a0527c380 Only make editor stuff work in editor 2023-07-14 20:05:05 -07:00
John Keiser bf70cd0ebf Get headers compiling in editor 2023-07-08 10:47:15 -07:00
John Keiser 53dad74d8e Don't put number_type into ondemand 2023-07-05 14:26:23 -07:00
Daniel Lemire cfd337bb50 This PR adds an 'at_end()' method. (#1978)
* This PR adds an 'at_end()' method.

* Adding 1111 }

* Tweaking test.
2023-03-30 10:47:13 -04:00
Daniel Lemire 37e87f6f99 Adding support for wobbly strings to On Demand and lossy trancoding from escaped (with replacement) (#1947)
* Adding support for wobbly strings.

* Minor tweak.

* Adopt substitution code point.

* Lossy

* Patching
2023-02-28 22:56:44 -05:00
Daniel Lemire b9234be80f complete number_in_string for documents, stringify the version macro and improving trailing content errors (#1934)
* This completes the '*_in_string' support within documents.

* Minor updates.

* Additional fixes.
2023-01-21 15:29:08 -05:00
Daniel Lemire db08d78ed0 Documents better the type method and makes is_null return an error condition in some instances (#1909) 2022-10-06 11:47:40 -04:00
strager 5510089d45 Improve build times for debug builds (#1859)
* Rename simdjson_really_inline -> simdjson_inline

I want to change the simdjson_really_inline macro to sometimes not force
inlining. After that upcoming change, the name simdjson_really_inline
will no longer makes sense.

Rename simdjson_really_inline to simdjson_inline. This patch should not
change semantics; simdjson_inline still forces inlining as before.

Some functions still need to be really inlined for ABI reasons.
(GCC's -Wpsabi complains otherwise.) Leave those functions marked as
simdjson_really_inline.

* Improve build times for debug builds

simdjson_inline is used for most simdjson functions. It forces inlining.
In unoptimized/debug builds, this can lead to a lot of machine code
being generated (especially with Address Sanitizer), causing slow
compilation.

Change simdjson_inline to force inlining only for optimized builds.

Sometimes, the programmer might want a slightly-optimized build and want
fast compilation (e.g. GCC's -Og mode). Allow simdjson users to define
the simdjson_inline macro themselves (e.g. on the command line:
-Dsimdjson_inline=inline) in cases where the default behavior is
undesired.

This patch reduced build times by over 75% for ondemand_object_tests.cpp
with GCC 9.4.0 and CMAKE_BUILD_TYPE=Debug on my AMD 5950X:

Before: 6.885 6.683 6.971 6.957 6.949 seconds (5 samples)
After:  1.492 1.551 1.494 1.490 1.531 seconds (5 samples)
2022-07-19 15:14:33 -04:00
Daniel Lemire 8c1bfe782b Adds 'current_depth()'. (#1824) 2022-05-17 18:06:42 -04:00
Daniel Lemire 730939f01c Better documentation regarding the direct get_number_type() method. (#1821) 2022-05-13 15:31:09 -04:00
Daniel Lemire b4b1927b0f Making the documentation more explicit. (#1765) 2021-12-21 09:51:20 -05:00
epoll-reactor 69ee84973a Fix some typos in source files. (#1761)
Co-authored-by: epoll-reactor <>
2021-12-06 16:56:15 -05:00
Daniel Lemire 1b01969bda Adding tests for issue 1745. (#1746)
* Adding tests for issue 1745.

* Tweaking the documentation so that it is clearer.
2021-11-03 10:12:18 -04:00
Daniel Lemire 6d308a08c5 Fixing issue 1736 (#1737)
* Fixing issue 1736

* Updating google benchmark.

* Minor trimming.

* Using the variable (to silence a warning).

* Adding assignment operator.
2021-10-20 12:15:35 -04:00
Daniel Lemire 91908ade4d Additional documentation following issue 1723 (#1724)
* Some extra documentation regarding issue 1723.

* Adding comments.

* Minor fix.

* [no ci] more documentation
2021-10-09 11:41:20 -04:00
Daniel Lemire af4ff7cc33 Adding fast "get_number_type()" function, bypassing "get_number()" (#1713)
* Adding fast "get_number_type" function, bypassing "get_number"

* Minor tweak.

* Adding missing get_number_type().
2021-09-07 14:34:40 -04:00
Nicolas Boyer c9179ad81d Add count_fields method for objects (#1712)
* Implement count_elements for object

* Add count_elements() for simdjson_result

* Add count_elements for documents(arrays,objects).

* Add tests for objects.

* Add tests for documents array. Typos.

* Renaming to count_fields() for objects.

* Update doc.

* Apply patch
2021-09-02 16:18:48 -04:00
Nicolas Boyer ed7343f7f2 Provide current location in JSON input (#1695)
* Setup.

* Add current_location().

* Make return simdjson_result and fix cast issues.

* Whitespace.

* Add broken JSON tests. Add null parser check.

* Remove unused variables.

* Alive fix.

* Fix merge issues.

* Simplification for out of bounds.

* More tests.

* Move pointer back for unrecoverable errors.

* Add new error OUT_OF_BOUNDS

* Remove unnecessary include and fix OUT_OF_BOUNDS.

* Add more tests. Fix unrecoverable errors.

* Fix tests.

* Modify one test.

* Update doc.

* Typos.

* Add read_me tests.

* Update doc.

* Add current_location for simdjson_result and document_reference

* Typos.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-08-27 13:41:59 -04:00
Daniel Lemire 35158257c6 Implementing get_number for the document instances. (#1706) 2021-08-27 10:26:01 -04:00
Daniel Lemire 0ad52a7e22 Renaming scalar to is_scalar. (#1698) 2021-08-21 10:23:22 -04:00
Daniel Lemire aa52cf6868 Alive fix. (#1700) 2021-08-21 10:22:59 -04:00
Daniel Lemire de4deb8c4e Makes it possible to cast a document to a value. (#1690)
* Makes it possible to cast a document to a value.
2021-08-11 20:02:30 -04:00
Daniel Lemire 06643fc9f5 Additional tests and document tuning (#1684)
* Additional example.

* Adds more tests.

* Actually using the variable.
2021-08-02 16:35:02 -04:00
Daniel Lemire eb93b98d6a verify and fix issue 1668 (#1673)
* Adding test.

* Verifies and fix issue 1668. This commit updates the previous behavior of the
On Demand stream support by return a value type (document_reference) instead
of a reference to a document. This allows us to bridge with the usually simdjson
error system, with its simdjson_result types.

* Minor reformat.

* Adds a test with initial tests passing.

* Adding an example.
2021-07-27 08:51:07 -04:00
Nicolas Boyer 7d887fdc1e Parse numbers inside strings (#1667)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Naive implementation for doubles in string.

* Add double from string in atom doc.

* Simplification (removed all *_from_string())

* Add int and uint parsing in string.

* Make duplicates instead.

* Make tests exceptionless.

* Add missing declarations.

* Add more tests (errors, JSON pointer).

* Add crypto json tests.

* Update doc.

* Update doc after review.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-27 08:50:44 -04:00
Daniel Lemire 47a62db559 Isolated jkeiser fix for issue 1632: make it so that INCORRECT_TYPE is a recoverable condition in On Demand (#1663) 2021-07-23 11:32:26 -04:00
Nicolas Boyer 5c590b8434 Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire 2dac3705d2 renames 'to_string' to 'to_json_string' and makes it ridiculously fast (#1642)
* Changing the name of the function to 'to_json_string' from 'to_string' to avoid confusion.

* Moving to a fast string_view model

* Making it exception-safe.

* Tweaking.

* Workaround for exceptions.

* more robust to_json_string (#1651)

* WIP.

* Fuzzing timeout  (bug fix) (#1650)

* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

* WIP.

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

Co-authored-by: Paul Dreik <github@pauldreik.se>

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-19 10:24:36 -04:00
Nicolas Boyer eb849662c0 Update basic.md to document JSON pointer for On Demand. (#1618)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-06-26 11:38:17 -04:00
Daniel Lemire 1c01fc35eb This better documents invalidation. (#1625)
* This better documents invalidation.

* Tweak.
2021-06-22 11:33:25 -04:00
Nicolas Boyer ce38fe7bea Add automatic rewind for at_pointer (#1624) 2021-06-21 15:17:24 -04:00
Nicolas Boyer a4803d50c5 Add JSON Pointer for On Demand (#1615)
* Add working JSON pointer for array of atoms.

* Add working JSON pointer for object with key-atom pairs.

* Add first version of JSON pointer.

* Update tests (2 tests).

* Make tests exceptionless.

* Fix builing issues.

* Add more tests. Add json_pointer validation in array-inl.h and object-inl.h and empty json_pointer in document-inl.h.

* Fix errors in tests.

* Review.

* Add missing comment.
2021-06-11 14:20:05 -04:00
Daniel Lemire 16e8db1f17 Adding 'count_elements' method. (#1577)
* Adding 'count_elements' method.

* Actually reporting errors.

* removing white space.

* Removing white space again.

* Adding an extra example.

* Prettier.

* Making the functionality more error-proof.

* Avoiding exceptions.

* Various fixes including extending count_elements to value types.

* Various fixes.

* Minor fixes.

* Correcting comment.

* Trimming white spaces.
2021-06-06 17:56:00 -04:00