Commit Graph

36 Commits

Author SHA1 Message Date
Daniel Lemire 24068a089e Merge branch 'master' into dlemire/backport_jkeiser_fix_for_issue1632 2021-07-22 12:56:18 -04:00
Nicolas Boyer 5c590b8434 Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire e8e731a311 Merge branch 'master' into dlemire/backport_jkeiser_fix_for_issue1632 2021-07-19 12:17:17 -04:00
Daniel Lemire 3c5c2d2404 This backports (isolate) a fix by jkeiser of issue 1632. 2021-07-19 11:13:37 -04:00
Daniel Lemire 2dac3705d2 renames 'to_string' to 'to_json_string' and makes it ridiculously fast (#1642)
* Changing the name of the function to 'to_json_string' from 'to_string' to avoid confusion.

* Moving to a fast string_view model

* Making it exception-safe.

* Tweaking.

* Workaround for exceptions.

* more robust to_json_string (#1651)

* WIP.

* Fuzzing timeout  (bug fix) (#1650)

* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

* WIP.

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

Co-authored-by: Paul Dreik <github@pauldreik.se>

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-19 10:24:36 -04:00
Nicolas Boyer eb849662c0 Update basic.md to document JSON pointer for On Demand. (#1618)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-06-26 11:38:17 -04:00
Daniel Lemire 1c01fc35eb This better documents invalidation. (#1625)
* This better documents invalidation.

* Tweak.
2021-06-22 11:33:25 -04:00
Nicolas Boyer ce38fe7bea Add automatic rewind for at_pointer (#1624) 2021-06-21 15:17:24 -04:00
Nicolas Boyer a4803d50c5 Add JSON Pointer for On Demand (#1615)
* Add working JSON pointer for array of atoms.

* Add working JSON pointer for object with key-atom pairs.

* Add first version of JSON pointer.

* Update tests (2 tests).

* Make tests exceptionless.

* Fix builing issues.

* Add more tests. Add json_pointer validation in array-inl.h and object-inl.h and empty json_pointer in document-inl.h.

* Fix errors in tests.

* Review.

* Add missing comment.
2021-06-11 14:20:05 -04:00
Daniel Lemire 16e8db1f17 Adding 'count_elements' method. (#1577)
* Adding 'count_elements' method.

* Actually reporting errors.

* removing white space.

* Removing white space again.

* Adding an extra example.

* Prettier.

* Making the functionality more error-proof.

* Avoiding exceptions.

* Various fixes including extending count_elements to value types.

* Various fixes.

* Minor fixes.

* Correcting comment.

* Trimming white spaces.
2021-06-06 17:56:00 -04:00
Daniel Lemire eb0ae041e3 Verification and bug fix of issue 1511 (#1602)
* Verification and bug fix.

* Removing comment.

* Removing spaces.

* Guarding exceptions.

* Tweaking the test
2021-06-06 17:55:33 -04:00
Daniel Lemire 19c3b1315a Rewind functionality. (#1539)
* Rewind functionality.


* Keeping just the document rewind.
2021-06-04 09:22:33 -04:00
Daniel Lemire 62cd5f7984 get_root_value is dead code that should have been removed. 2021-03-18 11:30:40 -04:00
Daniel Lemire 02f9b83353 This moves us to On Demand as the default front-end. (#1494)
* This moves us to On Demand as the default front-end.

* Made casting magical

* Adding another section

* Undoing my damage.
2021-03-12 14:19:11 -05:00
John Keiser bad582c2d3 Add value.raw_json_token() 2021-03-05 09:07:41 -08:00
John Keiser 9944db6d73 Move json_type to ondemand to prevent target mismatch inline errors 2021-03-02 18:31:17 -08:00
John Keiser 2ed24666b5 Add value.type() 2021-03-02 17:02:50 -08:00
Daniel Lemire ad37651726 Guarding undefined templates with a static_assert. (#1454)
* Guarding undefined templates with a static_assert.

* Fixing comments.

* Undeprecating (or whatever you want to call it).
2021-02-22 16:54:47 -05:00
Daniel Lemire 81609393f1 Fixing issue 1449. (#1451) 2021-02-21 16:33:05 -05:00
John Keiser 55faf4c5bc Recommend simdjson::ondemand over simdjson::builtin::ondemand (#1380)
Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-01-14 17:33:49 -05:00
John Keiser 3279c2f15b Remove unnecessary "try_get_XXX" methods
Also don't distinguish between & and && (instead, advance the first time
you encounter a scalar no matter what)
2021-01-11 15:19:26 -08:00
John Keiser 195acc3e45 Add find_field / find_field_unordered to object 2020-12-20 11:39:50 -08:00
John Keiser d670906ff3 Don't keep a separate at_start boolean in object 2020-12-15 11:29:31 -08:00
John Keiser f9c6dedca1 Add test for out-of-order parse asserts 2020-12-13 13:39:47 -08:00
John Keiser 73c7b7db3d Safety: check index of scalar values before use 2020-12-11 11:57:09 -08:00
John Keiser 3aa3175378 Add tests for recovering from partial child iteration 2020-12-06 16:19:06 -08:00
John Keiser 4c63956624 Enable object["x"]["y"] 2020-12-06 15:23:52 -08:00
John Keiser c89647af9e Make all values use same underlying iterator 2020-12-06 15:23:52 -08:00
John Keiser 1303a88769 Unconditionally track depth 2020-12-06 15:23:52 -08:00
Paul Dreik af4db55e66 remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
John Keiser 5b926b8196 Support array iteration over document 2020-10-06 11:29:45 -07:00
John Keiser 9f1786aeb1 Add .get(T) to value/document 2020-10-06 11:29:45 -07:00
John Keiser ba02cda55f Allow direct document iteration 2020-10-06 11:29:45 -07:00
John Keiser 88f0dc4726 Add API docs, ensure parser and field methods called with & 2020-10-06 11:29:45 -07:00
John Keiser 4859cb8528 Add SIMDJSON_ONDEMAND_SAFETY_RAILS 2020-10-06 11:29:45 -07:00
John Keiser a700848bae Move ondemand implementation to include/ 2020-10-04 12:47:30 -07:00