Commit Graph

38 Commits

Author SHA1 Message Date
Daniel Lemire 4e1e002cb0 This verifies and fixes issue 1834. (#1843) 2022-06-15 13:42:04 -04:00
Dirk Stolle 82433a67f9 fix a few typos (#1829) 2022-05-20 13:31:30 -04:00
Nicolas Boyer ed7343f7f2 Provide current location in JSON input (#1695)
* Setup.

* Add current_location().

* Make return simdjson_result and fix cast issues.

* Whitespace.

* Add broken JSON tests. Add null parser check.

* Remove unused variables.

* Alive fix.

* Fix merge issues.

* Simplification for out of bounds.

* More tests.

* Move pointer back for unrecoverable errors.

* Add new error OUT_OF_BOUNDS

* Remove unnecessary include and fix OUT_OF_BOUNDS.

* Add more tests. Fix unrecoverable errors.

* Fix tests.

* Modify one test.

* Update doc.

* Typos.

* Add read_me tests.

* Update doc.

* Add current_location for simdjson_result and document_reference

* Typos.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-08-27 13:41:59 -04:00
Daniel Lemire aa52cf6868 Alive fix. (#1700) 2021-08-21 10:22:59 -04:00
Daniel Lemire de4deb8c4e Makes it possible to cast a document to a value. (#1690)
* Makes it possible to cast a document to a value.
2021-08-11 20:02:30 -04:00
Daniel Lemire b79261eebc This cleans a bit the current code, especially with respect to EOF guards. (#1669)
* Upgrading the GitHub Actions.

* Upgrading appveyor

* Upgrading circle ci.

* Cleaning.
2021-07-25 10:36:22 -04:00
Daniel Lemire 47a62db559 Isolated jkeiser fix for issue 1632: make it so that INCORRECT_TYPE is a recoverable condition in On Demand (#1663) 2021-07-23 11:32:26 -04:00
Nicolas Boyer 5c590b8434 Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire 2dac3705d2 renames 'to_string' to 'to_json_string' and makes it ridiculously fast (#1642)
* Changing the name of the function to 'to_json_string' from 'to_string' to avoid confusion.

* Moving to a fast string_view model

* Making it exception-safe.

* Tweaking.

* Workaround for exceptions.

* more robust to_json_string (#1651)

* WIP.

* Fuzzing timeout  (bug fix) (#1650)

* prove pull request #1648 introduces an infinite loop

* Interesting bug!

* Tweak.

Co-authored-by: Paul Dreik <github@pauldreik.se>

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

* WIP.

* It should now work.

* Moving car examples to exception mode

* Simplifying somewhat.

* I forgot to abandon. Let us do that.

* Adding more tests.

Co-authored-by: Paul Dreik <github@pauldreik.se>

Co-authored-by: Paul Dreik <github@pauldreik.se>
2021-07-19 10:24:36 -04:00
Daniel Lemire 16e8db1f17 Adding 'count_elements' method. (#1577)
* Adding 'count_elements' method.

* Actually reporting errors.

* removing white space.

* Removing white space again.

* Adding an extra example.

* Prettier.

* Making the functionality more error-proof.

* Avoiding exceptions.

* Various fixes including extending count_elements to value types.

* Various fixes.

* Minor fixes.

* Correcting comment.

* Trimming white spaces.
2021-06-06 17:56:00 -04:00
Daniel Lemire 19c3b1315a Rewind functionality. (#1539)
* Rewind functionality.


* Keeping just the document rewind.
2021-06-04 09:22:33 -04:00
Daniel Lemire 1032f70ddf Verifies and fixes issue 1588 (#1589)
* Verifies and fix issue 1588

* Removing a trailing space.
2021-05-27 19:35:42 -04:00
John Keiser bcab8d3abf Check for end object/array at top level
This avoids a very unlikely buffer overrun that can occur in a particular kind of invalid JSON:
- the document is invalid with an unclosed top level array or object
- the last thing in the document is a number that ends at EOF
- the padding is filled entirely with numeric digits
2021-02-22 09:35:21 -08:00
John Keiser 3076de0405 Use SIMDJSON_DEVELOPMENT_CHECKS instead of SIMDJSON_PRODUCTION
Don't enable in retail
2021-02-20 11:46:01 -08:00
John Keiser 14315ec5cd Default SIMDJSON_PRODUCTION to OFF for bare header usage 2021-02-06 11:06:37 -08:00
John Keiser 9d693da852 Only set container depth when a container iteration starts 2021-02-05 17:20:24 -08:00
John Keiser 0d1c99a6ad Allow object lookup safety to be disabled
Use cmake -DSIMDJSON_API_USAGE_CHECKS=OFF ..
2021-02-05 10:18:01 -08:00
John Keiser e4626d233c Descend into fields at the value position, not the key 2021-02-05 10:18:01 -08:00
John Keiser 9934f65987 Store start index of each depth for safety 2021-02-05 10:17:28 -08:00
John Keiser 38da15b501 Rename checkpoint() -> position(), add token_position type 2021-01-11 15:19:24 -08:00
John Keiser 0f515785c6 Support reading scalars out of order 2021-01-11 15:17:46 -08:00
John Keiser 6fed8d2a26 Use active implementation for stage 1 on demand 2021-01-11 14:57:52 -08:00
John Keiser b8426584fc Make field lookup order-insensitive. 2020-12-19 13:57:29 -08:00
John Keiser f9c6dedca1 Add test for out-of-order parse asserts 2020-12-13 13:39:47 -08:00
John Keiser 73c7b7db3d Safety: check index of scalar values before use 2020-12-11 11:57:09 -08:00
John Keiser 3baba73cf5 Don't call functions in assume (VS2019 Clang doesn't like it) 2020-12-07 11:17:24 -08:00
John Keiser 3aa3175378 Add tests for recovering from partial child iteration 2020-12-06 16:19:06 -08:00
John Keiser c89647af9e Make all values use same underlying iterator 2020-12-06 15:23:52 -08:00
John Keiser 1303a88769 Unconditionally track depth 2020-12-06 15:23:52 -08:00
Paul Dreik af4db55e66 remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Daniel Lemire bb2bc98a22 Fix issue https://github.com/simdjson/simdjson/issues/1127 (#1224) 2020-10-13 09:18:54 -04:00
John Keiser 2ba67c2bc2 [WIP] && and & versions of each operator 2020-10-06 11:29:45 -07:00
John Keiser cae91983ec Fix issue with early destruction 2020-10-06 11:29:45 -07:00
John Keiser c7c1372833 Allow reuse of value to try multiple types 2020-10-06 11:29:45 -07:00
John Keiser 512a94afaa Move error to json_iterator 2020-10-06 11:29:45 -07:00
John Keiser 4859cb8528 Add SIMDJSON_ONDEMAND_SAFETY_RAILS 2020-10-06 11:29:45 -07:00
John Keiser c42b91980b Fix VC++ forward friend declarations 2020-10-06 11:29:45 -07:00
John Keiser a700848bae Move ondemand implementation to include/ 2020-10-04 12:47:30 -07:00