Commit Graph

2354 Commits

Author SHA1 Message Date
Daniel Lemire 450cb7b0f8 Merge branch 'dlemire/nopadding' of github.com:simdjson/simdjson into dlemire/nopadding 2021-07-23 23:40:41 -04:00
Daniel Lemire 2907912b1d Guarding the peek/advance. 2021-07-23 23:40:26 -04:00
Daniel Lemire b6aff44c42 Syncing the singleheader files. 2021-07-23 23:12:03 -04:00
Daniel Lemire ce11f41840 Merge branch 'dlemire/nopadding' of github.com:simdjson/simdjson into dlemire/nopadding 2021-07-23 23:06:32 -04:00
Daniel Lemire 7c8bc650a7 Turning on developer mode explicitly throughout. 2021-07-23 23:06:08 -04:00
Daniel Lemire 657489e387 More fixes. 2021-07-23 22:27:46 -04:00
Daniel Lemire 60b6a16edc Let us stress the memory overflow in more tests. 2021-07-23 21:56:29 -04:00
Daniel Lemire 2ceca3e510 More tweaking to fix problems exposed by the fuzzers. 2021-07-23 21:36:40 -04:00
Daniel Lemire d8b6a8c9d3 Merge branch 'dlemire/nopadding' of github.com:simdjson/simdjson into dlemire/nopadding 2021-07-23 20:35:20 -04:00
Daniel Lemire bbc9bd059a Some tweaking. 2021-07-23 20:35:12 -04:00
Daniel Lemire 68df233b4b Removing a couple of tests from circle ci. 2021-07-23 18:07:43 -04:00
Daniel Lemire dc66d6133a Fixing comment. 2021-07-23 17:48:21 -04:00
Daniel Lemire 13055a4601 Using an alternative instrinsic. 2021-07-23 17:47:05 -04:00
Daniel Lemire a7ff410532 Various fixes for the number parsing in the DOM front-end. Basically, we had
a mess whereas a SWAR optimization processing 8 digits would sometimes be called,
despite the fact that it would be unsafe and/or it would not be called, and we would
end up leaving performance on the floor.
2021-07-23 17:39:09 -04:00
Daniel Lemire d72cba29be Minor tweaks. 2021-07-23 16:25:08 -04:00
Daniel Lemire 757db1b2f0 Removing padding from DOM API. 2021-07-23 15:18:49 -04:00
Daniel Lemire 7175731e09 Removing the special-case 2021-07-23 14:07:32 -04:00
Daniel Lemire f6fc786b05 Disabling some change made to the DOM kernel which was evidently made to allow padding-less parsing. 2021-07-23 14:01:10 -04:00
Daniel Lemire 8ce1e52e4f Merge branch 'master' into dlemire/nopadding 2021-07-23 11:43:39 -04:00
Daniel Lemire 47a62db559 Isolated jkeiser fix for issue 1632: make it so that INCORRECT_TYPE is a recoverable condition in On Demand (#1663) 2021-07-23 11:32:26 -04:00
Daniel Lemire b12014f9e4 Let us try some setups in debug mode (for faster builds). 2021-07-23 11:20:51 -04:00
Daniel Lemire 1a433fc5b4 Enabling a broad range of tests. 2021-07-23 10:51:43 -04:00
Daniel Lemire 21845f1f6a Documenting iterate_many + removing remnants of padding in On Demand. 2021-07-23 10:37:08 -04:00
Daniel Lemire dc86daa4ec Removing the padding. 2021-07-22 17:32:06 -04:00
Daniel Lemire 910686ffa8 Merge branch 'dlemire/nopadding' of github.com:simdjson/simdjson into dlemire/nopadding 2021-07-22 15:41:01 -04:00
Daniel Lemire b1a3943a8b We should now be able to iterate over a string_view. 2021-07-22 15:40:45 -04:00
Daniel Lemire 70a0d62f51 Merge branch 'dlemire/backport_jkeiser_fix_for_issue1632' into dlemire/nopadding 2021-07-22 15:27:41 -04:00
Daniel Lemire bb0ed22ab8 Removing some of the padding in the tests. 2021-07-22 15:26:41 -04:00
Daniel Lemire d7770e9a6b Disabling 'json[N-1]' test. 2021-07-22 14:24:39 -04:00
Daniel Lemire 0e8b68683d Merge branch 'dlemire/backport_jkeiser_fix_for_issue1632' into dlemire/nopadding 2021-07-22 14:03:23 -04:00
Daniel Lemire 1fe50cb777 Trimming. 2021-07-22 14:02:30 -04:00
Daniel Lemire f2d419dda6 Basic guarding. 2021-07-22 13:42:49 -04:00
Daniel Lemire 57ca29e151 Merge branch 'dlemire/backport_jkeiser_fix_for_issue1632' into dlemire/nopadding 2021-07-22 13:23:08 -04:00
Daniel Lemire 01645cbd69 Update bug_report.md 2021-07-22 13:22:49 -04:00
Daniel Lemire 24068a089e Merge branch 'master' into dlemire/backport_jkeiser_fix_for_issue1632 2021-07-22 12:56:18 -04:00
Daniel Lemire e4bba2df9d Do not needlessly do perf checks. 2021-07-22 10:52:05 -04:00
Daniel Lemire ca9391c5a9 Let us run more CI tests. 2021-07-22 10:44:10 -04:00
Daniel Lemire 91b8fa4c4b Fixing a CI issue. 2021-07-22 10:38:59 -04:00
Daniel Lemire 081c3ca7c8 Merged. 2021-07-22 10:17:37 -04:00
Daniel Lemire bbeec66a93 Merge branch 'jkeiser/no-padding-scalar' into dlemire/nopadding 2021-07-21 13:45:10 -04:00
Daniel Lemire 7dcd43e82a Merge branch 'jkeiser/no-padding' into dlemire/nopadding 2021-07-21 11:19:43 -04:00
Daniel Lemire 3dbecab494 Trimming the documentation a bit. 2021-07-20 15:05:39 -04:00
Nicolas Boyer 5c590b8434 Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Daniel Lemire c3f7eff019 Adding some commentary 2021-07-19 17:08:51 -04:00
Daniel Lemire d0e2a7fb00 Minor fixes 2021-07-19 16:53:11 -04:00
Daniel Lemire 9cace04da3 Adding more tests. 2021-07-19 16:33:58 -04:00
Daniel Lemire 2e688a2825 Removing two other asserts. 2021-07-19 16:27:29 -04:00
Daniel Lemire 4501aebee8 Disabling a couple of asserts. 2021-07-19 16:23:48 -04:00
Daniel Lemire df24da314f Cleaning. 2021-07-19 15:58:17 -04:00
Daniel Lemire 8b2bff981f Messy temporary work. 2021-07-19 15:34:48 -04:00