1. Allows processing inclomplete, damaged, corrupted json to some extent.
2. Pariity with the Presto Java functionality.
3. Protected with SIMDJSON_EXPERIMENTAL_ALLOW_INCOMPLETE_JSON define.
4. Does not interfere with the normal path (can co-exist).
5. Tested in production forkflow.
* Add comma separated value parsing
* Fix failing tests
* Make tests work for exceptions
* Fix test
* Fix try catch making test fail
---------
Co-authored-by: Yong Xiang Ng <yxng@drwholdings.com>
* Add info and error logging
* Add tests for error logging
* Add logging for missing field
* Update docs for logging usage
* Fix style and pass by ref for string format args
* Make log_level explicit and simplify get log level from env
* Make log level int32_t
* Format enum class
* Fix ci
* Move enum to header
* Fix compilation for noexception build in test case
* Disable warnings and putenv
---------
Co-authored-by: Yong Xiang Ng <yxng@drwholdings.com>
* Rename simdjson_really_inline -> simdjson_inline
I want to change the simdjson_really_inline macro to sometimes not force
inlining. After that upcoming change, the name simdjson_really_inline
will no longer makes sense.
Rename simdjson_really_inline to simdjson_inline. This patch should not
change semantics; simdjson_inline still forces inlining as before.
Some functions still need to be really inlined for ABI reasons.
(GCC's -Wpsabi complains otherwise.) Leave those functions marked as
simdjson_really_inline.
* Improve build times for debug builds
simdjson_inline is used for most simdjson functions. It forces inlining.
In unoptimized/debug builds, this can lead to a lot of machine code
being generated (especially with Address Sanitizer), causing slow
compilation.
Change simdjson_inline to force inlining only for optimized builds.
Sometimes, the programmer might want a slightly-optimized build and want
fast compilation (e.g. GCC's -Og mode). Allow simdjson users to define
the simdjson_inline macro themselves (e.g. on the command line:
-Dsimdjson_inline=inline) in cases where the default behavior is
undesired.
This patch reduced build times by over 75% for ondemand_object_tests.cpp
with GCC 9.4.0 and CMAKE_BUILD_TYPE=Debug on my AMD 5950X:
Before: 6.885 6.683 6.971 6.957 6.949 seconds (5 samples)
After: 1.492 1.551 1.494 1.490 1.531 seconds (5 samples)
* Update basic.md to document JSON pointer for On Demand.
* Add automatic rewind for at_pointer
* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.
* Review
* Add test
* Add document_stream constructors and iterate_many
* Attempt to implement streaming.
* Kind of fixed next() for getting next document
* Temporary save.
* Putting in working order.
* Add working doc_index and add function next_document()
* Attempt to implement streaming.
* Re-anchoring json_iterator after a call to stage 1
* I am convinced it should be a 'while'.
* Add source() with test.
* Add truncated_bytes().
* Fix casting issues.
* Fix old style cast.
* Fix privacy issue.
* Fix privacy issues.
* Again
* .
* Add more tests. Add error() for iterator class.
* Fix source() to not included whitespaces between documents.
* Fixing CI.
* Fix source() for multiple batches. Add new tests.
* Fix batch_start when document has leading spaces. Add new tests for that.
* Add new tests.
* Temporary save.
* Working hacky multithread version.
* Small fix in header files.
* Correct version (not working).
* Adding a move assignment to ondemand::parser.
* Fix attempt by changing std::swap.
* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.
* Update doc and readme tests.
* Update basics.md
* Update readme_examples tests.
* Fix exceptions in test.
* Partial setup for amazon_cellphones.
* Benchmark with vectors.
* Benchmark with maps
* With vectors again.
* Fix for weighted average.
* DOM benchmark.
* Fix typos. Add On Demand benchmark.
* Add large amazon_cellphones benchmark for DOM
* Add benchmark for On demand.
* Fix broken read_me test.
* Add parser.threaded to enable/disable thread usage.
Co-authored-by: Daniel Lemire <lemire@gmail.com>
* Changing the name of the function to 'to_json_string' from 'to_string' to avoid confusion.
* Moving to a fast string_view model
* Making it exception-safe.
* Tweaking.
* Workaround for exceptions.
* more robust to_json_string (#1651)
* WIP.
* Fuzzing timeout (bug fix) (#1650)
* prove pull request #1648 introduces an infinite loop
* Interesting bug!
* Tweak.
Co-authored-by: Paul Dreik <github@pauldreik.se>
* It should now work.
* Moving car examples to exception mode
* Simplifying somewhat.
* I forgot to abandon. Let us do that.
* Adding more tests.
* WIP.
* It should now work.
* Moving car examples to exception mode
* Simplifying somewhat.
* I forgot to abandon. Let us do that.
* Adding more tests.
Co-authored-by: Paul Dreik <github@pauldreik.se>
Co-authored-by: Paul Dreik <github@pauldreik.se>
* Adding 'count_elements' method.
* Actually reporting errors.
* removing white space.
* Removing white space again.
* Adding an extra example.
* Prettier.
* Making the functionality more error-proof.
* Avoiding exceptions.
* Various fixes including extending count_elements to value types.
* Various fixes.
* Minor fixes.
* Correcting comment.
* Trimming white spaces.
This avoids a very unlikely buffer overrun that can occur in a particular kind of invalid JSON:
- the document is invalid with an unclosed top level array or object
- the last thing in the document is a number that ends at EOF
- the padding is filled entirely with numeric digits
Daniel Lemire