305 Commits

Author SHA1 Message Date
Daniel Lemire aaa12c8fb6 more thread testing (#1324)
* Adding a few tests.

* More tests.

* Trailing.

* More links/documentations.
2020-12-02 11:33:36 -05:00
Daniel Lemire 9304d88920 Prototype test for issue 1299: using parse_many, find the location of the end of the last document (#1301)
* Prototype test for issue 1299.

* This improves the documentation.

* Removing trailing white spaces.

* Removing trailing spaces

* Trailing.
2020-12-01 15:59:20 -05:00
Daniel Lemire d9c4191e8a This should fix an issue we have with unclosed strings with document_stream (#1321)
* This should fix an issue we have with unclosed strings.

* Patching the fallback kernel.

* Better guarding the code.

* Patching the fallback.
2020-11-30 13:43:57 -05:00
Daniel Lemire dc69bc28ae Trying to verify recent document stream issues. (#1318)
* Trying to verify recent document stream issues.

* Adding another one.

* More thorough tests.

* Removing trailing spaces.

* Working toward exposing some issues.

* Tweaking.
2020-11-27 17:04:10 -05:00
Daniel Lemire 1abb64f6f6 This fixes issue 1302 (#1303)
* This verifies issue 1302

* I believe that this fixes the issue.

* Let us do it differently.

* Better comments.
2020-11-27 14:19:36 -05:00
Daniel Lemire 5609851747 This will guard the batch_size so it cannot be so low as to accidentally burn through your CPU. (#1319)
* This will guard the batch_size so it cannot be so low as to accidentally burn through your CPU.
2020-11-27 09:37:54 -05:00
friendlyanon c805fc28a4 Remove git modules (#1258)
* Bump minimum CMake version

* Remove unnecessary git checks

* Move benchmark options where they are used

* Declare helper functions for dependencies

The custom solution here is tailored for fast configure times, but only
works for dependencies on Github.

* Import dependencies using the declared commands

* Remove git submodules

* Call target_link_libraries properly

target_link_libraries must not be called without a requirement
specifier.

* Fix includes for competition

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-04 13:34:29 -05:00
Paul Dreik af4db55e66 remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Paul Dreik 47669566da fix unintended early return in ondemand unit test loops (#1263)
fix uninteded early return in ondemand unit test loops

go through and fix warnings appearing in qtcreator,

qualify with std::, add const, abort on error

get rid of ulp_distance, not needed anymore when parsing is exact
2020-11-01 19:08:40 +01:00
Paul Dreik 0b82f07115 fix segfault in numberparsing #1273 (#1274)
This was a read overflow.
2020-11-01 18:27:21 +01:00
Paul Dreik f93fb21c95 optionally disable deprecated apis (#1271)
Introduce cmake option SIMDJSON_DISABLE_DEPRECATED_API (default Off)
which turns off deprecated simdjson api functions by setting the macro
 SIMDJSON_DISABLE_DEPRECATED_API.

For non-cmake users, users will have to set SIMDJSON_DISABLE_DEPRECATED_API
by some other means to disable the api.

Closes #1264
2020-11-01 06:38:52 +01:00
Daniel Lemire a8bf10ea5a Minor patch. 2020-10-30 14:51:50 -04:00
Daniel Lemire a75c07065f Fix for issue 1246. We document the relationship between parser instances and elements (#1250)
* Fix for issue 1246.

* Adopting John's wording.
2020-10-26 08:40:45 -04:00
Daniel Lemire 14039d05a9 Adding a new benchmark for ondemand: distinct user id (#1239)
* Adding a distinct user id benchmark

* reenabling everything

* Removing an unnecessary "value()".

* Better tests of the examples and some fixes.

* Guarding exception code.
2020-10-23 08:47:01 -04:00
Daniel Lemire 61fb4244f2 There are a couple of extra value() that should not be there (#1242) 2020-10-21 12:37:22 -04:00
Daniel Lemire 0d6919dd99 Reenable the on-demand tests and allows us to convert a raw string into a C++ string. (#1232)
* Reenable the on-demand tests and allows us to convert a raw string into a C++ string.

* Fixing a 1-byte buffer overrun.

* More documentation.

* Adding more tests.

* Enabling the new tests

* Committing a nicer example.

* Not yet happy but this should fix our failures.

* Duh.

* Ok. Making it easier to get string_view instances from field instances.

* It is a struct.

* Trying to satisfy VS.

* Adopting John's name.
2020-10-19 20:22:24 -04:00
Paul Dreik f1b4a54991 add fuzz element (#1204)
* add definitions for is_number and tie (by lemire)
* add fuzzer for element
* update fuzz documentation
* fix UB in creating an empty padded string
* don't bother null terminating padded_string, it is done by the std::memset already
*  refactor fuzz data splitting into a separate class
2020-10-17 05:48:50 +02:00
Daniel Lemire 07a6e098c8 This would allow users to find out what builtin is. (#1227)
* This would allow users to find out what builtin is.

* Trying another approach.

* Added instructions.

* Cleaning up the printout.

* Let us be less invasive.

* Adding a comment.
2020-10-15 21:58:42 -04:00
Daniel Lemire 3cd98df30d This adds new tests regarding ordering. (#1233)
* This adds new tests regarding ordering.

* Updating the documentation with more examples.

* Adding compilation tests.

* Pruning code for exceptions.

* Guarding exceptionless.
2020-10-15 16:41:14 -04:00
Daniel Lemire bb2bc98a22 Fix issue https://github.com/simdjson/simdjson/issues/1127 (#1224) 2020-10-13 09:18:54 -04:00
Daniel Lemire 37e6d1e9c7 new number parsing (#1222)
* Remove our dependency on strtod_l by bundling our own slow path.

* Ok. Let us drop strtod entirely.

* Trimming down the powers to -342.

* Removing useless line.

* Many more comments.

* Adding some DLL exports.

* Let the gods help those who rely on windows+gcc.

* Marking the subnormals as unlikely. This is pretty much "performance neutral", but it might help just a bit with twitter.json.
2020-10-10 12:47:49 -04:00
John Keiser 676a3d068c Add non-top-level array iteration test 2020-10-06 11:29:46 -07:00
John Keiser 5533f8d87b Add object iteration error tests 2020-10-06 11:29:46 -07:00
John Keiser 93af7b61ce Add array iteration error tests 2020-10-06 11:29:46 -07:00
John Keiser 364ad5529d Add basic error tests 2020-10-06 11:29:46 -07:00
John Keiser 1974a46fe0 Remove validate tests and unimplemented cast tests from ondemand 2020-10-06 11:29:46 -07:00
John Keiser 5327ab9903 Remove ondemand minify and format tests 2020-10-06 11:29:46 -07:00
John Keiser 235d191bae Add Twitter exception tests 2020-10-06 11:29:46 -07:00
John Keiser 4eb80ec75a Add DOM API exception tests 2020-10-06 11:29:45 -07:00
John Keiser 2900459222 Separate twitter tests from DOM API tests 2020-10-06 11:29:45 -07:00
John Keiser 9088792b0e Disable value["x"] (unsafe, convert to object first) 2020-10-06 11:29:45 -07:00
John Keiser b41fe7beab Add object indexing tests 2020-10-06 11:29:45 -07:00
John Keiser 00f9bb8a07 Add null tests 2020-10-06 11:29:45 -07:00
John Keiser a90e1637cb Add boolean tests 2020-10-06 11:29:45 -07:00
John Keiser 4bad5c0241 Add numeric value tests 2020-10-06 11:29:45 -07:00
John Keiser ce09d82fc7 Test strings 2020-10-06 11:29:45 -07:00
John Keiser 5b926b8196 Support array iteration over document 2020-10-06 11:29:45 -07:00
John Keiser c719ccdb48 Add tests for empty object/array 2020-10-06 11:29:45 -07:00
John Keiser 9f1786aeb1 Add .get(T) to value/document 2020-10-06 11:29:45 -07:00
John Keiser 0bb83e06bc Fix root number parsing 2020-10-06 11:29:45 -07:00
John Keiser 99bc591366 Don't enable fallback test unless fallback is explicitly selected 2020-10-06 11:29:45 -07:00
John Keiser 8b3c8820e0 Fix numberparsingcheck to define found_invalid_number before simdjson.h 2020-10-06 11:29:39 -07:00
John Keiser ebcb3c6b3b On-demand parse implementation 2020-10-04 12:47:29 -07:00
Daniel Lemire f1841e48b3 Minor fixes to some headers (tweak) (#1198) 2020-10-02 12:29:05 -04:00
Daniel Lemire 9865bb6904 Make it possible to check that an implementation is supported at runtime (#1197)
* Make it possible to check that an implementation is supported at runtime.

* add CI fuzzing on arm 64 bit

This adds fuzzing on drone.io arm64

For some reason, leak detection had to be disabled. If it is enabled, the fuzzer falsely reports a crash at the end of fuzzing.

Closes: #1188

* Guarding the implementation accesses.

* Better doc.

* Updating cxxopts.

* Make it possible to check that an implementation is supported at runtime.

* Guarding the implementation accesses.

* Better doc.

* Updating cxxopts.

* We need to accomodate cxxopts

Co-authored-by: Paul Dreik <github@pauldreik.se>
2020-10-02 11:04:51 -04:00
Daniel Lemire 8b5a89c136 Parsing floats with 19 significant digits should be fine. (#1191)
* Parsing floats with 19 significant digits should be fine.

* Adding more tests with very long mantissa.
2020-09-29 19:42:43 -04:00
Daniel Lemire da093c1982 Fixing "undefined behavior" issue in new fast_itoa functions (#1186)
* Fixing "undefined behavior" issue.

* Simplifying our custom atoi

* Fixing minor bug
2020-09-29 19:17:03 -04:00
Daniel Lemire 048fb6278a This adds two tests to verify a new fuzzer issue. (So far I could not verify.) (#1194) 2020-09-29 11:45:41 -04:00
Daniel Lemire 0e584fa4a5 Attempt to fix issue 1187. (#1192) 2020-09-27 12:04:47 -04:00
Daniel Lemire 60c139a844 Faster and more correct serialization (#1168)
* Adding new files.

* Better.

* Fixing minifier and adding tests.

* Adding benchmarks.

* Including the array header.

* Replacing old stream-based code by the new code.

* Doubling up the itoa.

* Hidden away to_chars in internal namespace.

* Removing the repetitions.

* Documented the atoi functions.

* Tuning the escape sequences.

* Moving the operators off the main namespace.

* Added more tests.

* Tweaking the implementation so that it works with and without exp.

* The string_builder template and mini_formatter class
 are not part of  our public API and are subject to change
 at any time!

* Adding a benchmark and some optimization.

* Cleaning.

* Strictly speaking, this header is needed.
2020-09-23 10:00:39 -04:00