Commit Graph

3129 Commits

Author SHA1 Message Date
John Keiser db6bf15361 Allow all array/object/etc. to be copied 2020-12-06 16:29:53 -08:00
John Keiser 3aa3175378 Add tests for recovering from partial child iteration 2020-12-06 16:19:06 -08:00
John Keiser 4c63956624 Enable object["x"]["y"] 2020-12-06 15:23:52 -08:00
John Keiser c89647af9e Make all values use same underlying iterator 2020-12-06 15:23:52 -08:00
John Keiser 1303a88769 Unconditionally track depth 2020-12-06 15:23:52 -08:00
John Keiser aa1eabbb56 Add benchmark that stops early 2020-12-06 15:23:51 -08:00
Daniel Lemire cbacec0760 Releasing 0.7.0. v0.7.0 2020-12-04 18:56:15 -05:00
0xflotus 82fcf141f7 fix: small typo error (#1328) 2020-12-03 13:16:45 -05:00
Daniel Lemire aaa12c8fb6 more thread testing (#1324)
* Adding a few tests.

* More tests.

* Trailing.

* More links/documentations.
2020-12-02 11:33:36 -05:00
Daniel Lemire 9b5150daef Fixing issue 1247 (#1325) 2020-12-01 19:09:55 -05:00
Daniel Lemire 9304d88920 Prototype test for issue 1299: using parse_many, find the location of the end of the last document (#1301)
* Prototype test for issue 1299.

* This improves the documentation.

* Removing trailing white spaces.

* Removing trailing spaces

* Trailing.
2020-12-01 15:59:20 -05:00
Paul Dreik 725ca010e7 add ndjson fuzzer (#1304)
* add ndjson fuzzer

* reproduce #1310 in the newly added unit test

Had to replace the input, because:
1)
the fuzzer uses the first part of the input to determine
the batch_size to use, so that has to be cut off

2)
the master now protects against low values of batch_size

I also made the test not return early, so the error is triggered.
2020-12-01 15:58:41 -05:00
Daniel Lemire 59c857e969 Merge branch 'master' of github.com:simdjson/simdjson 2020-12-01 15:31:44 -05:00
Daniel Lemire 45c9136443 Fixed odd issue. 2020-12-01 15:31:24 -05:00
Daniel Lemire bc4087ac96 Hunting for data races in document_stream (#1323)
* Adding matching tests.

* Actually adding thread testing actions.
2020-12-01 12:40:15 -05:00
Daniel Lemire d9c4191e8a This should fix an issue we have with unclosed strings with document_stream (#1321)
* This should fix an issue we have with unclosed strings.

* Patching the fallback kernel.

* Better guarding the code.

* Patching the fallback.
2020-11-30 13:43:57 -05:00
Daniel Lemire 3fa40b8dc2 Adding an example corresponding to issue 1316 (documentation enhancement) (#1317)
* Adding an example.

* Updated other doc file.

* Trying to take into account @jkeiser's comments.

* Some people prefer empty final lines.
2020-11-27 17:40:29 -05:00
Daniel Lemire dc69bc28ae Trying to verify recent document stream issues. (#1318)
* Trying to verify recent document stream issues.

* Adding another one.

* More thorough tests.

* Removing trailing spaces.

* Working toward exposing some issues.

* Tweaking.
2020-11-27 17:04:10 -05:00
Daniel Lemire 53577f11e1 Explicit worker disposal. 2020-11-27 15:39:43 -05:00
Daniel Lemire 1abb64f6f6 This fixes issue 1302 (#1303)
* This verifies issue 1302

* I believe that this fixes the issue.

* Let us do it differently.

* Better comments.
2020-11-27 14:19:36 -05:00
Mikhail Matrosov b442b29ccf Mark comparison operators as const (#1320)
Co-authored-by: Mikhail Matrosov <mikhail.matrosov@aimtech.com>
2020-11-27 14:18:42 -05:00
Daniel Lemire 5609851747 This will guard the batch_size so it cannot be so low as to accidentally burn through your CPU. (#1319)
* This will guard the batch_size so it cannot be so low as to accidentally burn through your CPU.
2020-11-27 09:37:54 -05:00
Paul Dreik 68a8004518 don't memcpy after failed alloc (#1315)
Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=27675
2020-11-21 07:52:24 +01:00
Paul Dreik 0b39e3a6cf add fuzzer for padded_string (#1312)
This also fixes an overflow problem.
2020-11-19 16:51:56 +01:00
Paul Dreik 924ad2d592 when amalagamating, replace the SIMDJSON_IMPLEMENTATION (#1306) 2020-11-17 20:50:49 +01:00
Paul Dreik 5202d07a77 add script and instructions for minimizing and cleansing fuzz crashes (#1305) 2020-11-17 20:49:29 +01:00
friendlyanon b632107a7a Use the org's own PPA for CMake (#1295)
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-10 13:55:59 -05:00
Paul Dreik f62ca21dd1 enable boost json (#1292)
* bump boost.json and see if it works in simdjson CI

* enable boost json

* clean up

* add boost json to deps

* use boost if std::string_view is available

* add build with c++20

* use docker image which has the proper libc++ installed
2020-11-10 13:55:04 -05:00
friendlyanon 553befa012 Disable Github CI in feature branches (#1298)
Forks that would like to contribute via PRs from feature branches
needlessly run CI on those branches on top of the PRs.
This is a waste of resources.

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-10 13:52:58 -05:00
Paul Dreik def624a50c update version tag (#1297) 2020-11-08 10:17:06 -05:00
Daniel Lemire 268f26a84f Tweak of the documentation. 2020-11-07 16:33:12 -05:00
Paul Dreik d9456b3030 fix numeric_limits usage problem on windows caused by the max macro (#1293)
See https://stackoverflow.com/questions/2789481/problem-calling-stdmax/2789509#2789509
2020-11-06 06:55:27 +01:00
Daniel Lemire f195f849d0 Helping contributors. (#1289) 2020-11-05 10:27:35 -05:00
friendlyanon 669e5b91b4 Improve CI workflows (#1291)
* Fix Cirrus CI

CMake is not installed at this point yet.

* Add caching in Circle CI

* Add caching to the MinGW Github workflows

* Fix Circle CI config

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-05 08:42:00 -05:00
Daniel Lemire a06e0958ca We do not want the amalgamate.py script to run *ever* if you do not have at least Python 3. (#1290) 2020-11-04 13:52:52 -05:00
friendlyanon a56149e79a Cache CMake dependencies in CI (#1282)
Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-04 13:35:33 -05:00
friendlyanon c805fc28a4 Remove git modules (#1258)
* Bump minimum CMake version

* Remove unnecessary git checks

* Move benchmark options where they are used

* Declare helper functions for dependencies

The custom solution here is tailored for fast configure times, but only
works for dependencies on Github.

* Import dependencies using the declared commands

* Remove git submodules

* Call target_link_libraries properly

target_link_libraries must not be called without a requirement
specifier.

* Fix includes for competition

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-04 13:34:29 -05:00
Daniel Lemire 218c274090 Updating main branch for legacy libc++ support (#1288)
* Updating main branch for legacy libc++ support

* Adopting

* Removing unnecessary math header.

* Updating the single-header files so we can pass the new tests.

* Portable infinite-value detection is hard.

* Working toward disabling boost json selectively.

* Selectively disabling Boost JSON

* More work toward selectively disabling boost json.
2020-11-04 12:24:42 -05:00
Tyler Kennedy 921c79f26f Add method to get the total number of slots used by an array in the t… (#1253)
* Add method to get the total number of slots used by an array in the tape.

* Whoops
2020-11-04 10:35:54 -05:00
Paul Dreik af4db55e66 remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Paul Dreik 9f78559cc8 detect trailing whitespace in a CI job (#1268) 2020-11-03 20:41:19 +01:00
Paul Dreik 23b4bc93aa move amalgamate from bash to python (#1278)
This is much faster (from 3.5 to 0.14 seconds)
2020-11-03 07:35:16 +01:00
Daniel Lemire 6321df078a This fixes very minor "mistakes". (#1281) 2020-11-02 20:04:04 -05:00
Convery e6258377d9 Additional define to silence the 32-bit portability warning. (#1279)
Under compilers like MSVC the `#pragma message` about portability will be issued for each translation module; regardless of whether or not `SIMDJSON_PORTABILITY_H` has already been defined. As such a new define `SIMDJSON_NO_PORTABILITY_WARNING`, can be defined prior to the inclusion of `<simdjson.h>` to silence it.
2020-11-02 08:37:13 -05:00
Jeremy Ong b3a26fd0cc Do not enable -fPIC when compiling on Windows (#1280)
Regardless of the compiler, Windows targets do not support -fPIC,
as position independent code is already implicitly enabled. Compiling
simdjson with Clang on Windows will error because -fPIC is an
unsupported option for target 'x86_64-pc-windows-msvc'.
2020-11-02 08:18:04 -05:00
Paul Dreik 47669566da fix unintended early return in ondemand unit test loops (#1263)
fix uninteded early return in ondemand unit test loops

go through and fix warnings appearing in qtcreator,

qualify with std::, add const, abort on error

get rid of ulp_distance, not needed anymore when parsing is exact
2020-11-01 19:08:40 +01:00
Paul Dreik 03a1bb0a5b detect if amalgamation is needed (#1269)
output amalgamation date to be locale and timezone independent
2020-11-01 19:07:15 +01:00
Paul Dreik 54ffbbe7db remove asserts from compute_float_64 (#1276) 2020-11-01 18:34:42 +01:00
Paul Dreik 0b82f07115 fix segfault in numberparsing #1273 (#1274)
This was a read overflow.
2020-11-01 18:27:21 +01:00
Paul Dreik 265db2e533 fix non ascii sources (#1275)
Master does not build because of non-ascii sources, merging without waiting for CI.
2020-11-01 11:14:01 +01:00