Commit Graph

95 Commits

Author SHA1 Message Date
Daniel Lemire 7c450fbb70 Adding clang 13 tests. (#1844) 2022-06-15 15:21:33 -04:00
Dirk Stolle 57d54792ba update actions/cache + actions/checkout in GitHub Actions to v3 (#1839) 2022-06-06 08:51:50 -04:00
Dirk Stolle 79879802f9 run tests with GCC 8, too (#1837) 2022-06-03 20:52:07 -04:00
Daniel Lemire fbe955e9a4 gcc12 without warnings. (#1836) 2022-06-02 22:53:39 -04:00
Daniel Lemire e8f370b085 Basic AVX-512 implementation (icelake or better) (#1813)
* Add cascadelake implementation, which use AVX512 Intrinsics to optimize performance(#1811)

Co-authored-by: mellonyou <fangzheng.zhang@intel.com>
Co-authored-by: wanweiqiangintel <weiqiang.wan@intel.com>
2022-05-25 11:14:02 -04:00
Daniel Lemire 9f0a2e0e86 Update to vs17 (required) (#1795)
* Update to vs17 (required)

* Moving to windows-latest.

* Dropping old vs
2022-03-19 08:00:06 -04:00
Daniel Lemire 3a93e45dd1 Removing scoop support. 2021-12-30 12:10:11 -05:00
DavidKorczynski 4aaf3f030c Add CIFuzz integration (#1757)
Signed-off-by: David Korczynski <david@adalogics.com>
2021-11-30 10:14:26 -05:00
Daniel Lemire 4a2f10ebcf Let us trim out these legacy systems from CI. 2021-10-12 18:24:03 -04:00
Daniel Lemire 9a32c48098 Cleaning up the single-header demo. (#1731)
* Cleaning up the single-header demo.

* Updating xcode
2021-10-09 11:34:21 -04:00
Daniel Lemire 18c003e8bf Adding CI for old LLVM in GitHub Actions (#1672)
* Adding CI for old LLVM

* Simplifying.
2021-07-26 08:37:29 -04:00
Daniel Lemire ec4316aa84 Adding macos testing. (#1670) 2021-07-25 12:25:31 -04:00
Daniel Lemire b79261eebc This cleans a bit the current code, especially with respect to EOF guards. (#1669)
* Upgrading the GitHub Actions.

* Upgrading appveyor

* Upgrading circle ci.

* Cleaning.
2021-07-25 10:36:22 -04:00
Daniel Lemire 01645cbd69 Update bug_report.md 2021-07-22 13:22:49 -04:00
Nicolas Boyer 5c590b8434 Bringing ndjson(document_stream) to On Demand (#1643)
* Update basic.md to document JSON pointer for On Demand.

* Add automatic rewind for at_pointer

* Remove DOM examples in basics.md and update documentation reflecting addition of at_pointer automatic rewinding.

* Review

* Add test

* Add document_stream constructors and iterate_many

* Attempt to implement streaming.

* Kind of fixed next() for getting next document

* Temporary save.

* Putting in working order.

* Add working doc_index and add function next_document()

* Attempt to implement streaming.

* Re-anchoring json_iterator after a call to stage 1

* I am convinced it should be a 'while'.

* Add source() with test.

* Add truncated_bytes().

* Fix casting issues.

* Fix old style cast.

* Fix privacy issue.

* Fix privacy issues.

* Again

* .

* Add more tests. Add error() for iterator class.

* Fix source() to not included whitespaces between documents.

* Fixing CI.

* Fix source() for multiple batches. Add new tests.

* Fix batch_start when document has leading spaces. Add new tests for that.

* Add new tests.

* Temporary save.

* Working hacky multithread version.

* Small fix in header files.

* Correct version (not working).

* Adding a move assignment to ondemand::parser.

* Fix attempt by changing std::swap.

* Moving DEFAULT_BATCH_SIZE and MINIMAL_BATCH_SIZE.

* Update doc and readme tests.

* Update basics.md

* Update readme_examples tests.

* Fix exceptions in test.

* Partial setup for amazon_cellphones.

* Benchmark with vectors.

* Benchmark with maps

* With vectors again.

* Fix for weighted average.

* DOM benchmark.

* Fix typos. Add On Demand benchmark.

* Add large amazon_cellphones benchmark for DOM

* Add benchmark for On demand.

* Fix broken read_me test.

* Add parser.threaded to enable/disable thread usage.

Co-authored-by: Daniel Lemire <lemire@gmail.com>
2021-07-20 14:17:23 -04:00
Paul Dreik 90efd79055 Get fuzzing working again (#1646)
* upload corpus to https://www.pauldreik.se/ from the x64 github action job (keep the github action cache)
* drop the github action cache (which was not working anyway) for power fuzzer and download the fuzz corpus from https://www.pauldreik.se/ instead
* resurrect arm64 fuzzing on drone CI, downloading the fuzz corpus from https://www.pauldreik.se/
* update the fuzzing documentation
2021-07-05 09:42:57 +02:00
Daniel Lemire b991a4c7f3 We should be more generously testing in debug mode. (#1635) 2021-06-24 12:44:28 -04:00
Daniel Lemire 40cba172ed Adds compile-test for Visual Studio + ARM and turn developer mode throughout CI. (#1609)
* Adds compile-test for Visual Studio + ARM and turn developer mode throughout CI.

* Correcting YAML error.

* Disabling google benchmarks under Windows ARM.

* Turning off exceptions under ARM.
2021-06-09 16:42:37 -04:00
Daniel Lemire f54bd69b5b Update bug_report.md 2021-06-07 09:57:43 -04:00
Daniel Lemire af5c8175b4 By default, we should not do the DOM checkperf… (#1571)
* By default, we should not do the DOM checkperf. These targets assume that main branch remains
compatible, an assumption that will break over time.
2021-05-15 15:28:59 -04:00
Daniel Lemire d539781cf3 This attempts to fix the fuzzers. (#1564)
* This attempts to fix the fuzzers.

* Retiring bintray.

* Disabling ARM fuzzing.
2021-05-07 22:59:26 -04:00
friendlyanon 5ec85197f8 CMake refactor stage1 (#1512)
* Remove CMP0025 policy

This policy is already set to NEW by the minimum required version.

* Use HOMEPAGE_URL in the project call

* Use VERSION in the project call

* Detect if this is the top project

* Port simdjson-user-cmakecache to a CMake script

* Create a developer mode

The SIMDJSON_DEVELOPER_MODE option set to ON will enable targets that
are only useful for developers of simdjson.

* Consolidate root CML commands into logical sections

* Warn about intended use of developer mode

* Prettify the just_ascii test

* Remove redundant CMake variables

* Inline CML contents from include and src

* Raise minimum CMake requirement to 3.14

* Define proper install rules

* Restore thread support variable

* Add BUILD_SHARED_LIBS as a top level only option

* Force developer mode to be on in CI

* Include flags earlier in developer mode

* Set CMAKE_BUILD_TYPE conditionally

CMAKE_BUILD_TYPE is used only by single configuration generators and is
otherwise completely ignored.

* Remove useless static/shared options

simdjson now uses the CMake builtin BUILD_SHARED_LIBS to switch the
built artifact's type.

* Remove unused CMAKE_MODULE_PATH variable

* Refactor implementation switching into a module

* Factor exception option out into a module

* Reformat simdjson-flags.cmake

* Rename simdjson-flags to developer-options

* Accumulate properties into an include module

This is done this way to avoid using utility targets that must be
exported and installed, which could potentially be misused by users of
the library.

* Port impl definitions to props

* Port exception options to props

* Lift normal options to the top

* Port developer options to props

* Remove simdjson-flags from benchmark

* Document the developer mode in HACKING

* Fix include path in installed config file

* Fix formatting of prop commands

* Fix tests that include .cpp files

* Change GCC AVX fixes back to compile options

* Deprecate SIMDJSON_BUILD_STATIC

* Always link fuzz targets to simdjson

* Install CMake from simdjson's debian repo

* Add gnupg for apt-key

* Make sure ASan link flags come first

* Pass CI env variable to cmake invocation

* Install package for apt-add-repository

* Remove return() from flush macro

* Use directory level commands instead of props

* Restore the github repository variable

* Set developer mode unconditionally for checkperf

The CI env variable is only set in the CI and this target is always run
in developer mode.

* Attempt to fix ODR violation in parsing checks

These tests were compiling the simdjson.cpp file again and linking to
the simdjson library target causes ODR violations.

Instead of linking to the target, just inherit its props.

* Move variables before the source dir

* Mark props to be flushed after adding more

* Use props for every command for the library

* Use keyword form for linking libs

* Handle deprecation of SIMDJSON_JUST_LIBRARY

* Handle deprecations in a separate module

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2021-04-23 09:24:56 -04:00
Paul Dreik a79bbd63a3 use github action cache instead of bintray (#1536)
* use github action cache instead of bintray

* add note on where to get the corpus

Co-authored-by: Paul Dreik <paul@simdjson>
2021-04-12 16:58:30 -04:00
Daniel Lemire dc2f767171 Ensuring that stage 1's json_block and cie are never copied and using explicit constructors (issue 1475) (#1478) 2021-03-04 07:59:36 -05:00
Daniel Lemire 9577c54999 Provide the CMake install the necessarily information (and flags) to hand Windows DLL and add Windows installation tests (#1457)
* This gives the CMake install the necessarily information (and flags) to know
whether we have a Windows DLL and in such cases how to handle the linkage.
2021-02-26 16:17:05 -05:00
Daniel Lemire 72c40be553 This adds tests without threads under linux. (#1460) 2021-02-26 16:16:10 -05:00
Daniel Lemire 6aa8877bf6 This adds tests for the no-exception variant of the library usage under linux. (#1459)
* This adds tests for the no-exception variant of the library usage under linux.

* Fixed name.

* Syntax fix.
2021-02-26 16:15:34 -05:00
Daniel Lemire 610b3ad302 Adds Visual Studio 2017 to CI (for real) and adapt our build/tests (#1444) 2021-02-15 19:49:12 -05:00
Daniel Lemire 0f72ff3a57 Adding Visual studio 2017 CI tests to GitHub actions. (#1438)
* Adding Visual studio 2017 CI tests to GitHub actions.

* Fixing the name.
2021-02-09 10:05:45 -05:00
Daniel Lemire 67afcd4edd Adding legacy Win32 / Visual Studio 2017 CI tests to GitHub Actions. (#1439) 2021-02-09 10:05:33 -05:00
Daniel Lemire a509e04f9b Let us actually test the find package functionality. (#1418)
* Let us actually test the find package functionality.

* Specifying the path.

* Fixing the path.
2021-01-28 13:59:35 -05:00
Daniel Lemire d6f33e4830 This adds a little test to see if we can compiler with very strict flags (conventional casts) (#1417)
* This adds a little test to see if we can compiler with very strict flags.

* Trimming a leftover old-style cast.

* More cleaning.

* A few more pedantic casts.
2021-01-27 18:37:30 -05:00
friendlyanon 451c393ef1 [skip ci] Skip CI for Github (#1336)
* Add conditional to github workflows

* [skip ci] Document and encourage skipping CI

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2021-01-11 19:08:01 -05:00
Daniel Lemire f29b70c63e Adding GitHub Actions VS - without exceptions. (#1362)
* Adding GitHub Actions VS - without exceptions.

* Rename.
2021-01-06 09:40:35 -05:00
John Keiser 158a3c53d8 Merge pull request #1351 from simdjson/jkeiser/unordered-lookup
Make `object["field"]` order-insensitive in On Demand
2020-12-24 13:33:09 -08:00
John Keiser 92443772d2 Enable all acceptance tests on mingw 2020-12-23 09:14:45 -08:00
John Keiser 041d59cc17 Create acceptance_tests, all_tests, etc. make targets
And use them for mingw build and test
2020-12-23 09:14:45 -08:00
Daniel Lemire 0b67847b56 Adding clang/msys-ci (#1354) 2020-12-22 17:53:53 -05:00
John Keiser f9c6dedca1 Add test for out-of-order parse asserts 2020-12-13 13:39:47 -08:00
Paul Dreik 5f7b2bac12 fuzz on power (#1326)
* first try

* use ubuntu 20.04, do the fuzzing

* new try at power fuzz

* hard code clang version

* setting env variables does not seem to work

* use fuzzer-no-link

* switch to Debian Buster for power fuzz

* use non-sanitizer build for power

* me not like yaml

* fix bad syntax
2020-12-07 18:12:36 -05:00
Daniel Lemire aaa12c8fb6 more thread testing (#1324)
* Adding a few tests.

* More tests.

* Trailing.

* More links/documentations.
2020-12-02 11:33:36 -05:00
Paul Dreik 725ca010e7 add ndjson fuzzer (#1304)
* add ndjson fuzzer

* reproduce #1310 in the newly added unit test

Had to replace the input, because:
1)
the fuzzer uses the first part of the input to determine
the batch_size to use, so that has to be cut off

2)
the master now protects against low values of batch_size

I also made the test not return early, so the error is triggered.
2020-12-01 15:58:41 -05:00
Daniel Lemire 59c857e969 Merge branch 'master' of github.com:simdjson/simdjson 2020-12-01 15:31:44 -05:00
Daniel Lemire 45c9136443 Fixed odd issue. 2020-12-01 15:31:24 -05:00
Daniel Lemire bc4087ac96 Hunting for data races in document_stream (#1323)
* Adding matching tests.

* Actually adding thread testing actions.
2020-12-01 12:40:15 -05:00
Daniel Lemire d9c4191e8a This should fix an issue we have with unclosed strings with document_stream (#1321)
* This should fix an issue we have with unclosed strings.

* Patching the fallback kernel.

* Better guarding the code.

* Patching the fallback.
2020-11-30 13:43:57 -05:00
Paul Dreik 0b39e3a6cf add fuzzer for padded_string (#1312)
This also fixes an overflow problem.
2020-11-19 16:51:56 +01:00
friendlyanon 553befa012 Disable Github CI in feature branches (#1298)
Forks that would like to contribute via PRs from feature branches
needlessly run CI on those branches on top of the PRs.
This is a waste of resources.

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-10 13:52:58 -05:00
Daniel Lemire f195f849d0 Helping contributors. (#1289) 2020-11-05 10:27:35 -05:00
friendlyanon 669e5b91b4 Improve CI workflows (#1291)
* Fix Cirrus CI

CMake is not installed at this point yet.

* Add caching in Circle CI

* Add caching to the MinGW Github workflows

* Fix Circle CI config

Co-authored-by: friendlyanon <friendlyanon@users.noreply.github.com>
2020-11-05 08:42:00 -05:00