Commit Graph

46 Commits

Author SHA1 Message Date
Daniel Lemire 37e87f6f99 Adding support for wobbly strings to On Demand and lossy trancoding from escaped (with replacement) (#1947)
* Adding support for wobbly strings.

* Minor tweak.

* Adopt substitution code point.

* Lossy

* Patching
2023-02-28 22:56:44 -05:00
strager 5510089d45 Improve build times for debug builds (#1859)
* Rename simdjson_really_inline -> simdjson_inline

I want to change the simdjson_really_inline macro to sometimes not force
inlining. After that upcoming change, the name simdjson_really_inline
will no longer makes sense.

Rename simdjson_really_inline to simdjson_inline. This patch should not
change semantics; simdjson_inline still forces inlining as before.

Some functions still need to be really inlined for ABI reasons.
(GCC's -Wpsabi complains otherwise.) Leave those functions marked as
simdjson_really_inline.

* Improve build times for debug builds

simdjson_inline is used for most simdjson functions. It forces inlining.
In unoptimized/debug builds, this can lead to a lot of machine code
being generated (especially with Address Sanitizer), causing slow
compilation.

Change simdjson_inline to force inlining only for optimized builds.

Sometimes, the programmer might want a slightly-optimized build and want
fast compilation (e.g. GCC's -Og mode). Allow simdjson users to define
the simdjson_inline macro themselves (e.g. on the command line:
-Dsimdjson_inline=inline) in cases where the default behavior is
undesired.

This patch reduced build times by over 75% for ondemand_object_tests.cpp
with GCC 9.4.0 and CMAKE_BUILD_TYPE=Debug on my AMD 5950X:

Before: 6.885 6.683 6.971 6.957 6.949 seconds (5 samples)
After:  1.492 1.551 1.494 1.490 1.531 seconds (5 samples)
2022-07-19 15:14:33 -04:00
Daniel Lemire 3fde8a4eac New release candidate (#1856)
* Patch for possible AVX-512 overflow.

* Updating the test for new padding.

* Preparing new version.

* replace binary integer literals with hex literals for C++11 compatibility (#1855)

Binary integer literals are a C++14 feature, so those are not supported
in C++11 and should be replaced by hexadecimal literals instead.

Fixes #1854.

Co-authored-by: Dirk Stolle <striezel-dev@web.de>
2022-07-04 21:54:19 -04:00
Daniel Lemire 1a195623a5 Improve string performance in ondemand by making the string processing runtime dispatched. (#1849)
* This should improve string performance in ondemand by making the string processing runtime dispatched.
2022-06-24 09:57:16 -04:00
Daniel Lemire 8eed8f5155 Document stream: truncate final unfinished document and give access to the number of truncated bytes. (#1534)
* Truncate final unclosed string.

* Adding more precise remarks.

* Better documentation and more robust code.

* ARM + PPC corrections.

* Patching ARM implementation with new stage1_mode parameter.

* Fixed most problems.

* Correcting white spaces and adding a remark.

* This adds the truncated_bytes() method to the stream instances.
2021-04-23 09:24:00 -04:00
Daniel Lemire dc2f767171 Ensuring that stage 1's json_block and cie are never copied and using explicit constructors (issue 1475) (#1478) 2021-03-04 07:59:36 -05:00
Paul Dreik af4db55e66 remove trailing whitespace (#1284) 2020-11-03 21:48:09 +01:00
Daniel Lemire bb2bc98a22 Fix issue https://github.com/simdjson/simdjson/issues/1127 (#1224) 2020-10-13 09:18:54 -04:00
John Keiser a700848bae Move ondemand implementation to include/ 2020-10-04 12:47:30 -07:00
John Keiser 9bbfd5804e Put haswell/westmere/fallback/amd64 in simdjson namespace 2020-10-04 12:47:30 -07:00
John Keiser b234d74f43 Remove unnamed namespace from ondemand 2020-10-04 12:47:30 -07:00
John Keiser ebcb3c6b3b On-demand parse implementation 2020-10-04 12:47:29 -07:00
John Keiser f0ec26992a Remove bit_or (bad perf on Windows) 2020-09-01 08:43:09 -07:00
John Keiser 62e8332b34 Use simd8x64 abstractions in classification 2020-09-01 08:43:09 -07:00
John Keiser 0925f71987 Simplify operator classification lookup on Intel 2020-09-01 08:43:07 -07:00
John Keiser 04d39c0961 Make tape_builder primary entry point to stage 2 2020-08-18 21:25:01 -07:00
Daniel Lemire 8a8eea53a2 Prefixing macros (issue 1035) (#1124)
* Renaming partially done.

* More prefixing.

* I thought that this was fixed.

* Missed one.

* Missed a few.

* Missed another one.

* Minor fixes.
2020-08-18 18:25:36 -04:00
Daniel Lemire 17f6d5208f Documenting and fixing the case where a string is immediately followed by a scalar (#1106)
* Documenting and fixing.

* More cleaning.

* Being a bit cleaner.
2020-08-14 16:19:57 -04:00
John Keiser ec28acba3d De-templatize stage2::structural_parser 2020-08-03 23:09:15 -07:00
John Keiser ee6647ce40 Make parse part of structural_parser 2020-08-03 17:50:51 -07:00
John Keiser 66a68ce264 Return errors immediately instead of using goto 2020-08-02 12:04:12 -07:00
John Keiser 379a4e6a01 namespace { -> unnamed namespace 2020-08-01 14:46:23 -07:00
John Keiser 4e944a9f3c Eliminate unused functions in fallback 2020-08-01 14:43:10 -07:00
John Keiser c7fa9b5fe8 Make entire implementation namespaces anonymous 2020-08-01 14:43:10 -07:00
John Keiser 65148b123b Put anonymous namespace in front of everything 2020-08-01 14:43:10 -07:00
John Keiser bf67c967d6 Inline jsoncharutils per-implementation 2020-07-24 02:56:41 -07:00
John Keiser 44b7a7145c Include bitmanip/simd everywhere 2020-07-24 02:56:39 -07:00
John Keiser 3867ee71ed Include files where they are used 2020-07-24 02:56:37 -07:00
John Keiser 464f4813e3 Define namespaces inside generic files 2020-07-24 02:56:36 -07:00
John Keiser af8b52e7e8 Target region for entire compilation of an implementation 2020-07-24 02:48:25 -07:00
Daniel Lemire 4beb2ed507 Make simd8 64 uncopyable and other Visual Studio optimizations (#1031)
* Working on making simd8x64 immutable


* Even less invasive
2020-07-21 18:11:21 -04:00
Daniel Lemire e9c91a1ce2 lookup4 (new UTF-8 validation) (#993)
* lookup4

* Self-document lookup4 and clean up extra bits

* Maintenance, to match against upcoming PR.

Co-authored-by: Daniel Lemire <lemire@gmai.com>
Co-authored-by: John Keiser <john@johnkeiser.com>
2020-07-20 18:20:07 -04:00
Daniel Lemire d0ce2f0b5a Fixing clang under visual studio (#1028)
* Lots of fixes

* Removing some lambdas

* Removing some functional programming.

Co-authored-by: Daniel Lemire <lemire@gmai.com>
2020-07-06 18:58:19 -04:00
Daniel Lemire a6cbf1f922 Going generic... 2020-06-22 16:25:11 -04:00
Daniel Lemire f03a6ab5a4 Tweaking. 2020-06-21 17:39:24 -04:00
Daniel Lemire 04139eb82e Ok. 2020-06-21 17:05:55 -04:00
Daniel Lemire 942ef3b7f2 Merge pull request #939 from simdjson/dlemire/lookup3
Introducing lookup3 (UTF-8 validation).
2020-06-17 11:19:09 -04:00
Daniel Lemire 71a889ed73 Introducing lookup3 (UTF-8 validation). 2020-06-16 19:08:25 -04:00
John Keiser 610c79fbf3 Don't use backslash branch on ARM 2020-06-13 07:51:28 -07:00
John Keiser a5beffda78 Remove streaming_structural_parser.h 2020-06-08 15:20:56 -07:00
John Keiser 0dbda65e44 Fix fallback implementation 2020-06-08 14:52:23 -07:00
John Keiser ed0c815735 Move unclosed array check to stage 2 2020-06-05 12:39:13 -07:00
Daniel Lemire 7a69da16e4 Fixing issue 906 (#912)
* Fixing issue 906

* Safe patching.

* Now with explanations.

* Bumping up memory allocation.

* Putting the patch back.

* fallback fixes.

Co-authored-by: Daniel Lemire <lemire@gmai.com>
2020-06-05 15:37:09 -04:00
John Keiser 1aab4752e2 Store all parser state in the implementation 2020-06-01 12:15:54 -07:00
John Keiser 84712a8bbc Store buf and len in parser implementation 2020-06-01 12:14:09 -07:00
John Keiser a3a9bde83e Move DOM parsing into concrete interface implementation 2020-06-01 12:14:09 -07:00