New release

Fix for issue 1981 (#1982 )
* Fix * Rewording. * Doing it differently.
2026-06-08 17:27:07 +00:00 · 2023-04-08 16:13:48 -04:00 · 2023-04-08 16:11:40 -04:00 · 2023-04-06 10:52:18 -04:00 · 2023-04-04 14:22:12 -04:00 · 2023-04-04 11:03:37 -04:00
46 changed files with 783 additions and 227 deletions
@@ -0,0 +1,24 @@
+name: Ubuntu 22.04 CI GCC 12 with GLIBCXX_ASSERTIONS
+on: [push, pull_request]
+
+jobs:
+  ubuntu-build:
+    if: >-
+      ! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
+      ! contains(toJSON(github.event.commits.*.message), '[skip github]')
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/cache@v3
+        with:
+          path: dependencies/.cache
+          key: ${{ hashFiles('dependencies/CMakeLists.txt') }}
+      - name: Install gcc12
+        run: sudo apt-get install -y g++-12
+      - name: Use cmake
+        run: |
+          mkdir build &&
+          cd build &&
+          CXX=g++-12 cmake -DCMAKE_BUILD_TYPE=Debug  -DSIMDJSON_GLIBCXX_ASSERTIONS=ON -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON ..  &&
+          cmake --build .   &&
+          ctest . -E avoid_
@@ -96,6 +96,7 @@
        "future": "cpp",
        "queue": "cpp",
        "shared_mutex": "cpp",
-        "ranges": "cpp"
+        "ranges": "cpp",
+        "span": "cpp"
    }
 }
@@ -3,7 +3,7 @@ cmake_minimum_required(VERSION 3.14)
 project(
    simdjson
    # The version number is modified by tools/release.py
-    VERSION 3.1.5
+    VERSION 3.1.7
    DESCRIPTION "Parsing gigabytes of JSON per second"
    HOMEPAGE_URL "https://simdjson.org/"
    LANGUAGES CXX C
@@ -251,7 +251,9 @@ add_subdirectory(singleheader)
 #
 add_subdirectory(tests)
 add_subdirectory(examples)
-add_subdirectory(benchmark)
+if(CMAKE_SIZEOF_VOID_P EQUAL 8) # we only include the benchmarks on 64-bit systems.
+  add_subdirectory(benchmark)
+endif()
 add_subdirectory(fuzz)

 #
@@ -38,7 +38,7 @@ PROJECT_NAME           = simdjson
 # could be handy for archiving the generated documentation or if some version
 # control system is used.

-PROJECT_NUMBER         = "3.1.5"
+PROJECT_NUMBER         = "3.1.7"

 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a
@@ -1,3 +1,4 @@
+
 Hacking simdjson
 ================

@@ -6,6 +7,16 @@ Here is wisdom about how to build, test and run simdjson from within the reposit
 If you plan to contribute to simdjson, please read our [CONTRIBUTING](https://github.com/simdjson/simdjson/blob/master/CONTRIBUTING.md) guide.


+  - [Design notes](#design-notes)
+  - [Developer mode](#developer-mode)
+  - [Directory Structure and Source](#directory-structure-and-source)
+  - [Runtime Dispatching](#runtime-dispatching)
+  - [Regenerating Single-Header Files](#regenerating-single-header-files)
+  - [Usage (CMake on 64-bit platforms like Linux, FreeBSD or macOS)](#usage-cmake-on-64-bit-platforms-like-linux-freebsd-or-macos)
+  - [Usage (CMake on 64-bit Windows using Visual Studio 2019)](#usage-cmake-on-64-bit-windows-using-visual-studio-2019)
+  - [Various References](#various-references)
+
+
 Design notes
 ------------------------------

@@ -193,7 +204,7 @@ point it gets included (but only once per header). singleheader/simdjson.cpp is
 src/simdjson.cpp the same way, except files under generic/ may be included and copy/pasted multiple
 times.

-### Usage (CMake on 64-bit platforms like Linux, FreeBSD or macOS)
+## Usage (CMake on 64-bit platforms like Linux, FreeBSD or macOS)

 Requirements: In addition to git, we require a recent version of CMake as well as bash.

@@ -256,7 +267,9 @@ Note that the name of directory (`build`) is arbitrary, you can name it as you w



-### Usage (CMake on 64-bit Windows using Visual Studio 2019)
+## Usage (CMake on 64-bit Windows using Visual Studio 2019 or better)
+
+Recent versions of Visual Studio support CMake natively, [please refer to the Visual Studio documentation](https://learn.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-170).

 We assume you have a common 64-bit Windows PC with at least Visual Studio 2019.

@@ -285,7 +298,7 @@ Furthermore, if you have installed LLVM clang on Windows, for example as a compo
 - `cmake --build . -config Release`


-### Various References
+## Various References

 - [How to implement atoi using SIMD?](https://stackoverflow.com/questions/35127060/how-to-implement-atoi-using-simd)
 - [Parsing JSON is a Minefield 💣](http://seriot.ch/parsing_json.php)
@@ -1,6 +1,7 @@

 [![Ubuntu 20.04 CI](https://github.com/simdjson/simdjson/workflows/Ubuntu%2020.04%20CI%20(GCC%209)/badge.svg)](https://simdjson.org/plots.html)
 ![VS16-CI](https://github.com/simdjson/simdjson/workflows/VS16-CI/badge.svg)
+[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/simdjson.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:simdjson)
 ![MinGW64-CI](https://github.com/simdjson/simdjson/workflows/MinGW64-CI/badge.svg)
 [![][license img]][license]

@@ -564,7 +564,7 @@ static void error_code_twitter_default_profile(State& state) noexcept {
    for (dom::element tweet : tweets) {
      dom::object user;
      if ((error = tweet["user"].get(user))) { return; }
-      bool default_profile;
+      bool default_profile{};
      if ((error = user["default_profile"].get(default_profile))) { return; }
      if (default_profile) {
        std::string_view screen_name;
@@ -100,9 +100,8 @@ simdjson2msgpack::to_msgpack(const simdjson::padded_string &json,
    recursive_processor_ref(val);
 #endif
  }
-  if (doc.current_location().error() == simdjson::SUCCESS) {
-    // Example of error detection - this won't be reached on twitter.json in the benchmark.
-    throw "There are unexpectedly tokens after the end of the json in the json2msgpack sample data";
+  if (!doc.at_end()) {
+     throw "There are unexpectedly tokens after the end of the json in the json2msgpack sample data";
  }
  return std::string_view(reinterpret_cast<char *>(buf), size_t(buff - buf));
 }
@@ -50,6 +50,23 @@ undefined behavior.")
  endif()
 endif()

+option(SIMDJSON_SANITIZE_MEMORY "Sanitize memory" OFF)
+
+
+if(SIMDJSON_SANITIZE_MEMORY)
+  message(STATUS "Setting the memory sanitizer.")
+  add_compile_options(
+      -fsanitize=memory -fno-sanitize-recover=all
+  )
+  link_libraries(
+      -fsanitize=memory -fno-sanitize-recover=all
+  )
+  # Ubuntu bug for GCC 5.0+ (safe for all versions)
+  if(CMAKE_COMPILER_IS_GNUCC)
+    link_libraries(-fuse-ld=gold)
+  endif()
+endif()
+
 if(SIMDJSON_SANITIZE_THREADS)
  message(STATUS "Setting both the thread sanitizer \
 and the undefined-behavior sanitizer.")
@@ -73,6 +90,8 @@ if(NOT is_multi_config AND NOT CMAKE_BUILD_TYPE)
  if(SIMDJSON_SANITIZE OR SIMDJSON_SANITIZE_UNDEFINED)
    message(STATUS "No build type selected and you have enabled the sanitizer, \
 default to Debug. Consider setting CMAKE_BUILD_TYPE.")
+    message(STATUS "Setting debug optimization flag to -O1 to help sanitizer.")
+    set(CMAKE_CXX_FLAGS_DEBUG "-O1" CACHE STRING "" FORCE)
    set(CMAKE_BUILD_TYPE Debug CACHE STRING "Choose the type of build." FORCE)
  else()
    message(STATUS "No build type selected, default to Release")
@@ -144,6 +163,11 @@ else()
  )
 endif()

+option(SIMDJSON_GLIBCXX_ASSERTIONS "Set _GLIBCXX_ASSERTIONS" OFF)
+if (SIMDJSON_GLIBCXX_ASSERTIONS)
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_GLIBCXX_ASSERTIONS")
+endif()
+
 #
 # Other optional flags
 #
@@ -23,6 +23,7 @@ An overview of what you need to know to use simdjson, with examples.
    - [Disabling Exceptions](#disabling-exceptions)
    - [Exceptions](#exceptions)
    - [Current location in document](#current-location-in-document)
+    - [Checking for trailing content](#checking-for-trailing-content)
  - [Rewinding](#rewinding)
  - [Direct Access to the Raw String](#direct-access-to-the-raw-string)
  - [Newline-Delimited JSON (ndjson) and JSON lines](#newline-delimited-json-ndjson-and-json-lines)
@@ -33,6 +34,7 @@ An overview of what you need to know to use simdjson, with examples.
  - [Standard Compliance](#standard-compliance)
  - [Backwards Compatibility](#backwards-compatibility)
  - [Examples](#examples)
+  - [Performance Tips](#performance-tips)


 Requirements
@@ -180,6 +182,21 @@ ondemand::document doc = parser.iterate(json, strlen(json), sizeof(json));
 The simdjson library will also accept `std::string` instances, as long as the `capacity()` of
 the string exceeds the `size()` by at least `SIMDJSON_PADDING`. You can increase the `capacity()` with the `reserve()` function of your strings.

+You can copy your data directly on a `simdjson::padded_string` as follows:
+
+```c++
+const char * data = "my data"; // 7 bytes
+simdjson::padded_string my_padded_data(data, 7); // copies to a padded buffer
+```
+
+Or as follows...
+
+```c++
+std::string data = "my data";
+simdjson::padded_string my_padded_data(data); // copies to a padded buffer
+```
+
+
 We recommend against creating many `std::string` or many `std::padding_string` instances in your application to store your JSON data.
 Consider reusing the same buffers and limiting memory allocations.

@@ -311,15 +328,19 @@ support for users who avoid exceptions. See [the simdjson error handling documen
  `double(element)`. This works for `std::string_view`, double, uint64_t, int64_t, bool,
  ondemand::object and ondemand::array. We also have explicit methods such as `get_string()`, `get_double()`,
  `get_uint64()`, `get_int64()`, `get_bool()`, `get_object()` and `get_array()`. After a cast or an explicit method,
-  the number, string or boolean will be parsed, or the initial `[` or `{` will be verified. An exception is thrown if
-  the cast is not possible. The `get_string()` returns a valid UTF-8 string, after
+  the number, string or boolean will be parsed, or the initial `{` or `[` will be verified for ondemand::object and ondemand::array. An exception is thrown if
+  the cast is not possible. Importantly, when getting an ondemand::object or ondemand::array instance, its content is
+  not validated: you are only guaranteed that the corresponding initial character (`{` or `[`) is present. Thus,
+  for example, you could have an ondemand::object instance pointing at the invalid JSON `{ "this is not a valid object" }`: the validation occurs as you access the content.
+  The `get_string()` returns a valid UTF-8 string, after
  unescaping characters as needed: unmatched surrogate pairs are treated as an error unless you
  pass `true` (`get_string(true)`) as a parameter to get replacement characters where errors
  occur. If you somehow need to access non-UTF-8 strings in a lossless manner
  (e.g., if you strings contain unpaired surrogates), you may use the `get_wobbly_string()` function to get a string in the [WTF-8 format](https://simonsapin.github.io/wtf-8).
-  Or you may pass `true` as a parameter to the
  When calling `get_uint64()` and `get_int64()`, if the number does not fit in a corresponding
-  64-bit integer type, it is also considered an error.
+  64-bit integer type, it is also considered an error. When parsing numbers or other scalar values, the library checks
+  that the value is followed by an expected character, thus you *may* get a number parsing error when accessing the digits
+  as an integer in the following strings: `{"number":12332a`, `{"number":12332\0`, `{"number":12332` (the digits appear at the end). We always abide by the [RFC 8259](https://www.tbray.org/ongoing/When/201x/2017/12/14/rfc8259.html) JSON specification so that, for example, numbers prefixed by the `+` sign are in error.

  > IMPORTANT NOTE: values can only be parsed once. Since documents are *iterators*, once you have
  > parsed a value (such as by casting to double), you cannot get at it again. It is an error to call
@@ -451,7 +472,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
  ```
  This examples also show how we can string several operations and only check for the error once, a strategy we call  *error chaining*.
  Though error chaining makes the code very compact, it also makes error reporting less precise: in this instance, you may get the
-  same error whether the field "str", "123" or "abc" is missing. If you need to break down error handling per operation, avoid error chaining.
+  same error whether the field "str", "123" or "abc" is missing. If you need to break down error handling per operation, avoid error chaining. Furthermore, you should be mindful that chaining that harm performance by encouraging redundancies: writing both `doc["str"]["123"]["abc"].get(value)` and `doc["str"]["123"]["zyw"].get(value)` in the same program may force multiple accesses to the same keys (`"str"` and `"123"`).
 * **Counting elements in arrays:** Sometimes it is useful to scan an array to determine its length prior to parsing it.
  For this purpose, `array` instances have a `count_elements` method. Users should be
  aware that the `count_elements` method can be costly since it requires scanning the
@@ -1023,7 +1044,7 @@ bool parse() {
    cout << "Make/Model: " << make << "/" << model << endl;

    // Casting a JSON element to an integer
-    uint64_t year;
+    uint64_t year{};
    error = car["year"].get(year);
    if(error) { std::cerr << error << std::endl; return false; }
    cout << "- This car is " << 2020 - year << " years old." << endl;
@@ -1124,7 +1145,7 @@ int main(void) {
 ### Current location in document

 Sometimes, it might be helpful to know the current location in the document during iteration. This is especially useful when encountering errors. The `current_location()` method on a
-`document` instances makes it easy to identify common JSON errors. Users can call the `current_location()` method on a validdocument instance to retrieve a `const char *` pointer to the current location in the document. This method also works even after an error has invalidated the document and the parser (e.g. `TAPE_ERROR`, `INCOMPLETE_ARRAY_OR_OBJECT`).
+`document` instances makes it easy to identify common JSON errors. Users can call the `current_location()` method on a valid document instance to retrieve a `const char *` pointer to the current location in the document. This method also works even after an error has invalidated the document and the parser (e.g. `TAPE_ERROR`, `INCOMPLETE_ARRAY_OR_OBJECT`).
 When the input was a `padding_string` or another null-terminated source, then you may
 use the `const char *` pointer as a C string. As an example, consider the following
 example where we used the exception-free simdjson interface:
@@ -1137,6 +1158,12 @@ int64_t i;
 auto error = doc["integer"].get_int64().get(i);    // Expect to get integer from "integer" key, but get TAPE_ERROR
 if (error) {
  std::cout << error << std::endl;    // Prints TAPE_ERROR error message
+  // Recover a pointer to the location of the first error:
+  const char * ptr;
+  doc.current_location().get(ptr);
+  // ptr points at 'false, "integer": -343} " which is the location of the error
+  //
+  // Because we pad simdjson::padded_string instances with null characters, you may also do the following:
  std::cout<< doc.current_location() << std::endl;  // Prints "false, "integer": -343} " (location of TAPE_ERROR)
 }
 ```
@@ -1213,6 +1240,32 @@ contains control characters that must be escaped and UNCLOSED_STRING if there
 is an unclosed string in the document. We do not provide location information for these
 errors.

+### Checking for trailing content
+
+The parser validates all parsed content, but your code may exhaust the content while
+not having processed the entire document. Thus, as a final optional step, you may
+call `at_end()` on the document instance. If it returns `false`, then you may
+conclude that you have trailing content and that your document is not valid JSON.
+You may then use `doc.current_location()` to obtain a pointer to the start of the trailing
+content.
+
+```C++
+  auto json = R"([1, 2] foo ])"_padded;
+  ondemand::parser parser;
+  ondemand::document doc = parser.iterate(json);
+  ondemand::array array = doc.get_array();
+  for (uint64_t values : array) {
+    std::cout << values << std::endl;
+  }
+  if(!doc.at_end()) {
+    // In this instance, we will be left pointing at 'foo' since we have consumed the array [1,2].
+    std::cerr << "trailing content at byte index " << doc.current_location() - json.data() << std::endl;
+  }
+```
+
+The `at_end()` method is equivalent to `doc.current_location().error() == simdjson::SUCCESS` but
+more convenient.
+
 Rewinding
 ----------

@@ -1817,3 +1870,28 @@ bool example() {
  return true;
 }
 ```
+
+
+Performance Tips
+--------
+
+
+- The On Demand front-end works best when doing a single pass over the input: avoid calling `count_elements`, `rewind` and similar methods.
+- If you are familiar with assembly language, you may use the online tool godbolt to explore the compiled code. The following example may work: [https://godbolt.org/z/xE4GWs573](https://godbolt.org/z/xE4GWs573).
+- Given a field `field` in an object, calling `field.key()` is often faster than `field.unescaped_key()` so if you do not need an unescaped `std::string_view` instance, prefer `field.key()`.
+- For release builds, we recommend setting `NDEBUG` pre-processor directive when compiling the `simdjson` library. Importantly, using the optimization flags `-O2` or `-O3` under GCC and LLVM clang does not set the `NDEBUG` directive, you must set it manually (e.g., `-DNDEBUG`).
+- For long streams of JSON documents, consider [`iterate_many`](iterate_many.md) and [`parse_many`](parse_many.md) for better performance.
+- If possible, refer to each object and array in your code once. For example, the following code repeatedly refers to the `"data"` key to create an object...
+	```C++
+	std::string_view make = o["data"]["make"];
+	std::string_view model = o["data"]["model"];
+	std::string_view year = o["data"]["year"];
+  ```
+  We expect that it is more efficient to access the `"data"` key once:
+  ```C++
+	simdjson::ondemand::object data = o["data"];
+	std::string_view model = data["model"];
+	std::string_view year = data["year"];
+	std::string_view rating = data["rating"];
+  ```
+- To better understand the operation of your On Demand parser, and whether it is performing as well as you think it should be, there is a  logger feature built in to simdjson! To use it, define the pre-processor directive `SIMDJSON_VERBOSE_LOGGING` prior to including the `simdjson.h` header, which enables logging in simdjson. Run your code. It may generate a lot of logging output; adding printouts from your application that show each section may be helpful. The log’s output will show step-by-step information on state, buffer pointer position, depth, and key retrieval status.
@@ -16,6 +16,7 @@ An overview of what you need to know to use simdjson, with examples.
 * [Server Loops: Long-Running Processes and Memory Capacity](#server-loops-long-running-processes-and-memory-capacity)
 * [Best Use of the DOM API](#best-use-of-the-dom-api)
 * [Padding and Temporary Copies](#padding-and-temporary-copies)
+* [Performance Tips](#performance-tips)

 DOM vs On Demand
 ----------------------------------------------
@@ -44,6 +45,20 @@ dom::parser parser;
 dom::element doc = parser.parse("[1,2,3]"_padded); // parse a string, the _padded suffix creates a simdjson::padded_string instance
 ```

+You can copy your data directly on a `simdjson::padded_string` as follows:
+
+```c++
+const char * data = "my data"; // 7 bytes
+simdjson::padded_string my_padded_data(data, 7); // copies to a padded buffer
+```
+
+Or as follows...
+
+```c++
+std::string data = "my data";
+simdjson::padded_string my_padded_data(data); // copies to a padded buffer
+```
+
 The parsed document resulting from the `parser.load` and `parser.parse` calls depends on the `parser` instance. Thus the `parser` instance must remain in scope. Furthermore, you must have at most one parsed document in play per `parser` instance.
 You cannot copy a `parser` instance, you may only move it.

@@ -644,3 +659,9 @@ simdjson::dom::element element = parser.parse(padded_json_copy.get(), json_len,
 ````

 Setting the `realloc_if_needed` parameter `false` in this manner may lead to better performance since copies are avoided, but it requires that the user takes more responsibilities: the simdjson library cannot verify that the input buffer was padded with SIMDJSON_PADDING extra bytes.
+
+Performance Tips
+---------------------
+
+- For release builds, we recommend setting `NDEBUG` pre-processor directive when compiling the `simdjson` library. Importantly, using the optimization flags `-O2` or `-O3` under GCC and LLVM clang does not set the `NDEBUG` directrive, you must set it manually (e.g., `-DNDEBUG`).
+- For long streams of JSON documents, consider [`iterate_many`](iterate_many.md) and [`parse_many`](parse_many.md) for better performance.
@@ -3,7 +3,10 @@ Performance Notes

 simdjson strives to be at its fastest *without tuning*, and generally achieves this. However, there
 are still some scenarios where tuning can enhance performance.
-
+Once your code is tested, we
+further encourage you to define `NDEBUG` in your Release builds to disable additional runtime
+testing and get the best performance.
+* [NDEBUG directive](#ndebug-directive)
 * [Reusing the parser for maximum efficiency](#reusing-the-parser-for-maximum-efficiency)
 * [Reusing string buffers](#reusing-string-buffers)
 * [Server Loops: Long-Running Processes and Memory Capacity](#server-loops-long-running-processes-and-memory-capacity)
@@ -12,6 +15,18 @@ are still some scenarios where tuning can enhance performance.
 * [Visual Studio](#visual-studio)
 * [Power Usage and Downclocking](#power-usage-and-downclocking)

+
+NDEBUG directive
+-------------
+
+In C/C++, the `NDEBUG` pre-processor directive is not set by default. When it is not set, the simdjson library does
+many additional checks that may impact negatively the performance. We recommend that, once your code
+is well tested, you define `NDEBUG` directive in your Release builds. The `NDEBUG` directive should be defined
+prior to including the `simdjson.h` header.
+
+The `NDEBUG` directive is generally independent from optimization flags. For example, setting `-O3` under
+GCC does not set the `NDEBUG` directive.
+
 Reusing the parser for maximum efficiency
 -----------------------------------------

@@ -9,6 +9,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #ifdef SIMDJSON_REGULAR_VISUAL_STUDIO
  unsigned long ret;
@@ -226,7 +226,7 @@ inline error_code parser::ensure_capacity(document& target_document, size_t desi
 }

 simdjson_inline void parser::set_max_capacity(size_t max_capacity) noexcept {
-  if(max_capacity < MINIMAL_DOCUMENT_CAPACITY) {
+  if(max_capacity > MINIMAL_DOCUMENT_CAPACITY) {
    _max_capacity = max_capacity;
  } else {
    _max_capacity = MINIMAL_DOCUMENT_CAPACITY;
@@ -20,7 +20,7 @@ inline std::string document::to_debug_string() noexcept {
  return iter.to_string();
 }

-inline simdjson_result<const char *> document::current_location() noexcept {
+inline simdjson_result<const char *> document::current_location() const noexcept {
  return iter.current_location();
 }

@@ -28,6 +28,11 @@ inline int32_t document::current_depth() const noexcept {
  return iter.depth();
 }

+inline bool document::at_end() const noexcept {
+  return iter.at_end();
+}
+
+
 inline bool document::is_alive() noexcept {
  return iter.is_alive();
 }
@@ -49,14 +54,24 @@ simdjson_inline simdjson_result<value> document::get_value() noexcept {
  // gets called.
  iter.assert_at_document_depth();
  switch (*iter.peek()) {
-    case '[':
-    case '{':
+    case '[': {
+      // The following lines check that the document ends with ].
+      auto value_iterator = get_root_value_iterator();
+      auto error = value_iterator.check_root_array();
+      if(error) { return error; }
      return value(get_root_value_iterator());
+    }
+    case '{': {
+      // The following lines would check that the document ends with }.
+      auto value_iterator = get_root_value_iterator();
+      auto error = value_iterator.check_root_object();
+      if(error) { return error; }
+      return value(get_root_value_iterator());
+    }
    default:
      // Unfortunately, scalar documents are a special case in simdjson and they cannot
      // be safely converted to value instances.
      return SCALAR_DOCUMENT_AS_VALUE;
-      // return value(get_root_value_iterator());
  }
 }
 simdjson_inline simdjson_result<array> document::get_array() & noexcept {
@@ -497,6 +512,12 @@ simdjson_inline simdjson_result<const char *> simdjson_result<SIMDJSON_IMPLEMENT
  return first.current_location();
 }

+simdjson_inline bool simdjson_result<SIMDJSON_IMPLEMENTATION::ondemand::document>::at_end() const noexcept {
+  if (error()) { return error(); }
+  return first.at_end();
+}
+
+
 simdjson_inline int32_t simdjson_result<SIMDJSON_IMPLEMENTATION::ondemand::document>::current_depth() const noexcept {
  if (error()) { return error(); }
  return first.current_depth();
@@ -511,7 +511,14 @@ public:
  /**
   * Returns the current location in the document if in bounds.
   */
-  inline simdjson_result<const char *> current_location() noexcept;
+  inline simdjson_result<const char *> current_location() const noexcept;
+
+  /**
+   * Returns true if this document has been fully parsed.
+   * If you have consumed the whole document and at_end() returns
+   * false, then there may be trailing content.
+   */
+  inline bool at_end() const noexcept;

  /**
   * Returns the current depth in the document if in bounds.
@@ -720,6 +727,7 @@ public:
  simdjson_inline simdjson_result<bool> is_scalar() noexcept;
  simdjson_inline simdjson_result<const char *> current_location() noexcept;
  simdjson_inline int32_t current_depth() const noexcept;
+  simdjson_inline bool at_end() const noexcept;
  simdjson_inline bool is_negative() noexcept;
  simdjson_inline simdjson_result<bool> is_integer() noexcept;
  simdjson_inline simdjson_result<SIMDJSON_IMPLEMENTATION::ondemand::number_type> get_number_type() noexcept;
@@ -208,7 +208,7 @@ inline std::string json_iterator::to_string() const noexcept {
          + std::string(" ]");
 }

-inline simdjson_result<const char *> json_iterator::current_location() noexcept {
+inline simdjson_result<const char *> json_iterator::current_location() const noexcept {
  if (!is_alive()) {    // Unrecoverable error
    if (!at_root()) {
      return reinterpret_cast<const char *>(token.peek(-1));
@@ -358,19 +358,16 @@ simdjson_inline error_code json_iterator::optional_error(error_code _error, cons
  return _error;
 }

-template<int N>
-simdjson_warn_unused simdjson_inline bool json_iterator::copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t (&tmpbuf)[N]) noexcept {
+
+simdjson_warn_unused simdjson_inline bool json_iterator::copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t *tmpbuf, size_t N) noexcept {
+  // This function is not expected to be called in performance-sensitive settings.
  // Let us guard against silly cases:
  if((N < max_len) || (N == 0)) { return false; }
-  // Truncate whitespace to fit the buffer.
-  if (max_len > N-1) {
-    // if (jsoncharutils::is_not_structural_or_whitespace(json[N-1])) { return false; }
-    max_len = N-1;
-  }
-
  // Copy to the buffer.
  std::memcpy(tmpbuf, json, max_len);
-  tmpbuf[max_len] = ' ';
+  if(N > max_len) { // We pad whatever remains with ' '.
+    std::memset(tmpbuf + max_len, ' ', N - max_len);
+  }
  return true;
 }

@@ -238,7 +238,12 @@ public:
   */
  simdjson_inline error_code optional_error(error_code error, const char *message) noexcept;

-  template<int N> simdjson_warn_unused simdjson_inline bool copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t (&tmpbuf)[N]) noexcept;
+  /**
+   * Take an input in json containing max_len characters and attempt to copy it over to tmpbuf, a buffer with
+   * N bytes of capacity. It will return false if N is too small (smaller than max_len) of if it is zero.
+   * The buffer (tmpbuf) is padded with space characters.
+   */
+  simdjson_warn_unused simdjson_inline bool copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t *tmpbuf, size_t N) noexcept;

  simdjson_inline token_position position() const noexcept;
  /**
@@ -261,7 +266,7 @@ public:
  /**
   * Returns the current location in the document if in bounds.
   */
-  inline simdjson_result<const char *> current_location() noexcept;
+  inline simdjson_result<const char *> current_location() const noexcept;

  /**
   * Updates this json iterator so that it is back at the beginning of the document,
@@ -34,12 +34,17 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
  return true;
 }

-simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_object() noexcept {
+simdjson_warn_unused simdjson_inline error_code value_iterator::check_root_object() noexcept {
  // When in streaming mode, we cannot expect peek_last() to be the last structural element of the
  // current document. It only works in the normal mode where we have indexed a single document.
  // Note that adding a check for 'streaming' is not expensive since we only have at most
  // one root element.
  if ( ! _json_iter->streaming() ) {
+    // The following lines do not fully protect against garbage content within the
+    // object: e.g., `{"a":2} foo }`. Users concerned with garbage content should
+    // call `at_end()` on the document instance at the end of the processing to
+    // ensure that the processing has finished at the end.
+    //
    if (*_json_iter->peek_last() != '}') {
      _json_iter->abandon();
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "missing } at end");
@@ -56,6 +61,12 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "the document is unbalanced");
    }
  }
+  return SUCCESS;
+}
+
+simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_object() noexcept {
+  auto error = check_root_object();
+  if(error) { return error; }
  return started_object();
 }

@@ -419,12 +430,17 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
  return true;
 }

-simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_array() noexcept {
+simdjson_warn_unused simdjson_inline error_code value_iterator::check_root_array() noexcept {
  // When in streaming mode, we cannot expect peek_last() to be the last structural element of the
  // current document. It only works in the normal mode where we have indexed a single document.
  // Note that adding a check for 'streaming' is not expensive since we only have at most
  // one root element.
  if ( ! _json_iter->streaming() ) {
+    // The following lines do not fully protect against garbage content within the
+    // array: e.g., `[1, 2] foo]`. Users concerned with garbage content should
+    // also call `at_end()` on the document instance at the end of the processing to
+    // ensure that the processing has finished at the end.
+    //
    if (*_json_iter->peek_last() != ']') {
      _json_iter->abandon();
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "missing ] at end");
@@ -441,6 +457,12 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "the document is unbalanced");
    }
  }
+  return SUCCESS;
+}
+
+simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_array() noexcept {
+  auto error = check_root_array();
+  if (error) { return error; }
  return started_array();
 }

@@ -551,7 +573,7 @@ simdjson_inline simdjson_result<bool> value_iterator::is_root_integer(bool check
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("is_root_integer");
  uint8_t tmpbuf[20+1]; // <20 digits> is the longest possible unsigned integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    return false; // if there are more than 20 characters, it cannot be represented as an integer.
  }
  auto answer = numberparsing::is_integer(tmpbuf);
@@ -569,7 +591,7 @@ simdjson_inline simdjson_result<SIMDJSON_IMPLEMENTATION::ondemand::number_type>
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -584,7 +606,7 @@ simdjson_inline simdjson_result<number> value_iterator::get_root_number(bool che
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -612,7 +634,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<uint64_t> value_iterator::g
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("uint64");
  uint8_t tmpbuf[20+1]; // <20 digits> is the longest possible unsigned integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -627,7 +649,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<uint64_t> value_iterator::g
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("uint64");
  uint8_t tmpbuf[20+1]; // <20 digits> is the longest possible unsigned integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -642,7 +664,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<int64_t> value_iterator::ge
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("int64");
  uint8_t tmpbuf[20+1]; // -<19 digits> is the longest possible integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -658,7 +680,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<int64_t> value_iterator::ge
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("int64");
  uint8_t tmpbuf[20+1]; // -<19 digits> is the longest possible integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -677,7 +699,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<double> value_iterator::get
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -696,7 +718,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<double> value_iterator::get
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -711,7 +733,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::get_r
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("bool");
  uint8_t tmpbuf[5+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) { return incorrect_type_error("Not a boolean"); }
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 5+1)) { return incorrect_type_error("Not a boolean"); }
  auto result = parse_bool(tmpbuf);
  if(result.error() == SUCCESS) {
    if (check_trailing && !_json_iter->is_single_token()) { return TRAILING_CONTENT; }
@@ -109,7 +109,14 @@ public:
   * @error TAPE_ERROR if there is no matching } at end of document
   */
  simdjson_warn_unused simdjson_inline simdjson_result<bool> start_root_object() noexcept;
-
+  /**
+   * Checks whether an object could be started from the root. May be called by start_root_object.
+   *
+   * @returns SUCCESS if it is possible to safely start an object from the root (document level).
+   * @error INCORRECT_TYPE if there is no opening {
+   * @error TAPE_ERROR if there is no matching } at end of document
+   */
+  simdjson_warn_unused simdjson_inline error_code check_root_object() noexcept;
  /**
   * Start an object iteration after the user has already checked and moved past the {.
   *
@@ -234,7 +241,14 @@ public:
   * @error TAPE_ERROR if there is no matching ] at end of document
   */
  simdjson_warn_unused simdjson_inline simdjson_result<bool> start_root_array() noexcept;
-
+  /**
+   * Checks whether an array could be started from the root. May be called by start_root_array.
+   *
+   * @returns SUCCESS if it is possible to safely start an array from the root (document level).
+   * @error INCORRECT_TYPE If there is no [.
+   * @error TAPE_ERROR if there is no matching ] at end of document
+   */
+  simdjson_warn_unused simdjson_inline error_code check_root_array() noexcept;
  /**
   * Start an array iteration, after the user has already checked and moved past the [.
   *
@@ -9,6 +9,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  return (int)_tzcnt_u64(input_num);
@@ -9,6 +9,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  return (int)_tzcnt_u64(input_num);
@@ -35,9 +35,8 @@ inline char *allocate_padded_buffer(size_t length) noexcept {
  if (padded_buffer == nullptr) {
    return nullptr;
  }
-  // We write zeroes in the padded region to avoid having uninitized
-  // garbage. If nothing else, garbage getting read might trigger a
-  // warning in a memory checking.
+  // We write nulls in the padded region to avoid having uninitialized
+  // content which may trigger warning for some sanitizers
  std::memset(padded_buffer + length, 0, totalpaddedlength - length);
  return padded_buffer;
 } // allocate_padded_buffer()
@@ -67,7 +66,7 @@ inline padded_string::padded_string(std::string_view sv_) noexcept
    : viable_size(sv_.size()), data_ptr(internal::allocate_padded_buffer(sv_.size())) {
  if(simdjson_unlikely(!data_ptr)) {
    //allocation failed or zero size
-    viable_size=0;
+    viable_size = 0;
    return;
  }
  if (sv_.size()) {
@@ -148,6 +148,19 @@ use a 64-bit target such as x64, 64-bit ARM or 64-bit PPC.")
 #define SIMDJSON_NO_SANITIZE_UNDEFINED
 #endif

+
+#if defined(__clang__) || defined(__GNUC__)
+#if defined(__has_feature)
+#  if __has_feature(memory_sanitizer)
+#define SIMDJSON_NO_SANITIZE_MEMORY __attribute__((no_sanitize("memory")))
+#  endif // if __has_feature(memory_sanitizer)
+#endif // defined(__has_feature)
+#endif
+// make sure it is defined as 'nothing' if it is unapplicable.
+#ifndef SIMDJSON_NO_SANITIZE_MEMORY
+#define SIMDJSON_NO_SANITIZE_MEMORY
+#endif
+
 #if SIMDJSON_VISUAL_STUDIO
 // This is one case where we do not distinguish between
 // regular visual studio and clang under visual studio.
@@ -9,6 +9,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  unsigned long ret;
@@ -4,7 +4,7 @@
 #define SIMDJSON_SIMDJSON_VERSION_H

 /** The version of simdjson being used (major.minor.revision) */
-#define SIMDJSON_VERSION "3.1.5"
+#define SIMDJSON_VERSION "3.1.7"

 namespace simdjson {
 enum {
@@ -19,7 +19,7 @@ enum {
  /**
   * The revision (major.minor.REVISION) of simdjson being used.
   */
-  SIMDJSON_VERSION_REVISION = 5
+  SIMDJSON_VERSION_REVISION = 7
 };
 } // namespace simdjson

@@ -9,6 +9,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  unsigned long ret;
@@ -1,4 +1,4 @@
-/* auto-generated on 2023-03-09 11:14:42 -0500. Do not edit! */
+/* auto-generated on 2023-04-08 16:11:40 -0400. Do not edit! */
 /* begin file src/simdjson.cpp */
 #include "simdjson.h"

@@ -1558,37 +1558,37 @@ namespace simdjson {
 namespace internal {

  SIMDJSON_DLLIMPORTEXPORT const error_code_info error_codes[] {
-    { SUCCESS, "No error" },
-    { CAPACITY, "This parser can't support a document that big" },
-    { MEMALLOC, "Error allocating memory, we're most likely out of memory" },
-    { TAPE_ERROR, "The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc." },
-    { DEPTH_ERROR, "The JSON document was too deep (too many nested objects and arrays)" },
-    { STRING_ERROR, "Problem while parsing a string" },
-    { T_ATOM_ERROR, "Problem while parsing an atom starting with the letter 't'" },
-    { F_ATOM_ERROR, "Problem while parsing an atom starting with the letter 'f'" },
-    { N_ATOM_ERROR, "Problem while parsing an atom starting with the letter 'n'" },
-    { NUMBER_ERROR, "Problem while parsing a number" },
-    { UTF8_ERROR, "The input is not valid UTF-8" },
-    { UNINITIALIZED, "Uninitialized" },
-    { EMPTY, "Empty: no JSON found" },
-    { UNESCAPED_CHARS, "Within strings, some characters must be escaped, we found unescaped characters" },
-    { UNCLOSED_STRING, "A string is opened, but never closed." },
-    { UNSUPPORTED_ARCHITECTURE, "simdjson does not have an implementation supported by this CPU architecture (perhaps it's a non-SIMD CPU?)." },
-    { INCORRECT_TYPE, "The JSON element does not have the requested type." },
-    { NUMBER_OUT_OF_RANGE, "The JSON number is too large or too small to fit within the requested type." },
-    { INDEX_OUT_OF_BOUNDS, "Attempted to access an element of a JSON array that is beyond its length." },
-    { NO_SUCH_FIELD, "The JSON field referenced does not exist in this object." },
-    { IO_ERROR, "Error reading the file." },
-    { INVALID_JSON_POINTER, "Invalid JSON pointer syntax." },
-    { INVALID_URI_FRAGMENT, "Invalid URI fragment syntax." },
-    { UNEXPECTED_ERROR, "Unexpected error, consider reporting this problem as you may have found a bug in simdjson" },
-    { PARSER_IN_USE, "Cannot parse a new document while a document is still in use." },
-    { OUT_OF_ORDER_ITERATION, "Objects and arrays can only be iterated when they are first encountered." },
-    { INSUFFICIENT_PADDING, "simdjson requires the input JSON string to have at least SIMDJSON_PADDING extra bytes allocated, beyond the string's length. Consider using the simdjson::padded_string class if needed." },
-    { INCOMPLETE_ARRAY_OR_OBJECT, "JSON document ended early in the middle of an object or array." },
-    { SCALAR_DOCUMENT_AS_VALUE, "A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. "},
-    { OUT_OF_BOUNDS, "Attempted to access location outside of document."},
-    { TRAILING_CONTENT, "Unexpected trailing content in the JSON input."}
+    { SUCCESS, "SUCCESS: No error" },
+    { CAPACITY, "CAPACITY: This parser can't support a document that big" },
+    { MEMALLOC, "MEMALLOC: Error allocating memory, we're most likely out of memory" },
+    { TAPE_ERROR, "TAPE_ERROR: The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc." },
+    { DEPTH_ERROR, "DEPTH_ERROR: The JSON document was too deep (too many nested objects and arrays)" },
+    { STRING_ERROR, "STRING_ERROR: Problem while parsing a string" },
+    { T_ATOM_ERROR, "T_ATOM_ERROR: Problem while parsing an atom starting with the letter 't'" },
+    { F_ATOM_ERROR, "F_ATOM_ERROR: Problem while parsing an atom starting with the letter 'f'" },
+    { N_ATOM_ERROR, "N_ATOM_ERROR: Problem while parsing an atom starting with the letter 'n'" },
+    { NUMBER_ERROR, "NUMBER_ERROR: Problem while parsing a number" },
+    { UTF8_ERROR, "UTF8_ERROR: The input is not valid UTF-8" },
+    { UNINITIALIZED, "UNINITIALIZED: Uninitialized" },
+    { EMPTY, "EMPTY: no JSON found" },
+    { UNESCAPED_CHARS, "UNESCAPED_CHARS: Within strings, some characters must be escaped, we found unescaped characters" },
+    { UNCLOSED_STRING, "UNCLOSED_STRING: A string is opened, but never closed." },
+    { UNSUPPORTED_ARCHITECTURE, "UNSUPPORTED_ARCHITECTURE: simdjson does not have an implementation supported by this CPU architecture. Please report this error to the core team as it should never happen." },
+    { INCORRECT_TYPE, "INCORRECT_TYPE: The JSON element does not have the requested type." },
+    { NUMBER_OUT_OF_RANGE, "NUMBER_OUT_OF_RANGE: The JSON number is too large or too small to fit within the requested type." },
+    { INDEX_OUT_OF_BOUNDS, "INDEX_OUT_OF_BOUNDS: Attempted to access an element of a JSON array that is beyond its length." },
+    { NO_SUCH_FIELD, "NO_SUCH_FIELD: The JSON field referenced does not exist in this object." },
+    { IO_ERROR, "IO_ERROR: Error reading the file." },
+    { INVALID_JSON_POINTER, "INVALID_JSON_POINTER: Invalid JSON pointer syntax." },
+    { INVALID_URI_FRAGMENT, "INVALID_URI_FRAGMENT: Invalid URI fragment syntax." },
+    { UNEXPECTED_ERROR, "UNEXPECTED_ERROR: Unexpected error, consider reporting this problem as you may have found a bug in simdjson" },
+    { PARSER_IN_USE, "PARSER_IN_USE: Cannot parse a new document while a document is still in use." },
+    { OUT_OF_ORDER_ITERATION, "OUT_OF_ORDER_ITERATION: Objects and arrays can only be iterated when they are first encountered." },
+    { INSUFFICIENT_PADDING, "INSUFFICIENT_PADDING: simdjson requires the input JSON string to have at least SIMDJSON_PADDING extra bytes allocated, beyond the string's length. Consider using the simdjson::padded_string class if needed." },
+    { INCOMPLETE_ARRAY_OR_OBJECT, "INCOMPLETE_ARRAY_OR_OBJECT: JSON document ended early in the middle of an object or array." },
+    { SCALAR_DOCUMENT_AS_VALUE, "SCALAR_DOCUMENT_AS_VALUE: A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. "},
+    { OUT_OF_BOUNDS, "OUT_OF_BOUNDS: Attempt to access location outside of document."},
+    { TRAILING_CONTENT, "TRAILING_CONTENT: Unexpected trailing content in the JSON input."}
  }; // error_messages[]

 } // namespace internal
@@ -1,4 +1,4 @@
-/* auto-generated on 2023-03-09 11:14:42 -0500. Do not edit! */
+/* auto-generated on 2023-04-08 16:11:40 -0400. Do not edit! */
 /* begin file include/simdjson.h */
 #ifndef SIMDJSON_H
 #define SIMDJSON_H
@@ -43,7 +43,7 @@
 #define SIMDJSON_SIMDJSON_VERSION_H

 /** The version of simdjson being used (major.minor.revision) */
-#define SIMDJSON_VERSION "3.1.5"
+#define SIMDJSON_VERSION "3.1.7"

 namespace simdjson {
 enum {
@@ -58,7 +58,7 @@ enum {
  /**
   * The revision (major.minor.REVISION) of simdjson being used.
   */
-  SIMDJSON_VERSION_REVISION = 5
+  SIMDJSON_VERSION_REVISION = 7
 };
 } // namespace simdjson

@@ -265,6 +265,19 @@ use a 64-bit target such as x64, 64-bit ARM or 64-bit PPC.")
 #define SIMDJSON_NO_SANITIZE_UNDEFINED
 #endif

+
+#if defined(__clang__) || defined(__GNUC__)
+#if defined(__has_feature)
+#  if __has_feature(memory_sanitizer)
+#define SIMDJSON_NO_SANITIZE_MEMORY __attribute__((no_sanitize("memory")))
+#  endif // if __has_feature(memory_sanitizer)
+#endif // defined(__has_feature)
+#endif
+// make sure it is defined as 'nothing' if it is unapplicable.
+#ifndef SIMDJSON_NO_SANITIZE_MEMORY
+#define SIMDJSON_NO_SANITIZE_MEMORY
+#endif
+
 #if SIMDJSON_VISUAL_STUDIO
 // This is one case where we do not distinguish between
 // regular visual studio and clang under visual studio.
@@ -3808,9 +3821,8 @@ inline char *allocate_padded_buffer(size_t length) noexcept {
  if (padded_buffer == nullptr) {
    return nullptr;
  }
-  // We write zeroes in the padded region to avoid having uninitized
-  // garbage. If nothing else, garbage getting read might trigger a
-  // warning in a memory checking.
+  // We write nulls in the padded region to avoid having uninitialized
+  // content which may trigger warning for some sanitizers
  std::memset(padded_buffer + length, 0, totalpaddedlength - length);
  return padded_buffer;
 } // allocate_padded_buffer()
@@ -3840,7 +3852,7 @@ inline padded_string::padded_string(std::string_view sv_) noexcept
    : viable_size(sv_.size()), data_ptr(internal::allocate_padded_buffer(sv_.size())) {
  if(simdjson_unlikely(!data_ptr)) {
    //allocation failed or zero size
-    viable_size=0;
+    viable_size = 0;
    return;
  }
  if (sv_.size()) {
@@ -8896,7 +8908,7 @@ inline error_code parser::ensure_capacity(document& target_document, size_t desi
 }

 simdjson_inline void parser::set_max_capacity(size_t max_capacity) noexcept {
-  if(max_capacity < MINIMAL_DOCUMENT_CAPACITY) {
+  if(max_capacity > MINIMAL_DOCUMENT_CAPACITY) {
    _max_capacity = max_capacity;
  } else {
    _max_capacity = MINIMAL_DOCUMENT_CAPACITY;
@@ -9845,6 +9857,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #ifdef SIMDJSON_REGULAR_VISUAL_STUDIO
  unsigned long ret;
@@ -13956,6 +13972,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  return (int)_tzcnt_u64(input_num);
@@ -16146,6 +16166,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  return (int)_tzcnt_u64(input_num);
@@ -18292,6 +18316,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  unsigned long ret;
@@ -20609,6 +20637,10 @@ namespace {
 // but the algorithms do not end up using the returned value.
 // Sadly, sanitizers are not smart enough to figure it out.
 SIMDJSON_NO_SANITIZE_UNDEFINED
+// This function can be used safely even if not all bytes have been
+// initialized.
+// See issue https://github.com/simdjson/simdjson/issues/1965
+SIMDJSON_NO_SANITIZE_MEMORY
 simdjson_inline int trailing_zeroes(uint64_t input_num) {
 #if SIMDJSON_REGULAR_VISUAL_STUDIO
  unsigned long ret;
@@ -23531,7 +23563,12 @@ public:
   */
  simdjson_inline error_code optional_error(error_code error, const char *message) noexcept;

-  template<int N> simdjson_warn_unused simdjson_inline bool copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t (&tmpbuf)[N]) noexcept;
+  /**
+   * Take an input in json containing max_len characters and attempt to copy it over to tmpbuf, a buffer with
+   * N bytes of capacity. It will return false if N is too small (smaller than max_len) of if it is zero.
+   * The buffer (tmpbuf) is padded with space characters.
+   */
+  simdjson_warn_unused simdjson_inline bool copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t *tmpbuf, size_t N) noexcept;

  simdjson_inline token_position position() const noexcept;
  /**
@@ -23554,7 +23591,7 @@ public:
  /**
   * Returns the current location in the document if in bounds.
   */
-  inline simdjson_result<const char *> current_location() noexcept;
+  inline simdjson_result<const char *> current_location() const noexcept;

  /**
   * Updates this json iterator so that it is back at the beginning of the document,
@@ -23718,7 +23755,14 @@ public:
   * @error TAPE_ERROR if there is no matching } at end of document
   */
  simdjson_warn_unused simdjson_inline simdjson_result<bool> start_root_object() noexcept;
-
+  /**
+   * Checks whether an object could be started from the root. May be called by start_root_object.
+   *
+   * @returns SUCCESS if it is possible to safely start an object from the root (document level).
+   * @error INCORRECT_TYPE if there is no opening {
+   * @error TAPE_ERROR if there is no matching } at end of document
+   */
+  simdjson_warn_unused simdjson_inline error_code check_root_object() noexcept;
  /**
   * Start an object iteration after the user has already checked and moved past the {.
   *
@@ -23843,7 +23887,14 @@ public:
   * @error TAPE_ERROR if there is no matching ] at end of document
   */
  simdjson_warn_unused simdjson_inline simdjson_result<bool> start_root_array() noexcept;
-
+  /**
+   * Checks whether an array could be started from the root. May be called by start_root_array.
+   *
+   * @returns SUCCESS if it is possible to safely start an array from the root (document level).
+   * @error INCORRECT_TYPE If there is no [.
+   * @error TAPE_ERROR if there is no matching ] at end of document
+   */
+  simdjson_warn_unused simdjson_inline error_code check_root_array() noexcept;
  /**
   * Start an array iteration, after the user has already checked and moved past the [.
   *
@@ -24947,7 +24998,14 @@ public:
  /**
   * Returns the current location in the document if in bounds.
   */
-  inline simdjson_result<const char *> current_location() noexcept;
+  inline simdjson_result<const char *> current_location() const noexcept;
+
+  /**
+   * Returns true if this document has been fully parsed.
+   * If you have consumed the whole document and at_end() returns
+   * false, then there may be trailing content.
+   */
+  inline bool at_end() const noexcept;

  /**
   * Returns the current depth in the document if in bounds.
@@ -25156,6 +25214,7 @@ public:
  simdjson_inline simdjson_result<bool> is_scalar() noexcept;
  simdjson_inline simdjson_result<const char *> current_location() noexcept;
  simdjson_inline int32_t current_depth() const noexcept;
+  simdjson_inline bool at_end() const noexcept;
  simdjson_inline bool is_negative() noexcept;
  simdjson_inline simdjson_result<bool> is_integer() noexcept;
  simdjson_inline simdjson_result<SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand::number_type> get_number_type() noexcept;
@@ -27878,7 +27937,7 @@ inline std::string json_iterator::to_string() const noexcept {
          + std::string(" ]");
 }

-inline simdjson_result<const char *> json_iterator::current_location() noexcept {
+inline simdjson_result<const char *> json_iterator::current_location() const noexcept {
  if (!is_alive()) {    // Unrecoverable error
    if (!at_root()) {
      return reinterpret_cast<const char *>(token.peek(-1));
@@ -28028,19 +28087,16 @@ simdjson_inline error_code json_iterator::optional_error(error_code _error, cons
  return _error;
 }

-template<int N>
-simdjson_warn_unused simdjson_inline bool json_iterator::copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t (&tmpbuf)[N]) noexcept {
+
+simdjson_warn_unused simdjson_inline bool json_iterator::copy_to_buffer(const uint8_t *json, uint32_t max_len, uint8_t *tmpbuf, size_t N) noexcept {
+  // This function is not expected to be called in performance-sensitive settings.
  // Let us guard against silly cases:
  if((N < max_len) || (N == 0)) { return false; }
-  // Truncate whitespace to fit the buffer.
-  if (max_len > N-1) {
-    // if (jsoncharutils::is_not_structural_or_whitespace(json[N-1])) { return false; }
-    max_len = N-1;
-  }
-
  // Copy to the buffer.
  std::memcpy(tmpbuf, json, max_len);
-  tmpbuf[max_len] = ' ';
+  if(N > max_len) { // We pad whatever remains with ' '.
+    std::memset(tmpbuf + max_len, ' ', N - max_len);
+  }
  return true;
 }

@@ -28094,12 +28150,17 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
  return true;
 }

-simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_object() noexcept {
+simdjson_warn_unused simdjson_inline error_code value_iterator::check_root_object() noexcept {
  // When in streaming mode, we cannot expect peek_last() to be the last structural element of the
  // current document. It only works in the normal mode where we have indexed a single document.
  // Note that adding a check for 'streaming' is not expensive since we only have at most
  // one root element.
  if ( ! _json_iter->streaming() ) {
+    // The following lines do not fully protect against garbage content within the
+    // object: e.g., `{"a":2} foo }`. Users concerned with garbage content should
+    // call `at_end()` on the document instance at the end of the processing to
+    // ensure that the processing has finished at the end.
+    //
    if (*_json_iter->peek_last() != '}') {
      _json_iter->abandon();
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "missing } at end");
@@ -28116,6 +28177,12 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "the document is unbalanced");
    }
  }
+  return SUCCESS;
+}
+
+simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_object() noexcept {
+  auto error = check_root_object();
+  if(error) { return error; }
  return started_object();
 }

@@ -28479,12 +28546,17 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
  return true;
 }

-simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_array() noexcept {
+simdjson_warn_unused simdjson_inline error_code value_iterator::check_root_array() noexcept {
  // When in streaming mode, we cannot expect peek_last() to be the last structural element of the
  // current document. It only works in the normal mode where we have indexed a single document.
  // Note that adding a check for 'streaming' is not expensive since we only have at most
  // one root element.
  if ( ! _json_iter->streaming() ) {
+    // The following lines do not fully protect against garbage content within the
+    // array: e.g., `[1, 2] foo]`. Users concerned with garbage content should
+    // also call `at_end()` on the document instance at the end of the processing to
+    // ensure that the processing has finished at the end.
+    //
    if (*_json_iter->peek_last() != ']') {
      _json_iter->abandon();
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "missing ] at end");
@@ -28501,6 +28573,12 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::start
      return report_error(INCOMPLETE_ARRAY_OR_OBJECT, "the document is unbalanced");
    }
  }
+  return SUCCESS;
+}
+
+simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::started_root_array() noexcept {
+  auto error = check_root_array();
+  if (error) { return error; }
  return started_array();
 }

@@ -28611,7 +28689,7 @@ simdjson_inline simdjson_result<bool> value_iterator::is_root_integer(bool check
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("is_root_integer");
  uint8_t tmpbuf[20+1]; // <20 digits> is the longest possible unsigned integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    return false; // if there are more than 20 characters, it cannot be represented as an integer.
  }
  auto answer = numberparsing::is_integer(tmpbuf);
@@ -28629,7 +28707,7 @@ simdjson_inline simdjson_result<SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand::numbe
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -28644,7 +28722,7 @@ simdjson_inline simdjson_result<number> value_iterator::get_root_number(bool che
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -28672,7 +28750,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<uint64_t> value_iterator::g
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("uint64");
  uint8_t tmpbuf[20+1]; // <20 digits> is the longest possible unsigned integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -28687,7 +28765,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<uint64_t> value_iterator::g
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("uint64");
  uint8_t tmpbuf[20+1]; // <20 digits> is the longest possible unsigned integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -28702,7 +28780,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<int64_t> value_iterator::ge
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("int64");
  uint8_t tmpbuf[20+1]; // -<19 digits> is the longest possible integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -28718,7 +28796,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<int64_t> value_iterator::ge
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("int64");
  uint8_t tmpbuf[20+1]; // -<19 digits> is the longest possible integer
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 20+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 20 characters");
    return NUMBER_ERROR;
  }
@@ -28737,7 +28815,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<double> value_iterator::get
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -28756,7 +28834,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<double> value_iterator::get
  // 1074 is the maximum number of significant fractional digits. Add 8 more digits for the biggest
  // number: -0.<fraction>e-308.
  uint8_t tmpbuf[1074+8+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) {
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 1074+8+1)) {
    logger::log_error(*_json_iter, start_position(), depth(), "Root number more than 1082 characters");
    return NUMBER_ERROR;
  }
@@ -28771,7 +28849,7 @@ simdjson_warn_unused simdjson_inline simdjson_result<bool> value_iterator::get_r
  auto max_len = peek_start_length();
  auto json = peek_root_scalar("bool");
  uint8_t tmpbuf[5+1];
-  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf)) { return incorrect_type_error("Not a boolean"); }
+  if (!_json_iter->copy_to_buffer(json, max_len, tmpbuf, 5+1)) { return incorrect_type_error("Not a boolean"); }
  auto result = parse_bool(tmpbuf);
  if(result.error() == SUCCESS) {
    if (check_trailing && !_json_iter->is_single_token()) { return TRAILING_CONTENT; }
@@ -29500,7 +29578,7 @@ inline std::string document::to_debug_string() noexcept {
  return iter.to_string();
 }

-inline simdjson_result<const char *> document::current_location() noexcept {
+inline simdjson_result<const char *> document::current_location() const noexcept {
  return iter.current_location();
 }

@@ -29508,6 +29586,11 @@ inline int32_t document::current_depth() const noexcept {
  return iter.depth();
 }

+inline bool document::at_end() const noexcept {
+  return iter.at_end();
+}
+
+
 inline bool document::is_alive() noexcept {
  return iter.is_alive();
 }
@@ -29529,14 +29612,24 @@ simdjson_inline simdjson_result<value> document::get_value() noexcept {
  // gets called.
  iter.assert_at_document_depth();
  switch (*iter.peek()) {
-    case '[':
-    case '{':
+    case '[': {
+      // The following lines check that the document ends with ].
+      auto value_iterator = get_root_value_iterator();
+      auto error = value_iterator.check_root_array();
+      if(error) { return error; }
      return value(get_root_value_iterator());
+    }
+    case '{': {
+      // The following lines would check that the document ends with }.
+      auto value_iterator = get_root_value_iterator();
+      auto error = value_iterator.check_root_object();
+      if(error) { return error; }
+      return value(get_root_value_iterator());
+    }
    default:
      // Unfortunately, scalar documents are a special case in simdjson and they cannot
      // be safely converted to value instances.
      return SCALAR_DOCUMENT_AS_VALUE;
-      // return value(get_root_value_iterator());
  }
 }
 simdjson_inline simdjson_result<array> document::get_array() & noexcept {
@@ -29977,6 +30070,12 @@ simdjson_inline simdjson_result<const char *> simdjson_result<SIMDJSON_BUILTIN_I
  return first.current_location();
 }

+simdjson_inline bool simdjson_result<SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand::document>::at_end() const noexcept {
+  if (error()) { return error(); }
+  return first.at_end();
+}
+
+
 simdjson_inline int32_t simdjson_result<SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand::document>::current_depth() const noexcept {
  if (error()) { return error(); }
  return first.current_depth();
@@ -4,37 +4,37 @@ namespace simdjson {
 namespace internal {

  SIMDJSON_DLLIMPORTEXPORT const error_code_info error_codes[] {
-    { SUCCESS, "No error" },
-    { CAPACITY, "This parser can't support a document that big" },
-    { MEMALLOC, "Error allocating memory, we're most likely out of memory" },
-    { TAPE_ERROR, "The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc." },
-    { DEPTH_ERROR, "The JSON document was too deep (too many nested objects and arrays)" },
-    { STRING_ERROR, "Problem while parsing a string" },
-    { T_ATOM_ERROR, "Problem while parsing an atom starting with the letter 't'" },
-    { F_ATOM_ERROR, "Problem while parsing an atom starting with the letter 'f'" },
-    { N_ATOM_ERROR, "Problem while parsing an atom starting with the letter 'n'" },
-    { NUMBER_ERROR, "Problem while parsing a number" },
-    { UTF8_ERROR, "The input is not valid UTF-8" },
-    { UNINITIALIZED, "Uninitialized" },
-    { EMPTY, "Empty: no JSON found" },
-    { UNESCAPED_CHARS, "Within strings, some characters must be escaped, we found unescaped characters" },
-    { UNCLOSED_STRING, "A string is opened, but never closed." },
-    { UNSUPPORTED_ARCHITECTURE, "simdjson does not have an implementation supported by this CPU architecture (perhaps it's a non-SIMD CPU?)." },
-    { INCORRECT_TYPE, "The JSON element does not have the requested type." },
-    { NUMBER_OUT_OF_RANGE, "The JSON number is too large or too small to fit within the requested type." },
-    { INDEX_OUT_OF_BOUNDS, "Attempted to access an element of a JSON array that is beyond its length." },
-    { NO_SUCH_FIELD, "The JSON field referenced does not exist in this object." },
-    { IO_ERROR, "Error reading the file." },
-    { INVALID_JSON_POINTER, "Invalid JSON pointer syntax." },
-    { INVALID_URI_FRAGMENT, "Invalid URI fragment syntax." },
-    { UNEXPECTED_ERROR, "Unexpected error, consider reporting this problem as you may have found a bug in simdjson" },
-    { PARSER_IN_USE, "Cannot parse a new document while a document is still in use." },
-    { OUT_OF_ORDER_ITERATION, "Objects and arrays can only be iterated when they are first encountered." },
-    { INSUFFICIENT_PADDING, "simdjson requires the input JSON string to have at least SIMDJSON_PADDING extra bytes allocated, beyond the string's length. Consider using the simdjson::padded_string class if needed." },
-    { INCOMPLETE_ARRAY_OR_OBJECT, "JSON document ended early in the middle of an object or array." },
-    { SCALAR_DOCUMENT_AS_VALUE, "A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. "},
-    { OUT_OF_BOUNDS, "Attempted to access location outside of document."},
-    { TRAILING_CONTENT, "Unexpected trailing content in the JSON input."}
+    { SUCCESS, "SUCCESS: No error" },
+    { CAPACITY, "CAPACITY: This parser can't support a document that big" },
+    { MEMALLOC, "MEMALLOC: Error allocating memory, we're most likely out of memory" },
+    { TAPE_ERROR, "TAPE_ERROR: The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc." },
+    { DEPTH_ERROR, "DEPTH_ERROR: The JSON document was too deep (too many nested objects and arrays)" },
+    { STRING_ERROR, "STRING_ERROR: Problem while parsing a string" },
+    { T_ATOM_ERROR, "T_ATOM_ERROR: Problem while parsing an atom starting with the letter 't'" },
+    { F_ATOM_ERROR, "F_ATOM_ERROR: Problem while parsing an atom starting with the letter 'f'" },
+    { N_ATOM_ERROR, "N_ATOM_ERROR: Problem while parsing an atom starting with the letter 'n'" },
+    { NUMBER_ERROR, "NUMBER_ERROR: Problem while parsing a number" },
+    { UTF8_ERROR, "UTF8_ERROR: The input is not valid UTF-8" },
+    { UNINITIALIZED, "UNINITIALIZED: Uninitialized" },
+    { EMPTY, "EMPTY: no JSON found" },
+    { UNESCAPED_CHARS, "UNESCAPED_CHARS: Within strings, some characters must be escaped, we found unescaped characters" },
+    { UNCLOSED_STRING, "UNCLOSED_STRING: A string is opened, but never closed." },
+    { UNSUPPORTED_ARCHITECTURE, "UNSUPPORTED_ARCHITECTURE: simdjson does not have an implementation supported by this CPU architecture. Please report this error to the core team as it should never happen." },
+    { INCORRECT_TYPE, "INCORRECT_TYPE: The JSON element does not have the requested type." },
+    { NUMBER_OUT_OF_RANGE, "NUMBER_OUT_OF_RANGE: The JSON number is too large or too small to fit within the requested type." },
+    { INDEX_OUT_OF_BOUNDS, "INDEX_OUT_OF_BOUNDS: Attempted to access an element of a JSON array that is beyond its length." },
+    { NO_SUCH_FIELD, "NO_SUCH_FIELD: The JSON field referenced does not exist in this object." },
+    { IO_ERROR, "IO_ERROR: Error reading the file." },
+    { INVALID_JSON_POINTER, "INVALID_JSON_POINTER: Invalid JSON pointer syntax." },
+    { INVALID_URI_FRAGMENT, "INVALID_URI_FRAGMENT: Invalid URI fragment syntax." },
+    { UNEXPECTED_ERROR, "UNEXPECTED_ERROR: Unexpected error, consider reporting this problem as you may have found a bug in simdjson" },
+    { PARSER_IN_USE, "PARSER_IN_USE: Cannot parse a new document while a document is still in use." },
+    { OUT_OF_ORDER_ITERATION, "OUT_OF_ORDER_ITERATION: Objects and arrays can only be iterated when they are first encountered." },
+    { INSUFFICIENT_PADDING, "INSUFFICIENT_PADDING: simdjson requires the input JSON string to have at least SIMDJSON_PADDING extra bytes allocated, beyond the string's length. Consider using the simdjson::padded_string class if needed." },
+    { INCOMPLETE_ARRAY_OR_OBJECT, "INCOMPLETE_ARRAY_OR_OBJECT: JSON document ended early in the middle of an object or array." },
+    { SCALAR_DOCUMENT_AS_VALUE, "SCALAR_DOCUMENT_AS_VALUE: A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. "},
+    { OUT_OF_BOUNDS, "OUT_OF_BOUNDS: Attempt to access location outside of document."},
+    { TRAILING_CONTENT, "TRAILING_CONTENT: Unexpected trailing content in the JSON input."}
  }; // error_messages[]

 } // namespace internal
@@ -59,28 +59,28 @@ private:

 template<typename T>
 bool cast_tester<T>::test_get(element element, T expected) {
-  T actual;
+  T actual{};
  ASSERT_SUCCESS(element.get(actual));
  return assert_equal(actual, expected);
 }

 template<typename T>
 bool cast_tester<T>::test_get(simdjson_result<element> element, T expected) {
-  T actual;
+  T actual{};
  ASSERT_SUCCESS(element.get(actual));
  return assert_equal(actual, expected);
 }

 template<typename T>
 bool cast_tester<T>::test_get_error(element element, error_code expected_error) {
-  T actual;
+  T actual{};
  ASSERT_EQUAL(element.get(actual), expected_error);
  return true;
 }

 template<typename T>
 bool cast_tester<T>::test_get_error(simdjson_result<element> element, error_code expected_error) {
-  T actual;
+  T actual{};
  ASSERT_EQUAL(element.get(actual), expected_error);
  return true;
 }
@@ -113,28 +113,28 @@ bool cast_tester<T>::test_get_t_error(simdjson_result<element> element, error_co

 template<typename T>
 bool cast_tester<T>::test_named_get(element element, T expected) {
-  T actual;
+  T actual{};
  ASSERT_SUCCESS(named_get(element).get(actual));
  return assert_equal(actual, expected);
 }

 template<typename T>
 bool cast_tester<T>::test_named_get(simdjson_result<element> element, T expected) {
-  T actual;
+  T actual{};
  ASSERT_SUCCESS(named_get(element).get(actual));
  return assert_equal(actual, expected);
 }

 template<typename T>
 bool cast_tester<T>::test_named_get_error(element element, error_code expected_error) {
-  T actual;
+  T actual{};
  ASSERT_EQUAL(named_get(element).get(actual), expected_error);
  return true;
 }

 template<typename T>
 bool cast_tester<T>::test_named_get_error(simdjson_result<element> element, error_code expected_error) {
-  T actual;
+  T actual{};
  ASSERT_EQUAL(named_get(element).get(actual), expected_error);
  return true;
 }
@@ -143,7 +143,7 @@ bool cast_tester<T>::test_named_get_error(simdjson_result<element> element, erro

 template<typename T>
 bool cast_tester<T>::test_implicit_cast(element element, T expected) {
-  T actual;
+  T actual{};
  try {
    actual = element;
  } catch(simdjson_error &e) {
@@ -155,7 +155,7 @@ bool cast_tester<T>::test_implicit_cast(element element, T expected) {

 template<typename T>
 bool cast_tester<T>::test_implicit_cast(simdjson_result<element> element, T expected) {
-  T actual;
+  T actual{};
  try {
    actual = element;
  } catch(simdjson_error &e) {
@@ -168,7 +168,7 @@ bool cast_tester<T>::test_implicit_cast(simdjson_result<element> element, T expe
 template<typename T>
 bool cast_tester<T>::test_implicit_cast_error(element element, error_code expected_error) {
  try {
-    simdjson_unused T actual;
+    simdjson_unused T actual{};
    actual = element;
    return false;
  } catch(simdjson_error &e) {
@@ -180,7 +180,7 @@ bool cast_tester<T>::test_implicit_cast_error(element element, error_code expect
 template<typename T>
 bool cast_tester<T>::test_implicit_cast_error(simdjson_result<element> element, error_code expected_error) {
  try {
-    simdjson_unused T actual;
+    simdjson_unused T actual{};
    actual = element;
    return false;
  } catch(simdjson_error &e) {
@@ -74,7 +74,7 @@ namespace number_tests {
    for (int m = 10; m < 20; m++) {
      for (int i = -1024; i < 1024; i++) {
        auto str = std::to_string(i);
-        int64_t actual;
+        int64_t actual{};
        ASSERT_SUCCESS(parser.parse(str).get(actual));
        if (actual != i) {
          std::cerr << "JSON '" << str << "' parsed to " << actual << " instead of " << i << std::endl;
@@ -956,6 +956,22 @@ namespace dom_api_tests {
    return true;
  }

+  bool issue1979() {
+    TEST_START();
+    auto json = R"({
+  "@avito-core/toggles:6.1.18": {
+    "add_model_review_from": true
+  }
+ })"_padded;
+    simdjson::dom::parser parser;
+    simdjson::dom::element doc;
+    ASSERT_SUCCESS(parser.parse(json).get(doc));
+    simdjson::dom::object main_object;
+    ASSERT_SUCCESS(doc.get_object().get(main_object));
+    ASSERT_SUCCESS(main_object["@avito-core/toggles:6.1.18"].get_object().error())
+    TEST_SUCCEED();
+  }
+
  bool document_object_index() {
    std::cout << "Running " << __func__ << std::endl;
    string json(R"({ "a": 1, "b": 2, "c/d": 3})");
@@ -978,14 +994,6 @@ namespace dom_api_tests {

    simdjson::error_code error;
    simdjson_unused element val;
-// This is disabled, see https://github.com/simdjson/simdjson/issues/1243
-//#ifndef _LIBCPP_VERSION // should work everywhere but with libc++, must include the <ciso646> header.
-//    std::tie(val,error) = object["d"];
-//    ASSERT_ERROR( error, NO_SUCH_FIELD );
-//    std::tie(std::ignore,error) = object["d"];
-//    ASSERT_ERROR( error, NO_SUCH_FIELD );
-//#endif
-    // tie(val, error) = object["d"]; fails with "no viable overloaded '='" on Apple clang version 11.0.0	    tie(val, error) = doc["d"];
    object["d"].tie(val, error);
    ASSERT_ERROR( error, NO_SUCH_FIELD );
    ASSERT_ERROR( object["d"].get(val), NO_SUCH_FIELD );
@@ -1254,6 +1262,7 @@ namespace dom_api_tests {
    #if SIMDJSON_ENABLE_DEPRECATED_API
        ParsedJson_Iterator_test() &&
    #endif
+           issue1979() &&
           object_iterator() &&
           array_iterator() &&
           object_iterator_empty() &&
@@ -634,7 +634,7 @@ namespace document_stream_tests {
    ASSERT_SUCCESS( odparser.parse_many(json.data(), json.length(), 50).get(odstream) );
    for (auto doc: odstream) {
      if(counter < 6) {
-        int64_t val;
+        int64_t val{};
        ASSERT_SUCCESS(doc.at_pointer("/4").get(val));
        ASSERT_EQUAL(val, 5);
      } else {
@@ -41,7 +41,7 @@ void basics_error_2() {
      cout << "Make/Model: " << make << "/" << model << endl;

      // Casting a JSON element to an integer
-      uint64_t year;
+      uint64_t year{};
      if ((error = car["year"].get(year))) { cerr << error << endl; exit(1); }
      cout << "- This car is " << 2020 - year << "years old." << endl;

@@ -132,7 +132,7 @@ void basics_error_2_cpp17() {
    cout << "Make/Model: " << make << "/" << model << endl;

    // Casting a JSON element to an integer
-    uint64_t year;
+    uint64_t year{};
    if ((error = car["year"].get(year))) { cerr << error << endl; exit(1); }
    cout << "- This car is " << 2020 - year << "years old." << endl;

@@ -6,6 +6,20 @@ using namespace simdjson;
 namespace array_tests {
  using namespace std;
  using simdjson::ondemand::json_type;
+  bool issue1977() {
+    TEST_START();
+    auto json = R"([1, 2] foo ])"_padded;
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(json).get(doc));
+    ondemand::array array;
+    ASSERT_SUCCESS(doc.get_array().get(array));
+    for (auto values : array) {
+      ASSERT_SUCCESS(values);
+    }
+    ASSERT_FALSE(doc.at_end());
+    TEST_SUCCEED();
+  }
  bool issue1588() {
    TEST_START();
    const auto json = R"({
@@ -134,7 +148,7 @@ namespace array_tests {
    ondemand::parser parser;
    ondemand::document doc;
    ASSERT_SUCCESS(parser.iterate(json).get(doc));
-    size_t count;
+    size_t count{};
    ASSERT_SUCCESS(doc.count_elements().get(count));
    ondemand::array arr;
    ASSERT_SUCCESS(doc.get_array().get(arr));
@@ -182,7 +196,7 @@ namespace array_tests {
    ASSERT_SUCCESS(doc.get_object().get(obj));
    ondemand::value v;
    ASSERT_SUCCESS(doc.find_field("test").get(v));
-    size_t count;
+    size_t count{};
    ASSERT_SUCCESS(v.count_elements().get(count));
    ASSERT_EQUAL(count, 3);
    ASSERT_SUCCESS(doc.find_field("joe").get(v));
@@ -200,7 +214,7 @@ namespace array_tests {
      ondemand::array array;
      ASSERT_RESULT( doc_result.type(), json_type::array );
      ASSERT_SUCCESS( doc_result.get_array().get(array) );
-      size_t count;
+      size_t count{};
      ASSERT_SUCCESS( array.count_elements().get(count) );
      ASSERT_EQUAL(count, expected_value.size());
      return true;
@@ -209,7 +223,7 @@ namespace array_tests {
      ondemand::array array;
      ASSERT_RESULT( doc_result.type(), json_type::array );
      ASSERT_SUCCESS( doc_result.get(array) );
-      size_t count;
+      size_t count{};
      ASSERT_SUCCESS( array.count_elements().get(count) );
      ASSERT_EQUAL(count, expected_value.size());
      size_t i = 0;
@@ -235,7 +249,7 @@ namespace array_tests {
      ondemand::array array;
      ASSERT_RESULT( doc_result.type(), json_type::array );
      ASSERT_SUCCESS( doc_result.get_array().get(array) );
-      size_t count;
+      size_t count{};
      ASSERT_SUCCESS( array.count_elements().get(count) );
      ASSERT_EQUAL(count, 0);
      return true;
@@ -244,7 +258,7 @@ namespace array_tests {
      ondemand::array array;
      ASSERT_RESULT( doc_result.type(), json_type::array );
      ASSERT_SUCCESS( doc_result.get(array) );
-      size_t count;
+      size_t count{};
      ASSERT_SUCCESS( array.count_elements().get(count) );
      ASSERT_EQUAL(count, 0);
      size_t i = 0;
@@ -269,7 +283,7 @@ namespace array_tests {
      ondemand::array array;
      ASSERT_RESULT( doc_result.type(), json_type::array );
      ASSERT_SUCCESS( doc_result.get(array) );
-      size_t count;
+      size_t count{};
      auto e = array.count_elements().get(count);
      if( e != TAPE_ERROR) {
        std::cout << e << "\n";
@@ -285,7 +299,7 @@ namespace array_tests {
    TEST_START();
    auto empty = R"( [] )"_padded;
    SUBTEST("ondemand::empty_doc_array", test_ondemand_doc(empty, [&](auto doc_result) {
-        size_t count;
+        size_t count{};
        ASSERT_RESULT( doc_result.type(), json_type::array );
        ASSERT_SUCCESS( doc_result.count_elements().get(count) );
        ASSERT_EQUAL( count, 0 );
@@ -293,7 +307,7 @@ namespace array_tests {
    }));
    auto basic = R"( [-1.234, 100000000000000, null, [1,2,3], {"t":true, "f":false}] )"_padded;
    SUBTEST("ondemand::basic_doc_array", test_ondemand_doc(basic, [&](auto doc_result) {
-        size_t count;
+        size_t count{};
        ASSERT_RESULT( doc_result.type(), json_type::array );
        ASSERT_SUCCESS( doc_result.count_elements().get(count) );
        ASSERT_EQUAL( count, 5 );
@@ -435,7 +449,7 @@ namespace array_tests {
    TEST_SUCCEED();
  }
  bool count_empty(simdjson::ondemand::array arr) {
-    size_t count;
+    size_t count{};
    ASSERT_SUCCESS(arr.count_elements().get(count));
    ASSERT_EQUAL(count, 0);
    bool is_empty;
@@ -466,7 +480,7 @@ namespace array_tests {
    for (auto d : data) {
      simdjson::ondemand::array arr;
      ASSERT_SUCCESS(d.get_array().get(arr));
-      size_t count;
+      size_t count{};
      ASSERT_SUCCESS(arr.count_elements().get(count));
      ASSERT_EQUAL(count, 4);
    }
@@ -830,6 +844,7 @@ namespace array_tests {

  bool run() {
    return
+           issue1977() &&
           issue1876() &&
           issue1742() &&
           empty_rewind_convoluted() &&
@@ -7,7 +7,7 @@ namespace document_stream_tests {

    template <typename T>
    bool process_doc(T &docref) {
-        int64_t val;
+        int64_t val{};
        ASSERT_SUCCESS(docref.at_pointer("/4").get(val));
        ASSERT_EQUAL(val, 5);
        return true;
@@ -193,6 +193,20 @@ namespace document_stream_tests {
        TEST_SUCCEED();
    }

+    bool issue1977() {
+        TEST_START();
+        std::string json = R"( 1111 })";
+        ondemand::parser odparser;
+        ondemand::document_stream odstream;
+        ASSERT_SUCCESS(odparser.iterate_many(json).get(odstream));
+
+        auto i = odstream.begin();
+        for (; i != odstream.end(); ++i) {
+            ASSERT_TRUE(false);
+        }
+        ASSERT_TRUE(i.current_index() == 0);
+        TEST_SUCCEED();
+    }

    bool issue1683() {
        TEST_START();
@@ -604,7 +618,7 @@ namespace document_stream_tests {
        ASSERT_SUCCESS( odparser.iterate_many(json.data(), json.length(), 50).get(odstream) );
        for (auto doc: odstream) {
            if(counter < 6) {
-                int64_t val;
+                int64_t val{};
                ASSERT_SUCCESS(doc.at_pointer("/4").get(val));
                ASSERT_EQUAL(val, 5);
            } else {
@@ -789,6 +803,7 @@ namespace document_stream_tests {

    bool run() {
        return
+            issue1977() &&
            string_with_trailing() &&
            uint64_with_trailing() &&
            int64_with_trailing() &&
@@ -19,9 +19,9 @@ namespace error_location_tests {
            const char* c;
            // Must call current_location first because get_int64() will consume values
            ASSERT_SUCCESS(doc.current_location().get(c));
-            ASSERT_EQUAL(*c,expected[count]);
+            ASSERT_EQUAL(*c, expected[count]);
            ASSERT_SUCCESS(value.get_int64().get(i));
-            ASSERT_EQUAL(i,expected_values[count]);
+            ASSERT_EQUAL(i, expected_values[count]);
            count++;
        }
        ASSERT_EQUAL(count,3);
@@ -64,12 +64,14 @@ namespace error_location_tests {
        ASSERT_SUCCESS(doc.at_pointer("/a/2/1").get(i));
        ASSERT_EQUAL(i, 4);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, ",5]], \"b\": {\"c\": [1.2, 2.3]}} ");
+        std::string expected = ",5]], \"b\": {\"c\": [1.2, 2.3]}} ";
+        ASSERT_EQUAL(std::string(ptr, expected.size()), expected);
        double d;
        ASSERT_SUCCESS(doc.at_pointer("/b/c/1").get(d));
        ASSERT_EQUAL(d, 2.3);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "]}} ");
+        expected = "]}} ";
+        ASSERT_EQUAL(std::string(ptr, expected.size()), expected);
        TEST_SUCCEED();
    }

@@ -83,11 +85,13 @@ namespace error_location_tests {
        double d;
        ASSERT_ERROR(doc.at_pointer("/b/c/0").get(d), NUMBER_ERROR);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "1.2., 2.3]}} ");
+        std::string expected = "1.2., 2.3]}} ";
+        ASSERT_EQUAL(std::string(ptr, expected.size()), expected);
        uint64_t i;
        ASSERT_ERROR(doc.at_pointer("/a/2/1").get(i), TAPE_ERROR);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "4,5]], \"b\": {\"c\": [1.2., 2.3]}} ");
+        expected = "4,5]], \"b\": {\"c\": [1.2., 2.3]}} ";
+        ASSERT_EQUAL(std::string(ptr, expected.size()), expected);
        TEST_SUCCEED();
    }

@@ -100,7 +104,7 @@ namespace error_location_tests {
        ASSERT_SUCCESS(parser.iterate(json).get(doc));
        ASSERT_ERROR(doc["a"], INCORRECT_TYPE);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "\xc3\x94\xc3\xb8\xe2\x84\xa6{\"a\":1, 3} ");
+        ASSERT_EQUAL(std::string(ptr, 18), "\xc3\x94\xc3\xb8\xe2\x84\xa6{\"a\":1, 3} ");
        TEST_SUCCEED();
    }

@@ -115,7 +119,7 @@ namespace error_location_tests {
        ASSERT_SUCCESS(doc.get_array().get(arr));
        ASSERT_ERROR(arr.count_elements(), TAPE_ERROR);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr - 2, "] ");
+        ASSERT_EQUAL(std::string(ptr - 2,2), "] ");
        TEST_SUCCEED();
    }

@@ -136,7 +140,7 @@ namespace error_location_tests {
        }
        ASSERT_EQUAL(count, 1);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "1.23, 2] ");
+        ASSERT_EQUAL(std::string(ptr, strlen("1.23, 2] ")), "1.23, 2] ");
        TEST_SUCCEED();
    }

@@ -149,7 +153,7 @@ namespace error_location_tests {
        ASSERT_SUCCESS(parser.iterate(json).get(doc));
        ASSERT_ERROR(doc["b"], TAPE_ERROR);
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "3.5, \"b\":5} ");
+        ASSERT_EQUAL(std::string(ptr, strlen("3.5, \"b\":5} ")), "3.5, \"b\":5} ");
        TEST_SUCCEED();
    }

@@ -164,7 +168,7 @@ namespace error_location_tests {
            ASSERT_ERROR(val, INCOMPLETE_ARRAY_OR_OBJECT);
        }
        ASSERT_SUCCESS(doc.current_location().get(ptr));
-        ASSERT_EQUAL(ptr, "[1,2,3 ");
+        ASSERT_EQUAL(std::string(ptr, strlen("[1,2,3 ")), "[1,2,3 ");
        TEST_SUCCEED();
    }

@@ -5,6 +5,29 @@ using namespace simdjson;

 namespace misc_tests {
  using namespace std;
+  bool issue1981_success() {
+    auto error_phrase = R"(false)"_padded;
+    TEST_START();
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(error_phrase).get(doc));
+    bool b;
+    ASSERT_SUCCESS( doc.get_bool().get(b));
+    ASSERT_FALSE(b);
+    TEST_SUCCEED();
+  }
+
+  bool issue1981_failure() {
+    auto error_phrase = R"(falseA)"_padded;
+    TEST_START();
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(error_phrase).get(doc));
+    bool b;
+    ASSERT_ERROR( doc.get_bool().get(b), INCORRECT_TYPE);
+    TEST_SUCCEED();
+  }
+
  bool replacement_char() {
    auto fun_phrase = R"( ["I \u2665 Unicode. Even broken \ud800 Unicode." ])"_padded;
    std::string_view expected_fun = "I \xe2\x99\xa5 Unicode. Even broken \xef\xbf\xbd Unicode.";
@@ -67,7 +90,7 @@ namespace misc_tests {
    ASSERT_SUCCESS(val.get_object().get(obj));
    ondemand::array arr;
    ASSERT_SUCCESS(obj["a"].get_array().get(arr));
-    size_t count;
+     size_t count{};
    ASSERT_SUCCESS(arr.count_elements().get(count));
    ASSERT_EQUAL(3,count);
    TEST_SUCCEED();
@@ -570,6 +593,8 @@ namespace misc_tests {

  bool run() {
    return
+           issue1981_success() &&
+           issue1981_failure() &&
           replacement_char() &&
           wobbly_tests() &&
           issue_uffff() &&
@@ -232,7 +232,7 @@ namespace number_tests {
    for(simdjson_result<ondemand::value> valr : arr) {
      ondemand::value val;
      ASSERT_SUCCESS(valr.get(val));
-      ondemand::number_type nt;
+      ondemand::number_type nt{};
      ASSERT_SUCCESS(val.get_number_type().get(nt));
      ASSERT_EQUAL(expectedtypes[counter], nt);
      ondemand::number num;
@@ -339,7 +339,7 @@ namespace number_tests {
    ondemand::document doc;
    padded_string docdata;
    ondemand::number number;
-    ondemand::number_type nt;
+    ondemand::number_type nt{};

    bool intvalue{};

@@ -10,7 +10,7 @@ namespace object_error_tests {
  bool assert_iterate_object(T &&object, const char **expected_key, V *expected, size_t N, simdjson::error_code *expected_error, size_t N2) {
    size_t count = 0;
    for (auto field : object) {
-      V actual;
+      V actual{};
      auto actual_error = field.value().get(actual);
      if (count >= N) {
        ASSERT((count - N) < N2, "Extra error reported");
@@ -6,6 +6,38 @@ using namespace simdjson;
 namespace object_tests {
  using namespace std;
  using simdjson::ondemand::json_type;
+
+  bool issue1979() {
+    TEST_START();
+    auto json = R"({
+  "@avito-core/toggles:6.1.18": {
+    "add_model_review_from": true
+  }
+ })"_padded;
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(json).get(doc));
+    ondemand::object object;
+    ASSERT_SUCCESS(doc.get_object().get(object));
+    ASSERT_SUCCESS(object["@avito-core/toggles:6.1.18"].get_object().error());
+    TEST_SUCCEED();
+  }
+
+  bool issue1977() {
+    TEST_START();
+    auto json = R"({"1": 2} foo })"_padded;
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(json).get(doc));
+    ondemand::object object;
+    ASSERT_SUCCESS(doc.get_object().get(object));
+    for (auto values : object) {
+      ASSERT_SUCCESS(values);
+    }
+    ASSERT_FALSE(doc.at_end());
+    TEST_SUCCEED();
+  }
+
  bool issue1745() {
    TEST_START();
    auto json = R"({
@@ -225,6 +257,25 @@ namespace object_tests {
  }

 #if SIMDJSON_EXCEPTIONS
+
+  bool issue1965() {
+    TEST_START();
+    std::string str = "{\"query\":\"ah\"}";
+    std::unique_ptr<char[]> buffer(new char[str.size() + simdjson::SIMDJSON_PADDING]);
+    memcpy(buffer.get(), str.data(), str.size());
+    simdjson::padded_string_view view(buffer.get(), str.size(), str.size() + simdjson::SIMDJSON_PADDING);
+    simdjson::ondemand::parser parser;
+    simdjson::ondemand::document doc = parser.iterate(view);
+    simdjson::ondemand::object root = doc.get_object();
+    simdjson::ondemand::value query = root.find_field("query");
+    simdjson::ondemand::raw_json_string raw = query.get_raw_json_string();
+    std::unique_ptr<uint8_t[]> dst_buffer(new uint8_t[3 + simdjson::SIMDJSON_PADDING]);
+    uint8_t * dst = dst_buffer.get();
+    std::string_view fieldstring = parser.unescape(raw, dst);
+    std::cout << fieldstring << std::endl;
+    TEST_SUCCEED();
+  }
+
  bool issue1745_with_exceptions() {
    TEST_START();
    auto json = R"({
@@ -1098,6 +1149,29 @@ namespace object_tests {
    TEST_SUCCEED();
  }

+
+  bool issue1974a() {
+    TEST_START();
+    padded_string bad_json = R"({"key":111)"_padded;
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(bad_json).get(doc));
+    ondemand::object object;
+    ASSERT_ERROR(doc.get_object().get(object), INCOMPLETE_ARRAY_OR_OBJECT);
+    TEST_SUCCEED();
+  }
+
+  bool issue1974b() {
+    TEST_START();
+    padded_string bad_json = R"({"key":111)"_padded;
+    ondemand::parser parser;
+    ondemand::document doc;
+    ASSERT_SUCCESS(parser.iterate(bad_json).get(doc));
+    ondemand::value val;
+    ASSERT_ERROR(doc.get_value().get(val), INCOMPLETE_ARRAY_OR_OBJECT);
+    TEST_SUCCEED();
+  }
+
  bool iterate_bad_doc_object_count() {
    TEST_START();
    padded_string bad_jsons[4] = {R"( {"a":5 "b":3} )"_padded, R"( {"a":5, 3} )"_padded, R"( {"a":5, "b": } )"_padded, R"( {"a":5, "b":3 )"_padded};
@@ -1107,7 +1181,7 @@ namespace object_tests {

    for (auto name : names) {
      SUBTEST("ondemand::" + name, test_ondemand_doc(bad_jsons[count], [&](auto doc_result) {
-        ASSERT_RESULT( doc_result.type(), json_type::object );
+        ASSERT_RESULT(doc_result.type(), json_type::object );
        ASSERT_ERROR(doc_result.count_fields(), errors[count]);
        return true;
      }));
@@ -1220,7 +1294,13 @@ namespace object_tests {
  }

  bool run() {
-    return
+    return issue1979() &&
+           issue1977() &&
+#if SIMDJSON_EXCEPTIONS
+           issue1965() &&
+#endif
+           issue1974a() &&
+           issue1974b() &&
           issue1876a() &&
           issue1876() &&
           test_strager() &&
@@ -5,10 +5,36 @@ using namespace std;
 using namespace simdjson;
 using error_code=simdjson::error_code;

+bool string1() {
+  const char * data = "my data"; // 7 bytes
+  simdjson::padded_string my_padded_data(data, 7); // copies to a padded buffer
+  std::cout << my_padded_data << std::endl;
+  return true;
+}
+
+bool string2() {
+  std::string data = "my data";
+  simdjson::padded_string my_padded_data(data); // copies to a padded buffer
+  std::cout << my_padded_data << std::endl;
+  return true;
+}


 #if SIMDJSON_EXCEPTIONS

+  bool at_end() {
+    auto json = R"([1, 2] foo ])"_padded;
+    ondemand::parser parser;
+    ondemand::document doc = parser.iterate(json);
+    ondemand::array array = doc.get_array();
+    for (uint64_t values : array) {
+      std::cout << values << std::endl;
+    }
+    if(!doc.at_end()) {
+      std::cerr << "trailing content at byte index " << doc.current_location() - json.data() << std::endl;
+    }
+    TEST_SUCCEED();
+  }
  bool number_tests() {
    ondemand::parser parser;
    padded_string docdata = R"([1.0, 3, 1, 3.1415,-13231232,9999999999999999999])"_padded;
@@ -576,7 +602,7 @@ bool using_the_parsed_json_no_exceptions() {
    cout << "Make/Model: " << make << "/" << model << endl;

    // Casting a JSON element to an integer
-    uint64_t year;
+    uint64_t year{};
    error = car["year"].get(year);
    if(error) { std::cerr << error << std::endl; return false; }
    cout << "- This car is " << 2020 - year << " years old." << endl;
@@ -973,10 +999,12 @@ bool current_location_tape_error() {
  int64_t i;
  ASSERT_ERROR(doc["integer"].get_int64().get(i), TAPE_ERROR);
  ASSERT_SUCCESS(doc.current_location().get(ptr));
-  ASSERT_EQUAL(ptr, "false, \"integer\": -343} ");
+  std::string expected = "false, \"integer\": -343} ";
+  ASSERT_EQUAL(std::string(ptr,expected.size()), expected);
  TEST_SUCCEED();
 }

+
 bool current_location_user_error() {
  TEST_START();
  auto json = R"( [1,2,3] )"_padded;
@@ -987,7 +1015,8 @@ bool current_location_user_error() {
  int64_t i;
  ASSERT_ERROR(doc["integer"].get_int64().get(i), INCORRECT_TYPE);
  ASSERT_SUCCESS(doc.current_location().get(ptr));
-  ASSERT_EQUAL(ptr, "[1,2,3] ");
+  std::string expected = "[1,2,3] ";
+  ASSERT_EQUAL(std::string(ptr, expected.size()), expected);
  TEST_SUCCEED();
 }

@@ -1021,7 +1050,8 @@ bool current_location_no_error() {
    auto error = val.get_object().get(obj);
    if (!error) {
      ASSERT_SUCCESS(doc.current_location().get(ptr));
-      ASSERT_EQUAL(ptr, "\"key\": \"value\"}, true] ");
+      std::string expected = "\"key\": \"value\"}, true] ";
+      ASSERT_EQUAL(std::string(ptr, expected.size()), expected);
    }
  }
  TEST_SUCCEED();
@@ -1205,6 +1235,7 @@ bool example1958() {
 bool run() {
  return true
 #if SIMDJSON_EXCEPTIONS
+    && at_end()
    && example1956() && example1958()
 //    && basics_1() // Fails because twitter.json isn't in current directory. Compile test only.
    &&  basics_treewalk()
@@ -18,7 +18,7 @@ namespace scalar_tests {
  bool test_scalar_value(const padded_string &json, const T &expected, bool test_twice=true) {
    std::cout << "- JSON: " << json << endl;
    SUBTEST( "simdjson_result<document>", test_ondemand_doc(json, [&](auto doc_result) {
-      T actual;
+      T actual{};
      ASSERT_RESULT( doc_result.type(), expected_json_type<T>() );
      ASSERT_SUCCESS( doc_result.get(actual) );
      ASSERT_EQUAL( actual, expected );
@@ -33,7 +33,7 @@ namespace scalar_tests {
    SUBTEST( "document", test_ondemand_doc(json, [&](auto doc_result) {
      ondemand::document doc;
      ASSERT_SUCCESS( std::move(doc_result).get(doc) );
-      T actual;
+      T actual{};
      ASSERT_RESULT( doc.type(), expected_json_type<T>() );
      ASSERT_SUCCESS( doc.get(actual) );
      ASSERT_EQUAL( actual, expected );
@@ -50,7 +50,7 @@ namespace scalar_tests {
      padded_string whitespace_json = std::string(json) + " ";
      std::cout << "- JSON: " << whitespace_json << endl;
      SUBTEST( "simdjson_result<document>", test_ondemand_doc(whitespace_json, [&](auto doc_result) {
-        T actual;
+        T actual{};
        ASSERT_RESULT( doc_result.type(), expected_json_type<T>() );
        ASSERT_SUCCESS( doc_result.get(actual) );
        ASSERT_EQUAL( actual, expected );
@@ -65,7 +65,7 @@ namespace scalar_tests {
      SUBTEST( "document", test_ondemand_doc(whitespace_json, [&](auto doc_result) {
        ondemand::document doc;
        ASSERT_SUCCESS( std::move(doc_result).get(doc) );
-        T actual;
+        T actual{};
        ASSERT_RESULT( doc.type(), expected_json_type<T>() );
        ASSERT_SUCCESS( doc.get(actual) );
        ASSERT_EQUAL( actual, expected );
@@ -85,7 +85,7 @@ namespace scalar_tests {
      SUBTEST( "simdjson_result<value>", test_ondemand_doc(array_json, [&](auto doc_result) {
        int count = 0;
        for (simdjson_result<ondemand::value> val_result : doc_result) {
-          T actual;
+          T actual{};
          ASSERT_RESULT( val_result.type(), expected_json_type<T>() );
          ASSERT_SUCCESS( val_result.get(actual) );
          ASSERT_EQUAL(actual, expected);
@@ -105,7 +105,7 @@ namespace scalar_tests {
        for (simdjson_result<ondemand::value> val_result : doc_result) {
          ondemand::value val;
          ASSERT_SUCCESS( val_result.get(val) );
-          T actual;
+          T actual{};
          ASSERT_RESULT( val.type(), expected_json_type<T>() );
          ASSERT_SUCCESS( val.get(actual) );
          ASSERT_EQUAL(actual, expected);
@@ -129,7 +129,7 @@ namespace scalar_tests {
      SUBTEST( "simdjson_result<value>", test_ondemand_doc(whitespace_array_json, [&](auto doc_result) {
        int count = 0;
        for (simdjson_result<ondemand::value> val_result : doc_result) {
-          T actual;
+          T actual{};
          ASSERT_RESULT( val_result.type(), expected_json_type<T>() );
          ASSERT_SUCCESS( val_result.get(actual) );
          ASSERT_EQUAL(actual, expected);
@@ -150,7 +150,7 @@ namespace scalar_tests {
        for (simdjson_result<ondemand::value> val_result : doc_result) {
          ondemand::value val;
          ASSERT_SUCCESS( val_result.get(val) );
-          T actual;
+          T actual{};
          ASSERT_RESULT( val.type(), expected_json_type<T>() );
          ASSERT_SUCCESS( val.get(actual) );
          ASSERT_EQUAL(actual, expected);
@@ -56,7 +56,7 @@ namespace twitter_tests {
        std::string_view screen_name;
        ASSERT_SUCCESS( user["screen_name"].get(screen_name) );

-        bool default_profile;
+        bool default_profile{};
        ASSERT_SUCCESS( user["default_profile"].get(default_profile) );
        if (default_profile) {
          default_users.insert(screen_name);
@@ -114,6 +114,7 @@ simdjson_inline bool assert_iterate_error(T &arr, simdjson::error_code expected,
 #define ASSERT_EQUAL(ACTUAL, EXPECTED)  do { if (!::assert_equal  ((ACTUAL), (EXPECTED), #ACTUAL)) { return false; } } while (0);
 #define ASSERT_RESULT(ACTUAL, EXPECTED) do { if (!::assert_result ((ACTUAL), (EXPECTED), #ACTUAL)) { return false; } } while (0);
 #define ASSERT_SUCCESS(ACTUAL)          do { if (!::assert_success((ACTUAL),             #ACTUAL)) { return false; } } while (0);
+#define ASSERT_FAILURE(ACTUAL)          do { if (::assert_success((ACTUAL),             #ACTUAL)) { return false; } } while (0);
 #define ASSERT_ERROR(ACTUAL, EXPECTED)  do { if (!::assert_error  ((ACTUAL), (EXPECTED), #ACTUAL)) { return false; } } while (0);
 #define ASSERT_TRUE(ACTUAL)             do { if (!::assert_true   ((ACTUAL),             #ACTUAL)) { return false; } } while (0);
 #define ASSERT_FALSE(ACTUAL)             do { if (!::assert_false   ((ACTUAL),             #ACTUAL)) { return false; } } while (0);
@@ -177,7 +177,7 @@ readmefile = maindir + os.sep + "README.md"
 readmedata = open(readmefile).read()
 m = pattern.search(readmedata)
 if m == None:
-    print(colored(255, 0, 0, 'I cannot find a link to the API documentation in your README?????'))
+    print('I cannot find a link to the API documentation in your README')
 else:
    detectedreadme = m.group(1)
    print("found a link to your API documentation in the README file: "+detectedreadme+" ("+toversionstring(*newversion)+")")
Author	SHA1	Message	Date
Daniel Lemire	14d927128b	New release	2023-04-08 16:13:48 -04:00
Daniel Lemire	48152a0326	Fix for issue 1981 (#1982 ) * Fix * Rewording. * Doing it differently.	2023-04-08 16:11:40 -04:00
Daniel Lemire	b5cf53232c	More documentation.	2023-04-06 10:52:18 -04:00
Daniel Lemire	b3d0b732e6	Update README.md	2023-04-04 14:22:12 -04:00
Daniel Lemire	3c29f40f2f	Trying to verify issue 1979 (#1980 )	2023-04-04 11:03:37 -04:00
Daniel Lemire	4a89d79984	Guarding the benchmarks.	2023-04-01 18:39:11 -04:00
Daniel Lemire	9cb639dcd7	Update basics.md	2023-03-30 12:35:38 -04:00
Daniel Lemire	4699dd0ce2	Update basics.md	2023-03-30 10:49:19 -04:00
Daniel Lemire	cfd337bb50	This PR adds an 'at_end()' method. (#1978 ) * This PR adds an 'at_end()' method. * Adding 1111 } * Tweaking test.	2023-03-30 10:47:13 -04:00
Daniel Lemire	9bef628982	It is no longer necessary to check separately for trailing content.	2023-03-24 20:04:04 -04:00
Daniel Lemire	59025bc8b1	Avoiding the issue where doc.get_value().get_object() differs from doc.get_object() in how errors are reported. (#1975 ) * Avoiding the issue where doc.get_value().get_object() differs from doc.get_object() in how errors are reported. * Minor tweaks	2023-03-24 12:44:18 -04:00
Daniel Lemire	ffe96dde19	Update HACKING.md	2023-03-22 09:02:24 -04:00
Daniel Lemire	f9664cf184	Update HACKING.md	2023-03-22 08:56:12 -04:00
Daniel Lemire	830323f383	Update HACKING.md	2023-03-22 08:55:26 -04:00
Daniel Lemire	eb8e15d9c0	Update HACKING.md	2023-03-22 08:53:44 -04:00
Daniel Lemire	3571f3a8d0	Additional documentation.	2023-03-18 16:38:16 -04:00
Daniel Lemire	4b13020d65	Patch release	2023-03-13 21:44:34 -04:00
Daniel Lemire	75c6c950e5	Ok.	2023-03-13 21:26:32 -04:00
Daniel Lemire	5ad7698af1	...	2023-03-13 17:07:25 -04:00
Daniel Lemire	f6f359d3eb	Very silly.	2023-03-13 17:05:53 -04:00
Daniel Lemire	185e157e0c	Silencing the stupid compiler warnings.	2023-03-13 16:06:10 -04:00
Daniel Lemire	3f3f6cde7c	Bad compiler	2023-03-13 13:33:20 -04:00
Daniel Lemire	005054916d	Bad compiler	2023-03-13 13:02:01 -04:00
Daniel Lemire	338224849d	Disabling memory sanitizer with one function. Might help with issue 1965 (#1966 ) * Might help with issue 1965 * Fix macro * Adding test. * Stupid compiler * Silly compiler * Stupid compilers * Unnecessary fixes * Update developer-options.cmake	2023-03-13 12:34:47 -04:00
Ashot Vardanian	ada52641b4	Fix: Defining maximal DOM capacity (#1970 )	2023-03-13 12:33:37 -04:00
Daniel Lemire	ede9d57f22	Adding tests with _GLIBCXX_ASSERTIONS (#1969 ) * Allowing users to specify -D_GLIBCXX_ASSERTIONS * Adding CI tests for GLIBCXX_ASSERTIONS * Adding curly brackets.	2023-03-13 12:32:43 -04:00