Updates

Right-justify int columns in markdown tables
2026-06-08 17:27:07 +00:00 · 2023-09-13 11:57:43 -04:00 · 2023-09-13 11:21:58 -04:00 · 2023-09-13 11:10:01 -04:00 · 2023-09-12 15:02:17 -04:00 · 2023-09-12 14:58:33 -04:00
303 changed files with 1176934 additions and 19529 deletions
@@ -1,2 +1,46 @@
 CompileFlags:
-    CompilationDatabase: build
+    CompilationDatabase: build
+    Add:
+        - -Werror -Wall -Wextra -Weffc++ -Wsign-compare -Wshadow -Wwrite-strings -Wpointer-arith -Winit-self -Wconversion -Wno-sign-conversion
+        - -Wundefined-inline
+Diagnostics:
+    Suppress:
+        - misc-unused-alias-decls
+        - misc-unused-using-decls
+        - misc-definitions-in-headers # TODO fix and remove these violations
+---
+If:
+    PathMatch:
+        - include/.*
+        - src/.*
+    PathExclude:
+        - include/simdjson.h
+        - src/simdjson.cpp
+CompileFlags:
+    Add:
+        - -Wno-unneeded-internal-declaration
+        - -Wno-undefined-internal # TODO fix and remove these violations
+        - -Wno-unused-function
+        - -Wno-unused-const-variable
+Diagnostics:
+    Suppress:
+        - pp_including_mainfile_in_preamble
+---
+# Amalgamated files that require or partly define an implementation
+If:
+    PathMatch:
+        - .*/(arm64|fallback|haswell|icelake|ppc64|westmere)/begin.h
+        - .*/generic/.*
+Diagnostics:
+    Suppress:
+        - pragma_attribute_no_pop_eof
+---
+# clang has a bad time detecting the push/pop together in src/ for some reason
+If:
+    PathMatch:
+        - include/simdjson/.*/end.h
+        - src/(arm64|fallback|haswell|icelake|ppc64|westmere).cpp
+Diagnostics:
+    Suppress:
+        - pragma_attribute_no_pop_eof
+        - pragma_attribute_stack_mismatch
@@ -49,7 +49,7 @@ It is fine to report bugs against our main branch, but if that is what you are d
 - Version [e.g. 22]
 - Optimization setting (e.g., -O3)

-We support up-to-date 64-bit ARM and x64 FreeBSD, macOS, Windows and Linux systems. Please ensure that your configuration is supported before labelling the issue as a bug. In particular, we do not support legacy 32-bit systems.
+We support up-to-date 64-bit ARM and x64 FreeBSD, macOS, Windows and Linux systems. Please ensure that your configuration is supported before labelling the issue as a bug.

 **Indicate whether you are willing or able to provide a bug fix as a pull request**

@@ -31,4 +31,4 @@ jobs:
          ./alpine.sh cmake --build build_for_alpine
      - name: test
        run: |
-          ./alpine.sh bash -c "cd build_for_alpine && ctest  -LE explicitonly"
+          ./alpine.sh bash -c "cd build_for_alpine && ctest  -LE explicitonly --output-on-failure"
@@ -34,6 +34,7 @@ jobs:
        sudo apt update
        sudo apt-get install --quiet ninja-build valgrind zip unzip lsb-release wget software-properties-common gnupg
        wget https://apt.llvm.org/llvm.sh
+        sudo apt-get purge --auto-remove llvm python3-lldb-14 llvm-14
        chmod +x llvm.sh
        sudo ./llvm.sh $CLANGVERSION

@@ -20,13 +20,13 @@ jobs:
          cd builddebug &&
          cmake -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          cmake  -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cmake --install . &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          cd ../tests/installation_tests/find &&
@@ -25,13 +25,13 @@ jobs:
          cd builddebug &&
          cmake -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          cmake  -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cmake --install . &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          cd ../tests/installation_tests/find &&
@@ -20,13 +20,13 @@ jobs:
          cd builddebug &&
          cmake -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DSIMDJSON_EXCEPTIONS=OFF -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          cmake  -DSIMDJSON_DEVELOPER_MODE=ON -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_EXCEPTIONS=OFF -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          make install  &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          mkdir testfindpackage &&
@@ -20,13 +20,13 @@ jobs:
          cd builddebug &&
          cmake -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF  -DSIMDJSON_ENABLE_THREADS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          cmake  -DSIMDJSON_DEVELOPER_MODE=ON -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_ENABLE_THREADS=OFF -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          make install  &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          mkdir testfindpackage &&
@@ -3,7 +3,7 @@ name: Ubuntu 20.04 CI (GCC 9) With Memory Sanitizer
 on: [push, pull_request]

 jobs:
-  ubuntu-build:
+  ubuntu-build-address-sanitizier:
    if: >-
      ! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
      ! contains(toJSON(github.event.commits.*.message), '[skip github]')
@@ -20,11 +20,23 @@ jobs:
          cd builddebug &&
          cmake -DSIMDJSON_SANITIZE=ON -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly
+          ctest --output-on-failure -LE explicitonly -j
+  ubuntu-build-undefined-sanitizer:
+    if: >-
+      ! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
+      ! contains(toJSON(github.event.commits.*.message), '[skip github]')
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/cache@v3
+        with:
+          path: dependencies/.cache
+          key: ${{ hashFiles('dependencies/CMakeLists.txt') }}
+          ctest --output-on-failure -LE explicitonly -j
      - name: Use cmake with undefined sanitizer
        run: |
          mkdir builddebugundefsani &&
          cd builddebugundefsani &&
          cmake -DSIMDJSON_SANITIZE_UNDEFINED=ON -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly
+          ctest --output-on-failure -LE explicitonly -j
@@ -14,19 +14,33 @@ jobs:
        with:
          path: dependencies/.cache
          key: ${{ hashFiles('dependencies/CMakeLists.txt') }}
+      - name: Use cmake to build just the library
+        run: |
+          mkdir buildjustlib &&
+          cd buildjustlib &&
+          cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSIMDJSON_DEVELOPER_MODE=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
+          cmake --build .   &&
+          cmake --install . &&
+          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp &&
+          c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson &&
+          cd ../tests/installation_tests/find &&
+          mkdir buildjustlib &&
+          cd buildjustlib &&
+          cmake -DCMAKE_INSTALL_PREFIX:PATH=../../../buildjustlib/destination .. &&
+          cmake --build .
      - name: Use cmake
        run: |
          mkdir builddebug &&
          cd builddebug &&
          cmake -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          cmake  -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cmake --install . &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          cd ../tests/installation_tests/find &&
@@ -14,12 +14,10 @@ jobs:
        with:
          path: dependencies/.cache
          key: ${{ hashFiles('dependencies/CMakeLists.txt') }}
-      - name: Install clang++-13
-        run: sudo apt-get install -y clang++-13
      - name: Use cmake
        run: |
          mkdir build &&
          cd build &&
          CXX=clang++-13 cmake -DSIMDJSON_DEVELOPER_MODE=ON ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly
+          ctest --output-on-failure -LE explicitonly -j
@@ -14,12 +14,10 @@ jobs:
        with:
          path: dependencies/.cache
          key: ${{ hashFiles('dependencies/CMakeLists.txt') }}
-      - name: Install clang++-14
-        run: sudo apt-get install -y clang++-14
      - name: Use cmake
        run: |
          mkdir build &&
          cd build &&
          CXX=clang++-14 cmake -DSIMDJSON_DEVELOPER_MODE=ON ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly
+          ctest --output-on-failure -LE explicitonly -j
@@ -20,13 +20,13 @@ jobs:
          cd builddebug &&
          CXX=g++-12 cmake -DSIMDJSON_CXX_STANDARD=20 -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          CXX=g++-12 cmake -DSIMDJSON_CXX_STANDARD=20  -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cmake --install . &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          cd ../tests/installation_tests/find &&
@@ -22,4 +22,4 @@ jobs:
          cd build &&
          CXX=g++-12 cmake -DSIMDJSON_DEVELOPER_MODE=ON ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly
+          ctest --output-on-failure -LE explicitonly -j
@@ -1,4 +1,4 @@
-name: Ubuntu 20.04 CI (GCC 9) with Thread Sanitizer
+name: Ubuntu 22.04 CI (GCC 11) with Thread Sanitizer

 on: [push, pull_request]

@@ -7,7 +7,7 @@ jobs:
    if: >-
      ! contains(toJSON(github.event.commits.*.message), '[skip ci]') &&
      ! contains(toJSON(github.event.commits.*.message), '[skip github]')
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v3
      - uses: actions/cache@v3
@@ -14,19 +14,33 @@ jobs:
        with:
          path: dependencies/.cache
          key: ${{ hashFiles('dependencies/CMakeLists.txt') }}
+      - name: Use cmake to build just the library
+        run: |
+          mkdir buildjustlib &&
+          cd buildjustlib &&
+          cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSIMDJSON_DEVELOPER_MODE=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
+          cmake --build .   &&
+          cmake --install . &&
+          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp &&
+          c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson &&
+          cd ../tests/installation_tests/find &&
+          mkdir buildjustlib &&
+          cd buildjustlib &&
+          cmake -DCMAKE_INSTALL_PREFIX:PATH=../../../buildjustlib/destination .. &&
+          cmake --build .
      - name: Use cmake
        run: |
          mkdir builddebug &&
          cd builddebug &&
          cmake -DCMAKE_BUILD_TYPE=Debug -DSIMDJSON_GOOGLE_BENCHMARKS=OFF -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cd .. &&
          mkdir build &&
          cd build &&
          cmake  -DSIMDJSON_GOOGLE_BENCHMARKS=ON -DSIMDJSON_DEVELOPER_MODE=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=destination ..  &&
          cmake --build .   &&
-          ctest -j --output-on-failure -LE explicitonly   &&
+          ctest --output-on-failure -LE explicitonly -j   &&
          cmake --install . &&
          echo -e '#include <simdjson.h>\nint main(int argc,char**argv) {simdjson::dom::parser parser;simdjson::dom::element tweets = parser.load(argv[1]); }' > tmp.cpp && c++ -Idestination/include -Ldestination/lib -std=c++17 -Wl,-rpath,destination/lib -o linkandrun tmp.cpp -lsimdjson && ./linkandrun jsonexamples/twitter.json &&
          cd ../tests/installation_tests/find &&
@@ -104,3 +104,6 @@ objs
 !.vscode/tasks.json
 !.vscode/launch.json
 !.vscode/extensions.json
+
+# clangd
+.cache
@@ -4,10 +4,15 @@

 	// List of extensions which should be recommended for users of this workspace.
 	"recommendations": [
-		// Syntax
-		"ms-vscode.cpptools",
-		"ms-vscode.cmake-tools",
+		// C++
+		"llvm-vs-code-extensions.vscode-clangd",
+		"xaver.clang-format",
+		// Python
 		"ms-python.python",
+		// .github/*
+		"github.vscode-github-actions",
+		// cmake
+		"ms-vscode.cmake-tools",
 		"twxs.cmake"
 	],
 	// List of extensions recommended by VS Code that should not be recommended for users of this workspace.
@@ -3,8 +3,9 @@
        {"column": 95 },
        {"column": 120 }
    ],
-    "files.trimTrailingWhitespace": true,
+    "files.trimTrailingWhitespace": false,
    "files.associations": {
+        ".clangd": "yaml",
        "array": "cpp",
        "iterator": "cpp",
        "chrono": "cpp",
@@ -98,5 +99,6 @@
        "shared_mutex": "cpp",
        "ranges": "cpp",
        "span": "cpp"
-    }
+    },
+    "editor.trimAutoWhitespace": true
 }
@@ -3,7 +3,7 @@ cmake_minimum_required(VERSION 3.14)
 project(
    simdjson
    # The version number is modified by tools/release.py
-    VERSION 3.2.1
+    VERSION 3.2.3
    DESCRIPTION "Parsing gigabytes of JSON per second"
    HOMEPAGE_URL "https://simdjson.org/"
    LANGUAGES CXX C
@@ -38,7 +38,7 @@ PROJECT_NAME           = simdjson
 # could be handy for archiving the generated documentation or if some version
 # control system is used.

-PROJECT_NUMBER         = "3.2.1"
+PROJECT_NUMBER         = "3.2.3"

 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a
@@ -73,26 +73,34 @@ simdjson's source structure, from the top level, looks like this:

 * **CMakeLists.txt:** The main build system.
 * **include:** User-facing declarations and inline definitions (most user-facing functions are inlined).
-  * simdjson.h: A "main include" that includes files from include/simdjson/. This is equivalent to
+  * simdjson.h: the `simdjson` namespace. A "main include" that includes files from include/simdjson/. This is equivalent to
    the distributed simdjson.h.
-  * simdjson/*.h: Declarations for public simdjson classes and functions.
-  * simdjson/*-inl.h: Definitions for public simdjson classes and functions.
+    * simdjson/*.h: Declarations for public simdjson classes and functions.
+    * simdjson/*-inl.h: Definitions for public simdjson classes and functions.
+    * simdjson/internal/*.h: the `simdjson::internal` namespace. Private classes and functions used by the rest of simdjson.
+  * simdjson/dom.h: the `simdjson::dom` namespace. Includes all public DOM classes.
+    * simdjson/dom/*.h: Declarations/definitions for individual DOM classes.
+  * simdjson/arm64|fallback|haswell|icelake|ppc64|westmere.h: `simdjson::<implementation>` namesapce. Common implementation-specific tools like number and string parsing, as well as minification.
+    * simdjson/arm64|fallback|haswell|icelake|ppc64|westmere/*.h: implementation-specific functions such as , etc.
+    * simdjson/generic/*.h: the bulk of the actual code, written generically and compiled for each implementation, using functions defined in the implementation's .h files.
+      * simdjson/generic/dependencies.h: dependencies on common, non-implementation-specific simdjson classes. This will be included before including amalgamated.h.
+      * simdjson/generic/amalgamated.h: all generic ondemand classes for an implementation.
+  * simdjson/ondemand.h: the `simdjson::ondemand` namespace. Includes all public ondemand classes.
+    * simdjson/builtin.h: the `simdjson::builtin` namespace. Aliased to the most universal implementation available.
+    * simdjson/builtin/ondemand.h: the `simdjson::builtin::ondemand` namespace.
+    * simdjson/arm64|fallback|haswell|icelake|ppc64|westmere/ondemand.h: the `simdjson::<implementation>::ondemand` namespace. on demand compiled for the specific implementation.
+    * simdjson/generic/ondemand/*.h: individual on demand classes, generically written.
+      * simdjson/generic/ondemand/dependencies.h: dependencies on common, non-implementation-specific simdjson classes. This will be included before including amalgamated.h.
+      * simdjson/generic/ondemand/amalgamated.h: all generic ondemand classes for an implementation.
 * **src:** The source files for non-inlined functionality (e.g. the architecture-specific parser
  implementations).
  * simdjson.cpp: A "main source" that includes all implementation files from src/. This is
    equivalent to the distributed simdjson.cpp.
-  * arm64/|fallback/|haswell/|ppc64/|westmere/: Architecture-specific implementations. All functions are
-    Each architecture defines its own namespace, e.g. simdjson::haswell.
-  * generic/: Generic implementations of the simdjson parser. These files may be included and
-    compiled multiple times, from whichever architectures use them. They assume they are already
-    enclosed in a namespace, e.g.:
-    ```c++
-    namespace simdjson {
-      namespace haswell {
-        #include "generic/stage1/json_structural_indexer.h"
-      }
-    }
-    ```
+  * *.cpp: other misc. implementations, such as `simdjson::implementation` and the minifier.
+  * arm64|fallback|haswell|icelake|ppc64|westmere.cpp: Architecture-specific parser implementations.
+    * generic/*.h: `simdjson::<implementation>` namespace. Generic implementation of the parser, particularly the `dom_parser_implementation`.
+    * generic/stage1/*.h: `simdjson::<implementation>::stage1` namespace. Generic implementation of the simd-heavy tokenizer/indexer pass of the simdjson parser. Used for the On Demand interface
+    * generic/stage2/*.h: `simdjson::<implementation>::stage2` namespace. Generic implementation of the tape creator, which consumes the index from stage 1 and actually parses numbers and string and such. Used for the DOM interface.

 Other important files and directories:
 * **.drone.yml:** Definitions for Drone CI.
@@ -1,8 +1,6 @@

 [![Ubuntu 20.04 CI](https://github.com/simdjson/simdjson/workflows/Ubuntu%2020.04%20CI%20(GCC%209)/badge.svg)](https://simdjson.org/plots.html)
-![VS16-CI](https://github.com/simdjson/simdjson/workflows/VS16-CI/badge.svg)
 [![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/simdjson.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:simdjson)
-![MinGW64-CI](https://github.com/simdjson/simdjson/workflows/MinGW64-CI/badge.svg)
 [![][license img]][license]

 [![Doxygen Documentation](https://img.shields.io/badge/docs-doxygen-green.svg)](https://simdjson.github.io/simdjson/)
@@ -445,7 +445,7 @@ struct benchmarker {
    return 100.0 * a / b;
  }

-  void print(bool tabbed_output) const {
+  void print(bool tabbed_output, bool stage1_only) const {
    if (tabbed_output) {
      char* filename_copy = reinterpret_cast<char*>(malloc(strlen(filename)+1));
      SIMDJSON_PUSH_DISABLE_WARNINGS
@@ -503,17 +503,21 @@ struct benchmarker {
          stats->blocks_with_16_structurals_flipped, percent(stats->blocks_with_16_structurals_flipped, stats->blocks));
      }
      printf("\n");
-      printf("All Stages (excluding allocation)\n");
-      print_aggregate("|    "   , all_stages_without_allocation.best);
-      // frequently, allocation is a tiny fraction of the running time so we omit it
-      if(allocate_stage.best.elapsed_sec() > 0.01 * all_stages_without_allocation.best.elapsed_sec()) {
-        printf("|- Allocation\n");
-        print_aggregate("|    ", allocate_stage.best);
+      if(!stage1_only) {
+        printf("All Stages (excluding allocation)\n");
+        print_aggregate("|    "   , all_stages_without_allocation.best);
+        // frequently, allocation is a tiny fraction of the running time so we omit it
+        if(allocate_stage.best.elapsed_sec() > 0.01 * all_stages_without_allocation.best.elapsed_sec()) {
+          printf("|- Allocation\n");
+          print_aggregate("|    ", allocate_stage.best);
+        }
      }
      printf("|- Stage 1\n");
      print_aggregate("|    ", stage1.best);
-      printf("|- Stage 2\n");
-      print_aggregate("|    ", stage2.best);
+      if(!stage1_only) {
+        printf("|- Stage 2\n");
+        print_aggregate("|    ", stage2.best);
+      }
      if (collector.has_events()) {
        double freq1 = (stage1.best.cycles() / stage1.best.elapsed_sec()) / 1000000000.0;
        double freq2 = (stage2.best.cycles() / stage2.best.elapsed_sec()) / 1000000000.0;
@@ -218,7 +218,7 @@ int main(int argc, char *argv[]) {
  if (!options.verbose) { progress.erase(); }

  for (size_t i=0; i<options.files.size(); i++) {
-    benchmarkers[i]->print(options.tabbed_output);
+    benchmarkers[i]->print(options.tabbed_output, options.stage1_only);
    delete benchmarkers[i];
  }

@@ -34,11 +34,15 @@
 #include <string>
 #include <vector>

-#include "linux-perf-events.h"
 #ifdef __linux__
+#include "linux-perf-events.h"
 #include <libgen.h>
 #endif

+#if __APPLE__ &&  __aarch64__
+#include "apple/apple_arm_events.h"
+#endif
+
 #include "simdjson.h"

 using std::string;
@@ -134,7 +138,7 @@ struct event_collector {

 #if defined(__linux__)
  LinuxEvents<PERF_TYPE_HARDWARE> linux_events;
-  event_collector(simdjson_unused bool quiet = false) : linux_events(vector<int>{
+  event_collector() : linux_events(vector<int>{
  #if SIMDJSON_SIMPLE_PERFORMANCE_COUNTERS
    PERF_COUNT_HW_CPU_CYCLES,
    PERF_COUNT_HW_INSTRUCTIONS,
@@ -149,8 +153,17 @@ struct event_collector {
  bool has_events() {
    return linux_events.is_working();
  }
+#elif __APPLE__ &&  __aarch64__
+  AppleEvents apple_events;
+  performance_counters diff;
+  event_collector() : diff(0) {
+    apple_events.setup_performance_counters();
+  }
+  bool has_events() {
+    return apple_events.setup_performance_counters();
+  }
 #else
-  event_collector(simdjson_unused bool _quiet = false) {}
+  event_collector() {}
  bool has_events() {
    return false;
  }
@@ -159,6 +172,8 @@ struct event_collector {
  simdjson_inline void start() {
 #if defined(__linux)
    linux_events.start();
+#elif __APPLE__ &&  __aarch64__
+    if(has_events()) { diff = apple_events.get_counters(); }
 #endif
    start_clock = steady_clock::now();
  }
@@ -166,6 +181,16 @@ struct event_collector {
    time_point<steady_clock> end_clock = steady_clock::now();
 #if defined(__linux)
    linux_events.end(count.event_counts);
+#elif __APPLE__ &&  __aarch64__
+    if(has_events()) {
+      performance_counters end = apple_events.get_counters();
+      diff = end - diff;
+    }
+    count.event_counts[0] = diff.cycles;
+    count.event_counts[1] = diff.instructions;
+    count.event_counts[2] = diff.missed_branches;
+    count.event_counts[3] = 0;
+    count.event_counts[4] = 0;
 #endif
    count.elapsed = end_clock - start_clock;
    return count;
@@ -19,7 +19,7 @@ void maybe_display_implementation() {
 template<typename B, typename R> static void run_json_benchmark(benchmark::State &state) {
  maybe_display_implementation();

-  event_collector collector(true);
+  event_collector collector;
  event_aggregate events;

  // Warmup and equality check (make sure the data is right!)
@@ -117,7 +117,12 @@ set(CMAKE_CXX_EXTENSIONS OFF)
 set(CMAKE_MACOSX_RPATH OFF)
 set(CMAKE_THREAD_PREFER_PTHREAD ON)
 set(THREADS_PREFER_PTHREAD_FLAG ON)
+set(SIMDJSON_STRUCTURAL_INDEXER_STEP CACHE STRING "the SIMDJSON_STRUCTURAL_INDEXER_STEP variable")

+if(SIMDJSON_STRUCTURAL_INDEXER_STEP)
+  message(STATUS "Setting SIMDJSON_STRUCTURAL_INDEXER_STEP to ${SIMDJSON_STRUCTURAL_INDEXER_STEP}.")
+  add_compile_definitions(SIMDJSON_STRUCTURAL_INDEXER_STEP=${SIMDJSON_STRUCTURAL_INDEXER_STEP})
+endif()
 # LTO seems to create all sorts of fun problems. Let us
 # disable temporarily.
 #include(CheckIPOSupported)
@@ -355,7 +355,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
 * **Field Access:** To get the value of the "foo" field in an object, use `object["foo"]`. This will
  scan through the object looking for the field with the matching string, doing a character-by-character
  comparison. It may generate the error `simdjson::NO_SUCH_FIELD` if there is no such key in the object, it may throw an exception (see [Error Handling](#error-handling)). For efficiency reason, you should avoid looking up the same field repeatedly: e.g., do
-  not do `object["foo"]` followed by `object["foo"]` with the same `object` instance. Keep in mind that On Demand does not buffer or save the result of the parsing: if you repeatedly access `object["foo"]`, then it must repeatedly seek the key and parse the content. The library does not provide a distinct function to check if a key is present, instead we recommend you attempt to access the key: e.g., by doing `ondemand::value val{}; if(!object["foo"].get(val)) {...}`, you have that `val` contains the requested value inside the if clause.  It is your responsibility as a user to temporarily keep a reference to the value (`auto v = object["foo"]`), or to consume the content and store it in your own data structures. If you consume an
+  not do `object["foo"]` followed by `object["foo"]` with the same `object` instance. Keep in mind that On Demand does not buffer or save the result of the parsing: if you repeatedly access `object["foo"]`, then it must repeatedly seek the key and parse the content. The library does not provide a distinct function to check if a key is present, instead we recommend you attempt to access the key: e.g., by doing `ondemand::value val{}; if (!object["foo"].get(val)) {...}`, you have that `val` contains the requested value inside the if clause.  It is your responsibility as a user to temporarily keep a reference to the value (`auto v = object["foo"]`), or to consume the content and store it in your own data structures. If you consume an
  object twice: `std::string_view(object["foo"]` followed by `std::string_view(object["foo"]` then your code
  is in error. Furthermore, you can only consume one field at a time, on the same object. The
  value instance you get from  `content["bids"]` becomes invalid when you call `content["asks"]`.
@@ -381,7 +381,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
  >    // parses and writes out the key, after unescaping it,
  >    // to a string buffer. It causes a performance penalty.
  >    std::string_view keyv = field.unescaped_key();
-  >    if(keyv == "key") { std::cout << uint64_t(field.value()); }
+  >    if (keyv == "key") { std::cout << uint64_t(field.value()); }
  >  }
  > ```
  >
@@ -445,7 +445,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
  > {
  >   ondemand::parser parser;
  >   for (ondemand::object car : parser.iterate(cars_json)) {
-  >     if(uint64_t(car["year"]) > 2000) {
+  >     if (uint64_t(car["year"]) > 2000) {
  >       arrays.push_back(simdjson::to_json_string(car["tire_pressure"]));
  >     }
  >   }
@@ -454,7 +454,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
  > std::ostringstream oss;
  > oss << "[";
  > for(size_t i = 0; i < arrays.size(); i++) {
-  >   if(i>0) { oss << ","; }
+  >   if (i>0) { oss << ","; }
  >   oss << arrays[i];
  > }
  > oss << "]";
@@ -597,7 +597,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
    case ondemand::json_type::null:
      // We check that the value is indeed null
      // otherwise: an error is thrown.
-      if(element.is_null()) {
+      if (element.is_null()) {
        cout << "null";
      }
      break;
@@ -910,11 +910,11 @@ bool simple_error_example() {
    ondemand::parser parser;
    auto json = R"({"bad number":3.14.1 })"_padded;
    ondemand::document doc;
-    if( parser.iterate(json).get(doc) != SUCCESS ) { return false; }
+    if (parser.iterate(json).get(doc) != SUCCESS) { return false; }
    double x;
    auto error = doc["bad number"].get_double().get(x);
    // returns "simdjson::NUMBER_ERROR"
-    if(error != SUCCESS) {
+    if (error != SUCCESS) {
      std::cout << error << std::endl;
      return false;
    }
@@ -976,10 +976,10 @@ it selects the key `"count"` within that object.
 int main(void) {
  simdjson::ondemand::parser parser;
  auto error = padded_string::load("twitter.json").get(json);
-  if(error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  simdjson::ondemand::document tweets;
  error = parser.iterate(json).get(tweets);
-  if( error ) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  simdjson::ondemand::value res;
  error = tweets["search_metadata"]["count"].get(res);
  if (error != SUCCESS) {
@@ -1010,12 +1010,12 @@ int main(void) {
  simdjson::ondemand::document tweets;
  padded_string json;
  auto error = padded_string::load("twitter.json").get(json);
-  if(error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  error = parser.iterate(json).get(tweets);
-  if(error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  uint64_t identifier;
  error = tweets["statuses"].at(0)["id"].get(identifier);
-  if(error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  std::cout << identifier << std::endl;
 }
 ```
@@ -1039,40 +1039,40 @@ bool parse() {

  // Iterating through an array of objects
  auto error = parser.iterate(cars_json).get(doc);
-  if(error) { std::cerr << error << std::endl; return false; }
+  if (error) { std::cerr << error << std::endl; return false; }
  ondemand::array cars; // invalid until the get() succeeds
  error = doc.get_array().get(cars);

  for (auto car_value : cars) {
    ondemand::object car; // invalid until the get() succeeds
    error = car_value.get_object().get(car);
-    if(error) { std::cerr << error << std::endl; return false; }
+    if (error) { std::cerr << error << std::endl; return false; }

    // Accessing a field by name
    std::string_view make;
    std::string_view model;
    error = car["make"].get(make);
-    if(error) { std::cerr << error << std::endl; return false; }
+    if (error) { std::cerr << error << std::endl; return false; }
    error = car["model"].get(model);
-    if(error) { std::cerr << error << std::endl; return false; }
+    if (error) { std::cerr << error << std::endl; return false; }

    cout << "Make/Model: " << make << "/" << model << endl;

    // Casting a JSON element to an integer
    uint64_t year{};
    error = car["year"].get(year);
-    if(error) { std::cerr << error << std::endl; return false; }
+    if (error) { std::cerr << error << std::endl; return false; }
    cout << "- This car is " << 2020 - year << " years old." << endl;

    // Iterating through an array of floats
    double total_tire_pressure = 0;
    ondemand::array pressures;
    error = car["tire_pressure"].get_array().get(pressures);
-    if(error) { std::cerr << error << std::endl; return false; }
+    if (error) { std::cerr << error << std::endl; return false; }
    for (auto tire_pressure_value : pressures) {
      double tire_pressure;
      error = tire_pressure_value.get_double().get(tire_pressure);
-      if(error) { std::cerr << error << std::endl; return false; }
+      if (error) { std::cerr << error << std::endl; return false; }
      total_tire_pressure += tire_pressure;
    }
    cout << "- Average tire pressure: " << (total_tire_pressure / 4) << endl;
@@ -1088,7 +1088,7 @@ after you have initialized them and checked that there is no error:
    ondemand::object car; // invalid until the get() succeeds
    // the `car` instance should not use used before it is initialized
    error = car_value.get_object().get(car);
-    if(error) {
+    if (error) {
      // the `car` instance should not use used
    } else {
      // the `car` instance can be safely used
@@ -1102,20 +1102,20 @@ having to handle exceptions.
  ondemand::parser parser;
  ondemand::document doc;
  auto error = parser.iterate(json).get(doc);
-  if(error) { return false; }
+  if (error) { return false; }
  ondemand::object object; // invalid until the get() succeeds
  error = doc.get_object().get(object);
-  if(error) { return false; }
+  if (error) { return false; }
  for(auto field : object) {
    // We could replace 'field.key() with field.unescaped_key(),
    // and ondemand::raw_json_string by std::string_view.
    ondemand::raw_json_string keyv;
    error = field.key().get(keyv);
-    if(error) { return false; }
-    if(keyv == "key") {
+    if (error) { return false; }
+    if (keyv == "key") {
      uint64_t intvalue;
      error = field.value().get(intvalue);
-      if(error) { return false; }
+      if (error) { return false; }
      std::cout << intvalue;
    }
  }
@@ -1294,7 +1294,7 @@ content.
  for (uint64_t values : array) {
    std::cout << values << std::endl;
  }
-  if(!doc.at_end()) {
+  if (!doc.at_end()) {
    // In this instance, we will be left pointing at 'foo' since we have consumed the array [1,2].
    std::cerr << "trailing content at byte index " << doc.current_location() - json.data() << std::endl;
  }
@@ -1324,7 +1324,7 @@ before printout the data.

  auto doc = parser.iterate(cars_json);
  for (simdjson_unused ondemand::object car : doc) {
-    if(car["make"] == "Toyota") { count++; }
+    if (car["make"] == "Toyota") { count++; }
  }
  std::cout << "We have " << count << " Toyota cars.\n";
  doc.rewind(); // requires simdjson 1.0 or better
@@ -1392,18 +1392,18 @@ ondemand::parser parser;
 ondemand::document_stream stream;
 size_t counter{0};
 auto error = parser.iterate_many(json, 50).get(stream);
-if( error ) { /* handle the error */ }
+if (error) { /* handle the error */ }
 for (auto doc: stream) {
-  if(counter < 6) {
+  if (counter < 6) {
    int64_t val;
    error = doc.at_pointer("/4").get(val);
-    if( error ) { /* handle the error */ }
+    if (error) { /* handle the error */ }
    std::cout << "5 = " << val << std::endl;
  } else {
    ondemand::value val;
    error = doc.at_pointer("/4").get(val);
    // error == simdjson::CAPACITY
-    if(error) {
+    if (error) {
      std::cerr << error << std::endl;
      // We left 293 bytes unprocessed at the tail end of the input.
      std::cout << " unprocessed bytes at the end: " << stream.truncated_bytes() << std::endl;
@@ -105,7 +105,7 @@ Once you have an element, you can navigate it with idiomatic C++ iterators, oper
  with the `size()` method.
 * **Checking an Element Type:** You can check an element's type with `element.type()`. It
  returns an `element_type` with values such as `simdjson::dom::element_type::ARRAY`, `simdjson::dom::element_type::OBJECT`, `simdjson::dom::element_type::INT64`,  `simdjson::dom::element_type::UINT64`,`simdjson::dom::element_type::DOUBLE`, `simdjson::dom::element_type::STRING`, `simdjson::dom::element_type::BOOL` or, `simdjson::dom::element_type::NULL_VALUE`.
-* **Output to streams and strings:** Given a document or an element (or node) out of a JSON document, you can output a minified string version using the C++ stream idiom (`out << element`). You can also request the construction of a minified string version (`simdjson::minify(element)`). Numbers are serialized as 64-bit floating-point numbers (`double`).
+* **Output to streams and strings:** Given a document or an element (or node) out of a JSON document, you can output a minified string version using the C++ stream idiom (`out << element`). You can also request the construction of a minified string version (`simdjson::minify(element)`) or a prettified string version (`simdjson::prettify(element)`). Numbers are serialized as 64-bit floating-point numbers (`double`).

 ### Examples

@@ -329,10 +329,10 @@ int main(void) {
  simdjson::dom::parser parser;
  simdjson::dom::element tweets; // invalid until the get() succeeds
  auto error = parser.load("twitter.json").get(tweets);
-  if(error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  uint64_t identifier;
  error = tweets["statuses"].at(0)["id"].get(identifier);
-  if(error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
+  if (error) { std::cerr << error << std::endl; return EXIT_FAILURE; }
  std::cout << identifier << std::endl;
  return EXIT_SUCCESS;
 }
@@ -101,8 +101,8 @@ by comparing it with the null pointer.

 ```c++
 auto my_implementation = simdjson::get_available_implementations()["haswell"];
-if(! my_implementation) { exit(1); }
-if(! my_implementation->supported_by_runtime_system()) { exit(1); }
+if (! my_implementation) { exit(1); }
+if (! my_implementation->supported_by_runtime_system()) { exit(1); }
 simdjson::get_active_implementation() = my_implementation;
 ```

@@ -113,7 +113,7 @@ You should call `supported_by_runtime_system()` to compare the processor's featu

 ```c++
 for (auto implementation : simdjson::get_available_implementations()) {
-  if(implementation->supported_by_runtime_system()) {
+  if (implementation->supported_by_runtime_system()) {
    cout << implementation->name() << ": " << implementation->description() << endl;
  }
 }
@@ -17,6 +17,9 @@ Contents

 - [Motivations](#motivations)
 - [How it works](#how-it-works)
+  - [Context](#context)
+  - [Design](#design)
+  - [Threads](#threads)
 - [Support](#support)
 - [API](#api)
 - [Use cases](#use-cases)
@@ -193,12 +196,12 @@ Let us illustrate the idea with code:
    simdjson::ondemand::parser parser;
    simdjson::ondemand::document_stream stream;
    auto error = parser.iterate_many(json).get(stream);
-    if( error ) { /* do something */ }
+    if (error) { /* do something */ }
    auto i = stream.begin();
 	 size_t count{0};
    for(; i != stream.end(); ++i) {
        auto doc = *i;
-        if(!i.error()) {
+        if (!i.error()) {
          std::cout << "got full document at " << i.current_index() << std::endl;
          std::cout << i.source() << std::endl;
          count++;
@@ -234,7 +237,7 @@ Consider the following example where a truncated document (`{"key":"intentionall
    simdjson::ondemand::parser parser;
    simdjson::ondemand::document_stream stream;
    auto error = parser.iterate_many(json,json.size()).get(stream);
-    if(error) { std::cerr << error << std::endl; return; }
+    if (error) { std::cerr << error << std::endl; return; }
    for(auto i = stream.begin(); i != stream.end(); ++i) {
       std::cout << i.source() << std::endl;
    }
@@ -266,7 +269,7 @@ Example:
    // we pass 'true' to the allow_comma parameter, the batch size will be set to at least
    // the document size.
    auto error = parser.iterate_many(json, 32, true).get(doc_stream);
-    if(error) { std::cerr << error << std::endl; return; }
+    if (error) { std::cerr << error << std::endl; return; }
    for (auto doc : doc_stream) {
        std::cout << doc.type() << std::endl;
    }
@@ -679,11 +679,11 @@ in production systems:
    ondemand::object c1 = parent["child1"];
    // c1 owns the focus
    //
-    if(std::string_view(c1["name"]) != "John") { ... }
+    if (std::string_view(c1["name"]) != "John") { ... }
    // c2 attempts to grab the focus from parent but fails
    ondemand::object c2 = parent["child2"];
    // c2 is now in an unsafe state and the following line would be unsafe
-    // if(std::string_view(c2["name"]) != "Daniel") { return false; }
+    // if (std::string_view(c2["name"]) != "Daniel") { return false; }
 ```

    A correct usage is given by the following example:
@@ -697,7 +697,7 @@ in production systems:
    {
      ondemand::object c1 = parent["child1"];
      // c1 grabbed the focus from parent
-      if(std::string_view(c1["name"]) != "John") { return false; }
+      if (std::string_view(c1["name"]) != "John") { return false; }
    }
    // c1 went out of scope, so its destructor was called and the focus
    // was handed back to parent.
@@ -705,7 +705,7 @@ in production systems:
      ondemand::object c2 = parent["child2"];
      // c2 grabbed the focus from parent
      // the following is safe:
-      if(std::string_view(c2["name"]) != "Daniel") { return false; }
+      if (std::string_view(c2["name"]) != "Daniel") { return false; }
    }
 ```

@@ -184,12 +184,12 @@ Let us illustrate the idea with code:
    simdjson::dom::parser parser;
    simdjson::dom::document_stream stream;
    auto error = parser.parse_many(json).get(stream);
-    if( error ) { /* do something */ }
+    if (error) { /* do something */ }
    auto i = stream.begin();
    size_t count{0};
    for(; i != stream.end(); ++i) {
        auto doc = *i;
-        if(!doc.error()) {
+        if (!doc.error()) {
          std::cout << "got full document at " << i.current_index() << std::endl;
          std::cout << i.source() << std::endl;
          count++;
@@ -225,7 +225,7 @@ Consider the following example where a truncated document (`{"key":"intentionall
    simdjson::dom::parser parser;
    simdjson::dom::document_stream stream;
    auto error = parser.parse_many(json,json.size()).get(stream);
-    if(error) { std::cerr << error << std::endl; return; }
+    if (error) { std::cerr << error << std::endl; return; }
    for(auto doc : stream) {
       std::cout << doc << std::endl;
    }
@@ -0,0 +1,277 @@
+number of iterations 3000 
+
+| Original (jsonexamples/twitter-padded-numbers.json) 773
+|- 18-digit numbers (jsonexamples/twitter18.json) 757
+|- 8-digit numbers (jsonexamples/twitter8.json) 766
+|- 1-digit numbers (jsonexamples/twitter1.json) 544
+|  |- strings -> no backslashes (jsonexamples/twitter1-nobackslash.json) 499
+|     |- strings -> no utf8 (jsonexamples/twitter1-nobackslash-noutf8.json) 489
+|        |- strings -> empty strings (jsonexamples/twitter1-emptystring.json) 351
+|           |- no number/bool/null -> all strings (jsonexamples/twitter-allstrings.json) 216
+|              |- no objects -> all arrays (jsonexamples/twitter-allstrings-arrays.json) 427
+|                 |- arrays -> no nested arrays (jsonexamples/twitter-allstrings-flatarray.json) 3
+
+jsonexamples/twitter-padded-numbers.json
+========================================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8      2289 ( 22.2 %) - escape       604 (  5.9 %) - 0 structurals      1270 ( 12.3 %) - 1+ structurals      9037 ( 87.7 %) - 8+ structurals      2576 ( 25.0 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8      1096 ( 10.6 %) - escape       646 (  6.3 %) - 0 structurals       932 (  9.0 %) - 1+ structurals       932 (  9.0 %) - 8+ structurals      2843 ( 27.6 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  20.7212 ns per block ( 97.66%) -   0.3238 ns per byte -   3.8647 ns per structural -   3.0885 GB/s
+|    Cycles       :  64.1524 per block    ( 97.86%) -   1.0024 per byte    -  11.9652 per structural    -    3.096 GHz est. frequency
+|    Instructions : 200.2565 per block    (100.00%) -   3.1291 per byte    -  37.3502 per structural    -    3.122 per cycle
+|    Misses       :     873 branch misses ( 93.08%) - 0 cache misses (  0.00%) - 28032.00 cache references
+|- Stage 1
+|    Speed        :   6.0073 ns per block ( 28.31%) -   0.0939 ns per byte -   1.1204 ns per structural -  10.6534 GB/s
+|    Cycles       :  18.6130 per block    ( 28.39%) -   0.2908 per byte    -   3.4715 per structural    -    3.098 GHz est. frequency
+|    Instructions :  61.8107 per block    ( 30.87%) -   0.9658 per byte    -  11.5284 per structural    -    3.321 per cycle
+|    Misses       :      93 branch misses (  9.92%) - 0 cache misses (  0.00%) - 12229.00 cache references
+|- Stage 2
+|    Speed        :  14.6555 ns per block ( 69.07%) -   0.2290 ns per byte -   2.7334 ns per structural -   4.3668 GB/s
+|    Cycles       :  45.3587 per block    ( 69.19%) -   0.7088 per byte    -   8.4599 per structural    -    3.095 GHz est. frequency
+|    Instructions : 138.4458 per block    ( 69.13%) -   2.1633 per byte    -  25.8217 per structural    -    3.052 per cycle
+|    Misses       :     773 branch misses ( 82.42%) - 0 cache misses (  0.00%) - 15925.00 cache references
+
+4682.2 documents parsed per second (best)
+
+jsonexamples/twitter18.json
+===========================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8      2289 ( 22.2 %) - escape       604 (  5.9 %) - 0 structurals      1270 ( 12.3 %) - 1+ structurals      9037 ( 87.7 %) - 8+ structurals      2585 ( 25.1 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8      1096 ( 10.6 %) - escape       646 (  6.3 %) - 0 structurals       932 (  9.0 %) - 1+ structurals       932 (  9.0 %) - 8+ structurals      2847 ( 27.6 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  21.1739 ns per block ( 93.63%) -   0.3309 ns per byte -   3.9492 ns per structural -   3.0225 GB/s
+|    Cycles       :  65.5570 per block    ( 97.40%) -   1.0244 per byte    -  12.2271 per structural    -    3.096 GHz est. frequency
+|    Instructions : 205.6160 per block    (100.00%) -   3.2129 per byte    -  38.3498 per structural    -    3.136 per cycle
+|    Misses       :     856 branch misses ( 97.20%) - 0 cache misses (  0.00%) - 28297.00 cache references
+|- Stage 1
+|    Speed        :   5.9968 ns per block ( 26.52%) -   0.0937 ns per byte -   1.1185 ns per structural -  10.6720 GB/s
+|    Cycles       :  18.5808 per block    ( 27.61%) -   0.2903 per byte    -   3.4655 per structural    -    3.098 GHz est. frequency
+|    Instructions :  61.8107 per block    ( 30.06%) -   0.9658 per byte    -  11.5284 per structural    -    3.327 per cycle
+|    Misses       :     103 branch misses ( 11.70%) - 0 cache misses (  0.00%) - 12271.00 cache references
+|- Stage 2
+|    Speed        :  15.0964 ns per block ( 66.75%) -   0.2359 ns per byte -   2.8157 ns per structural -   4.2393 GB/s
+|    Cycles       :  46.7271 per block    ( 69.43%) -   0.7301 per byte    -   8.7151 per structural    -    3.095 GHz est. frequency
+|    Instructions : 143.8053 per block    ( 69.94%) -   2.2470 per byte    -  26.8213 per structural    -    3.078 per cycle
+|    Misses       :     757 branch misses ( 85.96%) - 0 cache misses (  0.00%) - 16107.00 cache references
+
+4582.1 documents parsed per second (best)
+
+jsonexamples/twitter8.json
+==========================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8      2289 ( 22.2 %) - escape       604 (  5.9 %) - 0 structurals      1270 ( 12.3 %) - 1+ structurals      9037 ( 87.7 %) - 8+ structurals      2576 ( 25.0 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8      1096 ( 10.6 %) - escape       646 (  6.3 %) - 0 structurals       932 (  9.0 %) - 1+ structurals       932 (  9.0 %) - 8+ structurals      2843 ( 27.6 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  20.6976 ns per block ( 97.48%) -   0.3234 ns per byte -   3.8603 ns per structural -   3.0920 GB/s
+|    Cycles       :  64.0733 per block    ( 97.72%) -   1.0012 per byte    -  11.9504 per structural    -    3.096 GHz est. frequency
+|    Instructions : 200.2565 per block    (100.00%) -   3.1291 per byte    -  37.3502 per structural    -    3.125 per cycle
+|    Misses       :     860 branch misses ( 91.48%) - 0 cache misses (  0.00%) - 28700.00 cache references
+|- Stage 1
+|    Speed        :   6.0103 ns per block ( 28.31%) -   0.0939 ns per byte -   1.1210 ns per structural -  10.6481 GB/s
+|    Cycles       :  18.6231 per block    ( 28.40%) -   0.2910 per byte    -   3.4734 per structural    -    3.099 GHz est. frequency
+|    Instructions :  61.8107 per block    ( 30.87%) -   0.9658 per byte    -  11.5284 per structural    -    3.319 per cycle
+|    Misses       :     102 branch misses ( 10.85%) - 0 cache misses (  0.00%) - 11933.00 cache references
+|- Stage 2
+|    Speed        :  14.6407 ns per block ( 68.95%) -   0.2288 ns per byte -   2.7307 ns per structural -   4.3712 GB/s
+|    Cycles       :  45.3059 per block    ( 69.10%) -   0.7079 per byte    -   8.4501 per structural    -    3.095 GHz est. frequency
+|    Instructions : 138.4458 per block    ( 69.13%) -   2.1633 per byte    -  25.8217 per structural    -    3.056 per cycle
+|    Misses       :     766 branch misses ( 81.48%) - 0 cache misses (  0.00%) - 16566.00 cache references
+
+4687.6 documents parsed per second (best)
+
+jsonexamples/twitter1.json
+==========================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8      2289 ( 22.2 %) - escape       604 (  5.9 %) - 0 structurals      1270 ( 12.3 %) - 1+ structurals      9037 ( 87.7 %) - 8+ structurals      2584 ( 25.1 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8      1096 ( 10.6 %) - escape       646 (  6.3 %) - 0 structurals       932 (  9.0 %) - 1+ structurals       932 (  9.0 %) - 8+ structurals      2859 ( 27.7 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  20.1200 ns per block ( 93.76%) -   0.3144 ns per byte -   3.7526 ns per structural -   3.1808 GB/s
+|    Cycles       :  62.2865 per block    ( 97.52%) -   0.9733 per byte    -  11.6172 per structural    -    3.096 GHz est. frequency
+|    Instructions : 194.5666 per block    (100.00%) -   3.0402 per byte    -  36.2889 per structural    -    3.124 per cycle
+|    Misses       :     673 branch misses ( 97.88%) - 0 cache misses (  0.00%) - 28320.00 cache references
+|- Stage 1
+|    Speed        :   6.0046 ns per block ( 27.98%) -   0.0938 ns per byte -   1.1199 ns per structural -  10.6582 GB/s
+|    Cycles       :  18.6140 per block    ( 29.14%) -   0.2909 per byte    -   3.4717 per structural    -    3.100 GHz est. frequency
+|    Instructions :  61.8107 per block    ( 31.77%) -   0.9658 per byte    -  11.5284 per structural    -    3.321 per cycle
+|    Misses       :     113 branch misses ( 16.43%) - 0 cache misses (  0.00%) - 12175.00 cache references
+|- Stage 2
+|    Speed        :  14.0716 ns per block ( 65.57%) -   0.2199 ns per byte -   2.6245 ns per structural -   4.5480 GB/s
+|    Cycles       :  43.5464 per block    ( 68.18%) -   0.6804 per byte    -   8.1219 per structural    -    3.095 GHz est. frequency
+|    Instructions : 132.7559 per block    ( 68.23%) -   2.0744 per byte    -  24.7605 per structural    -    3.049 per cycle
+|    Misses       :     544 branch misses ( 79.11%) - 0 cache misses (  0.00%) - 16212.00 cache references
+
+4822.1 documents parsed per second (best)
+
+jsonexamples/twitter1-nobackslash.json
+======================================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8      2289 ( 22.2 %) - escape         0 (  0.0 %) - 0 structurals      1270 ( 12.3 %) - 1+ structurals      9037 ( 87.7 %) - 8+ structurals      2584 ( 25.1 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8      1096 ( 10.6 %) - escape         0 (  0.0 %) - 0 structurals       932 (  9.0 %) - 1+ structurals       932 (  9.0 %) - 8+ structurals      2859 ( 27.7 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  19.3473 ns per block ( 94.75%) -   0.3023 ns per byte -   3.6085 ns per structural -   3.3078 GB/s
+|    Cycles       :  59.8930 per block    ( 97.21%) -   0.9359 per byte    -  11.1707 per structural    -    3.096 GHz est. frequency
+|    Instructions : 191.2882 per block    (100.00%) -   2.9890 per byte    -  35.6774 per structural    -    3.194 per cycle
+|    Misses       :     624 branch misses ( 99.02%) - 2 cache misses ( 30.96%) - 28632.00 cache references
+|- Stage 1
+|    Speed        :   5.9519 ns per block ( 29.15%) -   0.0930 ns per byte -   1.1101 ns per structural -  10.7526 GB/s
+|    Cycles       :  18.4460 per block    ( 29.94%) -   0.2882 per byte    -   3.4404 per structural    -    3.099 GHz est. frequency
+|    Instructions :  61.1661 per block    ( 31.98%) -   0.9558 per byte    -  11.4082 per structural    -    3.316 per cycle
+|    Misses       :      92 branch misses ( 14.60%) - 1 cache misses ( 15.48%) - 12277.00 cache references
+|- Stage 2
+|    Speed        :  13.3283 ns per block ( 65.28%) -   0.2083 ns per byte -   2.4859 ns per structural -   4.8016 GB/s
+|    Cycles       :  41.2453 per block    ( 66.95%) -   0.6445 per byte    -   7.6927 per structural    -    3.095 GHz est. frequency
+|    Instructions : 130.1221 per block    ( 68.02%) -   2.0332 per byte    -  24.2693 per structural    -    3.155 per cycle
+|    Misses       :     499 branch misses ( 79.19%) - 1 cache misses ( 15.48%) - 16358.00 cache references
+
+5014.7 documents parsed per second (best)
+
+jsonexamples/twitter1-nobackslash-noutf8.json
+=============================================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals      1270 ( 12.3 %) - 1+ structurals      9037 ( 87.7 %) - 8+ structurals      2584 ( 25.1 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals       932 (  9.0 %) - 1+ structurals       932 (  9.0 %) - 8+ structurals      2859 ( 27.7 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  18.9506 ns per block ( 97.78%) -   0.2961 ns per byte -   3.5345 ns per structural -   3.3771 GB/s
+|    Cycles       :  58.6756 per block    ( 97.97%) -   0.9168 per byte    -  10.9437 per structural    -    3.096 GHz est. frequency
+|    Instructions : 186.6244 per block    (100.00%) -   2.9161 per byte    -  34.8076 per structural    -    3.181 per cycle
+|    Misses       :     634 branch misses ( 99.08%) - 0 cache misses (  0.00%) - 28596.00 cache references
+|- Stage 1
+|    Speed        :   5.4613 ns per block ( 28.18%) -   0.0853 ns per byte -   1.0186 ns per structural -  11.7184 GB/s
+|    Cycles       :  16.9236 per block    ( 28.26%) -   0.2644 per byte    -   3.1565 per structural    -    3.099 GHz est. frequency
+|    Instructions :  56.5024 per block    ( 30.28%) -   0.8829 per byte    -  10.5383 per structural    -    3.339 per cycle
+|    Misses       :     130 branch misses ( 20.32%) - 0 cache misses (  0.00%) - 12365.00 cache references
+|- Stage 2
+|    Speed        :  13.4108 ns per block ( 69.19%) -   0.2096 ns per byte -   2.5013 ns per structural -   4.7721 GB/s
+|    Cycles       :  41.5073 per block    ( 69.31%) -   0.6486 per byte    -   7.7416 per structural    -    3.095 GHz est. frequency
+|    Instructions : 130.1221 per block    ( 69.72%) -   2.0332 per byte    -  24.2693 per structural    -    3.135 per cycle
+|    Misses       :     489 branch misses ( 76.42%) - 0 cache misses (  0.00%) - 16279.00 cache references
+
+5119.7 documents parsed per second (best)
+
+jsonexamples/twitter1-emptystring.json
+======================================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals      1269 ( 12.3 %) - 1+ structurals      9038 ( 87.7 %) - 8+ structurals      2856 ( 27.7 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals       928 (  9.0 %) - 1+ structurals       928 (  9.0 %) - 8+ structurals      3123 ( 30.3 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  18.0587 ns per block ( 93.58%) -   0.2822 ns per byte -   3.3682 ns per structural -   3.5439 GB/s
+|    Cycles       :  55.9116 per block    ( 97.35%) -   0.8736 per byte    -  10.4282 per structural    -    3.096 GHz est. frequency
+|    Instructions : 183.3181 per block    (100.00%) -   2.8644 per byte    -  34.1909 per structural    -    3.279 per cycle
+|    Misses       :     473 branch misses (101.19%) - 1 cache misses ( 15.11%) - 18833.00 cache references
+|- Stage 1
+|    Speed        :   5.4026 ns per block ( 28.00%) -   0.0844 ns per byte -   1.0077 ns per structural -  11.8457 GB/s
+|    Cycles       :  16.7461 per block    ( 29.16%) -   0.2617 per byte    -   3.1233 per structural    -    3.100 GHz est. frequency
+|    Instructions :  56.5028 per block    ( 30.82%) -   0.8829 per byte    -  10.5384 per structural    -    3.374 per cycle
+|    Misses       :     112 branch misses ( 23.96%) - 0 cache misses (  0.00%) - 10717.00 cache references
+|- Stage 2
+|    Speed        :  12.6022 ns per block ( 65.30%) -   0.1969 ns per byte -   2.3505 ns per structural -   5.0783 GB/s
+|    Cycles       :  39.0031 per block    ( 67.91%) -   0.6094 per byte    -   7.2745 per structural    -    3.095 GHz est. frequency
+|    Instructions : 126.8154 per block    ( 69.18%) -   1.9816 per byte    -  23.6525 per structural    -    3.251 per cycle
+|    Misses       :     351 branch misses ( 75.09%) - 1 cache misses ( 15.11%) - 8120.00 cache references
+
+5372.6 documents parsed per second (best)
+
+jsonexamples/twitter-allstrings.json
+====================================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals      1269 ( 12.3 %) - 1+ structurals      9038 ( 87.7 %) - 8+ structurals      2853 ( 27.7 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals       928 (  9.0 %) - 1+ structurals       928 (  9.0 %) - 8+ structurals      3107 ( 30.1 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  19.6482 ns per block ( 97.20%) -   0.3070 ns per byte -   3.6646 ns per structural -   3.2572 GB/s
+|    Cycles       :  60.8255 per block    ( 97.62%) -   0.9504 per byte    -  11.3446 per structural    -    3.096 GHz est. frequency
+|    Instructions : 183.8463 per block    (100.00%) -   2.8727 per byte    -  34.2895 per structural    -    3.023 per cycle
+|    Misses       :     300 branch misses ( 96.97%) - 0 cache misses (  0.00%) - 18361.00 cache references
+|- Stage 1
+|    Speed        :   5.4053 ns per block ( 26.74%) -   0.0845 ns per byte -   1.0081 ns per structural -  11.8399 GB/s
+|    Cycles       :  16.7545 per block    ( 26.89%) -   0.2618 per byte    -   3.1249 per structural    -    3.100 GHz est. frequency
+|    Instructions :  56.5028 per block    ( 30.73%) -   0.8829 per byte    -  10.5384 per structural    -    3.372 per cycle
+|    Misses       :      98 branch misses ( 31.68%) - 1 cache misses ( 18.41%) - 10786.00 cache references
+|- Stage 2
+|    Speed        :  14.0367 ns per block ( 69.44%) -   0.2193 ns per byte -   2.6180 ns per structural -   4.5593 GB/s
+|    Cycles       :  43.4371 per block    ( 69.71%) -   0.6787 per byte    -   8.1015 per structural    -    3.095 GHz est. frequency
+|    Instructions : 127.3447 per block    ( 69.27%) -   1.9898 per byte    -  23.7513 per structural    -    2.932 per cycle
+|    Misses       :     216 branch misses ( 69.82%) - 0 cache misses (  0.00%) - 7835.00 cache references
+
+4937.9 documents parsed per second (best)
+
+jsonexamples/twitter-allstrings-arrays.json
+===========================================
+    10306 blocks -     659626 bytes - 55262 structurals (  8.4 %)
+special blocks with: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals      1269 ( 12.3 %) - 1+ structurals      9038 ( 87.7 %) - 8+ structurals      2853 ( 27.7 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals       928 (  9.0 %) - 1+ structurals       928 (  9.0 %) - 8+ structurals      3107 ( 30.1 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  23.6768 ns per block ( 96.29%) -   0.3700 ns per byte -   4.4160 ns per structural -   2.7030 GB/s
+|    Cycles       :  72.7530 per block    ( 95.79%) -   1.1368 per byte    -  13.5693 per structural    -    3.073 GHz est. frequency
+|    Instructions : 225.0789 per block    (100.00%) -   3.5170 per byte    -  41.9798 per structural    -    3.094 per cycle
+|    Misses       :     547 branch misses (104.38%) - 0 cache misses (  0.00%) - 17259.00 cache references
+|- Stage 1
+|    Speed        :   5.4063 ns per block ( 21.99%) -   0.0845 ns per byte -   1.0083 ns per structural -  11.8376 GB/s
+|    Cycles       :  16.7598 per block    ( 22.07%) -   0.2619 per byte    -   3.1259 per structural    -    3.100 GHz est. frequency
+|    Instructions :  56.5028 per block    ( 25.10%) -   0.8829 per byte    -  10.5384 per structural    -    3.371 per cycle
+|    Misses       :     101 branch misses ( 19.27%) - 0 cache misses (  0.00%) - 10570.00 cache references
+|- Stage 2
+|    Speed        :  18.2062 ns per block ( 74.04%) -   0.2845 ns per byte -   3.3957 ns per structural -   3.5152 GB/s
+|    Cycles       :  55.7937 per block    ( 73.46%) -   0.8718 per byte    -  10.4062 per structural    -    3.065 GHz est. frequency
+|    Instructions : 168.5761 per block    ( 74.90%) -   2.6341 per byte    -  31.4414 per structural    -    3.021 per cycle
+|    Misses       :     427 branch misses ( 81.48%) - 0 cache misses (  0.00%) - 6797.00 cache references
+
+4097.7 documents parsed per second (best)
+
+jsonexamples/twitter-allstrings-flatarray.json
+==============================================
+    10306 blocks -     659626 bytes - 49890 structurals (  7.6 %)
+special blocks with: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals      1279 ( 12.4 %) - 1+ structurals      9028 ( 87.6 %) - 8+ structurals      2100 ( 20.4 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals       946 (  9.2 %) - 1+ structurals       946 (  9.2 %) - 8+ structurals      2668 ( 25.9 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  20.5314 ns per block ( 98.77%) -   0.3208 ns per byte -   4.2417 ns per structural -   3.1171 GB/s
+|    Cycles       :  63.5535 per block    ( 98.95%) -   0.9931 per byte    -  13.1298 per structural    -    3.095 GHz est. frequency
+|    Instructions : 208.9967 per block    (100.00%) -   3.2657 per byte    -  43.1776 per structural    -    3.289 per cycle
+|    Misses       :      94 branch misses ( 99.52%) - 0 cache misses (  0.00%) - 15420.00 cache references
+|- Stage 1
+|    Speed        :   5.3985 ns per block ( 25.97%) -   0.0844 ns per byte -   1.1153 ns per structural -  11.8548 GB/s
+|    Cycles       :  16.7318 per block    ( 26.05%) -   0.2614 per byte    -   3.4567 per structural    -    3.099 GHz est. frequency
+|    Instructions :  56.4917 per block    ( 27.03%) -   0.8827 per byte    -  11.6709 per structural    -    3.376 per cycle
+|    Misses       :      87 branch misses ( 92.11%) - 0 cache misses (  0.00%) - 9889.00 cache references
+|- Stage 2
+|    Speed        :  15.0691 ns per block ( 72.49%) -   0.2355 ns per byte -   3.1132 ns per structural -   4.2470 GB/s
+|    Cycles       :  46.6318 per block    ( 72.61%) -   0.7286 per byte    -   9.6339 per structural    -    3.095 GHz est. frequency
+|    Instructions : 152.5050 per block    ( 72.97%) -   2.3830 per byte    -  31.5067 per structural    -    3.270 per cycle
+|    Misses       :       3 branch misses (  3.18%) - 0 cache misses (  0.00%) - 5610.00 cache references
+
+4725.5 documents parsed per second (best)
+
+jsonexamples/twitter-allstrings-flatobjects.json
+================================================
+    10306 blocks -     659626 bytes - 56091 structurals (  8.5 %)
+special blocks with: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals      1269 ( 12.3 %) - 1+ structurals      9038 ( 87.7 %) - 8+ structurals      2973 ( 28.8 %) - 16+ structurals         0 (  0.0 %)
+special block flips: utf8         0 (  0.0 %) - escape         0 (  0.0 %) - 0 structurals       928 (  9.0 %) - 1+ structurals       928 (  9.0 %) - 8+ structurals      3093 ( 30.0 %) - 16+ structurals         0 (  0.0 %)
+
+All Stages (excluding allocation)
+|    Speed        :  19.4292 ns per block ( 97.71%) -   0.3036 ns per byte -   3.5702 ns per structural -   3.2939 GB/s
+|    Cycles       :  60.1577 per block    ( 97.90%) -   0.9400 per byte    -  11.0543 per structural    -    3.096 GHz est. frequency
+|    Instructions : 181.6747 per block    (100.00%) -   2.8388 per byte    -  33.3836 per structural    -    3.020 per cycle
+|    Misses       :     508 branch misses (100.57%) - 0 cache misses (  0.00%) - 17421.00 cache references
+|- Stage 1
+|    Speed        :   5.4138 ns per block ( 27.22%) -   0.0846 ns per byte -   0.9948 ns per structural -  11.8213 GB/s
+|    Cycles       :  16.7833 per block    ( 27.31%) -   0.2622 per byte    -   3.0840 per structural    -    3.100 GHz est. frequency
+|    Instructions :  56.5028 per block    ( 31.10%) -   0.8829 per byte    -  10.3827 per structural    -    3.367 per cycle
+|    Misses       :      99 branch misses ( 19.60%) - 0 cache misses (  0.00%) - 10517.00 cache references
+|- Stage 2
+|    Speed        :  13.9445 ns per block ( 70.12%) -   0.2179 ns per byte -   2.5624 ns per structural -   4.5895 GB/s
+|    Cycles       :  43.1562 per block    ( 70.23%) -   0.6743 per byte    -   7.9302 per structural    -    3.095 GHz est. frequency
+|    Instructions : 125.1719 per block    ( 68.90%) -   1.9559 per byte    -  23.0010 per structural    -    2.900 per cycle
+|    Misses       :     402 branch misses ( 79.59%) - 0 cache misses (  0.00%) - 6973.00 cache references
+
+4993.6 documents parsed per second (best)
+
@@ -34,8 +34,24 @@
    }
 */

+#include "simdjson/common_defs.h"
+
+// This provides the public API for simdjson.
+// DOM and ondemand are amalgamated separately, in simdjson.h
 #include "simdjson/simdjson_version.h"
+
+#include "simdjson/base.h"
+
+#include "simdjson/error.h"
+#include "simdjson/error-inl.h"
+#include "simdjson/implementation.h"
+#include "simdjson/minify.h"
+#include "simdjson/padded_string.h"
+#include "simdjson/padded_string-inl.h"
+#include "simdjson/padded_string_view.h"
+#include "simdjson/padded_string_view-inl.h"
+
 #include "simdjson/dom.h"
-#include "simdjson/builtin.h"
+#include "simdjson/ondemand.h"

 #endif // SIMDJSON_H
@@ -1,34 +1,8 @@
 #ifndef SIMDJSON_ARM64_H
 #define SIMDJSON_ARM64_H

-#include "simdjson/implementation-base.h"
-
-#if SIMDJSON_IMPLEMENTATION_ARM64
-
-namespace simdjson {
-/**
- * Implementation for NEON (ARMv8).
- */
-namespace arm64 {
-} // namespace arm64
-} // namespace simdjson
-
-#include "simdjson/arm64/implementation.h"
-
 #include "simdjson/arm64/begin.h"
-
-// Declarations
-#include "simdjson/generic/dom_parser_implementation.h"
-#include "simdjson/arm64/intrinsics.h"
-#include "simdjson/arm64/bitmanipulation.h"
-#include "simdjson/arm64/bitmask.h"
-#include "simdjson/arm64/simd.h"
-#include "simdjson/generic/jsoncharutils.h"
-#include "simdjson/generic/atomparsing.h"
-#include "simdjson/arm64/stringparsing.h"
-#include "simdjson/arm64/numberparsing.h"
+#include "simdjson/generic/amalgamated.h"
 #include "simdjson/arm64/end.h"

-#endif // SIMDJSON_IMPLEMENTATION_ARM64
-
-#endif // SIMDJSON_ARM64_H
+#endif // SIMDJSON_ARM64_H
@@ -0,0 +1,26 @@
+#ifndef SIMDJSON_ARM64_BASE_H
+#define SIMDJSON_ARM64_BASE_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+namespace simdjson {
+/**
+ * Implementation for NEON (ARMv8).
+ */
+namespace arm64 {
+
+class implementation;
+
+namespace {
+namespace simd {
+template <typename T> struct simd8;
+template <typename T> struct simd8x64;
+} // namespace simd
+} // unnamed namespace
+
+} // namespace arm64
+} // namespace simdjson
+
+#endif // SIMDJSON_ARM64_BASE_H
@@ -1 +1,10 @@
 #define SIMDJSON_IMPLEMENTATION arm64
+#include "simdjson/arm64/base.h"
+#include "simdjson/arm64/intrinsics.h"
+#include "simdjson/arm64/bitmanipulation.h"
+#include "simdjson/arm64/bitmask.h"
+#include "simdjson/arm64/numberparsing_defs.h"
+#include "simdjson/arm64/simd.h"
+#include "simdjson/arm64/stringparsing_defs.h"
+
+#define SIMDJSON_SKIP_BACKSLASH_SHORT_CIRCUIT 1
@@ -1,8 +1,13 @@
 #ifndef SIMDJSON_ARM64_BITMANIPULATION_H
 #define SIMDJSON_ARM64_BITMANIPULATION_H

+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
+#include "simdjson/arm64/intrinsics.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
+namespace arm64 {
 namespace {

 // We sometimes call trailing_zero on inputs that are zero,
@@ -95,7 +100,7 @@ simdjson_inline bool add_overflow(uint64_t value1, uint64_t value2, uint64_t *re
 }

 } // unnamed namespace
-} // namespace SIMDJSON_IMPLEMENTATION
+} // namespace arm64
 } // namespace simdjson

 #endif // SIMDJSON_ARM64_BITMANIPULATION_H
@@ -1,8 +1,12 @@
 #ifndef SIMDJSON_ARM64_BITMASK_H
 #define SIMDJSON_ARM64_BITMASK_H

+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
+namespace arm64 {
 namespace {

 //
@@ -1 +1,6 @@
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+#undef SIMDJSON_SKIP_BACKSLASH_SHORT_CIRCUIT
 #undef SIMDJSON_IMPLEMENTATION
@@ -1,17 +1,15 @@
 #ifndef SIMDJSON_ARM64_IMPLEMENTATION_H
 #define SIMDJSON_ARM64_IMPLEMENTATION_H

+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
 #include "simdjson/base.h"
-#include "simdjson/internal/isadetection.h"
+#include "simdjson/implementation.h"
+#include "simdjson/internal/instruction_set.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
 namespace arm64 {

-namespace {
-using namespace simdjson;
-using namespace simdjson::dom;
-}
-
 /**
 * @private
 */
@@ -1,6 +1,10 @@
 #ifndef SIMDJSON_ARM64_INTRINSICS_H
 #define SIMDJSON_ARM64_INTRINSICS_H

+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 // This should be the correct header whether
 // you use visual studio or other compilers.
 #include <arm_neon.h>
@@ -1,27 +0,0 @@
-#ifndef SIMDJSON_ARM64_NUMBERPARSING_H
-#define SIMDJSON_ARM64_NUMBERPARSING_H
-
-namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
-namespace numberparsing {
-
-// we don't have SSE, so let us use a scalar function
-// credit: https://johnnylee-sde.github.io/Fast-numeric-string-to-int/
-/** @private */
-static simdjson_inline uint32_t parse_eight_digits_unrolled(const uint8_t *chars) {
-  uint64_t val;
-  std::memcpy(&val, chars, sizeof(uint64_t));
-  val = (val & 0x0F0F0F0F0F0F0F0F) * 2561 >> 8;
-  val = (val & 0x00FF00FF00FF00FF) * 6553601 >> 16;
-  return uint32_t((val & 0x0000FFFF0000FFFF) * 42949672960001 >> 32);
-}
-
-} // namespace numberparsing
-} // namespace SIMDJSON_IMPLEMENTATION
-} // namespace simdjson
-
-#define SIMDJSON_SWAR_NUMBER_PARSING 1
-
-#include "simdjson/generic/numberparsing.h"
-
-#endif // SIMDJSON_ARM64_NUMBERPARSING_H
@@ -0,0 +1,56 @@
+#ifndef SIMDJSON_ARM64_NUMBERPARSING_DEFS_H
+#define SIMDJSON_ARM64_NUMBERPARSING_DEFS_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
+#include "simdjson/arm64/intrinsics.h"
+#include "simdjson/internal/numberparsing_tables.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+#include <cstring>
+
+#if _M_ARM64
+// __umulh requires intrin.h
+#include <intrin.h>
+#endif // _M_ARM64
+
+namespace simdjson {
+namespace arm64 {
+namespace numberparsing {
+
+// we don't have SSE, so let us use a scalar function
+// credit: https://johnnylee-sde.github.io/Fast-numeric-string-to-int/
+/** @private */
+static simdjson_inline uint32_t parse_eight_digits_unrolled(const uint8_t *chars) {
+  uint64_t val;
+  std::memcpy(&val, chars, sizeof(uint64_t));
+  val = (val & 0x0F0F0F0F0F0F0F0F) * 2561 >> 8;
+  val = (val & 0x00FF00FF00FF00FF) * 6553601 >> 16;
+  return uint32_t((val & 0x0000FFFF0000FFFF) * 42949672960001 >> 32);
+}
+
+simdjson_inline internal::value128 full_multiplication(uint64_t value1, uint64_t value2) {
+  internal::value128 answer;
+#if SIMDJSON_REGULAR_VISUAL_STUDIO || SIMDJSON_IS_32BITS
+#ifdef _M_ARM64
+  // ARM64 has native support for 64-bit multiplications, no need to emultate
+  answer.high = __umulh(value1, value2);
+  answer.low = value1 * value2;
+#else
+  answer.low = _umul128(value1, value2, &answer.high); // _umul128 not available on ARM64
+#endif // _M_ARM64
+#else // SIMDJSON_REGULAR_VISUAL_STUDIO || SIMDJSON_IS_32BITS
+  __uint128_t r = (static_cast<__uint128_t>(value1)) * value2;
+  answer.low = uint64_t(r);
+  answer.high = uint64_t(r >> 64);
+#endif
+  return answer;
+}
+
+} // namespace numberparsing
+} // namespace arm64
+} // namespace simdjson
+
+#define SIMDJSON_SWAR_NUMBER_PARSING 1
+
+#endif // SIMDJSON_ARM64_NUMBERPARSING_DEFS_H
@@ -0,0 +1,8 @@
+#ifndef SIMDJSON_ARM64_ONDEMAND_H
+#define SIMDJSON_ARM64_ONDEMAND_H
+
+#include "simdjson/arm64/begin.h"
+#include "simdjson/generic/ondemand/amalgamated.h"
+#include "simdjson/arm64/end.h"
+
+#endif // SIMDJSON_ARM64_ONDEMAND_H
@@ -1,14 +1,14 @@
 #ifndef SIMDJSON_ARM64_SIMD_H
 #define SIMDJSON_ARM64_SIMD_H

-#include "simdjson/base.h"
-#include "simdjson/internal/simdprune_tables.h"
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
 #include "simdjson/arm64/bitmanipulation.h"
-#include <type_traits>
-
+#include "simdjson/internal/simdprune_tables.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
+namespace arm64 {
 namespace {
 namespace simd {

@@ -527,7 +527,7 @@ simdjson_inline int8x16_t make_int8x16_t(int8_t x1,  int8_t x2,  int8_t x3,  int

 } // namespace simd
 } // unnamed namespace
-} // namespace SIMDJSON_IMPLEMENTATION
+} // namespace arm64
 } // namespace simdjson

 #endif // SIMDJSON_ARM64_SIMD_H
@@ -1,12 +1,14 @@
-#ifndef SIMDJSON_ARM64_STRINGPARSING_H
-#define SIMDJSON_ARM64_STRINGPARSING_H
+#ifndef SIMDJSON_ARM64_STRINGPARSING_DEFS_H
+#define SIMDJSON_ARM64_STRINGPARSING_DEFS_H

-#include "simdjson/base.h"
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/arm64/base.h"
 #include "simdjson/arm64/simd.h"
 #include "simdjson/arm64/bitmanipulation.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
+namespace arm64 {
 namespace {

 using namespace simd;
@@ -45,7 +47,7 @@ simdjson_inline backslash_and_quote backslash_and_quote::copy_and_find(const uin
 }

 } // unnamed namespace
-} // namespace SIMDJSON_IMPLEMENTATION
+} // namespace arm64
 } // namespace simdjson

-#endif // SIMDJSON_ARM64_STRINGPARSING_H
+#endif // SIMDJSON_ARM64_STRINGPARSING_DEFS_H
@@ -1,26 +1,60 @@
+/**
+ * @file Base declarations for all simdjson headers
+ * @private
+ */
 #ifndef SIMDJSON_BASE_H
 #define SIMDJSON_BASE_H

-#include "simdjson/compiler_check.h"
 #include "simdjson/common_defs.h"
+#include "simdjson/compiler_check.h"
+#include "simdjson/error.h"
 #include "simdjson/portability.h"

-SIMDJSON_PUSH_DISABLE_WARNINGS
-SIMDJSON_DISABLE_UNDESIRED_WARNINGS
+/**
+ * @brief The top level simdjson namespace, containing everything the library provides.
+ */
+namespace simdjson {

-// Public API
-#include "simdjson/simdjson_version.h"
-#include "simdjson/error.h"
-#include "simdjson/minify.h"
-#include "simdjson/padded_string.h"
-#include "simdjson/padded_string_view.h"
-#include "simdjson/implementation.h"
+SIMDJSON_PUSH_DISABLE_UNUSED_WARNINGS

-// Inline functions
-#include "simdjson/error-inl.h"
-#include "simdjson/padded_string-inl.h"
-#include "simdjson/padded_string_view-inl.h"
+/** The maximum document size supported by simdjson. */
+constexpr size_t SIMDJSON_MAXSIZE_BYTES = 0xFFFFFFFF;

-SIMDJSON_POP_DISABLE_WARNINGS
+/**
+ * The amount of padding needed in a buffer to parse JSON.
+ *
+ * The input buf should be readable up to buf + SIMDJSON_PADDING
+ * this is a stopgap; there should be a better description of the
+ * main loop and its behavior that abstracts over this
+ * See https://github.com/simdjson/simdjson/issues/174
+ */
+constexpr size_t SIMDJSON_PADDING = 64;

-#endif // SIMDJSON_BASE_H
+/**
+ * By default, simdjson supports this many nested objects and arrays.
+ *
+ * This is the default for parser::max_depth().
+ */
+constexpr size_t DEFAULT_MAX_DEPTH = 1024;
+
+SIMDJSON_POP_DISABLE_UNUSED_WARNINGS
+
+class implementation;
+struct padded_string;
+class padded_string_view;
+enum class stage1_mode;
+
+namespace internal {
+
+template<typename T>
+class atomic_ptr;
+class dom_parser_implementation;
+class escape_json_string;
+class tape_ref;
+struct value128;
+enum class tape_type;
+
+} // namespace internal
+} // namespace simdjson
+
+#endif // SIMDJSON_BASE_H
@@ -1,64 +1,29 @@
 #ifndef SIMDJSON_BUILTIN_H
 #define SIMDJSON_BUILTIN_H

-#include "simdjson/implementations.h"
+#include "simdjson/builtin/base.h"
+#include "simdjson/builtin/implementation.h"

-// Determine the best builtin implementation
-#ifndef SIMDJSON_BUILTIN_IMPLEMENTATION
-#if SIMDJSON_CAN_ALWAYS_RUN_ICELAKE
-#define SIMDJSON_BUILTIN_IMPLEMENTATION icelake
-#elif SIMDJSON_CAN_ALWAYS_RUN_HASWELL
-#define SIMDJSON_BUILTIN_IMPLEMENTATION haswell
-#elif SIMDJSON_CAN_ALWAYS_RUN_WESTMERE
-#define SIMDJSON_BUILTIN_IMPLEMENTATION westmere
-#elif SIMDJSON_CAN_ALWAYS_RUN_ARM64
-#define SIMDJSON_BUILTIN_IMPLEMENTATION arm64
-#elif SIMDJSON_CAN_ALWAYS_RUN_PPC64
-#define SIMDJSON_BUILTIN_IMPLEMENTATION ppc64
-#elif SIMDJSON_CAN_ALWAYS_RUN_FALLBACK
-#define SIMDJSON_BUILTIN_IMPLEMENTATION fallback
+#include "simdjson/generic/dependencies.h"
+
+#define SIMDJSON_CONDITIONAL_INCLUDE
+
+#if SIMDJSON_BUILTIN_IMPLEMENTATION_IS(arm64)
+#include "simdjson/arm64.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(fallback)
+#include "simdjson/fallback.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(haswell)
+#include "simdjson/haswell.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(icelake)
+#include "simdjson/icelake.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(ppc64)
+#include "simdjson/ppc64.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(westmere)
+#include "simdjson/westmere.h"
 #else
-#error "All possible implementations (including fallback) have been disabled! simdjson will not run."
+#error Unknown SIMDJSON_BUILTIN_IMPLEMENTATION
 #endif
-#endif // SIMDJSON_BUILTIN_IMPLEMENTATION

-#define SIMDJSON_IMPLEMENTATION SIMDJSON_BUILTIN_IMPLEMENTATION
-
-// ondemand is only compiled as part of the builtin implementation at present
-
-// Interface declarations
-#include "simdjson/generic/implementation_simdjson_result_base.h"
-#include "simdjson/generic/ondemand.h"
-
-// Inline definitions
-#include "simdjson/generic/implementation_simdjson_result_base-inl.h"
-#include "simdjson/generic/ondemand-inl.h"
-
-#undef SIMDJSON_IMPLEMENTATION
-
-namespace simdjson {
-  /**
-   * Represents the best statically linked simdjson implementation that can be used by the compiling
-   * program.
-   *
-   * Detects what options the program is compiled against, and picks the minimum implementation that
-   * will work on any computer that can run the program. For example, if you compile with g++
-   * -march=westmere, it will pick the westmere implementation. The haswell implementation will
-   * still be available, and can be selected at runtime, but the builtin implementation (and any
-   * code that uses it) will use westmere.
-   */
-  namespace builtin = SIMDJSON_BUILTIN_IMPLEMENTATION;
-  /**
-   * @copydoc simdjson::SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand
-   */
-  namespace ondemand = SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand;
-  /**
-   * Function which returns a pointer to an implementation matching the "builtin" implementation.
-   * The builtin implementation is the best statically linked simdjson implementation that can be used by the compiling
-   * program. If you compile with g++ -march=haswell, this will return the haswell implementation.
-   * It is handy to be able to check what builtin was used: builtin_implementation()->name().
-   */
-  const implementation * builtin_implementation();
-} // namespace simdjson
+#undef SIMDJSON_CONDITIONAL_INCLUDE

 #endif // SIMDJSON_BUILTIN_H
@@ -0,0 +1,37 @@
+#ifndef SIMDJSON_BUILTIN_BASE_H
+#define SIMDJSON_BUILTIN_BASE_H
+
+#include "simdjson/base.h"
+#include "simdjson/implementation_detection.h"
+
+namespace simdjson {
+#if SIMDJSON_BUILTIN_IMPLEMENTATION_IS(arm64)
+  namespace arm64 {}
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(fallback)
+  namespace fallback {}
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(haswell)
+  namespace haswell {}
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(icelake)
+  namespace icelake {}
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(ppc64)
+  namespace ppc64 {}
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(westmere)
+  namespace westmere {}
+#else
+#error Unknown SIMDJSON_BUILTIN_IMPLEMENTATION
+#endif
+
+  /**
+   * Represents the best statically linked simdjson implementation that can be used by the compiling
+   * program.
+   *
+   * Detects what options the program is compiled against, and picks the minimum implementation that
+   * will work on any computer that can run the program. For example, if you compile with g++
+   * -march=westmere, it will pick the westmere implementation. The haswell implementation will
+   * still be available, and can be selected at runtime, but the builtin implementation (and any
+   * code that uses it) will use westmere.
+   */
+  namespace builtin = SIMDJSON_BUILTIN_IMPLEMENTATION;
+} // namespace simdjson
+
+#endif // SIMDJSON_BUILTIN_BASE_H
@@ -0,0 +1,38 @@
+#ifndef SIMDJSON_BUILTIN_IMPLEMENTATION_H
+#define SIMDJSON_BUILTIN_IMPLEMENTATION_H
+
+#include "simdjson/builtin/base.h"
+
+#include "simdjson/generic/dependencies.h"
+
+#define SIMDJSON_CONDITIONAL_INCLUDE
+
+#if SIMDJSON_BUILTIN_IMPLEMENTATION_IS(arm64)
+#include "simdjson/arm64/implementation.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(fallback)
+#include "simdjson/fallback/implementation.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(haswell)
+#include "simdjson/haswell/implementation.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(icelake)
+#include "simdjson/icelake/implementation.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(ppc64)
+#include "simdjson/ppc64/implementation.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(westmere)
+#include "simdjson/westmere/implementation.h"
+#else
+#error Unknown SIMDJSON_BUILTIN_IMPLEMENTATION
+#endif
+
+#undef SIMDJSON_CONDITIONAL_INCLUDE
+
+namespace simdjson {
+  /**
+   * Function which returns a pointer to an implementation matching the "builtin" implementation.
+   * The builtin implementation is the best statically linked simdjson implementation that can be used by the compiling
+   * program. If you compile with g++ -march=haswell, this will return the haswell implementation.
+   * It is handy to be able to check what builtin was used: builtin_implementation()->name().
+   */
+  const implementation * builtin_implementation();
+} // namespace simdjson
+
+#endif // SIMDJSON_BUILTIN_IMPLEMENTATION_H
@@ -0,0 +1,36 @@
+#ifndef SIMDJSON_BUILTIN_ONDEMAND_H
+#define SIMDJSON_BUILTIN_ONDEMAND_H
+
+#include "simdjson/builtin.h"
+#include "simdjson/builtin/base.h"
+
+#include "simdjson/generic/ondemand/dependencies.h"
+
+#define SIMDJSON_CONDITIONAL_INCLUDE
+
+#if SIMDJSON_BUILTIN_IMPLEMENTATION_IS(arm64)
+#include "simdjson/arm64/ondemand.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(fallback)
+#include "simdjson/fallback/ondemand.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(haswell)
+#include "simdjson/haswell/ondemand.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(icelake)
+#include "simdjson/icelake/ondemand.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(ppc64)
+#include "simdjson/ppc64/ondemand.h"
+#elif SIMDJSON_BUILTIN_IMPLEMENTATION_IS(westmere)
+#include "simdjson/westmere/ondemand.h"
+#else
+#error Unknown SIMDJSON_BUILTIN_IMPLEMENTATION
+#endif
+
+#undef SIMDJSON_CONDITIONAL_INCLUDE
+
+namespace simdjson {
+  /**
+   * @copydoc simdjson::SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand
+   */
+  namespace ondemand = SIMDJSON_BUILTIN_IMPLEMENTATION::ondemand;
+} // namespace simdjson
+
+#endif // SIMDJSON_BUILTIN_ONDEMAND_H
@@ -2,10 +2,10 @@
 #define SIMDJSON_COMMON_DEFS_H

 #include <cassert>
+#include "simdjson/compiler_check.h"
 #include "simdjson/portability.h"

 namespace simdjson {
-
 namespace internal {
 /**
 * @private
@@ -20,7 +20,6 @@ char *to_chars(char *first, const char *last, double value);
 */
 double from_chars(const char *first) noexcept;
 double from_chars(const char *first, const char* end) noexcept;
-
 }

 #ifndef SIMDJSON_EXCEPTIONS
@@ -31,26 +30,6 @@ double from_chars(const char *first, const char* end) noexcept;
 #endif
 #endif

-/** The maximum document size supported by simdjson. */
-constexpr size_t SIMDJSON_MAXSIZE_BYTES = 0xFFFFFFFF;
-
-/**
- * The amount of padding needed in a buffer to parse JSON.
- *
- * The input buf should be readable up to buf + SIMDJSON_PADDING
- * this is a stopgap; there should be a better description of the
- * main loop and its behavior that abstracts over this
- * See https://github.com/simdjson/simdjson/issues/174
- */
-constexpr size_t SIMDJSON_PADDING = 64;
-
-/**
- * By default, simdjson supports this many nested objects and arrays.
- *
- * This is the default for parser::max_depth().
- */
-constexpr size_t DEFAULT_MAX_DEPTH = 1024;
-
 } // namespace simdjson

 #if defined(__GNUC__)
@@ -105,6 +84,9 @@ constexpr size_t DEFAULT_MAX_DEPTH = 1024;
  #define SIMDJSON_DISABLE_STRICT_OVERFLOW_WARNING
  #define SIMDJSON_POP_DISABLE_WARNINGS __pragma(warning( pop ))

+  #define SIMDJSON_PUSH_DISABLE_UNUSED_WARNINGS
+  #define SIMDJSON_POP_DISABLE_UNUSED_WARNINGS
+
 #else // SIMDJSON_REGULAR_VISUAL_STUDIO

  #define simdjson_really_inline inline __attribute__((always_inline))
@@ -150,7 +132,8 @@ constexpr size_t DEFAULT_MAX_DEPTH = 1024;
    SIMDJSON_DISABLE_GCC_WARNING(-Wshadow) \
    SIMDJSON_DISABLE_GCC_WARNING(-Wunused-parameter) \
    SIMDJSON_DISABLE_GCC_WARNING(-Wunused-variable) \
-    SIMDJSON_DISABLE_GCC_WARNING(-Wmaybe-uninitialized)
+    SIMDJSON_DISABLE_GCC_WARNING(-Wmaybe-uninitialized) \
+    SIMDJSON_DISABLE_GCC_WARNING(-Wformat-security)
  #endif // __clang__

  #define SIMDJSON_PRAGMA(P) _Pragma(#P)
@@ -164,6 +147,10 @@ constexpr size_t DEFAULT_MAX_DEPTH = 1024;
  #define SIMDJSON_DISABLE_STRICT_OVERFLOW_WARNING SIMDJSON_DISABLE_GCC_WARNING(-Wstrict-overflow)
  #define SIMDJSON_POP_DISABLE_WARNINGS _Pragma("GCC diagnostic pop")

+  #define SIMDJSON_PUSH_DISABLE_UNUSED_WARNINGS SIMDJSON_PUSH_DISABLE_WARNINGS \
+    SIMDJSON_DISABLE_GCC_WARNING(-Wunused)
+  #define SIMDJSON_POP_DISABLE_UNUSED_WARNINGS SIMDJSON_POP_DISABLE_WARNINGS
+


 #endif // MSC_VER
@@ -322,7 +309,6 @@ namespace std {
 # define simdjson_fallthrough do {} while (0)  /* fallthrough */
 #endif // simdjson_fallthrough

-
 #if SIMDJSON_DEVELOPMENT_CHECKS
 #define SIMDJSON_DEVELOPMENT_ASSERT(expr) do { assert ((expr)); } while (0)
 #else
@@ -333,4 +319,25 @@ namespace std {
 #define SIMDJSON_UTF8VALIDATION 1
 #endif

+#ifdef __has_include
+// How do we detect that a compiler supports vbmi2?
+// For sure if the following header is found, we are ok?
+#if __has_include(<avx512vbmi2intrin.h>)
+#define SIMDJSON_COMPILER_SUPPORTS_VBMI2 1
+#endif
+#endif
+
+#ifdef _MSC_VER
+#if _MSC_VER >= 1920
+// Visual Studio 2019 and up support VBMI2 under x64 even if the header
+// avx512vbmi2intrin.h is not found.
+#define SIMDJSON_COMPILER_SUPPORTS_VBMI2 1
+#endif
+#endif
+
+// By default, we allow AVX512.
+#ifndef SIMDJSON_AVX512_ALLOWED
+#define SIMDJSON_AVX512_ALLOWED 1
+#endif
+
 #endif // SIMDJSON_COMMON_DEFS_H
@@ -32,4 +32,12 @@
 #error simdjson requires a compiler compliant with the C++11 standard
 #endif

+#ifndef SIMDJSON_IF_CONSTEXPR
+#if SIMDJSON_CPLUSPLUS17
+#define SIMDJSON_IF_CONSTEXPR if constexpr
+#else
+#define SIMDJSON_IF_CONSTEXPR if
+#endif
+#endif
+
 #endif // SIMDJSON_COMPILER_CHECK_H
@@ -1,11 +1,7 @@
 #ifndef SIMDJSON_DOM_H
 #define SIMDJSON_DOM_H

-#include "simdjson/base.h"
-
-SIMDJSON_PUSH_DISABLE_WARNINGS
-SIMDJSON_DISABLE_UNDESIRED_WARNINGS
-
+#include "simdjson/dom/base.h"
 #include "simdjson/dom/array.h"
 #include "simdjson/dom/document_stream.h"
 #include "simdjson/dom/document.h"
@@ -30,6 +26,4 @@ SIMDJSON_DISABLE_UNDESIRED_WARNINGS
 #include "simdjson/internal/tape_ref-inl.h"
 #include "simdjson/dom/serialization-inl.h"

-SIMDJSON_POP_DISABLE_WARNINGS
-
 #endif // SIMDJSON_DOM_H
@@ -1,11 +1,15 @@
-#ifndef SIMDJSON_INLINE_ARRAY_H
-#define SIMDJSON_INLINE_ARRAY_H
+#ifndef SIMDJSON_ARRAY_INL_H
+#define SIMDJSON_ARRAY_INL_H

-// Inline implementations go in here.
+#include <utility>

+#include "simdjson/dom/base.h"
 #include "simdjson/dom/array.h"
 #include "simdjson/dom/element.h"
-#include <utility>
+#include "simdjson/error-inl.h"
+#include "simdjson/internal/tape_ref-inl.h"
+
+#include <limits>

 namespace simdjson {

@@ -170,4 +174,4 @@ static_assert(std::ranges::sized_range<simdjson::simdjson_result<simdjson::dom::
 #endif // SIMDJSON_EXCEPTIONS
 #endif // defined(__cpp_lib_ranges)

-#endif // SIMDJSON_INLINE_ARRAY_H
+#endif // SIMDJSON_ARRAY_INL_H
@@ -1,21 +1,12 @@
 #ifndef SIMDJSON_DOM_ARRAY_H
 #define SIMDJSON_DOM_ARRAY_H

-#include "simdjson/common_defs.h"
-#include "simdjson/error.h"
+#include "simdjson/dom/base.h"
 #include "simdjson/internal/tape_ref.h"

 namespace simdjson {
-
-namespace internal {
-template<typename T>
-class string_builder;
-}
 namespace dom {

-class document;
-class element;
-
 /**
 * JSON array.
 */
@@ -0,0 +1,54 @@
+#ifndef SIMDJSON_DOM_BASE_H
+#define SIMDJSON_DOM_BASE_H
+
+#include "simdjson/base.h"
+
+namespace simdjson {
+
+/**
+ * @brief A DOM API on top of the simdjson parser.
+ */
+namespace dom {
+
+/** The default batch size for parser.parse_many() and parser.load_many() */
+static constexpr size_t DEFAULT_BATCH_SIZE = 1000000;
+/**
+ * Some adversary might try to set the batch size to 0 or 1, which might cause problems.
+ * We set a minimum of 32B since anything else is highly likely to be an error. In practice,
+ * most users will want a much larger batch size.
+ *
+ * All non-negative MINIMAL_BATCH_SIZE values should be 'safe' except that, obviously, no JSON
+ * document can ever span 0 or 1 byte and that very large values would create memory allocation issues.
+ */
+static constexpr size_t MINIMAL_BATCH_SIZE = 32;
+
+/**
+ * It is wasteful to allocate memory for tiny documents (e.g., 4 bytes).
+ */
+static constexpr size_t MINIMAL_DOCUMENT_CAPACITY = 32;
+
+class array;
+class document;
+class document_stream;
+class element;
+class key_value_pair;
+class object;
+class parser;
+
+#ifdef SIMDJSON_THREADS_ENABLED
+struct stage1_worker;
+#endif // SIMDJSON_THREADS_ENABLED
+
+} // namespace dom
+
+namespace internal {
+
+template<typename T>
+class string_builder;
+class tape_ref;
+
+} // namespace internal
+
+} // namespace simdjson
+
+#endif // SIMDJSON_DOM_BASE_H
@@ -1,13 +1,14 @@
-#ifndef SIMDJSON_INLINE_DOCUMENT_H
-#define SIMDJSON_INLINE_DOCUMENT_H
+#ifndef SIMDJSON_DOCUMENT_INL_H
+#define SIMDJSON_DOCUMENT_INL_H

 // Inline implementations go in here.

+#include "simdjson/dom/base.h"
 #include "simdjson/dom/document.h"
-#include "simdjson/dom/element.h"
-#include "simdjson/internal/tape_ref.h"
+#include "simdjson/dom/element-inl.h"
+#include "simdjson/internal/tape_ref-inl.h"
 #include "simdjson/internal/jsonformatutils.h"
-#include <ostream>
+
 #include <cstring>

 namespace simdjson {
@@ -155,4 +156,4 @@ inline bool document::dump_raw_tape(std::ostream &os) const noexcept {
 } // namespace dom
 } // namespace simdjson

-#endif // SIMDJSON_INLINE_DOCUMENT_H
+#endif // SIMDJSON_DOCUMENT_INL_H
@@ -1,15 +1,13 @@
 #ifndef SIMDJSON_DOM_DOCUMENT_H
 #define SIMDJSON_DOM_DOCUMENT_H

-#include "simdjson/common_defs.h"
+#include "simdjson/dom/base.h"
+
 #include <memory>
-#include <ostream>

 namespace simdjson {
 namespace dom {

-class element;
-
 /**
 * A parsed JSON document.
 *
@@ -1,14 +1,18 @@
-#ifndef SIMDJSON_INLINE_DOCUMENT_STREAM_H
-#define SIMDJSON_INLINE_DOCUMENT_STREAM_H
+#ifndef SIMDJSON_DOCUMENT_STREAM_INL_H
+#define SIMDJSON_DOCUMENT_STREAM_INL_H

+#include "simdjson/dom/base.h"
 #include "simdjson/dom/document_stream.h"
-#include <algorithm>
-#include <limits>
-#include <stdexcept>
+#include "simdjson/dom/element-inl.h"
+#include "simdjson/dom/parser-inl.h"
+#include "simdjson/error-inl.h"
+#include "simdjson/internal/dom_parser_implementation.h"
+
 namespace simdjson {
 namespace dom {

 #ifdef SIMDJSON_THREADS_ENABLED
+
 inline void stage1_worker::finish() {
  // After calling "run" someone would call finish() to wait
  // for the end of the processing.
@@ -337,4 +341,4 @@ simdjson_inline dom::document_stream::iterator simdjson_result<dom::document_str
 #endif // SIMDJSON_EXCEPTIONS

 } // namespace simdjson
-#endif // SIMDJSON_INLINE_DOCUMENT_STREAM_H
+#endif // SIMDJSON_DOCUMENT_STREAM_INL_H
@@ -1,9 +1,9 @@
 #ifndef SIMDJSON_DOCUMENT_STREAM_H
 #define SIMDJSON_DOCUMENT_STREAM_H

-#include "simdjson/common_defs.h"
+#include "simdjson/dom/base.h"
 #include "simdjson/dom/parser.h"
-#include "simdjson/error.h"
+
 #ifdef SIMDJSON_THREADS_ENABLED
 #include <thread>
 #include <mutex>
@@ -13,7 +13,6 @@
 namespace simdjson {
 namespace dom {

-
 #ifdef SIMDJSON_THREADS_ENABLED
 /** @private Custom worker class **/
 struct stage1_worker {
@@ -1,11 +1,17 @@
-#ifndef SIMDJSON_INLINE_ELEMENT_H
-#define SIMDJSON_INLINE_ELEMENT_H
+#ifndef SIMDJSON_ELEMENT_INL_H
+#define SIMDJSON_ELEMENT_INL_H

-#include "simdjson/dom/array.h"
+#include "simdjson/dom/base.h"
 #include "simdjson/dom/element.h"
+#include "simdjson/dom/document.h"
 #include "simdjson/dom/object.h"
-#include <cstring>
-#include <utility>
+#include "simdjson/internal/tape_type.h"
+
+#include "simdjson/dom/object-inl.h"
+#include "simdjson/error-inl.h"
+
+#include <ostream>
+#include <limits>

 namespace simdjson {

@@ -438,4 +444,4 @@ inline std::ostream& operator<<(std::ostream& out, element_type type) {

 } // namespace simdjson

-#endif // SIMDJSON_INLINE_ELEMENT_H
+#endif // SIMDJSON_ELEMENT_INL_H
@@ -1,20 +1,11 @@
 #ifndef SIMDJSON_DOM_ELEMENT_H
 #define SIMDJSON_DOM_ELEMENT_H

-#include "simdjson/common_defs.h"
-#include "simdjson/error.h"
-#include "simdjson/internal/tape_ref.h"
-#include <ostream>
+#include "simdjson/dom/base.h"
+#include "simdjson/dom/array.h"

 namespace simdjson {
-namespace internal {
-template<typename T>
-class string_builder;
-}
 namespace dom {
-class array;
-class document;
-class object;

 /**
 * The actual concrete type of a JSON element
@@ -537,7 +528,6 @@ public:
 #endif // SIMDJSON_EXCEPTIONS
 };

-
 } // namespace simdjson

 #endif // SIMDJSON_DOM_DOCUMENT_H
@@ -3,9 +3,11 @@
 #ifndef SIMDJSON_DOM_JSONPARSER_H
 #define SIMDJSON_DOM_JSONPARSER_H

-#include "simdjson/dom/document.h"
-#include "simdjson/dom/parsedjson.h"
-#include "simdjson/jsonioutil.h"
+#include "simdjson/dom/base.h"
+#include "simdjson/dom/parser.h"
+#include "simdjson/dom/element.h"
+
+#include "simdjson/dom/parser-inl.h"

 namespace simdjson {

@@ -1,11 +1,14 @@
-#ifndef SIMDJSON_INLINE_OBJECT_H
-#define SIMDJSON_INLINE_OBJECT_H
+#ifndef SIMDJSON_OBJECT_INL_H
+#define SIMDJSON_OBJECT_INL_H

-#include "simdjson/dom/element.h"
+#include "simdjson/dom/base.h"
 #include "simdjson/dom/object.h"
-#include "simdjson/portability.h"
+#include "simdjson/dom/document.h"
+
+#include "simdjson/dom/element-inl.h"
+#include "simdjson/error-inl.h"
+
 #include <cstring>
-#include <string>

 namespace simdjson {

@@ -253,4 +256,4 @@ static_assert(std::ranges::sized_range<simdjson::simdjson_result<simdjson::dom::
 #endif // SIMDJSON_EXCEPTIONS
 #endif // defined(__cpp_lib_ranges)

-#endif // SIMDJSON_INLINE_OBJECT_H
+#endif // SIMDJSON_OBJECT_INL_H
@@ -1,21 +1,13 @@
 #ifndef SIMDJSON_DOM_OBJECT_H
 #define SIMDJSON_DOM_OBJECT_H

-#include "simdjson/common_defs.h"
-#include "simdjson/error.h"
+#include "simdjson/dom/base.h"
+#include "simdjson/dom/element.h"
 #include "simdjson/internal/tape_ref.h"

 namespace simdjson {
-namespace internal {
-template<typename T>
-class string_builder;
-}
 namespace dom {

-class document;
-class element;
-class key_value_pair;
-
 /**
 * JSON object.
 */
@@ -3,7 +3,7 @@
 #ifndef SIMDJSON_DOM_PARSEDJSON_H
 #define SIMDJSON_DOM_PARSEDJSON_H

-#include "simdjson/dom/document.h"
+#include "simdjson/dom/base.h"

 namespace simdjson {

@@ -1,9 +1,17 @@
-#ifndef SIMDJSON_INLINE_PARSEDJSON_ITERATOR_H
-#define SIMDJSON_INLINE_PARSEDJSON_ITERATOR_H
+#ifndef SIMDJSON_PARSEDJSON_ITERATOR_INL_H
+#define SIMDJSON_PARSEDJSON_ITERATOR_INL_H

+#include "simdjson/dom/base.h"
 #include "simdjson/dom/parsedjson_iterator.h"
-#include "simdjson/portability.h"
+#include "simdjson/internal/jsonformatutils.h"
+
+#include "simdjson/dom/parser-inl.h"
+#include "simdjson/internal/tape_ref-inl.h"
+
 #include <cstring>
+#include <iterator>
+#include <limits>
+#include <ostream>

 #ifndef SIMDJSON_DISABLE_DEPRECATED_API

@@ -197,6 +205,12 @@ void dom::parser::Iterator::to_start_scope() {
  current_type = uint8_t(current_val >> 56);
 }

+inline void dom::parser::Iterator::rewind() {
+    while (up())
+    ;
+}
+
+
 bool dom::parser::Iterator::next() {
  size_t npos;
  if ((current_type == '[') || (current_type == '{')) {
@@ -365,6 +379,48 @@ bool dom::parser::Iterator::move_to(const char *pointer,
  return found;
 }

+inline bool dom::parser::Iterator::move_to(const std::string &pointer) {
+  return move_to(pointer.c_str(), uint32_t(pointer.length()));
+}
+
+inline int64_t dom::parser::Iterator::get_integer() const {
+    if (location + 1 >= tape_length) {
+    return 0; // default value in case of error
+    }
+    return static_cast<int64_t>(doc.tape[location + 1]);
+}
+
+inline uint64_t dom::parser::Iterator::get_unsigned_integer() const {
+    if (location + 1 >= tape_length) {
+    return 0; // default value in case of error
+    }
+    return doc.tape[location + 1];
+}
+
+inline const char * dom::parser::Iterator::get_string() const {
+    return reinterpret_cast<const char *>(
+        doc.string_buf.get() + (current_val & internal::JSON_VALUE_MASK) + sizeof(uint32_t));
+}
+
+inline uint32_t dom::parser::Iterator::get_string_length() const {
+    uint32_t answer;
+    std::memcpy(&answer,
+        reinterpret_cast<const char *>(doc.string_buf.get() +
+                                        (current_val & internal::JSON_VALUE_MASK)),
+        sizeof(uint32_t));
+    return answer;
+}
+
+inline double dom::parser::Iterator::get_double() const {
+    if (location + 1 >= tape_length) {
+    return std::numeric_limits<double>::quiet_NaN(); // default value in
+                                                    // case of error
+    }
+    double answer;
+    std::memcpy(&answer, &doc.tape[location + 1], sizeof(answer));
+    return answer;
+}
+
 bool dom::parser::Iterator::relative_move_to(const char *pointer,
                                                            uint32_t length) {
  if (length == 0) {
@@ -486,4 +542,4 @@ SIMDJSON_POP_DISABLE_WARNINGS
 #endif // SIMDJSON_DISABLE_DEPRECATED_API


-#endif // SIMDJSON_INLINE_PARSEDJSON_ITERATOR_H
+#endif // SIMDJSON_PARSEDJSON_ITERATOR_INL_H
@@ -3,16 +3,8 @@
 #ifndef SIMDJSON_DOM_PARSEDJSON_ITERATOR_H
 #define SIMDJSON_DOM_PARSEDJSON_ITERATOR_H

-#include <cstring>
-#include <string>
-#include <ostream>
-#include <iterator>
-#include <limits>
-#include <stdexcept>
-
-#include "simdjson/dom/document.h"
-#include "simdjson/dom/parsedjson.h"
-#include "simdjson/internal/jsonformatutils.h"
+#include "simdjson/dom/base.h"
+#include "simdjson/dom/parser.h"

 #ifndef SIMDJSON_DISABLE_DEPRECATED_API

@@ -52,51 +44,23 @@ public:
  }

  // get the int64_t value at this node; valid only if get_type is "l"
-  inline int64_t get_integer() const {
-      if (location + 1 >= tape_length) {
-      return 0; // default value in case of error
-      }
-      return static_cast<int64_t>(doc.tape[location + 1]);
-  }
+  inline int64_t get_integer() const;

  // get the value as uint64; valid only if  if get_type is "u"
-  inline uint64_t get_unsigned_integer() const {
-      if (location + 1 >= tape_length) {
-      return 0; // default value in case of error
-      }
-      return doc.tape[location + 1];
-  }
+  inline uint64_t get_unsigned_integer() const;

  // get the string value at this node (NULL ended); valid only if get_type is "
  // note that tabs, and line endings are escaped in the returned value (see
  // print_with_escapes) return value is valid UTF-8, it may contain NULL chars
  // within the string: get_string_length determines the true string length.
-  inline const char *get_string() const {
-      return reinterpret_cast<const char *>(
-          doc.string_buf.get() + (current_val & internal::JSON_VALUE_MASK) + sizeof(uint32_t));
-  }
+  inline const char *get_string() const;

  // return the length of the string in bytes
-  inline uint32_t get_string_length() const {
-      uint32_t answer;
-      std::memcpy(&answer,
-          reinterpret_cast<const char *>(doc.string_buf.get() +
-                                          (current_val & internal::JSON_VALUE_MASK)),
-          sizeof(uint32_t));
-      return answer;
-  }
+  inline uint32_t get_string_length() const;

  // get the double value at this node; valid only if
  // get_type() is "d"
-  inline double get_double() const {
-      if (location + 1 >= tape_length) {
-      return std::numeric_limits<double>::quiet_NaN(); // default value in
-                                                      // case of error
-      }
-      double answer;
-      std::memcpy(&answer, &doc.tape[location + 1], sizeof(answer));
-      return answer;
-  }
+  inline double get_double() const;

  inline bool is_object_or_array() const { return is_object() || is_array(); }

@@ -190,9 +154,7 @@ public:
  // "If a referenced member name is not unique in an object, the member that
  // is referenced is undefined, and evaluation fails". Here we just return
  // the first corresponding value.
-  inline bool move_to(const std::string &pointer) {
-      return move_to(pointer.c_str(), uint32_t(pointer.length()));
-  }
+  inline bool move_to(const std::string &pointer);

  private:
  // Almost the same as move_to(), except it searches from the current
@@ -240,10 +202,7 @@ public:
  // a scope is a series of nodes at the same level
  inline void to_start_scope();

-  inline void rewind() {
-      while (up())
-      ;
-  }
+  inline void rewind();



@@ -1,12 +1,16 @@
-#ifndef SIMDJSON_INLINE_PARSER_H
-#define SIMDJSON_INLINE_PARSER_H
+#ifndef SIMDJSON_PARSER_INL_H
+#define SIMDJSON_PARSER_INL_H

+#include "simdjson/dom/base.h"
 #include "simdjson/dom/document_stream.h"
-#include "simdjson/dom/parser.h"
 #include "simdjson/implementation.h"
-#include "simdjson/internal/jsonformatutils.h"
-#include "simdjson/portability.h"
-#include <cstdio>
+#include "simdjson/internal/dom_parser_implementation.h"
+
+#include "simdjson/error-inl.h"
+#include "simdjson/padded_string-inl.h"
+#include "simdjson/dom/document_stream-inl.h"
+#include "simdjson/dom/element-inl.h"
+
 #include <climits>

 namespace simdjson {
@@ -236,4 +240,4 @@ simdjson_inline void parser::set_max_capacity(size_t max_capacity) noexcept {
 } // namespace dom
 } // namespace simdjson

-#endif // SIMDJSON_INLINE_PARSER_H
+#endif // SIMDJSON_PARSER_INL_H
@@ -1,41 +1,13 @@
 #ifndef SIMDJSON_DOM_PARSER_H
 #define SIMDJSON_DOM_PARSER_H

-#include "simdjson/common_defs.h"
+#include "simdjson/dom/base.h"
 #include "simdjson/dom/document.h"
-#include "simdjson/error.h"
-#include "simdjson/internal/dom_parser_implementation.h"
-#include "simdjson/internal/tape_ref.h"
-#include "simdjson/padded_string.h"
-#include "simdjson/portability.h"
-#include <memory>
-#include <ostream>
-#include <string>

 namespace simdjson {

 namespace dom {

-class document_stream;
-class element;
-
-/** The default batch size for parser.parse_many() and parser.load_many() */
-static constexpr size_t DEFAULT_BATCH_SIZE = 1000000;
-/**
- * Some adversary might try to set the batch size to 0 or 1, which might cause problems.
- * We set a minimum of 32B since anything else is highly likely to be an error. In practice,
- * most users will want a much larger batch size.
- *
- * All non-negative MINIMAL_BATCH_SIZE values should be 'safe' except that, obviously, no JSON
- * document can ever span 0 or 1 byte and that very large values would create memory allocation issues.
- */
-static constexpr size_t MINIMAL_BATCH_SIZE = 32;
-
-/**
- * It is wasteful to allocate memory for tiny documents (e.g., 4 bytes).
- */
-static constexpr size_t MINIMAL_DOCUMENT_CAPACITY = 32;
-
 /**
 * A persistent document parser.
 *
@@ -2,10 +2,16 @@
 #ifndef SIMDJSON_SERIALIZATION_INL_H
 #define SIMDJSON_SERIALIZATION_INL_H

+#include "simdjson/dom/base.h"
 #include "simdjson/dom/serialization.h"
+#include "simdjson/dom/parser.h"
+#include "simdjson/internal/tape_type.h"

-#include <cinttypes>
-#include <type_traits>
+#include "simdjson/dom/array-inl.h"
+#include "simdjson/dom/object-inl.h"
+#include "simdjson/internal/tape_ref-inl.h"
+
+#include <cstring>

 namespace simdjson {
 namespace dom {
@@ -17,12 +23,46 @@ inline bool parser::print_json(std::ostream &os) const noexcept {
  os << answer;
  return true;
 }
+
+inline std::ostream& operator<<(std::ostream& out, simdjson::dom::element value) {
+    simdjson::internal::string_builder<> sb;
+    sb.append(value);
+    return (out << sb.str());
 }
+#if SIMDJSON_EXCEPTIONS
+inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<simdjson::dom::element> x) {
+    if (x.error()) { throw simdjson::simdjson_error(x.error()); }
+    return (out << x.value());
+}
+#endif
+inline std::ostream& operator<<(std::ostream& out, simdjson::dom::array value)  {
+    simdjson::internal::string_builder<> sb;
+    sb.append(value);
+    return (out << sb.str());
+}
+#if SIMDJSON_EXCEPTIONS
+inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<simdjson::dom::array> x) {
+    if (x.error()) { throw simdjson::simdjson_error(x.error()); }
+    return (out << x.value());
+}
+#endif
+inline std::ostream& operator<<(std::ostream& out, simdjson::dom::object value)   {
+    simdjson::internal::string_builder<> sb;
+    sb.append(value);
+    return (out << sb.str());
+}
+#if SIMDJSON_EXCEPTIONS
+inline std::ostream& operator<<(std::ostream& out,  simdjson::simdjson_result<simdjson::dom::object> x) {
+    if (x.error()) { throw  simdjson::simdjson_error(x.error()); }
+    return (out << x.value());
+}
+#endif
+
+} // namespace dom
+
 /***
 * Number utility functions
 **/
-
-
 namespace {
 /**@private
 * Escape sequence like \b or \u0001
@@ -41,7 +81,7 @@ struct escape_sequence {
 * adds the to_chars functions which would do as well, but
 * we want to support C++11.
 */
-char *fast_itoa(char *output, int64_t value) noexcept {
+static char *fast_itoa(char *output, int64_t value) noexcept {
  // This is a standard implementation of itoa.
  char buffer[20];
  uint64_t value_positive;
@@ -83,7 +123,7 @@ char *fast_itoa(char *output, int64_t value) noexcept {
 * adds the to_chars functions which would do as well, but
 * we want to support C++11.
 */
-char *fast_itoa(char *output, uint64_t value) noexcept {
+static char *fast_itoa(char *output, uint64_t value) noexcept {
  // This is a standard implementation of itoa.
  char buffer[20];
  const char *const end_buffer = buffer + 20;
@@ -100,6 +140,8 @@ char *fast_itoa(char *output, uint64_t value) noexcept {
  std::memcpy(output, write_pointer, len);
  return output + len;
 }
+
+
 } // anonymous namespace
 namespace internal {

@@ -107,19 +149,22 @@ namespace internal {
 * Minifier/formatter code.
 **/

-simdjson_inline void mini_formatter::number(uint64_t x) {
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::number(uint64_t x) {
  char number_buffer[24];
  char *newp = fast_itoa(number_buffer, x);
  buffer.insert(buffer.end(), number_buffer, newp);
 }

-simdjson_inline void mini_formatter::number(int64_t x) {
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::number(int64_t x) {
  char number_buffer[24];
  char *newp = fast_itoa(number_buffer, x);
  buffer.insert(buffer.end(), number_buffer, newp);
 }

-simdjson_inline void mini_formatter::number(double x) {
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::number(double x) {
  char number_buffer[24];
  // Currently, passing the nullptr to the second argument is
  // safe because our implementation does not check the second
@@ -128,31 +173,51 @@ simdjson_inline void mini_formatter::number(double x) {
  buffer.insert(buffer.end(), number_buffer, newp);
 }

-simdjson_inline void mini_formatter::start_array() { one_char('['); }
-simdjson_inline void mini_formatter::end_array() { one_char(']'); }
-simdjson_inline void mini_formatter::start_object() { one_char('{'); }
-simdjson_inline void mini_formatter::end_object() { one_char('}'); }
-simdjson_inline void mini_formatter::comma() { one_char(','); }
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::start_array() { one_char('['); }


-simdjson_inline void mini_formatter::true_atom() {
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::end_array() { one_char(']'); }
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::start_object() { one_char('{'); }
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::end_object() { one_char('}'); }
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::comma() { one_char(','); }
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::true_atom() {
  const char * s = "true";
  buffer.insert(buffer.end(), s, s + 4);
 }
-simdjson_inline void mini_formatter::false_atom() {
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::false_atom() {
  const char * s = "false";
  buffer.insert(buffer.end(), s, s + 5);
 }
-simdjson_inline void mini_formatter::null_atom() {
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::null_atom() {
  const char * s = "null";
  buffer.insert(buffer.end(), s, s + 4);
 }
-simdjson_inline void mini_formatter::one_char(char c) { buffer.push_back(c); }
-simdjson_inline void mini_formatter::key(std::string_view unescaped) {
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::one_char(char c) { buffer.push_back(c); }
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::key(std::string_view unescaped) {
  string(unescaped);
  one_char(':');
 }
-simdjson_inline void mini_formatter::string(std::string_view unescaped) {
+
+template<class formatter>
+simdjson_inline void base_formatter<formatter>::string(std::string_view unescaped) {
  one_char('\"');
  size_t i = 0;
  // Fast path for the case where we have no control character, no ", and no backslash.
@@ -231,14 +296,46 @@ simdjson_inline void mini_formatter::string(std::string_view unescaped) {
  one_char('\"');
 }

-inline void mini_formatter::clear() {
+
+template<class formatter>
+inline void base_formatter<formatter>::clear() {
  buffer.clear();
 }

-simdjson_inline std::string_view mini_formatter::str() const {
+template<class formatter>
+simdjson_inline std::string_view base_formatter<formatter>::str() const {
  return std::string_view(buffer.data(), buffer.size());
 }

+simdjson_inline void mini_formatter::print_newline() {
+    return;
+}
+
+simdjson_inline void mini_formatter::print_indents(size_t depth) {
+    (void)depth;
+    return;
+}
+
+simdjson_inline void mini_formatter::print_space() {
+    return;
+}
+
+simdjson_inline void pretty_formatter::print_newline() {
+    one_char('\n');
+}
+
+simdjson_inline void pretty_formatter::print_indents(size_t depth) {
+    if(this->indent_step <= 0) {
+        return;
+    }
+    for(size_t i = 0; i < this->indent_step * depth; i++) {
+        one_char(' ');
+    }
+}
+
+simdjson_inline void pretty_formatter::print_space() {
+    one_char(' ');
+}

 /***
 * String building code.
@@ -258,11 +355,16 @@ inline void string_builder<serializer>::append(simdjson::dom::element value) {
    // print commas after each value
    if (after_value) {
      format.comma();
+      format.print_newline();
    }
+
+    format.print_indents(depth);
+
    // If we are in an object, print the next key and :, and skip to the next
    // value.
    if (is_object[depth]) {
      format.key(iter.get_string_view());
+      format.print_space();
      iter.json_index++;
    }
    switch (iter.tape_ref_type()) {
@@ -291,6 +393,7 @@ inline void string_builder<serializer>::append(simdjson::dom::element value) {

      is_object[depth] = false;
      after_value = false;
+      format.print_newline();
      continue;
    }

@@ -318,6 +421,7 @@ inline void string_builder<serializer>::append(simdjson::dom::element value) {

      is_object[depth] = true;
      after_value = false;
+      format.print_newline();
      continue;
    }

@@ -362,17 +466,21 @@ inline void string_builder<serializer>::append(simdjson::dom::element value) {
    // Handle multiple ends in a row
    while (depth != 0 && (iter.tape_ref_type() == tape_type::END_ARRAY ||
                          iter.tape_ref_type() == tape_type::END_OBJECT)) {
+      format.print_newline();
+      depth--;
+      format.print_indents(depth);
      if (iter.tape_ref_type() == tape_type::END_ARRAY) {
        format.end_array();
      } else {
        format.end_object();
      }
-      depth--;
      iter.json_index++;
    }

    // Stop when we're at depth 0
  } while (depth != 0);
+
+  format.print_newline();
 }

 template <class serializer>
@@ -1,13 +1,10 @@
 #ifndef SIMDJSON_SERIALIZATION_H
 #define SIMDJSON_SERIALIZATION_H

-#include "simdjson/common_defs.h"
-#include "simdjson/dom/document.h"
-#include "simdjson/error.h"
-#include "simdjson/internal/dom_parser_implementation.h"
-#include "simdjson/internal/tape_ref.h"
-#include "simdjson/padded_string.h"
-#include "simdjson/portability.h"
+#include "simdjson/dom/base.h"
+#include "simdjson/dom/element.h"
+#include "simdjson/dom/object.h"
+
 #include <vector>

 namespace simdjson {
@@ -19,7 +16,90 @@ namespace simdjson {
 */
 namespace internal {

-class mini_formatter;
+template<class formatter>
+class base_formatter {
+public:
+  /** Add a comma **/
+  simdjson_inline void comma();
+  /** Start an array, prints [ **/
+  simdjson_inline void start_array();
+  /** End an array, prints ] **/
+  simdjson_inline void end_array();
+  /** Start an array, prints { **/
+  simdjson_inline void start_object();
+  /** Start an array, prints } **/
+  simdjson_inline void end_object();
+  /** Prints a true **/
+  simdjson_inline void true_atom();
+  /** Prints a false **/
+  simdjson_inline void false_atom();
+  /** Prints a null **/
+  simdjson_inline void null_atom();
+  /** Prints a number **/
+  simdjson_inline void number(int64_t x);
+  /** Prints a number **/
+  simdjson_inline void number(uint64_t x);
+  /** Prints a number **/
+  simdjson_inline void number(double x);
+  /** Prints a key (string + colon) **/
+  simdjson_inline void key(std::string_view unescaped);
+  /** Prints a string. The string is escaped as needed. **/
+  simdjson_inline void string(std::string_view unescaped);
+  /** Clears out the content. **/
+  simdjson_inline void clear();
+  /**
+   * Get access to the buffer, it is owned by the instance, but
+   * the user can make a copy.
+   **/
+  simdjson_inline std::string_view str() const;
+
+  /** Prints one character **/
+  simdjson_inline void one_char(char c);
+
+  simdjson_inline void call_print_newline() {
+      this->print_newline();
+  }
+
+  simdjson_inline void call_print_indents(size_t depth) {
+      this->print_indents(depth);
+  }
+
+  simdjson_inline void call_print_space() {
+      this->print_space();
+  }
+
+protected:
+  // implementation details (subject to change)
+  /** Backing buffer **/
+  std::vector<char> buffer{}; // not ideal!
+};
+
+
+/**
+ * @private This is the class that we expect to use with the string_builder
+ * template. It tries to produce a compact version of the JSON element
+ * as quickly as possible.
+ */
+class mini_formatter : public base_formatter<mini_formatter> {
+public:
+  simdjson_inline void print_newline();
+
+  simdjson_inline void print_indents(size_t depth);
+
+  simdjson_inline void print_space();
+};
+
+class pretty_formatter : public base_formatter<pretty_formatter> {
+public:
+  simdjson_inline void print_newline();
+
+  simdjson_inline void print_indents(size_t depth);
+
+  simdjson_inline void print_space();
+
+protected:
+  int indent_step = 4;
+};

 /**
 * @private The string_builder template allows us to construct
@@ -55,56 +135,6 @@ private:
  formatter format{};
 };

-/**
- * @private This is the class that we expect to use with the string_builder
- * template. It tries to produce a compact version of the JSON element
- * as quickly as possible.
- */
-class mini_formatter {
-public:
-  mini_formatter() = default;
-  /** Add a comma **/
-  simdjson_inline void comma();
-  /** Start an array, prints [ **/
-  simdjson_inline void start_array();
-  /** End an array, prints ] **/
-  simdjson_inline void end_array();
-  /** Start an array, prints { **/
-  simdjson_inline void start_object();
-  /** Start an array, prints } **/
-  simdjson_inline void end_object();
-  /** Prints a true **/
-  simdjson_inline void true_atom();
-  /** Prints a false **/
-  simdjson_inline void false_atom();
-  /** Prints a null **/
-  simdjson_inline void null_atom();
-  /** Prints a number **/
-  simdjson_inline void number(int64_t x);
-  /** Prints a number **/
-  simdjson_inline void number(uint64_t x);
-  /** Prints a number **/
-  simdjson_inline void number(double x);
-  /** Prints a key (string + colon) **/
-  simdjson_inline void key(std::string_view unescaped);
-  /** Prints a string. The string is escaped as needed. **/
-  simdjson_inline void string(std::string_view unescaped);
-  /** Clears out the content. **/
-  simdjson_inline void clear();
-  /**
-   * Get access to the buffer, it is owned by the instance, but
-   * the user can make a copy.
-   **/
-  simdjson_inline std::string_view str() const;
-
-private:
-  // implementation details (subject to change)
-  /** Prints one character **/
-  simdjson_inline void one_char(char c);
-  /** Backing buffer **/
-  std::vector<char> buffer{}; // not ideal!
-};
-
 } // internal

 namespace dom {
@@ -116,16 +146,9 @@ namespace dom {
 * @param value The element.
 * @throw if there is an error with the underlying output stream. simdjson itself will not throw.
 */
-inline std::ostream& operator<<(std::ostream& out, simdjson::dom::element value) {
-    simdjson::internal::string_builder<> sb;
-    sb.append(value);
-    return (out << sb.str());
-}
+inline std::ostream& operator<<(std::ostream& out, simdjson::dom::element value);
 #if SIMDJSON_EXCEPTIONS
-inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<simdjson::dom::element> x) {
-    if (x.error()) { throw simdjson::simdjson_error(x.error()); }
-    return (out << x.value());
-}
+inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<simdjson::dom::element> x);
 #endif
 /**
 * Print JSON to an output stream.
@@ -134,16 +157,9 @@ inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<sim
 * @param value The array.
 * @throw if there is an error with the underlying output stream. simdjson itself will not throw.
 */
-inline std::ostream& operator<<(std::ostream& out, simdjson::dom::array value)  {
-    simdjson::internal::string_builder<> sb;
-    sb.append(value);
-    return (out << sb.str());
-}
+inline std::ostream& operator<<(std::ostream& out, simdjson::dom::array value);
 #if SIMDJSON_EXCEPTIONS
-inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<simdjson::dom::array> x) {
-    if (x.error()) { throw simdjson::simdjson_error(x.error()); }
-    return (out << x.value());
-}
+inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<simdjson::dom::array> x);
 #endif
 /**
 * Print JSON to an output stream.
@@ -152,16 +168,9 @@ inline std::ostream& operator<<(std::ostream& out, simdjson::simdjson_result<sim
 * @param value The object.
 * @throw if there is an error with the underlying output stream. simdjson itself will not throw.
 */
-inline std::ostream& operator<<(std::ostream& out, simdjson::dom::object value)   {
-    simdjson::internal::string_builder<> sb;
-    sb.append(value);
-    return (out << sb.str());
-}
+inline std::ostream& operator<<(std::ostream& out, simdjson::dom::object value);
 #if SIMDJSON_EXCEPTIONS
-inline std::ostream& operator<<(std::ostream& out,  simdjson::simdjson_result<simdjson::dom::object> x) {
-    if (x.error()) { throw  simdjson::simdjson_error(x.error()); }
-    return (out << x.value());
-}
+inline std::ostream& operator<<(std::ostream& out,  simdjson::simdjson_result<simdjson::dom::object> x);
 #endif
 } // namespace dom

@@ -212,6 +221,38 @@ std::string minify(simdjson_result<T> x) {
 }
 #endif

+/**
+ * Prettifies a JSON element or document, printing the valid JSON with indentation.
+ *
+ *   dom::parser parser;
+ *   element doc = parser.parse("   [ 1 , 2 , 3 ] "_padded);
+ *
+ *   // Prints:
+ *   // {
+ *   //     [
+ *   //         1,
+ *   //         2,
+ *   //         3
+ *   //     ]
+ *   // }
+ *   cout << prettify(doc) << endl;
+ *
+ */
+template <class T>
+std::string prettify(T x)  {
+    simdjson::internal::string_builder<simdjson::internal::pretty_formatter> sb;
+    sb.append(x);
+    std::string_view answer = sb.str();
+    return std::string(answer.data(), answer.size());
+}
+
+#if SIMDJSON_EXCEPTIONS
+template <class T>
+std::string prettify(simdjson_result<T> x) {
+    if (x.error()) { throw simdjson_error(x.error()); }
+    return to_string(x.value());
+}
+#endif

 } // namespace simdjson

@@ -1,11 +1,10 @@
-#ifndef SIMDJSON_INLINE_ERROR_H
-#define SIMDJSON_INLINE_ERROR_H
+#ifndef SIMDJSON_ERROR_INL_H
+#define SIMDJSON_ERROR_INL_H

-#include <cstring>
-#include <string>
-#include <utility>
 #include "simdjson/error.h"

+#include <iostream>
+
 namespace simdjson {
 namespace internal {
  // We store the error code so we can validate the error message is associated with the right code
@@ -182,4 +181,4 @@ simdjson_inline simdjson_result<T>::simdjson_result() noexcept

 } // namespace simdjson

-#endif // SIMDJSON_INLINE_ERROR_H
+#endif // SIMDJSON_ERROR_INL_H
@@ -1,8 +1,10 @@
 #ifndef SIMDJSON_ERROR_H
 #define SIMDJSON_ERROR_H

-#include "simdjson/common_defs.h"
+#include "simdjson/base.h"
+
 #include <string>
+#include <ostream>

 namespace simdjson {

@@ -1,30 +1,8 @@
 #ifndef SIMDJSON_FALLBACK_H
 #define SIMDJSON_FALLBACK_H

-#include "simdjson/implementation-base.h"
-
-#if SIMDJSON_IMPLEMENTATION_FALLBACK
-
-namespace simdjson {
-/**
- * Fallback implementation (runs on any machine).
- */
-namespace fallback {
-} // namespace fallback
-} // namespace simdjson
-
-#include "simdjson/fallback/implementation.h"
-
 #include "simdjson/fallback/begin.h"
-
-// Declarations
-#include "simdjson/generic/dom_parser_implementation.h"
-#include "simdjson/fallback/bitmanipulation.h"
-#include "simdjson/generic/jsoncharutils.h"
-#include "simdjson/generic/atomparsing.h"
-#include "simdjson/fallback/stringparsing.h"
-#include "simdjson/fallback/numberparsing.h"
+#include "simdjson/generic/amalgamated.h"
 #include "simdjson/fallback/end.h"

-#endif // SIMDJSON_IMPLEMENTATION_FALLBACK
-#endif // SIMDJSON_FALLBACK_H
+#endif // SIMDJSON_FALLBACK_H
@@ -0,0 +1,19 @@
+#ifndef SIMDJSON_FALLBACK_BASE_H
+#define SIMDJSON_FALLBACK_BASE_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+namespace simdjson {
+/**
+ * Fallback implementation (runs on any machine).
+ */
+namespace fallback {
+
+class implementation;
+
+} // namespace fallback
+} // namespace simdjson
+
+#endif // SIMDJSON_FALLBACK_BASE_H
@@ -1 +1,5 @@
 #define SIMDJSON_IMPLEMENTATION fallback
+#include "simdjson/fallback/base.h"
+#include "simdjson/fallback/bitmanipulation.h"
+#include "simdjson/fallback/stringparsing_defs.h"
+#include "simdjson/fallback/numberparsing_defs.h"
@@ -1,11 +1,12 @@
 #ifndef SIMDJSON_FALLBACK_BITMANIPULATION_H
 #define SIMDJSON_FALLBACK_BITMANIPULATION_H

-#include "simdjson/base.h"
-#include <limits>
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/fallback/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
+namespace fallback {
 namespace {

 #if defined(_MSC_VER) && !defined(_M_ARM64) && !defined(_M_X64)
@@ -41,7 +42,7 @@ simdjson_inline int leading_zeroes(uint64_t input_num) {
 }

 } // unnamed namespace
-} // namespace SIMDJSON_IMPLEMENTATION
+} // namespace fallback
 } // namespace simdjson

 #endif // SIMDJSON_FALLBACK_BITMANIPULATION_H
@@ -1 +1,5 @@
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/fallback/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 #undef SIMDJSON_IMPLEMENTATION
@@ -1,16 +1,14 @@
 #ifndef SIMDJSON_FALLBACK_IMPLEMENTATION_H
 #define SIMDJSON_FALLBACK_IMPLEMENTATION_H

+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/fallback/base.h"
 #include "simdjson/implementation.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
 namespace fallback {

-namespace {
-using namespace simdjson;
-using namespace simdjson::dom;
-}
-
 /**
 * @private
 */
@@ -24,7 +22,7 @@ public:
  simdjson_warn_unused error_code create_dom_parser_implementation(
    size_t capacity,
    size_t max_length,
-    std::unique_ptr<internal::dom_parser_implementation>& dst
+    std::unique_ptr<simdjson::internal::dom_parser_implementation>& dst
  ) const noexcept final;
  simdjson_warn_unused error_code minify(const uint8_t *buf, size_t len, uint8_t *dst, size_t &dst_len) const noexcept final;
  simdjson_warn_unused bool validate_utf8(const char *buf, size_t len) const noexcept final;
@@ -1,36 +0,0 @@
-#ifndef SIMDJSON_FALLBACK_NUMBERPARSING_H
-#define SIMDJSON_FALLBACK_NUMBERPARSING_H
-
-#ifdef JSON_TEST_NUMBERS // for unit testing
-void found_invalid_number(const uint8_t *buf);
-void found_integer(int64_t result, const uint8_t *buf);
-void found_unsigned_integer(uint64_t result, const uint8_t *buf);
-void found_float(double result, const uint8_t *buf);
-#endif
-
-namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
-namespace numberparsing {
-// credit: https://johnnylee-sde.github.io/Fast-numeric-string-to-int/
-/** @private */
-static simdjson_inline uint32_t parse_eight_digits_unrolled(const char *chars) {
-  uint64_t val;
-  memcpy(&val, chars, sizeof(uint64_t));
-  val = (val & 0x0F0F0F0F0F0F0F0F) * 2561 >> 8;
-  val = (val & 0x00FF00FF00FF00FF) * 6553601 >> 16;
-  return uint32_t((val & 0x0000FFFF0000FFFF) * 42949672960001 >> 32);
-}
-/** @private */
-static simdjson_inline uint32_t parse_eight_digits_unrolled(const uint8_t *chars) {
-  return parse_eight_digits_unrolled(reinterpret_cast<const char *>(chars));
-}
-
-} // namespace numberparsing
-} // namespace SIMDJSON_IMPLEMENTATION
-} // namespace simdjson
-
-#define SIMDJSON_SWAR_NUMBER_PARSING 1
-
-#include "simdjson/generic/numberparsing.h"
-
-#endif // SIMDJSON_FALLBACK_NUMBERPARSING_H
@@ -0,0 +1,80 @@
+#ifndef SIMDJSON_FALLBACK_NUMBERPARSING_DEFS_H
+#define SIMDJSON_FALLBACK_NUMBERPARSING_DEFS_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/fallback/base.h"
+#include "simdjson/internal/numberparsing_tables.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+#include <cstring>
+
+#ifdef JSON_TEST_NUMBERS // for unit testing
+void found_invalid_number(const uint8_t *buf);
+void found_integer(int64_t result, const uint8_t *buf);
+void found_unsigned_integer(uint64_t result, const uint8_t *buf);
+void found_float(double result, const uint8_t *buf);
+#endif
+
+namespace simdjson {
+namespace fallback {
+namespace numberparsing {
+
+// credit: https://johnnylee-sde.github.io/Fast-numeric-string-to-int/
+/** @private */
+static simdjson_inline uint32_t parse_eight_digits_unrolled(const char *chars) {
+  uint64_t val;
+  memcpy(&val, chars, sizeof(uint64_t));
+  val = (val & 0x0F0F0F0F0F0F0F0F) * 2561 >> 8;
+  val = (val & 0x00FF00FF00FF00FF) * 6553601 >> 16;
+  return uint32_t((val & 0x0000FFFF0000FFFF) * 42949672960001 >> 32);
+}
+
+/** @private */
+static simdjson_inline uint32_t parse_eight_digits_unrolled(const uint8_t *chars) {
+  return parse_eight_digits_unrolled(reinterpret_cast<const char *>(chars));
+}
+
+#if SIMDJSON_IS_32BITS // _umul128 for x86, arm
+// this is a slow emulation routine for 32-bit
+//
+static simdjson_inline uint64_t __emulu(uint32_t x, uint32_t y) {
+  return x * (uint64_t)y;
+}
+static simdjson_inline uint64_t _umul128(uint64_t ab, uint64_t cd, uint64_t *hi) {
+  uint64_t ad = __emulu((uint32_t)(ab >> 32), (uint32_t)cd);
+  uint64_t bd = __emulu((uint32_t)ab, (uint32_t)cd);
+  uint64_t adbc = ad + __emulu((uint32_t)ab, (uint32_t)(cd >> 32));
+  uint64_t adbc_carry = !!(adbc < ad);
+  uint64_t lo = bd + (adbc << 32);
+  *hi = __emulu((uint32_t)(ab >> 32), (uint32_t)(cd >> 32)) + (adbc >> 32) +
+        (adbc_carry << 32) + !!(lo < bd);
+  return lo;
+}
+#endif
+
+/** @private */
+simdjson_inline internal::value128 full_multiplication(uint64_t value1, uint64_t value2) {
+  internal::value128 answer;
+#if SIMDJSON_REGULAR_VISUAL_STUDIO || SIMDJSON_IS_32BITS
+#ifdef _M_ARM64
+  // ARM64 has native support for 64-bit multiplications, no need to emultate
+  answer.high = __umulh(value1, value2);
+  answer.low = value1 * value2;
+#else
+  answer.low = _umul128(value1, value2, &answer.high); // _umul128 not available on ARM64
+#endif // _M_ARM64
+#else // SIMDJSON_REGULAR_VISUAL_STUDIO || SIMDJSON_IS_32BITS
+  __uint128_t r = (static_cast<__uint128_t>(value1)) * value2;
+  answer.low = uint64_t(r);
+  answer.high = uint64_t(r >> 64);
+#endif
+  return answer;
+}
+
+} // namespace numberparsing
+} // namespace fallback
+} // namespace simdjson
+
+#define SIMDJSON_SWAR_NUMBER_PARSING 1
+
+#endif // SIMDJSON_FALLBACK_NUMBERPARSING_DEFS_H
@@ -0,0 +1,8 @@
+#ifndef SIMDJSON_FALLBACK_ONDEMAND_H
+#define SIMDJSON_FALLBACK_ONDEMAND_H
+
+#include "simdjson/fallback/begin.h"
+#include "simdjson/generic/ondemand/amalgamated.h"
+#include "simdjson/fallback/end.h"
+
+#endif // SIMDJSON_FALLBACK_ONDEMAND_H
@@ -1,10 +1,12 @@
-#ifndef SIMDJSON_FALLBACK_STRINGPARSING_H
-#define SIMDJSON_FALLBACK_STRINGPARSING_H
+#ifndef SIMDJSON_FALLBACK_STRINGPARSING_DEFS_H
+#define SIMDJSON_FALLBACK_STRINGPARSING_DEFS_H

-#include "simdjson/base.h"
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#include "simdjson/fallback/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
+namespace fallback {
 namespace {

 // Holds backslashes and quotes locations.
@@ -28,7 +30,7 @@ simdjson_inline backslash_and_quote backslash_and_quote::copy_and_find(const uin
 }

 } // unnamed namespace
-} // namespace SIMDJSON_IMPLEMENTATION
+} // namespace fallback
 } // namespace simdjson

-#endif // SIMDJSON_FALLBACK_STRINGPARSING_H
+#endif // SIMDJSON_FALLBACK_STRINGPARSING_DEFS_H
@@ -0,0 +1,12 @@
+#if defined(SIMDJSON_CONDITIONAL_INCLUDE) && !defined(SIMDJSON_GENERIC_DEPENDENCIES_H)
+#error simdjson/generic/dependencies.h must be included before simdjson/generic/amalgamated.h!
+#endif
+
+#include "simdjson/generic/base.h"
+#include "simdjson/generic/jsoncharutils.h"
+#include "simdjson/generic/atomparsing.h"
+#include "simdjson/generic/dom_parser_implementation.h"
+#include "simdjson/generic/implementation_simdjson_result_base.h"
+#include "simdjson/generic/numberparsing.h"
+
+#include "simdjson/generic/implementation_simdjson_result_base-inl.h"
@@ -1,3 +1,13 @@
+#ifndef SIMDJSON_GENERIC_ATOMPARSING_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_ATOMPARSING_H
+#include "simdjson/generic/base.h"
+#include "simdjson/generic/jsoncharutils.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+#include <cstring>
+
 namespace simdjson {
 namespace SIMDJSON_IMPLEMENTATION {
 namespace {
@@ -63,3 +73,5 @@ simdjson_inline bool is_valid_null_atom(const uint8_t *src, size_t len) {
 } // unnamed namespace
 } // namespace SIMDJSON_IMPLEMENTATION
 } // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_ATOMPARSING_H
@@ -0,0 +1,46 @@
+#ifndef SIMDJSON_GENERIC_BASE_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_BASE_H
+#include "simdjson/base.h"
+// If we haven't got an implementation yet, we're in the editor, editing a generic file! Just
+// use the most advanced one we can so the most possible stuff can be tested.
+#ifndef SIMDJSON_IMPLEMENTATION
+#include "simdjson/implementation_detection.h"
+#if SIMDJSON_IMPLEMENTATION_ICELAKE
+#include "simdjson/icelake/begin.h"
+#elif SIMDJSON_IMPLEMENTATION_HASWELL
+#include "simdjson/haswell/begin.h"
+#elif SIMDJSON_IMPLEMENTATION_WESTMERE
+#include "simdjson/westmere/begin.h"
+#elif SIMDJSON_IMPLEMENTATION_ARM64
+#include "simdjson/arm64/begin.h"
+#elif SIMDJSON_IMPLEMENTATION_PPC64
+#include "simdjson/ppc64/begin.h"
+#elif SIMDJSON_IMPLEMENTATION_FALLBACK
+#include "simdjson/fallback/begin.h"
+#else
+#error "All possible implementations (including fallback) have been disabled! simdjson will not run."
+#endif
+#endif // SIMDJSON_IMPLEMENTATION
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
+namespace simdjson {
+namespace SIMDJSON_IMPLEMENTATION {
+
+struct open_container;
+class dom_parser_implementation;
+
+/**
+ * The type of a JSON number
+ */
+enum class number_type {
+    floating_point_number=1, /// a binary64 number
+    signed_integer,          /// a signed integer that fits in a 64-bit word using two's complement
+    unsigned_integer         /// a positive integer larger or equal to 1<<63
+};
+
+} // namespace SIMDJSON_IMPLEMENTATION
+} // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_BASE_H
@@ -0,0 +1,20 @@
+#ifdef SIMDJSON_CONDITIONAL_INCLUDE
+#error simdjson/generic/dependencies.h must be included before defining SIMDJSON_CONDITIONAL_INCLUDE!
+#endif
+
+#ifndef SIMDJSON_GENERIC_DEPENDENCIES_H
+#define SIMDJSON_GENERIC_DEPENDENCIES_H
+
+// Internal headers needed for generics.
+// All includes referencing simdjson headers *not* under simdjson/generic must be here!
+// Otherwise, amalgamation will fail.
+#include "simdjson/base.h"
+#include "simdjson/implementation.h"
+#include "simdjson/implementation_detection.h"
+#include "simdjson/internal/instruction_set.h"
+#include "simdjson/internal/dom_parser_implementation.h"
+#include "simdjson/internal/jsoncharutils_tables.h"
+#include "simdjson/internal/numberparsing_tables.h"
+#include "simdjson/internal/simdprune_tables.h"
+
+#endif // SIMDJSON_GENERIC_DEPENDENCIES_H
@@ -1,5 +1,10 @@
-#include "simdjson/base.h"
-#include "simdjson/internal/isadetection.h"
+#ifndef SIMDJSON_GENERIC_DOM_PARSER_IMPLEMENTATION_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_DOM_PARSER_IMPLEMENTATION_H
+#include "simdjson/generic/base.h"
+#include "simdjson/internal/dom_parser_implementation.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
 namespace SIMDJSON_IMPLEMENTATION {
@@ -80,3 +85,5 @@ inline simdjson_warn_unused error_code dom_parser_implementation::set_max_depth(

 } // namespace SIMDJSON_IMPLEMENTATION
 } // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_DOM_PARSER_IMPLEMENTATION_H
@@ -1,3 +1,11 @@
+#ifndef SIMDJSON_GENERIC_IMPLEMENTATION_SIMDJSON_RESULT_BASE_INL_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_IMPLEMENTATION_SIMDJSON_RESULT_BASE_INL_H
+#include "simdjson/generic/base.h"
+#include "simdjson/generic/implementation_simdjson_result_base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 namespace simdjson {
 namespace SIMDJSON_IMPLEMENTATION {

@@ -78,3 +86,5 @@ simdjson_inline implementation_simdjson_result_base<T>::implementation_simdjson_

 } // namespace SIMDJSON_IMPLEMENTATION
 } // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_IMPLEMENTATION_SIMDJSON_RESULT_BASE_INL_H
@@ -1,3 +1,10 @@
+#ifndef SIMDJSON_GENERIC_IMPLEMENTATION_SIMDJSON_RESULT_BASE_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_IMPLEMENTATION_SIMDJSON_RESULT_BASE_H
+#include "simdjson/generic/base.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 namespace simdjson {
 namespace SIMDJSON_IMPLEMENTATION {

@@ -123,3 +130,5 @@ protected:

 } // namespace SIMDJSON_IMPLEMENTATION
 } // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_IMPLEMENTATION_SIMDJSON_RESULT_BASE_H
@@ -1,5 +1,11 @@
+#ifndef SIMDJSON_GENERIC_JSONCHARUTILS_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_JSONCHARUTILS_H
+#include "simdjson/generic/base.h"
 #include "simdjson/internal/jsoncharutils_tables.h"
 #include "simdjson/internal/numberparsing_tables.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE

 namespace simdjson {
 namespace SIMDJSON_IMPLEMENTATION {
@@ -90,27 +96,9 @@ static simdjson_inline uint64_t _umul128(uint64_t ab, uint64_t cd, uint64_t *hi)
 }
 #endif

-using internal::value128;
-
-simdjson_inline value128 full_multiplication(uint64_t value1, uint64_t value2) {
-  value128 answer;
-#if SIMDJSON_REGULAR_VISUAL_STUDIO || SIMDJSON_IS_32BITS
-#ifdef _M_ARM64
-  // ARM64 has native support for 64-bit multiplications, no need to emultate
-  answer.high = __umulh(value1, value2);
-  answer.low = value1 * value2;
-#else
-  answer.low = _umul128(value1, value2, &answer.high); // _umul128 not available on ARM64
-#endif // _M_ARM64
-#else // SIMDJSON_REGULAR_VISUAL_STUDIO || SIMDJSON_IS_32BITS
-  __uint128_t r = (static_cast<__uint128_t>(value1)) * value2;
-  answer.low = uint64_t(r);
-  answer.high = uint64_t(r >> 64);
-#endif
-  return answer;
-}
-
 } // namespace jsoncharutils
 } // unnamed namespace
 } // namespace SIMDJSON_IMPLEMENTATION
 } // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_JSONCHARUTILS_H
@@ -1,20 +1,20 @@
+#ifndef SIMDJSON_GENERIC_NUMBERPARSING_H
+
+#ifndef SIMDJSON_CONDITIONAL_INCLUDE
+#define SIMDJSON_GENERIC_NUMBERPARSING_H
+#include "simdjson/generic/base.h"
+#include "simdjson/generic/jsoncharutils.h"
 #include "simdjson/internal/numberparsing_tables.h"
+#endif // SIMDJSON_CONDITIONAL_INCLUDE
+
 #include <limits>
+#include <ostream>
+#include <cstring>

 namespace simdjson {
 namespace SIMDJSON_IMPLEMENTATION {
-/// @private
 namespace numberparsing {

-/**
- * The type of a JSON number
- */
-enum class number_type {
-    floating_point_number=1, /// a binary64 number
-    signed_integer,          /// a signed integer that fits in a 64-bit word using two's complement
-    unsigned_integer         /// a positive integer larger or equal to 1<<63
-};
-
 #ifdef JSON_TEST_NUMBERS
 #define INVALID_NUMBER(SRC) (found_invalid_number((SRC)), NUMBER_ERROR)
 #define WRITE_INTEGER(VALUE, SRC, WRITER) (found_integer((VALUE), (SRC)), (WRITER).append_s64((VALUE)))
@@ -165,7 +165,7 @@ simdjson_inline bool compute_float_64(int64_t power, uint64_t i, bool negative,
  // with a returned value of type value128 with a "low component" corresponding to the
  // 64-bit least significant bits of the product and with a "high component" corresponding
  // to the 64-bit most significant bits of the product.
-  simdjson::internal::value128 firstproduct = jsoncharutils::full_multiplication(i, simdjson::internal::power_of_five_128[index]);
+  simdjson::internal::value128 firstproduct = full_multiplication(i, simdjson::internal::power_of_five_128[index]);
  // Both i and power_of_five_128[index] have their most significant bit set to 1 which
  // implies that the either the most or the second most significant bit of the product
  // is 1. We pack values in this manner for efficiency reasons: it maximizes the use
@@ -186,8 +186,7 @@ simdjson_inline bool compute_float_64(int64_t power, uint64_t i, bool negative,
    // power_of_five_128[index]. Usually, that's good enough to approximate i * 5^q
    // to the desired approximation using one multiplication. Sometimes it does not suffice.
    // Then we store the next most significant 64 bits in power_of_five_128[index + 1], and
-    // then we get a better approximation to i * 5^q. In very rare cases, even that
-    // will not suffice, though it is seemingly very hard to find such a scenario.
+    // then we get a better approximation to i * 5^q.
    //
    // That's for when q>=0. The logic for q<0 is somewhat similar but it is somewhat
    // more complicated.
@@ -199,15 +198,12 @@ simdjson_inline bool compute_float_64(int64_t power, uint64_t i, bool negative,
    // with a returned value of type value128 with a "low component" corresponding to the
    // 64-bit least significant bits of the product and with a "high component" corresponding
    // to the 64-bit most significant bits of the product.
-    simdjson::internal::value128 secondproduct = jsoncharutils::full_multiplication(i, simdjson::internal::power_of_five_128[index + 1]);
+    simdjson::internal::value128 secondproduct = full_multiplication(i, simdjson::internal::power_of_five_128[index + 1]);
    firstproduct.low += secondproduct.high;
    if(secondproduct.high > firstproduct.low) { firstproduct.high++; }
-    // At this point, we might need to add at most one to firstproduct, but this
-    // can only change the value of firstproduct.high if firstproduct.low is maximal.
-    if(simdjson_unlikely(firstproduct.low  == 0xFFFFFFFFFFFFFFFF)) {
-      // This is very unlikely, but if so, we need to do much more work!
-      return false;
-    }
+    // As it has been proven by Noble Mushtak and Daniel Lemire in "Fast Number Parsing Without
+    // Fallback" (https://arxiv.org/abs/2212.06644), at this point we are sure that the product
+    // is sufficiently accurate, and more computation is not needed.
  }
  uint64_t lower = firstproduct.low;
  uint64_t upper = firstproduct.high;
@@ -1250,6 +1246,8 @@ simdjson_unused simdjson_inline simdjson_result<double> parse_double_in_string(c
 } // unnamed namespace
 #endif // SIMDJSON_SKIPNUMBERPARSING

+} // namespace numberparsing
+
 inline std::ostream& operator<<(std::ostream& out, number_type type) noexcept {
    switch (type) {
        case number_type::signed_integer: out << "integer in [-9223372036854775808,9223372036854775808)"; break;
@@ -1260,6 +1258,7 @@ inline std::ostream& operator<<(std::ostream& out, number_type type) noexcept {
    return out;
 }

-} // namespace numberparsing
 } // namespace SIMDJSON_IMPLEMENTATION
 } // namespace simdjson
+
+#endif // SIMDJSON_GENERIC_NUMBERPARSING_H
@@ -1,16 +0,0 @@
-#include "simdjson/generic/ondemand/json_type-inl.h"
-#include "simdjson/generic/ondemand/logger-inl.h"
-#include "simdjson/generic/ondemand/raw_json_string-inl.h"
-#include "simdjson/generic/ondemand/token_iterator-inl.h"
-#include "simdjson/generic/ondemand/json_iterator-inl.h"
-#include "simdjson/generic/ondemand/value_iterator-inl.h"
-#include "simdjson/generic/ondemand/array_iterator-inl.h"
-#include "simdjson/generic/ondemand/object_iterator-inl.h"
-#include "simdjson/generic/ondemand/array-inl.h"
-#include "simdjson/generic/ondemand/document-inl.h"
-#include "simdjson/generic/ondemand/value-inl.h"
-#include "simdjson/generic/ondemand/field-inl.h"
-#include "simdjson/generic/ondemand/object-inl.h"
-#include "simdjson/generic/ondemand/parser-inl.h"
-#include "simdjson/generic/ondemand/document_stream-inl.h"
-#include "simdjson/generic/ondemand/serialization-inl.h"
@@ -1,36 +0,0 @@
-namespace simdjson {
-namespace SIMDJSON_IMPLEMENTATION {
-/**
- * A fast, simple, DOM-like interface that parses JSON as you use it.
- *
- * Designed for maximum speed and a lower memory profile.
- */
-namespace ondemand {
-
-/** Represents the depth of a JSON value (number of nested arrays/objects). */
-using depth_t = int32_t;
-
-/** @copydoc simdjson::SIMDJSON_IMPLEMENTATION::numberparsing::number_type */
-using number_type = simdjson::SIMDJSON_IMPLEMENTATION::numberparsing::number_type;
-
-} // namespace ondemand
-} // namespace SIMDJSON_IMPLEMENTATION
-} // namespace simdjson
-
-#include "simdjson/generic/ondemand/json_type.h"
-#include "simdjson/generic/ondemand/token_position.h"
-#include "simdjson/generic/ondemand/logger.h"
-#include "simdjson/generic/ondemand/raw_json_string.h"
-#include "simdjson/generic/ondemand/token_iterator.h"
-#include "simdjson/generic/ondemand/json_iterator.h"
-#include "simdjson/generic/ondemand/value_iterator.h"
-#include "simdjson/generic/ondemand/array_iterator.h"
-#include "simdjson/generic/ondemand/object_iterator.h"
-#include "simdjson/generic/ondemand/array.h"
-#include "simdjson/generic/ondemand/document.h"
-#include "simdjson/generic/ondemand/value.h"
-#include "simdjson/generic/ondemand/field.h"
-#include "simdjson/generic/ondemand/object.h"
-#include "simdjson/generic/ondemand/parser.h"
-#include "simdjson/generic/ondemand/document_stream.h"
-#include "simdjson/generic/ondemand/serialization.h"
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
John Keiser	d1de135ebd	Updates	2023-09-13 11:57:43 -04:00
John Keiser	0edf1008c9	Updates	2023-09-13 11:21:58 -04:00
John Keiser	6a0d7bcb55	Right-justify int columns in markdown tables	2023-09-13 11:10:01 -04:00
John Keiser	6f196d0af0	Add results	2023-09-12 15:02:17 -04:00
John Keiser	c3414a1234	Show more transitions	2023-09-12 14:58:33 -04:00
John Keiser	5d2107b470	Add jsonexamples generator and miss reduction script	2023-09-12 14:36:11 -04:00
John Keiser	90aa198430	Moar jsonexamples	2023-09-12 14:36:11 -04:00
John Keiser	02b488b88d	All branch tests for twitter.json	2023-09-12 14:36:10 -04:00
John Keiser	47e3e95867	Branch tests with twitter.json.	2023-09-12 14:36:10 -04:00
Daniel Lemire	588c0677f5	Redesigning visit_primitive so that it is optimized for strings and numbers.	2023-09-05 11:05:15 -04:00
Daniel Lemire	68ba9a1b2a	Adding a few spaces (minor formatting) (#2059 )	2023-08-30 20:23:58 -04:00
Daniel Lemire	6fed6bd29b	Update README.md	2023-08-30 19:26:34 -04:00
Piotr Rżysko	cc24bb4114	Removed number parsing fallback (#2056 )	2023-08-29 18:28:21 -04:00
John Keiser	14ec0ca8f4	Merge pull request #2052 from simdjson/jkeiser/structural_indexer_step Write out 4 structural indexes at a time instead of 8	2023-08-29 15:24:54 -07:00
John Keiser	8dabd02c3a	Default to step = 4	2023-08-29 15:48:52 -04:00
Daniel Lemire	c9692005fd	This allows us to pass SIMDJSON_STRUCTURAL_INDEXER_STEP as a command-line option. (#2055 )	2023-08-25 13:47:20 -04:00
Daniel Lemire	2e0a9397c9	Merge branch 'master' into jkeiser/structural_indexer_step	2023-08-25 11:34:13 -04:00
Daniel Lemire	f7922075c3	Adding support for Apple event counters (#2054 )	2023-08-25 11:33:24 -04:00
Daniel Lemire	b2e20e04c9	Standard compatibility fixes (#2053 ) * Standard compatibility fixes * missing commit * Should work. * Fix. * Fix. * Should work now. --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me>	2023-08-25 10:18:02 -04:00
John Keiser	adc9d18efd	Change step for structural_indexes to 2	2023-08-23 19:02:05 -07:00
Daniel Lemire	bf849e3619	Preparing release	2023-08-22 14:05:14 -04:00
Daniel Lemire	c09e8dc225	Icelake processes 64 bytes at a time when processing strings. (#2050 ) Co-authored-by: Daniel Lemire <dlemire@lemire.me>	2023-08-18 14:37:10 -04:00
Daniel Lemire	5bf585ef02	Fixing failing thread sanitizer (#2051 )	2023-08-18 14:35:12 -04:00
Daniel Lemire	caff09cafc	Update iterate_many.md	2023-08-14 12:18:09 -04:00
Daniel Lemire	f043db4dca	Preparing release.	2023-08-02 16:01:39 -04:00
Daniel Lemire	e3bd0828d1	Fix for issue 2045. (#2046 ) * Fix for issue 2045. * Retauring single-header code.	2023-08-02 16:00:45 -04:00
John Keiser	9408298e0e	Merge pull request #2042 from simdjson/jkeiser/escape3 New escape algorithm with significantly less data dependency	2023-08-02 10:38:09 -07:00
John Keiser	573bbac2d2	Reamalgamate again	2023-08-02 08:28:35 -07:00
John Keiser	ac78c625df	simdjson_inline -> simdjson_really_inline	2023-08-02 08:28:05 -07:00
John Keiser	390a66c6e8	Make next_structurals inline	2023-08-02 08:28:05 -07:00
John Keiser	210e0a56a1	Reamalgamate	2023-08-02 08:28:05 -07:00
John Keiser	084f662f32	Don't use C++20 designated initializers	2023-08-02 08:28:05 -07:00
John Keiser	c878ff2500	Fix compiler warnings	2023-08-02 08:28:05 -07:00
John Keiser	06afe9ecb3	New escape algorithm with significantly less data dependency	2023-08-02 08:28:05 -07:00
Daniel Lemire	e658502adb	We are having issues with llvm.sh (#2044 )	2023-08-02 11:24:28 -04:00
Daniel Lemire	6e7d415040	These lines should not be necessary. (#2043 )	2023-08-01 19:33:27 -04:00
John Keiser	2ccd5b9c97	Merge pull request #2039 from simdjson/jkeiser/more-headers Disallow including headers from implementation files	2023-07-21 12:09:57 -07:00
Daniel Lemire	1cdcbf79b3	We do actually support legacy 32-bit systems, although not with fast kernels. (#2041 )	2023-07-21 13:53:10 -04:00
Calum McConnell	7013ed98cc	Peek(0) is an error on 32-bit systems, where pointers are indistinguishable from int32_t (#2040 )	2023-07-21 13:52:53 -04:00
John Keiser	b383c717d7	Have amalgamate.py verify it's actually amalgamating all the files	2023-07-20 15:11:52 -07:00
John Keiser	b8743bced0	Fix amalgamate.py under python 3.7	2023-07-20 14:14:44 -07:00
John Keiser	ae9ba2bed2	Rename SIMDJSON_AMALGAMATED to SIMDJSON_CONDITIONAL_INCLUDE	2023-07-20 11:43:19 -07:00
Daniel Lemire	939844d79f	Adding tests for issue 2037. (#2038 ) * Adding tests for issue 2037. * Fixed typo * Simplifying.	2023-07-20 14:11:33 -04:00
John Keiser	4f4e81668e	Change all include paths to <> instead of "" to avoid relative path includes	2023-07-20 10:48:25 -07:00
John Keiser	bb54946b78	Disallow including headers from implementation files	2023-07-20 10:01:22 -07:00
John Keiser	4dec6ed5a7	Merge pull request #2036 from simdjson/jkeiser/more-header-fixes A few more header fixes	2023-07-18 10:26:40 -07:00
John Keiser	1dfd48195d	Merge pull request #2035 from simdjson/jkeiser/sanitizer-ci Run sanitizers in parallel	2023-07-17 23:37:34 -07:00
John Keiser	23afc6b695	Merge branch 'master' into jkeiser/sanitizer-ci	2023-07-17 16:09:20 -07:00
John Keiser	e3cac71afe	Enforce that dependencies.h and generic/*.h include the same dependencies.	2023-07-17 16:07:42 -07:00
John Keiser	cbe562c1d5	Fix header define names for inline files	2023-07-17 13:55:07 -07:00
John Keiser	30437d0ff6	A few header fixes for dom and ondemand	2023-07-17 13:53:50 -07:00
John Keiser	1e97af7dfb	Merge pull request #2031 from simdjson/jkeiser/vscode-green Get headers compiling in editor	2023-07-17 13:07:20 -07:00
John Keiser	80bc9e5051	Run sanitizers in parallel	2023-07-17 12:57:50 -07:00
John Keiser	a74e87df17	Merge pull request #2026 from Cuda-Chen/pretty-print-for-dom Add pretty print for DOM	2023-07-17 12:38:20 -07:00
John Keiser	3274299647	Merge pull request #2030 from yongxiangng/lazy-log-error Add info and error logging, defer log message evaluation	2023-07-17 12:38:05 -07:00
John Keiser	22ca25faa6	Reamalgamate; make amalgamation normalize paths	2023-07-17 11:27:43 -07:00
John Keiser	37399fd41b	Merge pull request #2034 from simdjson/add_add_emul_for_32bits Adding missing umul	2023-07-17 10:02:55 -07:00
Daniel Lemire	a170145d5b	Missing header.	2023-07-17 11:48:07 -04:00
Daniel Lemire	7d1d635883	Adding missing umul	2023-07-17 10:04:57 -04:00
John Keiser	ddfa8ae928	Reamalgamate	2023-07-16 22:53:42 -07:00
John Keiser	25dfb078c7	A few random header include ordering fixes	2023-07-16 22:50:54 -07:00
John Keiser	9f0710e3ad	Move full_multiplication to <implementation>/numberparsing_defs.h	2023-07-16 22:34:21 -07:00
John Keiser	19b9cd9b05	Explain new directory structure	2023-07-16 21:42:50 -07:00
John Keiser	7540248396	Include simdprune_tables.h in generic implementations	2023-07-16 19:19:10 -07:00
John Keiser	ab09e96de9	Only use amalgamated.h in generic/	2023-07-16 19:18:58 -07:00
John Keiser	66252e4925	Honor --output-on-failure again	2023-07-15 17:44:44 -07:00
John Keiser	944778c186	Reamalgamate	2023-07-15 17:30:42 -07:00
John Keiser	62868a2f53	Catch more headers without which it won't compile	2023-07-15 17:16:18 -07:00
John Keiser	3c446c551b	Move AMALGAMATED up to include implementation.cpp	2023-07-15 17:04:14 -07:00
John Keiser	efe3d4b7d7	Few more missing headers, make it even stricter	2023-07-15 16:55:27 -07:00
John Keiser	28db262b47	Make SIMDJSON_AMALGAMATED checking much stricter	2023-07-15 16:46:15 -07:00
John Keiser	bf076bfb5f	Begin amalgamation early in simdjson.cpp	2023-07-15 15:55:38 -07:00
John Keiser	7bf1ecff9e	Mark amalgamation early in	2023-07-15 15:53:09 -07:00
John Keiser	e25a40facf	Check if amalgamation rules are not being followed while compiling	2023-07-15 15:47:14 -07:00
John Keiser	a442842f0d	Use SIMDJSON_AMALGAMATED (which works in-editor and out) instead of editor-specific tweaks	2023-07-15 14:53:02 -07:00
John Keiser	421ed4f390	Try to make output on failure actually work	2023-07-14 20:57:24 -07:00
John Keiser	7dff5fa37c	Include intrin.h from jsoncharutils.h since it's needed	2023-07-14 20:53:47 -07:00
John Keiser	9a0527c380	Only make editor stuff work in editor	2023-07-14 20:05:05 -07:00
John Keiser	b160cf0610	Make simdjson.cpp compile on VSCode	2023-07-08 11:53:58 -07:00
John Keiser	553ee69b7c	Make implementation .h/.cpp compile	2023-07-08 11:42:57 -07:00
John Keiser	bf70cd0ebf	Get headers compiling in editor	2023-07-08 10:47:15 -07:00
Cuda-Chen	cf0551edf5	Add pretty print for DOM Add pretty print for DOM with documentation. Currently, the indentation is fixed to four spaces. Close #1329.	2023-07-08 21:09:57 +08:00
yongxiangng	ddb094d550	Move should log into separate if clause	2023-07-08 13:07:25 +08:00
Yong Xiang Ng	975fdda03c	Add fine grain logging	2023-07-07 12:11:27 +08:00