mirror of
https://github.com/simdjson/simdjson
synced 2026-06-08 17:27:07 +00:00
lemire
2590 lines
276 KiB
HTML
2590 lines
276 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
|
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
|
|
<meta name="generator" content="Doxygen 1.9.8"/>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
|
<title>simdjson: The Basics</title>
|
|
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="jquery.js"></script>
|
|
<script type="text/javascript" src="dynsections.js"></script>
|
|
<link href="navtree.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="resize.js"></script>
|
|
<script type="text/javascript" src="navtreedata.js"></script>
|
|
<script type="text/javascript" src="navtree.js"></script>
|
|
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="search/searchdata.js"></script>
|
|
<script type="text/javascript" src="search/search.js"></script>
|
|
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
|
<link href="doxygen-awesome.css" rel="stylesheet" type="text/css"/>
|
|
<link href="doxygen-awesome-sidebar-only.css" rel="stylesheet" type="text/css"/>
|
|
<link href="doxygen-awesome-sidebar-only-darkmode-toggle.css" rel="stylesheet" type="text/css"/>
|
|
</head>
|
|
<body>
|
|
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
|
<div id="titlearea">
|
|
<table cellspacing="0" cellpadding="0">
|
|
<tbody>
|
|
<tr id="projectrow">
|
|
<td id="projectlogo"><img alt="Logo" src="logotiny.png"/></td>
|
|
<td id="projectalign">
|
|
<div id="projectname">simdjson<span id="projectnumber"> 4.6.4</span>
|
|
</div>
|
|
<div id="projectbrief">Ridiculously Fast JSON</div>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<!-- end header part -->
|
|
<!-- Generated by Doxygen 1.9.8 -->
|
|
<script type="text/javascript">
|
|
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
|
|
var searchBox = new SearchBox("searchBox", "search/",'.html');
|
|
/* @license-end */
|
|
</script>
|
|
<script type="text/javascript" src="menudata.js"></script>
|
|
<script type="text/javascript" src="menu.js"></script>
|
|
<script type="text/javascript">
|
|
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
|
|
$(function() {
|
|
initMenu('',true,false,'search.php','Search');
|
|
$(document).ready(function() { init_search(); });
|
|
});
|
|
/* @license-end */
|
|
</script>
|
|
<div id="main-nav"></div>
|
|
</div><!-- top -->
|
|
<div id="side-nav" class="ui-resizable side-nav-resizable">
|
|
<div id="nav-tree">
|
|
<div id="nav-tree-contents">
|
|
<div id="nav-sync" class="sync"></div>
|
|
</div>
|
|
</div>
|
|
<div id="splitbar" style="-moz-user-select:none;"
|
|
class="ui-resizable-handle">
|
|
</div>
|
|
</div>
|
|
<script type="text/javascript">
|
|
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
|
|
$(document).ready(function(){initNavTree('md_doc_2basics.html',''); initResizable(); });
|
|
/* @license-end */
|
|
</script>
|
|
<div id="doc-content">
|
|
<!-- window showing the filter options -->
|
|
<div id="MSearchSelectWindow"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
|
</div>
|
|
|
|
<!-- iframe showing the search results (closed by default) -->
|
|
<div id="MSearchResultsWindow">
|
|
<div id="MSearchResults">
|
|
<div class="SRPage">
|
|
<div id="SRIndex">
|
|
<div id="SRResults"></div>
|
|
<div class="SRStatus" id="Loading">Loading...</div>
|
|
<div class="SRStatus" id="Searching">Searching...</div>
|
|
<div class="SRStatus" id="NoMatches">No Matches</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div><div class="header">
|
|
<div class="headertitle"><div class="title">The Basics</div></div>
|
|
</div><!--header-->
|
|
<div class="contents">
|
|
<div class="textblock"><p>An overview of what you need to know to use simdjson to parse JSON documents, with examples. <a href="https://github.com/simdjson/simdjson/blob/master/doc/builder.md">Our documentation regarding the generation (serialization) of JSON documents is in a separate document</a>.</p>
|
|
<ul>
|
|
<li>Requirements</li>
|
|
<li>Including simdjson</li>
|
|
<li>Using simdjson with package managers</li>
|
|
<li>Using simdjson as a CMake dependency</li>
|
|
<li>Versions</li>
|
|
<li>The basics: loading and parsing JSON documents</li>
|
|
<li>Documents are iterators<ul>
|
|
<li>Parser, document and JSON scope</li>
|
|
</ul>
|
|
</li>
|
|
<li>string_view</li>
|
|
<li>Avoiding pitfalls: enable development checks</li>
|
|
<li>Using the parsed JSON<ul>
|
|
<li>Using the parsed JSON: additional examples</li>
|
|
</ul>
|
|
</li>
|
|
<li>Adding support for custom types<ul>
|
|
<li>1. Specialize `simdjson::ondemand::value::get` to get custom types (pre-C++20)</li>
|
|
<li>2. Use `tag_invoke` for custom types (C++20)</li>
|
|
<li>3. Using static reflection (C++26)<ul>
|
|
<li>Special cases</li>
|
|
</ul>
|
|
</li>
|
|
<li>The simdjson::from shortcut (experimental, C++20)</li>
|
|
</ul>
|
|
</li>
|
|
<li>Minifying JSON strings without parsing</li>
|
|
<li>UTF-8 validation (alone)</li>
|
|
<li>JSON Pointer</li>
|
|
<li>JSONPath<ul>
|
|
<li>Using `for_each_at_path_with_wildcard` for JSONPath Queries (On-Demand)<ul>
|
|
<li>Example Usage</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li>Compile-Time JSONPath and JSON Pointer (C++26 Reflection)</li>
|
|
<li>Error handling<ul>
|
|
<li>Error handling examples without exceptions</li>
|
|
<li>Disabling exceptions</li>
|
|
<li>Exceptions</li>
|
|
<li>Current location in document</li>
|
|
<li>Checking for trailing content</li>
|
|
</ul>
|
|
</li>
|
|
<li>Rewinding</li>
|
|
<li>Newline-Delimited JSON (ndjson) and JSON lines</li>
|
|
<li>Parsing numbers inside strings</li>
|
|
<li>Dynamic Number Types</li>
|
|
<li>Raw strings from keys</li>
|
|
<li>General direct access to the raw JSON string<ul>
|
|
<li>Raw JSON string for objects and arrays</li>
|
|
</ul>
|
|
</li>
|
|
<li>Storing directly into an existing string instance</li>
|
|
<li>Thread safety</li>
|
|
<li>Standard compliance</li>
|
|
<li>Backwards compatibility</li>
|
|
<li>Examples</li>
|
|
<li>Performance tips</li>
|
|
<li>Further reading</li>
|
|
</ul>
|
|
<h1><a class="anchor" id="autotoc_md0"></a>
|
|
Requirements</h1>
|
|
<p>The simdjson library is widely deployed in popular systems such as the Node.js runtime environment.</p>
|
|
<ul>
|
|
<li>A recent compiler (LLVM clang 6 or better, GNU GCC 7.4 or better, Xcode 11 or better) on POSIX systems such as macOS, FreeBSD or Linux. We require that the compiler supports the C++11 standard or better. We test the library on a big-endian system (IBM s390x with Linux).</li>
|
|
<li>Visual Studio 2017 or better. We support the LLVM clang compiler under Visual Studio (clang-cl) as well as the regular Visual Studio compiler. For better release performance (both compile time and execution time), we recommend Visual Studio users adopt LLVM (clang-cl). We discourage using GCC under Windows: there <a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412">is a long-running bug with GCC under Windows</a>.</li>
|
|
</ul>
|
|
<p>Support for AVX-512 require a processor with AVX512-VBMI2 support (Ice Lake or better, AMD Zen 4 or better) under a 64-bit system and a recent compiler (LLVM clang 6 or better, GCC 8 or better, Visual Studio 2019 or better). You need a correspondingly recent assembler such as gas (2.30+) or nasm (2.14+): recent compilers usually come with recent assemblers. If you mix a recent compiler with an incompatible/old assembler (e.g., when using a recent compiler with an old Linux distribution), you may get errors at build time because the compiler produces instructions that the assembler does not recognize: you should update your assembler to match your compiler (e.g., upgrade binutils to version 2.30 or better under Linux) or use an older compiler matching the capabilities of your assembler.</p>
|
|
<h1><a class="anchor" id="autotoc_md1"></a>
|
|
Including simdjson</h1>
|
|
<p>To include simdjson, copy <a href="/singleheader/simdjson.h">simdjson.h</a> and <a href="/singleheader/simdjson.cpp">simdjson.cpp</a> into your project. Then include it in your project with:</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
<div class="line"><span class="keyword">using namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a>; <span class="comment">// optional</span></div>
|
|
<div class="ttc" id="anamespacesimdjson_html"><div class="ttname"><a href="namespacesimdjson.html">simdjson</a></div><div class="ttdoc">The top level simdjson namespace, containing everything the library provides.</div><div class="ttdef"><b>Definition</b> <a href="arm64_2base_8h_source.html#l00008">base.h:8</a></div></div>
|
|
<div class="ttc" id="asimdjson_8h_html"><div class="ttname"><a href="simdjson_8h.html">simdjson.h</a></div></div>
|
|
</div><!-- fragment --><p>Under most systems, you can compile with:</p>
|
|
<div class="fragment"><div class="line">c++ myproject.cpp simdjson.cpp</div>
|
|
</div><!-- fragment --><p>Note:</p><ul>
|
|
<li>We recommend that you use simdjson by copying the single-header <code><a class="el" href="simdjson_8h.html">simdjson.h</a></code> file along with the source file <code>simdjson.cpp</code> directly into your project, as they are part of <a href="https://github.com/simdjson/simdjson/releases">every release</a> as assets. In this manner, you only have to compile <code>simdjson.cpp</code> as any other source file: it works well in every development environment. However, you may also use simdjson as a git submodule (<a href="https://github.com/simdjson/cmakedemo">example</a>), using FetchContent (<a href="https://github.com/simdjson/cmake_demo_single_file">example</a>), with ExternalProject_Add (<a href="https://github.com/simdjson/cmakedemo_externalproject">example</a>) or with CPM (<a href="https://github.com/cpm-cmake/CPM.cmake/tree/master/examples/simdjson">example</a>).</li>
|
|
<li>Users on macOS and other platforms where default compilers do not provide C++11 compliant by default should request it with the appropriate flag (e.g., <code>c++ -std=c++11 myproject.cpp simdjson.cpp</code>).</li>
|
|
<li>The library relies on <a class="el" href="md_doc_2implementation-selection.html">runtime CPU detection</a>: avoid specifying an architecture at compile time (e.g., <code>-march-native</code>) if you want your binaries to run everywhere.</li>
|
|
</ul>
|
|
<h1><a class="anchor" id="autotoc_md2"></a>
|
|
Using simdjson with package managers</h1>
|
|
<p>You can install the simdjson library on your system or in your project using multiple package managers such as MSYS2, the conan package manager, vcpkg, brew, the apt package manager (debian-based Linux systems), the FreeBSD package manager (FreeBSD), and so on. E.g., <a href="https://github.com/simdjson/simdjson-vcpkg">we provide a complete example with vcpkg</a> that works under Windows. <a href="https://github.com/simdjson/simdjson/wiki/Installing-simdjson-with-a-package-manager">Visit our wiki for more details</a>.</p>
|
|
<p>The following Linux distributions provide simdjson packages: Alpine, RedHat, Rocky Linux, Debian, Fedora, and Ubuntu.</p>
|
|
<h1><a class="anchor" id="autotoc_md3"></a>
|
|
Using simdjson as a CMake dependency</h1>
|
|
<p>You can include the simdjson library as a CMake dependency by including the following lines in your <code>CMakeLists.txt</code>:</p>
|
|
<div class="fragment"><div class="line">include(FetchContent)</div>
|
|
<div class="line"> </div>
|
|
<div class="line">FetchContent_Declare(</div>
|
|
<div class="line"> simdjson</div>
|
|
<div class="line"> GIT_REPOSITORY https://github.com/simdjson/simdjson.git</div>
|
|
<div class="line"> GIT_TAG tags/v3.6.0</div>
|
|
<div class="line"> GIT_SHALLOW TRUE)</div>
|
|
<div class="line"> </div>
|
|
<div class="line">FetchContent_MakeAvailable(simdjson)</div>
|
|
</div><!-- fragment --><p>You should provide <code>GIT_TAG</code> with the release you need. If you omit <code>GIT_TAG ...</code>, you will work from the main branch of simdjson: we recommend that if you are working on production code, you always work from a release.</p>
|
|
<p>Elsewhere in your project, you can declare dependencies on simdjson with lines such as these:</p>
|
|
<div class="fragment"><div class="line">add_executable(myprogram myprogram.cpp)</div>
|
|
<div class="line">target_link_libraries(myprogram simdjson)</div>
|
|
</div><!-- fragment --><p>We recommend CMake version 3.15 or better.</p>
|
|
<p>See <a href="https://github.com/simdjson/cmake_demo_single_file">our CMake demonstration</a>. It works under Linux, FreeBSD, macOS and Windows (including Visual Studio).</p>
|
|
<p>The CMake build in simdjson can be tailored with a few variables. You can see the available variables and their default values by entering the <code>cmake -LA</code> command.</p>
|
|
<h1><a class="anchor" id="autotoc_md4"></a>
|
|
Versions</h1>
|
|
<p>Users are discouraged from building production code from the project's main branch. The main branch is used for development: it may contain new features but also additional bugs.</p>
|
|
<p>Users should pick a release. They should also access the documentation matching the release that they have chosen. Note that new features may be added over time.</p>
|
|
<p>Our releases are tagged using semantic versioning: the tags are made of three numbers prefixed by the letter <code>v</code> and separated by periods.</p>
|
|
<p>You can always find the latest release at the following hyperlink:</p>
|
|
<p><a href="https://github.com/simdjson/simdjson/releases/latest/">https://github.com/simdjson/simdjson/releases/latest/</a></p>
|
|
<p>The archive you download at this location contains its own corresponding documentation.</p>
|
|
<p>You can also choose to browse a specific version of the documentation and the code using GitHub, by appending the version number to the hyperlink, like so:</p>
|
|
<p><a href="https://github.com/simdjson/simdjson/blob/vx.y.z/doc/basics.md">https://github.com/simdjson/simdjson/blob/vx.y.z/doc/basics.md</a></p>
|
|
<p>where <code>x.y.z</code> should correspond to the version number you have chosen.</p>
|
|
<h1><a class="anchor" id="autotoc_md5"></a>
|
|
The basics: loading and parsing JSON documents</h1>
|
|
<p>The simdjson library allows you to navigate and validate JSON documents (<a href="https://www.tbray.org/ongoing/When/201x/2017/12/14/rfc8259.html">RFC 8259</a>). As required by the standard, your JSON document should be in a Unicode (UTF-8) string. The whole string, from the beginning to the end, needs to be valid: we do not attempt to tolerate bad inputs before or after a document.</p>
|
|
<p>For efficiency reasons, simdjson requires a string with a few bytes (<code><a class="el" href="namespacesimdjson.html#aecdd750132f0eb123a6d61113b4197bf" title="The amount of padding needed in a buffer to parse JSON.">simdjson::SIMDJSON_PADDING</a></code>) at the end, these bytes may be read but their content does not affect the parsing. In practice, it means that the JSON inputs should be stored in a memory region with <code><a class="el" href="namespacesimdjson.html#aecdd750132f0eb123a6d61113b4197bf" title="The amount of padding needed in a buffer to parse JSON.">simdjson::SIMDJSON_PADDING</a></code> extra bytes at the end. You do not have to set these bytes to specific values though you may want to if you want to avoid runtime warnings with some sanitizers. We expect the user of the library to load the data (from disk or from the network) into a padded buffer. To make this easy, we provide the <code>padded_string::load</code> function which loads files from disk in a padded buffer. <a href="https://github.com/simdjson/curltostring">You can similarly fetch a file from a URL to a padded string</a> using our <code><a class="el" href="classsimdjson_1_1padded__string__builder.html" title="Builder for constructing padded_string incrementally.">simdjson::padded_string_builder</a></code>. Advanced users may want to read the section Free Padding in <a class="el" href="md_doc_2performance.html">our performance notes</a>.</p>
|
|
<p>The simdjson library offers a tree-like <a href="https://en.wikipedia.org/wiki/API">API</a>, which you can access by creating a <code>ondemand::parser</code> and calling the <code>iterate()</code> method. The iterate method quickly indexes the input string and may detect some errors. The following example illustrates how to get started with an input JSON file (<code>"twitter.json"</code>):</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> json = padded_string::load(<span class="stringliteral">"twitter.json"</span>); <span class="comment">// load JSON file 'twitter.json'.</span></div>
|
|
<div class="line">ondemand::document doc = parser.iterate(json); <span class="comment">// position a pointer at the beginning of the JSON data</span></div>
|
|
</div><!-- fragment --><p>(Windows users compiling with C++17 or better may use <code>wchar_t</code> strings to support non-ASCII filenames: <code>padded_string::load(L"twitter.json")</code>.)</p>
|
|
<p>If you prefer not to create your own <code>ondemand::parser</code> instance, you can access a thread-local version by calling <code>ondemand::parser.get_parser()</code>.</p>
|
|
<div class="fragment"><div class="line">ondemand::document doc = ondemand::parser.get_parser().iterate(json);</div>
|
|
</div><!-- fragment --><p>However, you should be careful because a parser instance can only be used for one document at a time, thus it is only applicable when you are only parsing one document per thread at any one time.</p>
|
|
<p>You can also create a padded string—and call <code>iterate()</code>:</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> json = <span class="stringliteral">"[1,2,3]"</span>_padded; <span class="comment">// The _padded suffix creates a simdjson::padded_string instance</span></div>
|
|
<div class="line">ondemand::document doc = parser.iterate(json); <span class="comment">// parse a string</span></div>
|
|
</div><!-- fragment --><p>If you have a buffer of your own with enough padding already (SIMDJSON_PADDING extra bytes allocated), you can use <code>padded_string_view</code> to pass it in:</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keywordtype">char</span> json[3+SIMDJSON_PADDING];</div>
|
|
<div class="line">strcpy(json, <span class="stringliteral">"[1]"</span>);</div>
|
|
<div class="line">ondemand::document doc = parser.iterate(json, strlen(json), <span class="keyword">sizeof</span>(json));</div>
|
|
</div><!-- fragment --><p>The simdjson library will also accept <code>std::string</code> instances. If the provided reference is non-const, it will allocate padding as needed.</p>
|
|
<p>You can copy your data directly on a <code><a class="el" href="structsimdjson_1_1padded__string.html" title="String with extra allocation for ease of use with parser::parse()">simdjson::padded_string</a></code> as follows:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">const</span> <span class="keywordtype">char</span> * data = <span class="stringliteral">"my data"</span>; <span class="comment">// 7 bytes</span></div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> my_padded_data(data, 7); <span class="comment">// copies to a padded buffer</span></div>
|
|
<div class="ttc" id="astructsimdjson_1_1padded__string_html"><div class="ttname"><a href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a></div><div class="ttdoc">String with extra allocation for ease of use with parser::parse()</div><div class="ttdef"><b>Definition</b> <a href="padded__string_8h_source.html#l00023">padded_string.h:23</a></div></div>
|
|
</div><!-- fragment --><p>Or as follows...</p>
|
|
<div class="fragment"><div class="line">std::string data = <span class="stringliteral">"my data"</span>;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> my_padded_data(data); <span class="comment">// copies to a padded buffer</span></div>
|
|
</div><!-- fragment --><p>You can then parse the JSON data from the <code><a class="el" href="structsimdjson_1_1padded__string.html" title="String with extra allocation for ease of use with parser::parse()">simdjson::padded_string</a></code> instance:</p>
|
|
<div class="fragment"><div class="line">ondemand::document doc = parser.iterate(my_padded_data);</div>
|
|
</div><!-- fragment --><p>Whenever you pass an <code>std::string</code> reference to <code>parser::iterate</code>, the parser will access the bytes beyond the end of the string but before the end of the allocated memory (<code>std::string::capacity()</code>). If you are using a sanitizer that checks for reading uninitialized bytes or <code>std::string</code>'s container-overflow checks, you may encounter sanitizer warnings. You can safely ignore these warnings. Or you can call <code><a class="el" href="namespacesimdjson.html#ac4131fd8cef9c8046d0bc9aee8d44d16" title="Create a padded_string_view from a string.">simdjson::pad</a>(std::string&)</code> to pad the string with <code>SIMDJSON_PADDING</code> spaces: this function returns a <code>simdjson::padding_string_view</code> which can be be passed to the parser's iterator function:</p>
|
|
<div class="fragment"><div class="line">std::string json = <span class="stringliteral">"[1]"</span>;</div>
|
|
<div class="line">ondemand::document doc = parser.iterate(<a class="code hl_function" href="namespacesimdjson.html#ac4131fd8cef9c8046d0bc9aee8d44d16">simdjson::pad</a>(json));</div>
|
|
<div class="ttc" id="anamespacesimdjson_html_ac4131fd8cef9c8046d0bc9aee8d44d16"><div class="ttname"><a href="namespacesimdjson.html#ac4131fd8cef9c8046d0bc9aee8d44d16">simdjson::pad</a></div><div class="ttdeci">padded_string_view pad(std::string &s) noexcept</div><div class="ttdoc">Create a padded_string_view from a string.</div><div class="ttdef"><b>Definition</b> <a href="padded__string__view-inl_8h_source.html#l00072">padded_string_view-inl.h:72</a></div></div>
|
|
</div><!-- fragment --><p>We recommend against creating many <code>std::string</code> or many <code>std::padding_string</code> instances in your application to store your JSON data. Consider reusing the same buffers and limiting memory allocations.</p>
|
|
<p>By default, the simdjson library throws exceptions (<code>simdjson_error</code>) on errors. We omit <code>try</code>-<code>catch</code> clauses from our illustrating examples: if you omit <code>try</code>-<code>catch</code> in your code, an uncaught exception will halt your program. It is also possible to use simdjson without generating exceptions, and you may even build the library without exception support at all. See Error handling for details.</p>
|
|
<p>Some users may want to browse code along with the compiled assembly. You want to check out the following lists of examples:</p>
|
|
<ul>
|
|
<li><a href="https://godbolt.org/z/98Kx9Kqjn">simdjson examples with errors handled through exceptions</a></li>
|
|
<li><a href="https://godbolt.org/z/PKG7GdbPo">simdjson examples with errors without exceptions</a></li>
|
|
</ul>
|
|
<p><em>Windows-specific</em>: Windows users who need to read files with non-ANSI characters in the name should set their code page to UTF-8 (65001). This should be the default with Windows 11 and better. Further, they may use the AreFileApisANSI function to determine whether the filename is interpreted using the ANSI or the system default OEM codepage, and they may call SetFileApisToOEM accordingly.</p>
|
|
<p><b>Advanced feature:</b> On non-Windows systems, you can use memory-file mapping to create a <code><a class="el" href="classsimdjson_1_1padded__string__view.html" title="User-provided string that promises it has extra padded bytes at the end for use with parser::parse().">simdjson::padded_string_view</a></code> from a file on disk.</p>
|
|
<div class="fragment"><div class="line"><span class="comment">// If the macro _WIN32 is defined, this will not work since we do not support memory-file mapping</span></div>
|
|
<div class="line"><span class="comment">// under Windows at this time.</span></div>
|
|
<div class="line"><a class="code hl_class" href="classsimdjson_1_1padded__memory__map.html">simdjson::padded_memory_map</a> map(myfilename);</div>
|
|
<div class="line"><span class="keywordflow">if</span> (!map.is_valid()) { <span class="comment">/* handle error */</span> }</div>
|
|
<div class="line"><a class="code hl_class" href="classsimdjson_1_1padded__string__view.html">simdjson::padded_string_view</a> view = map.view(); <span class="comment">// view is usable while padded_memory_map is in scope</span></div>
|
|
<div class="line">ondemand::document doc = parser.iterate(view); <span class="comment">// parse the JSON</span></div>
|
|
<div class="ttc" id="aclasssimdjson_1_1padded__memory__map_html"><div class="ttname"><a href="classsimdjson_1_1padded__memory__map.html">simdjson::padded_memory_map</a></div><div class="ttdoc">A class representing a memory-mapped file with padding.</div><div class="ttdef"><b>Definition</b> <a href="padded__string_8h_source.html#l00286">padded_string.h:286</a></div></div>
|
|
<div class="ttc" id="aclasssimdjson_1_1padded__string__view_html"><div class="ttname"><a href="classsimdjson_1_1padded__string__view.html">simdjson::padded_string_view</a></div><div class="ttdoc">User-provided string that promises it has extra padded bytes at the end for use with parser::parse().</div><div class="ttdef"><b>Definition</b> <a href="padded__string__view_8h_source.html#l00018">padded_string_view.h:18</a></div></div>
|
|
</div><!-- fragment --><h1><a class="anchor" id="autotoc_md6"></a>
|
|
Documents are iterators</h1>
|
|
<p>The simdjson library relies on an approach to parsing JSON that we call "On-Demand". A <code>document</code> is <em>not</em> a fully-parsed JSON value; rather, it is an <b>iterator</b> over the JSON text. This means that while you iterate an array, or search for a field in an object, it is actually walking through the original JSON text, merrily reading commas and colons and brackets to make sure you get where you are going. This is the key to On-Demand's performance: since it's just an iterator, it lets you parse values as you use them. And particularly, it lets you <em>skip</em> values you do not want to use. On-Demand is also ideally suited when you want to capture part of the document without parsing it immediately (e.g., see General direct access to the raw JSON string).</p>
|
|
<p>We refer to "On-Demand" as a front-end component since it is an interface between the low-level parsing functions and the user. It hides much of the complexity of parsing JSON documents.</p>
|
|
<h2><a class="anchor" id="autotoc_md7"></a>
|
|
Parser, document and JSON scope</h2>
|
|
<p>For code safety, you should keep (1) the <code>parser</code> instance, (2) the input string and (3) the document instance alive throughout your parsing. Additionally, you should follow the following rules:</p>
|
|
<ul>
|
|
<li>A <code>parser</code> may have at most one document open at a time, since it holds allocated memory used for the parsing.</li>
|
|
<li>By design, you should only have one <code>document</code> instance per JSON document. Thus, if you must pass a document instance to a function, you should avoid passing it by value: choose to pass it by reference instance to avoid the copy. In any case, the <code>document</code> class does not have a copy constructor.</li>
|
|
</ul>
|
|
<p>During the <code>iterate</code> call, the original JSON text is never modified–only read. After you are done with the document, the source (whether file or string) can be safely discarded.</p>
|
|
<p>For best performance, a <code>parser</code> instance should be reused over several files: otherwise you will needlessly reallocate memory, an expensive process. It is also possible to avoid entirely memory allocations during parsing when using simdjson. <a class="el" href="md_doc_2performance.html">See our performance notes for details</a>.</p>
|
|
<p>If you need to have several documents active at once, you should have several parser instances.</p>
|
|
<h1><a class="anchor" id="autotoc_md8"></a>
|
|
string_view</h1>
|
|
<p>The simdjson library builds on compilers supporting the <a href="https://en.wikipedia.org/wiki/C%2B%2B11">C++11 standard</a>. It is also a strict requirement: we have no plan to support older C++ compilers.</p>
|
|
<p>We represent parsed Unicode (UTF-8) strings in simdjson using the <code>std::string_view</code> class. It avoids the need to copy the data, as would be necessary with the <code>std::string</code> class. It also avoids the pitfalls of null-terminated C strings. It makes it easier for our users to copy the data into their own favorite class instances (e.g., alternatives to <code>std::string</code>).</p>
|
|
<p>A <code>std::string_view</code> instance is effectively just a pointer to a region in memory representing a string. In simdjson, we return <code>std::string_view</code> instances that either point within the input string you parsed (see General direct access to the raw JSON string), or to a temporary string buffer inside our parser class instances that is valid until the parser object is destroyed or you use it to parse another document. When using <code>std::string_view</code> instances, it is your responsibility to ensure that <code>std::string_view</code> instance does not outlive the pointed-to memory (e.g., either the input buffer or the parser instance). Furthermore, some operations reset the string buffer inside our parser instances: e.g., when we parse a new document. Thus a <code>std::string_view</code> instance is often best viewed as a temporary string value that is tied to the document you are parsing. At the cost of some memory allocation, you may convert your <code>std::string_view</code> instances for long-term storage into <code>std::string</code> instances: <code>std::string mycopy(view)</code> (C++17) or <code>std::string mycopy(view.begin(), view.end())</code> (prior to C++17). For convenience, we also allow storing an escaped string directly into an existing string instance.</p>
|
|
<p>The <code>std::string_view</code> class has become standard as part of C++17 but it is not always available on compilers that only supports C++11. When we detect that <code>string_view</code> is natively available, we define the macro <code>SIMDJSON_HAS_STRING_VIEW</code>.</p>
|
|
<p>When we detect that it is unavailable, we use <a href="https://github.com/martinmoene/string-view-lite">string-view-lite</a> as a substitute. In such cases, we use the type alias <code>using string_view = nonstd::string_view;</code> to offer the same API, irrespective of the compiler and standard library. The macro <code>SIMDJSON_HAS_STRING_VIEW</code> will be <em>undefined</em> to indicate that we emulate <code>string_view</code>.</p>
|
|
<p>Some users prefer to use non-JSON native encoding formats such as UTF-16 or UTF-32. Users may transcode the UTF-8 strings produced by the simdjson library to other formats. See the <a href="https://github.com/simdutf/simdutf">simdutf library</a>, for example.</p>
|
|
<h1><a class="anchor" id="autotoc_md9"></a>
|
|
Avoiding pitfalls: enable development checks</h1>
|
|
<p>We recommend that you first compile and run your code in debug mode:</p>
|
|
<ul>
|
|
<li>under Visual Studio, it means having the <code>_DEBUG</code> macro defined,</li>
|
|
<li>for many other compilers, it means leaving the <code>__OPTIMIZE__</code> macro undefined.</li>
|
|
</ul>
|
|
<p>The simdjson code will set <code>SIMDJSON_DEVELOPMENT_CHECKS=1</code> in debug mode. Because the C++ standard does not provide a direct way of checking for a debug build, and because you may want the checks while building with optimizations, you can set the macro <code>SIMDJSON_DEVELOPMENT_CHECKS</code> to 1 prior to including the <code><a class="el" href="simdjson_8h.html">simdjson.h</a></code> header to enable these additional checks: just make sure you remove the definition once your code has been tested. When <code>SIMDJSON_DEVELOPMENT_CHECKS</code> is set to 1, the simdjson library runs additional (expensive) tests on your code to help ensure that you are using the library in a safe manner. We add asserts which may halt your program, helping you find the bad programming pattern.</p>
|
|
<p>When <code>SIMDJSON_DEVELOPMENT_CHECKS</code>, some of our data structures contain extra data for tracking explicitly potential programming mistakes. Thus you should not relying on the size (<code>sizeof</code>) of our data structures to be constant: they may change depending on the compiler settings.</p>
|
|
<p>Once your code has been tested, you can then run it in Release mode: under Visual Studio, it means having the <code>_DEBUG</code> macro undefined, and, for other compilers, it means setting <code>__OPTIMIZE__</code> to a positive integer. You can also forcefully disable these checks by setting <code>SIMDJSON_DEVELOPMENT_CHECKS</code> to 0.</p>
|
|
<p>Once your code is tested, we further encourage you to define <code>NDEBUG</code> in your release builds to disable additional runtime testing and get the best performance. We disable these checks on a best-effort basis but the C++ standard does not provide a direct way to check for a release build.</p>
|
|
<p>Warnign: Mixing debug and release simdjson code is unsafe: you either build all your code using simdjson in release mode or all of it in debug mode.</p>
|
|
<h1><a class="anchor" id="autotoc_md10"></a>
|
|
Using the parsed JSON</h1>
|
|
<p>Once you have a document (<code>simdjson::ondemand::document</code>), you can navigate it with idiomatic C++ iterators, operators and casts. Besides the document instances and native types (<code>double</code>, <code>uint64_t</code>, <code>int64_t</code>, <code>bool</code>), we also access Unicode (UTF-8) strings (<code>std::string_view</code>), objects (<code>simdjson::ondemand::object</code>) and arrays (<code>simdjson::ondemand::array</code>). We also have a generic ephemeral type (<code>simdjson::ondemand::value</code>) which represents a potential array or object, or scalar type (<code>double</code>, <code>uint64_t</code>, <code>int64_t</code>, <code>bool</code>, <code>null</code>, string) inside an array or an object. Both generic types (<code>simdjson::ondemand::document</code> and <code>simdjson::ondemand::value</code>) have a <code>type()</code> method returning a <code>json_type</code> value describing indicating the type (<code>json_type::array</code>, <code>json_type::object</code>, <code>json_type::number</code>, <code>json_type::string</code>, <code>json_type::boolean</code>, <code>json_type::null</code>, and <code>json_type::unknown</code> for unrecognized types). The <code>type()</code> method does not consume nor validate the value: e.g., you must still call <code>is_null()</code> to check that the value is a <code>null</code> even if <code>json_type::null</code> is returned. Starting with simdjson 4.0, we return <code>json_type::unknown</code> for bad tokens such as the <code>NaN</code> token in <code>{"key":NaN}</code>. A <code>json_type::unknown</code> type value indicates an error in the JSON document but you might still be able to proceed, see General direct access to the raw JSON string. A generic value (<code>simdjson::ondemand::value</code>) is only valid temporarily, as soon as you access other values, other keys in objects, etc. it becomes invalid: you should therefore consume the value immediately by converting it to a scalar type, an array or an object.</p>
|
|
<p>Advanced users who need to determine the number types (integer or float) dynamically, should review our section dynamic number types. Indeed, we have an additional <code>ondemand::number</code> type which may represent either integers or floating-point values, depending on how the numbers are formatted. floating-point values followed by an integer.</p>
|
|
<p>We invite you to keep the following rules in mind:</p><ol type="1">
|
|
<li>While you are accessing the document, the <code>document</code> instance should remain in scope: it is your "iterator" which keeps track of where you are in the JSON document. By design, there is one and only one <code>document</code> instance per JSON document.</li>
|
|
<li>Because On-Demand is really just an iterator, you must fully consume the current object or array before accessing a sibling object or array.</li>
|
|
<li>Values can only be consumed once, you should get the values and store them if you plan to need them multiple times. You are expected to access the keys of an object just once. You are expected to go through the values of an array just once.</li>
|
|
</ol>
|
|
<p>The simdjson library makes generous use of <code>std::string_view</code> instances. If you are unfamiliar with <code>std::string_view</code> in C++, make sure to read the section on std::string_view. They behave much like an immutable <code>std::string</code> but they require no memory allocation. You can create a <code>std::string</code> instance from a <code>std::string_view</code> when you need it.</p>
|
|
<p>The following specific instructions indicate how to use the JSON when exceptions are enabled, but simdjson has full, idiomatic support for users who avoid exceptions. See <a href="basics.md#error-handling">the simdjson error handling documentation</a> for more.</p>
|
|
<ul>
|
|
<li><b>Validate What You Use:</b> When calling <code>iterate</code>, the document is quickly indexed. If it is not a valid Unicode (UTF-8) string or if there is an unclosed string, an error may be reported right away. However, it is not fully validated. On-Demand only fully validates the values you use and the structure leading to it. It means that at every step as you traverse the document, you may encounter an error. You can handle errors either with exceptions or with error codes.</li>
|
|
<li><b>Extracting Values:</b> You can cast a JSON element to a native type: <code>double(element)</code>. This works for <code>std::string_view</code>, double, uint64_t, int64_t, bool, ondemand::object and ondemand::array. We also have explicit methods such as <code>get_string()</code>, <code>get_double()</code>, <code>get_uint64()</code>, <code>get_int64()</code>, <code>get_uint32()</code>, <code>get_int32()</code>, <code>get_bool()</code>, <code>get_object()</code> and <code>get_array()</code>. After a cast or an explicit method, the number, string or boolean will be parsed, or the initial <code>{</code> or <code>[</code> will be verified for <code>ondemand::object</code> and <code>ondemand::array</code>. An exception may be thrown if the cast is not possible: the error code is <code><a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9a1de02af1d67e8350f612eee0dcf634b4" title="JSON element has a different type than user expected.">simdjson::INCORRECT_TYPE</a></code> (see Error handling). Importantly, when getting an ondemand::object or ondemand::array instance, its content is not validated: you are only guaranteed that the corresponding initial character (<code>{</code> or <code>[</code>) is present. Thus, for example, you could have an ondemand::object instance pointing at the invalid JSON <code>{ "this is not a valid object" }</code>: the validation occurs as you access the content. The <code>get_string()</code> returns a valid UTF-8 string, after unescaping characters as needed: unmatched surrogate pairs are treated as an error unless you pass <code>true</code> (<code>get_string(true)</code>) as a parameter to get replacement characters where errors occur. If you somehow need to access non-UTF-8 strings in a lossless manner (e.g., if you strings contain unpaired surrogates), you may use the <code>get_wobbly_string()</code> function to get a string in the <a href="https://simonsapin.github.io/wtf-8">WTF-8 format</a>. When calling <code>get_uint64()</code>, <code>get_int64()</code>, <code>get_uint32()</code> or <code>get_int32()</code>, if the number does not fit in the corresponding integer type, it is also considered an error (<code>NUMBER_OUT_OF_RANGE</code>). When parsing numbers or other scalar values, the library checks that the value is followed by an expected character, thus you <em>may</em> get a number parsing error when accessing the digits as an integer in the following strings: <code>{"number":12332a</code>, <code>{"number":12332\0</code>, <code>{"number":12332</code> (the digits appear at the end). We always abide by the <a href="https://www.tbray.org/ongoing/When/201x/2017/12/14/rfc8259.html">RFC 8259</a> JSON specification so that, for example, numbers prefixed by the <code>+</code> sign are in error.</li>
|
|
</ul>
|
|
<blockquote class="doxtable">
|
|
<p>‍IMPORTANT NOTE: values can only be parsed once. Since documents are <em>iterators</em>, once you have parsed a value (such as by casting to double), you cannot get at it again. It is an error to call <code>get_string()</code> twice on an object (or to cast an object twice to <code>std::string_view</code>). </p>
|
|
</blockquote>
|
|
<p>* <b>Array Iteration:</b> To iterate through an array, use <code>for (auto value : array) { ... }</code>. This will step through each value in the JSON array.</p>
|
|
<p>To iterate through an array, you should be at the beginning of the array: to warn you, an OUT_OF_ORDER_ITERATION error is generated when development checks are active. If you need to access an array more than once, you may call <code>reset()</code> on it although we discourage this practice. Keep in mind that you should consume each value at most once.</p>
|
|
<p>If you know the type of the value, you can cast it right there, too! <code>for (double value : array) { ... }</code>.</p>
|
|
<p>You may also use explicit iterators: <code>for(auto i = array.begin(); i != array.end(); i++) {}</code>. You can check that an array is empty with the condition <code>auto i = array.begin(); if (i == array.end()) {...}</code>. You should derefence (<code>*i</code>) an iterator at most once before incrementing it (<code>i++</code>), when compiling in debug mode with development checks, we add asserts to help you identify such a mistake.</p><ul>
|
|
<li><p class="startli"><b>Object Iteration:</b> You can iterate through an object's fields, as well: <code>for (auto field : object) { ... }</code>.</p><ul>
|
|
<li><code>field.unescaped_key()</code> will get you the unescaped key string as a <code>std::string_view</code> instance. E.g., the JSON string <code>"\u00e1"</code> becomes the Unicode string <code>á</code>. Optionally, you pass <code>true</code> as a parameter to the <code>unescaped_key</code> method if you want invalid escape sequences to be replaced by a default replacement character (e.g., <code>\ud800\ud801\ud811</code>): otherwise bad escape sequences lead to an immediate error.</li>
|
|
<li><code>field.escaped_key()</code> will get you the key string as as a <code>std::string_view</code> instance, but unlike <code>unescaped_key()</code>, the key is not processed, so no unescaping is done. E.g., the JSON string <code>"\u00e1"</code> becomes the Unicode string <code>\u00e1</code>. We expect that <code>escaped_key()</code> is faster than <code>field.unescaped_key()</code>.</li>
|
|
<li><code>field.value()</code> will get you the value, which you can then use all these other methods on.</li>
|
|
</ul>
|
|
<p class="startli">To iterate through an object, you should be at the beginning of the object: to warn you, an OUT_OF_ORDER_ITERATION error is generated when development checks are active. If you need to access an object more than once, you may call <code>reset()</code> on it although we discourage this practice. Keep in mind that you should consume each value at most once.</p>
|
|
<p class="startli">When you are iterating through an object, you are advancing through its keys and values. You should not also access the object or other objects. E.g. within a loop over <code>myobject</code>, you should not be accessing <code>myobject</code>. The following is an anti-pattern: <code>for(auto value: myobject) {myobject["mykey"]}</code>.</p>
|
|
<p class="startli">We discourage using the object iterators explicitly: <code>for(auto i = object.begin(); i != object.end(); i++) { auto field = *i; .... }</code>. In addition to the usual requirement to check against <code>end()</code> prior to dereferencing, you must also always dereference the pointer (<code>*it</code>) exactly once before you increment it (<code>it++</code>). You must also only deference the iterator once (never more than once). When compiling in debug mode with development checks, we add asserts to help check whether you correctly dereferenced the pointer before incrementing it.</p>
|
|
<p class="startli">You should never reset an object as you are iterating through it. The following is an anti-pattern: <code>for(auto value: myobject) {myobject.reset()}</code>.</p>
|
|
</li>
|
|
<li><b>Array Index:</b> Because it is forward-only, you cannot look up an array element by index. Instead, you should iterate through the array and keep an index yourself. Exceptionally, if need a single value out of the array, you may use an array access (e.g., <code>array[1]</code>). You should never reset an array as you are iterating through it. The following is an anti-pattern: <code>for(auto value: myarray) {myarray.reset()}</code>.</li>
|
|
<li><b>Field Access:</b> To get the value of the "foo" field in an object, use <code>object["foo"]</code>. This will scan through the object looking for the field with the matching string, doing a character-by-character comparison. It may generate the error <code><a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9a4a2c71f8eca438b3f0553a2811bdab9a" title="JSON field not found in object.">simdjson::NO_SUCH_FIELD</a></code> if there is no such key in the object, it may throw an exception (see Error handling). The returned value is only valid so long as you do not access another field: normally, you should therefore grab the value right after accessing a key (i.e., convert it to number, string, object, array...). For efficiency reason, you should avoid looking up the same field repeatedly: e.g., do not do <code>object["foo"]</code> followed by <code>object["foo"]</code> with the same <code>object</code> instance. Generally, you should not mix and match iterating through an object (<code>for(auto field : object) {...}</code>) and key accesses (<code>object["foo"]</code>): if you need to iterate through an object after a key access, you need to call <code>reset()</code> on the object. Whenever you call <code>reset()</code>, you need to keep in mind that though you can iterate over the array repeatedly, values should be consumedonly once (e.g., repeatedly calling <code>unescaped_key()</code> on the same key is forbidden). Keep in mind that On-Demand does not buffer or save the result of the parsing: if you repeatedly access <code>object["foo"]</code>, then it must repeatedly seek the key and parse the content. The library does not provide a distinct function to check if a key is present, instead we recommend you attempt to access the key: e.g., by doing <code>ondemand::value val{}; if (!object["foo"].get(val)) {...}</code>, you have that <code>val</code> contains the requested value inside the if clause. It is your responsibility as a user to temporarily keep a reference to the value (<code>auto v = object["foo"]</code>), or to consume the content and store it in your own data structures. If you consume an object twice: <code>std::string_view(object["foo"]</code> followed by <code>std::string_view(object["foo"]</code> then your code is in error. Furthermore, you can only consume one field at a time, on the same object. The value instance you get from <code>content["bids"]</code> becomes invalid when you call <code>content["asks"]</code>. If you have retrieved <code>content["bids"].get_array()</code> and you later call <code>content["asks"].get_array()</code>, then the first array should no longer be accessed: it would be unsafe to do so. You can detect such mistakes by first compiling and running the code with development checks: an OUT_OF_ORDER_ITERATION error is generated.</li>
|
|
</ul>
|
|
<blockquote class="doxtable">
|
|
<p>‍NOTE: JSON allows you to escape characters in keys. E.g., the key <code>"date"</code> may be written as <code>"\u0064\u0061\u0074\u0065"</code>. By default, simdjson does <em>not</em> unescape keys when matching. Thus if you search for the key <code>"date"</code> and the JSON document uses <code>"\u0064\u0061\u0074\u0065"</code> as a key, it will not be recognized. This is not generally a problem. Nevertheless, if you do need to support escaped keys, the method <code>unescaped_key()</code> provides the desired unescaped keys by parsing and writing out the unescaped keys to a string buffer and returning a <code>std::string_view</code> instance. The <code>unescaped_key</code> takes an optional Boolean value: passing it true will decode invalid Unicode sequences with replacement, meaning that the decoding always succeeds but bogus Unicode replacement characters are inserted. In general, you should expect a performance penalty when using <code>unescaped_key()</code> compared to <code>key()</code> because of the string processing: the <code>key()</code> function just points inside the source JSON document. As a compromise, you may use <code>escaped_key()</code><code> which returns a</code>std::string_view<code>instance pointing directly in the document, like</code>key()<code>, although, unlike</code>key()`, it has to determine the location of the final quote character.</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"({"k\u0065y": 1})"_padded; </span><span class="comment">// R"( ... )" is a C++ raw string literal.</span></div>
|
|
<div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line">ondemand::object <span class="keywordtype">object</span> = doc.get_object();</div>
|
|
<div class="line"><span class="keywordflow">for</span>(<span class="keyword">auto</span> field : object) {</div>
|
|
<div class="line"> <span class="comment">// parses and writes out the key, after unescaping it,</span></div>
|
|
<div class="line"> <span class="comment">// to a string buffer. It causes a performance penalty.</span></div>
|
|
<div class="line"> <span class="comment">// If you do not expect that unescaping is useful, you</span></div>
|
|
<div class="line"> <span class="comment">// may replace field.unescaped_key() with</span></div>
|
|
<div class="line"> <span class="comment">// field.escaped_key().</span></div>
|
|
<div class="line"> std::string_view keyv = field.unescaped_key();</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (keyv == <span class="stringliteral">"key"</span>) { std::cout << uint64_t(field.value()); }</div>
|
|
<div class="line"> }</div>
|
|
</div><!-- fragment --><p>By default, field lookup is order-insensitive, so you can look up values in any order. However, we still encourage you to look up fields in the order you expect them in the JSON, as it is still faster.</p>
|
|
<p>If you want to enforce finding fields in order, you can use <code>object.find_field("foo")</code> instead. This will only look forward, and will fail to find fields in the wrong order: for example, this will fail:</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( { "x": 1, "y": 2 } )"_padded; </span><span class="comment">// R"( ... )" is a C++ raw string literal.</span></div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="keywordtype">double</span> y = doc.find_field(<span class="stringliteral">"y"</span>); <span class="comment">// The cursor is now after the 2 (at })</span></div>
|
|
<div class="line"><span class="keywordtype">double</span> x = doc.find_field(<span class="stringliteral">"x"</span>); <span class="comment">// This fails, because there are no more fields after "y"</span></div>
|
|
</div><!-- fragment --><p>By contrast, using the default (order-insensitive) lookup succeeds:</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( { "x": 1, "y": 2 } )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="keywordtype">double</span> y = doc[<span class="stringliteral">"y"</span>]; <span class="comment">// The cursor is now after the 2 (at })</span></div>
|
|
<div class="line"><span class="keywordtype">double</span> x = doc[<span class="stringliteral">"x"</span>]; <span class="comment">// Success: [] loops back around to find "x"</span></div>
|
|
</div><!-- fragment --> </blockquote>
|
|
<p>* <b>Output to strings:</b> Given a document, a value, an array or an object in a JSON document, you can output a JSON string version suitable to be parsed again as JSON content: <code>simdjson::to_json_string(element)</code>. A call to <code>to_json_string</code> consumes fully the element: if you apply it on a document, the internal pointer is advanced to the end of the document. The <code><a class="el" href="namespacesimdjson.html#a2cda17221b852e5c398bad5bf350d270" title="Create a string-view instance out of a document instance.">simdjson::to_json_string</a></code> does not allocate memory. The <code>to_json_string</code> function should not be confused with retrieving the value of a string instance which are escaped and represented using a lightweight <code>std::string_view</code> instance pointing at an internal string buffer inside the parser instance. To illustrate, the first of the following two code segments will print the unescaped string <code>"test"</code> complete with the quote whereas the second one will print the escaped content of the string (without the quotes). </p><blockquote class="doxtable">
|
|
<p>‍<code>cpp // serialize a JSON to an escaped std::string instance so that it can be parsed again as JSON auto silly_json = R"( { "test": "result" } )"_padded; ondemand::document doc = parser.iterate(silly_json); std::cout << <a class="el" href="namespacesimdjson.html#a2cda17221b852e5c398bad5bf350d270" title="Create a string-view instance out of a document instance.">simdjson::to_json_string</a>(doc["test"]) << std::endl; // Requires simdjson 1.0 or better @icode @endicode cpp // retrieves an unescaped string value as a string_view instance auto silly_json = R"( { "test": "result" } )"_padded; ondemand::document doc = parser.iterate(silly_json); std::cout << std::string_view(doc["test"]) << std::endl; @icode You can use `to_json_string` to efficiently extract components of a JSON document to reconstruct a new JSON document, as in the following example: @endicode cpp auto cars_json = R"( [
|
|
{ "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] },
|
|
{ "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] },
|
|
{ "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] }
|
|
] )"_padded; std::vector<std::string_view> arrays; // We are going to collect string_view instances which point inside the `cars_json` string // and are therefore valid as long as `cars_json` remains in scope. { ondemand::parser parser; for (ondemand::object car : parser.iterate(cars_json)) { if (uint64_t(car["year"]) > 2000) { arrays.push_back(<a class="el" href="namespacesimdjson.html#a2cda17221b852e5c398bad5bf350d270" title="Create a string-view instance out of a document instance.">simdjson::to_json_string</a>(car["tire_pressure"])); } } } // We can now convert to a JSON string: std::ostringstream oss; oss << "["; for(size_t i = 0; i < arrays.size(); i++) { if (i>0) { oss << ","; } oss << arrays[i]; } oss << "]"; auto json_string = oss.str(); // json_string == "[[ 40.1, 39.9, 37.7, 40.4 ],[ 30.1, 31.0, 28.6, 28.7 ]]" </code> </p>
|
|
</blockquote>
|
|
<p>* <b>Extracting Values (without exceptions):</b> You can use a variant usage of <code>get()</code> with error codes to avoid exceptions. You first declare the variable of the appropriate type (<code>double</code>, <code>uint64_t</code>, <code>int64_t</code>, <code>bool</code>, <code>ondemand::object</code> and <code>ondemand::array</code>) and pass it by reference to <code>get()</code> which gives you back an error code: e.g.,</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">auto</span> abstract_json = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"> { "str" : { "123" : {"abc" : 3.14 } } }</span></div>
|
|
<div class="line"><span class="stringliteral">)"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordtype">double</span> value;</div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(abstract_json);</div>
|
|
<div class="line"><span class="keyword">auto</span> error = doc[<span class="stringliteral">"str"</span>][<span class="stringliteral">"123"</span>][<span class="stringliteral">"abc"</span>].get(value);</div>
|
|
<div class="line"><span class="keywordflow">if</span> (error) { std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error) << std::endl; <span class="keywordflow">return</span> EXIT_FAILURE; }</div>
|
|
<div class="line">cout << value << endl; <span class="comment">// Prints 3.14</span></div>
|
|
<div class="ttc" id="anamespacesimdjson_html_a65b00a7a7a9c8fafe9eb3b9b413a36fa"><div class="ttname"><a href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a></div><div class="ttdeci">const char * error_message(error_code error) noexcept</div><div class="ttdoc">It is the convention throughout the code that the macro SIMDJSON_DEVELOPMENT_CHECKS determines whethe...</div><div class="ttdef"><b>Definition</b> <a href="error-inl_8h_source.html#l00025">error-inl.h:25</a></div></div>
|
|
</div><!-- fragment --><p> This examples also show how we can string several operations and only check for the error once, a strategy we call <em>error chaining</em>. Though error chaining makes the code very compact, it also makes error reporting less precise: in this instance, you may get the same error whether the field "str", "123" or "abc" is missing. If you need to break down error handling per operation, avoid error chaining. Furthermore, you should be mindful that chaining that harm performance by encouraging redundancies: writing both <code>doc["str"]["123"]["abc"].get(value)</code> and <code>doc["str"]["123"]["zyw"].get(value)</code> in the same program may force multiple accesses to the same keys (<code>"str"</code> and <code>"123"</code>).</p><ul>
|
|
<li><p class="startli"><b>Counting elements in arrays:</b> Sometimes it is useful to scan an array to determine its length prior to parsing it. For this purpose, <code>array</code> instances have a <code>count_elements</code> method. Users should be aware that the <code>count_elements</code> method can be costly since it requires scanning the whole array. You should only call <code>count_elements</code> as a last resort as it may require scanning the document twice or more. You should never use the <code>count_elements</code> as part of an attempt to iterate through the array: use a <code>for</code> loop to iterate through arrays. In the spirit of On-Demand, the <code>count_elements</code> function does not validate the values in the array: they are validated when they are consumed. You may use it as follows if your document is itself an array:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">auto</span> cars_json = R<span class="stringliteral">"( [ 40.1, 39.9, 37.7, 40.4 ] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(cars_json);</div>
|
|
<div class="line"><span class="keywordtype">size_t</span> count = doc.count_elements(); <span class="comment">// requires simdjson 1.0 or better</span></div>
|
|
<div class="line">std::vector<double> values(count);</div>
|
|
<div class="line"><span class="keywordtype">size_t</span> index = 0;</div>
|
|
<div class="line"><span class="keywordflow">for</span>(<span class="keywordtype">double</span> x : doc) { values[index++] = x; }</div>
|
|
</div><!-- fragment --><p> If you access an array inside a document, you can use the <code>count_elements</code> method as follow. You should not let the array instance go out of scope before consuming it after calling the <code>count_elements</code> method: </p><div class="fragment"><div class="line">C++</div>
|
|
<div class="line"> ondemand::parser parser;</div>
|
|
<div class="line"> auto cars_json = R"( { "test":[ { "val1":1, "val2":2 }, { "val1":1, "val2":2 } ] } )"_padded;</div>
|
|
<div class="line"> auto doc = parser.iterate(cars_json);</div>
|
|
<div class="line"> auto test_array = doc.find_field("test").get_array();</div>
|
|
<div class="line"> size_t count = test_array.count_elements(); // requires simdjson 1.0 or better</div>
|
|
<div class="line"> std::cout << "Number of elements: " << count << std::endl;</div>
|
|
<div class="line"> for(ondemand::object elem: test_array) {</div>
|
|
<div class="line"> std::cout << simdjson::to_json_string(elem);</div>
|
|
<div class="line"> }</div>
|
|
</div><!-- fragment --></li>
|
|
<li><p class="startli"><b>Counting fields in objects:</b> Other times, it is useful to scan an object to determine the number of fields prior to parsing it. For this purpose, <code>object</code> instances have a <code>count_fields</code> method. Again, users should be aware that the <code>count_fields</code> method can be costly since it requires scanning the whole objects. You should only call <code>count_fields</code> as a last resort as it may require scanning the document twice or more. You may use it as follows if your document is itself an object:</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( { "test":{ "val1":1, "val2":2 } } )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="keywordtype">size_t</span> count = doc.count_fields(); <span class="comment">// requires simdjson 1.0 or better</span></div>
|
|
<div class="line">std::cout << <span class="stringliteral">"Number of fields: "</span> << count << std::endl; <span class="comment">// Prints "Number of fields: 1"</span></div>
|
|
</div><!-- fragment --><p> Similarly to <code>count_elements</code>, you should not let an object instance go out of scope before consuming it after calling the <code>count_fields</code> method. If you access an object inside a document, you can use the <code>count_fields</code> method as follow. </p><div class="fragment"><div class="line">C++</div>
|
|
<div class="line"> ondemand::parser parser;</div>
|
|
<div class="line"> auto json = R"( { "test":{ "val1":1, "val2":2 } } )"_padded;</div>
|
|
<div class="line"> auto doc = parser.iterate(json);</div>
|
|
<div class="line"> auto test_object = doc.find_field("test").get_object();</div>
|
|
<div class="line"> size_t count = test_object.count_fields(); // requires simdjson 1.0 or better</div>
|
|
<div class="line"> std::cout << "Number of fields: " << count << std::endl; // Prints "Number of fields: 2"</div>
|
|
</div><!-- fragment --></li>
|
|
<li><p class="startli"><b>Tree Walking and JSON Element Types:</b> Sometimes you don't necessarily have a document with a known type, and are trying to generically inspect or walk over JSON elements. You can also represent arbitrary JSON values with <code>ondemand::value</code> instances: it can represent anything except a scalar document (lone number, string, null or Boolean). You can check for scalar documents with the method <code>scalar()</code>. You can cast a document that is either an array or an object to an <code>ondemand::value</code> instance immediately after you create the document instance: you cannot create an <code>ondemand::value</code> instance from a document that has already been accessed as it would mean that you would have two instances of the object or array simultaneously (see rewinding). You can query the type of a document or a value with the <code>type()</code> method. The <code>type()</code> method does not consume or validate documents and values, but it tells you whether they are</p><ul>
|
|
<li>arrays (<code>json_type::array</code>),</li>
|
|
<li>objects (<code>json_type::object</code>)</li>
|
|
<li>numbers (<code>json_type::number</code>),</li>
|
|
<li>strings (<code>json_type::string</code>),</li>
|
|
<li>Booleans (<code>json_type::boolean</code>),</li>
|
|
<li>null (<code>json_type::null</code>).</li>
|
|
</ul>
|
|
<p class="startli">You must still validate and consume the values (e.g., call <code>is_null()</code>) after calling <code>type()</code>. You may also access the raw JSON string. For example, the following is a quick and dirty recursive function that verbosely prints the JSON document as JSON. This example also illustrates lifecycle requirements: the <code>document</code> instance holds the iterator. The document must remain in scope while you are accessing instances of <code>value</code>, <code>object</code> and <code>array</code>. </p><div class="fragment"><div class="line"><span class="keywordtype">void</span> recursive_print_json(ondemand::value element) {</div>
|
|
<div class="line"> <span class="keywordtype">bool</span> add_comma;</div>
|
|
<div class="line"> <span class="keywordflow">switch</span> (element.type()) {</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::array:</div>
|
|
<div class="line"> cout << <span class="stringliteral">"["</span>;</div>
|
|
<div class="line"> add_comma = <span class="keyword">false</span>;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> child : element.get_array()) {</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (add_comma) {</div>
|
|
<div class="line"> cout << <span class="stringliteral">","</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="comment">// We need the call to value() to get</span></div>
|
|
<div class="line"> <span class="comment">// an ondemand::value type.</span></div>
|
|
<div class="line"> recursive_print_json(child.value());</div>
|
|
<div class="line"> add_comma = <span class="keyword">true</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> cout << <span class="stringliteral">"]"</span>;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::object:</div>
|
|
<div class="line"> cout << <span class="stringliteral">"{"</span>;</div>
|
|
<div class="line"> add_comma = <span class="keyword">false</span>;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> field : element.get_object()) {</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (add_comma) {</div>
|
|
<div class="line"> cout << <span class="stringliteral">","</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="comment">// key() returns the key as it appears in the raw</span></div>
|
|
<div class="line"> <span class="comment">// JSON document, if we want the unescaped key,</span></div>
|
|
<div class="line"> <span class="comment">// we should do field.unescaped_key().</span></div>
|
|
<div class="line"> <span class="comment">// We could also use field.escaped_key() if we want</span></div>
|
|
<div class="line"> <span class="comment">// a std::string_view instance, but we do not need</span></div>
|
|
<div class="line"> <span class="comment">// escaping.</span></div>
|
|
<div class="line"> cout << <span class="stringliteral">"\""</span> << field.key() << <span class="stringliteral">"\": "</span>;</div>
|
|
<div class="line"> recursive_print_json(field.value());</div>
|
|
<div class="line"> add_comma = <span class="keyword">true</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> cout << <span class="stringliteral">"}\n"</span>;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::number:</div>
|
|
<div class="line"> <span class="comment">// assume it fits in a double</span></div>
|
|
<div class="line"> cout << element.get_double();</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::string:</div>
|
|
<div class="line"> <span class="comment">// get_string() would return escaped string, but</span></div>
|
|
<div class="line"> <span class="comment">// we are happy with unescaped string.</span></div>
|
|
<div class="line"> cout << <span class="stringliteral">"\""</span> << element.get_raw_json_string() << <span class="stringliteral">"\""</span>;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::boolean:</div>
|
|
<div class="line"> cout << element.get_bool();</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::null:</div>
|
|
<div class="line"> <span class="comment">// We check that the value is indeed null</span></div>
|
|
<div class="line"> <span class="comment">// otherwise: an error is thrown.</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span> (element.is_null()) {</div>
|
|
<div class="line"> cout << <span class="stringliteral">"null"</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::json_type::unknown:</div>
|
|
<div class="line"> cout << <span class="stringliteral">"unknown"</span>; <span class="comment">// indicates an error</span></div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="keywordtype">void</span> basics_treewalk() {</div>
|
|
<div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = R<span class="stringliteral">"( [</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::value val = doc;</span></div>
|
|
<div class="line"><span class="stringliteral"> recursive_print_json(val);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::cout << std::endl;</span></div>
|
|
<div class="line"><span class="stringliteral">}</span></div>
|
|
</div><!-- fragment --></li>
|
|
</ul>
|
|
<h2><a class="anchor" id="autotoc_md11"></a>
|
|
Using the parsed JSON: additional examples</h2>
|
|
<p>Let us review these concepts with some additional examples. For simplicity, we omit the include clauses (<code>#include "simdjson.h"</code>) as well as namespace-using clauses (<code>using namespace simdjson;</code>).</p>
|
|
<p>The first example illustrates how we can chain operations. In this instance, we repeatedly select keys using the bracket operator (<code>doc["str"]</code>) and then finally request a number (using <code>get_double()</code>). It is safe to write code in this manner: if any step causes an error, the error status propagates and an exception is thrown at the end. You do not need to constantly check for errors.</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">auto</span> abstract_json = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"> { "str" : { "123" : {"abc" : 3.14 } } }</span></div>
|
|
<div class="line"><span class="stringliteral">)"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(abstract_json);</div>
|
|
<div class="line">cout << doc[<span class="stringliteral">"str"</span>][<span class="stringliteral">"123"</span>][<span class="stringliteral">"abc"</span>].get_double() << endl; <span class="comment">// Prints 3.14</span></div>
|
|
</div><!-- fragment --><p>In the following example, we start with a JSON document that contains an array of objects. We iterate through the objects using a for-loop. Within each object, we use the bracket operator (e.g., <code>car["make"]</code>) to select values. We also show how we can iterate through an array, corresponding to the key <code>tire_pressure</code>, that is contained inside each object.</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> cars_json = R<span class="stringliteral">"( [</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// Iterating through an array of objects</span></div>
|
|
<div class="line"><span class="keywordflow">for</span> (ondemand::object car : parser.iterate(cars_json)) {</div>
|
|
<div class="line"> <span class="comment">// Accessing a field by name</span></div>
|
|
<div class="line"> cout << <span class="stringliteral">"Make/Model: "</span> << std::string_view(car[<span class="stringliteral">"make"</span>]) << <span class="stringliteral">"/"</span> << std::string_view(car[<span class="stringliteral">"model"</span>]) << endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="comment">// Casting a JSON element to an integer</span></div>
|
|
<div class="line"> uint64_t year = car[<span class="stringliteral">"year"</span>];</div>
|
|
<div class="line"> cout << <span class="stringliteral">"- This car is "</span> << 2020 - year << <span class="stringliteral">"years old."</span> << endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="comment">// Iterating through an array of floats</span></div>
|
|
<div class="line"> <span class="keywordtype">double</span> total_tire_pressure = 0;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keywordtype">double</span> tire_pressure : car[<span class="stringliteral">"tire_pressure"</span>]) {</div>
|
|
<div class="line"> total_tire_pressure += tire_pressure;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> cout << <span class="stringliteral">"- Average tire pressure: "</span> << (total_tire_pressure / 4) << endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>The previous example had an array of objects, but we can use essentially the same approach with an object of objects.</p>
|
|
<div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> cars_json = R<span class="stringliteral">"( {</span></div>
|
|
<div class="line"><span class="stringliteral"> "identifier1":{ "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> "identifier2":{ "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> "identifier3":{ "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">} )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// Iterating through an array of objects</span></div>
|
|
<div class="line">ondemand::document doc = parser.iterate(cars_json);</div>
|
|
<div class="line"><span class="keywordflow">for</span> (ondemand::field key_car : doc.get_object()) {</div>
|
|
<div class="line"> <span class="comment">// If I need a string_view and/or, I can use key_car.unescaped_key() instead, but</span></div>
|
|
<div class="line"> <span class="comment">// key_car.key() will be more performant otherwise.</span></div>
|
|
<div class="line"> <span class="comment">// If we want a std::string_view instance but we do not care about escaping, we</span></div>
|
|
<div class="line"> <span class="comment">// can also use key_car.escaped_key().</span></div>
|
|
<div class="line"> cout << <span class="stringliteral">"identifier : "</span> << key_car.key() << std::endl;</div>
|
|
<div class="line"> <span class="comment">// I can now access the subobject:</span></div>
|
|
<div class="line"> ondemand::object car = key_car.value();</div>
|
|
<div class="line"> <span class="comment">// Accessing a field by name</span></div>
|
|
<div class="line"> cout << <span class="stringliteral">"Make/Model: "</span> << std::string_view(car[<span class="stringliteral">"make"</span>]) << <span class="stringliteral">"/"</span> << std::string_view(car[<span class="stringliteral">"model"</span>]) << endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="comment">// Casting a JSON element to an integer</span></div>
|
|
<div class="line"> uint64_t year = car[<span class="stringliteral">"year"</span>];</div>
|
|
<div class="line"> cout << <span class="stringliteral">"- This car is "</span> << 2020 - year << <span class="stringliteral">"years old."</span> << endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="comment">// Iterating through an array of floats</span></div>
|
|
<div class="line"> <span class="keywordtype">double</span> total_tire_pressure = 0;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keywordtype">double</span> tire_pressure : car[<span class="stringliteral">"tire_pressure"</span>]) {</div>
|
|
<div class="line"> total_tire_pressure += tire_pressure;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> cout << <span class="stringliteral">"- Average tire pressure: "</span> << (total_tire_pressure / 4) << endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>The following example illustrates how you may also iterate through object values, effectively visiting all key-value pairs in the object.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <iostream></span></div>
|
|
<div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
<div class="line"><span class="keyword">using namespace </span>std;</div>
|
|
<div class="line"><span class="keyword">using namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a>;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="comment">// ...</span></div>
|
|
<div class="line"> </div>
|
|
<div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> points_json = R<span class="stringliteral">"( [</span></div>
|
|
<div class="line"><span class="stringliteral"> { "12345" : {"x":12.34, "y":56.78, "z": 9998877} },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "12545" : {"x":11.44, "y":12.78, "z": 11111111} }</span></div>
|
|
<div class="line"><span class="stringliteral"> ] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// Parse and iterate through an array of objects</span></div>
|
|
<div class="line"><span class="keywordflow">for</span> (ondemand::object points : parser.iterate(points_json)) {</div>
|
|
<div class="line"> <span class="comment">// Iterating through an object, you iterate through key-value pairs (a 'field').</span></div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> point : points) {</div>
|
|
<div class="line"> <span class="comment">// Get the key corresponding the the field 'point'.</span></div>
|
|
<div class="line"> cout << <span class="stringliteral">"id: "</span> << std::string_view(point.unescaped_key()) << <span class="stringliteral">": ("</span>;</div>
|
|
<div class="line"> <span class="comment">// Get the value corresponding the the field 'point'.</span></div>
|
|
<div class="line"> ondemand::object xyz = point.value();</div>
|
|
<div class="line"> cout << xyz[<span class="stringliteral">"x"</span>].get_double() << <span class="stringliteral">", "</span>;</div>
|
|
<div class="line"> cout << xyz[<span class="stringliteral">"y"</span>].get_double() << <span class="stringliteral">", "</span>;</div>
|
|
<div class="line"> cout << xyz[<span class="stringliteral">"z"</span>].get_int64() << endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><h1><a class="anchor" id="autotoc_md12"></a>
|
|
Adding support for custom types</h1>
|
|
<p>There are 3 main ways provided by simdjson to deserialize a value into a custom type:</p>
|
|
<ol type="1">
|
|
<li>Provide a <a href="https://en.cppreference.com/w/cpp/language/template_specialization#Members_of_specializations"><b>template specialization</b> for member functions</a><ol type="a">
|
|
<li>Specialize <code>simdjson::ondemand::document::get</code> for the whole document</li>
|
|
<li>Specialize <code>simdjson::ondemand::value::get</code> for each value</li>
|
|
</ol>
|
|
</li>
|
|
<li>Using <code>tag_invoke</code> *(the recommended way if your system supports C++20 or better)*</li>
|
|
<li>Using static reflection (requires C++26 or better)</li>
|
|
</ol>
|
|
<p>We describe all of them in the following sections. Most users who have systems compatible with C++20 or better should skip ahead to using `tag_invoke` for custom types (C++20) as it is more powerful and simpler. The C++26 approach is even simpler.</p>
|
|
<h2><a class="anchor" id="autotoc_md13"></a>
|
|
1. Specialize <tt>simdjson::ondemand::value::get</tt> to get custom types (pre-C++20)</h2>
|
|
<p>Suppose you have your own types, such as a <code>Car</code> struct:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">struct </span>Car {</div>
|
|
<div class="line"> std::string make;</div>
|
|
<div class="line"> std::string model;</div>
|
|
<div class="line"> int64_t year;</div>
|
|
<div class="line"> std::vector<double> tire_pressure;</div>
|
|
<div class="line">};</div>
|
|
</div><!-- fragment --><p>You might want to write code that automatically parses the JSON content to your custom type:</p>
|
|
<div class="fragment"><div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = R<span class="stringliteral">"( [ { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 30.1, 31.0 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 29.8, 30.0 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> Car c(val);</div>
|
|
<div class="line"> std::cout << c.make << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
</div><!-- fragment --><p>We may do so by providing additional template definitions to the <code>ondemand::value</code> type. We may start by providing a definition for <code>std::vector<double></code> as follows. Observe how we guard the code with <code>#if !SIMDJSON_SUPPORTS_CONCEPTS</code>: that is because the necessary code is automatically provided by simdjson if C++20 (and concepts) are available. See Use `tag_invoke` for custom types if you have C++20 support.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#if !SIMDJSON_SUPPORTS_CONCEPTS</span></div>
|
|
<div class="line"><span class="comment">// The code is unnecessary with C++20:</span></div>
|
|
<div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<std::vector<double></a>></div>
|
|
<div class="line">simdjson::ondemand::value::get() noexcept {</div>
|
|
<div class="line"> ondemand::array array;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_array().get(array);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> std::vector<double> vec;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> v : array) {</div>
|
|
<div class="line"> <span class="keywordtype">double</span> val;</div>
|
|
<div class="line"> error = v.get_double().get(val);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> vec.push_back(val);</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> vec;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="preprocessor">#endif</span></div>
|
|
<div class="ttc" id="astructsimdjson_1_1simdjson__result_html"><div class="ttname"><a href="structsimdjson_1_1simdjson__result.html">simdjson::simdjson_result</a></div><div class="ttdoc">The result of a simdjson operation that could fail.</div><div class="ttdef"><b>Definition</b> <a href="error_8h_source.html#l00280">error.h:280</a></div></div>
|
|
</div><!-- fragment --><p>We may then provide support for our <code>Car</code> struct:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<Car></a> simdjson::ondemand::value::get() noexcept {</div>
|
|
<div class="line"> ondemand::object obj;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_object().get(obj);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> Car car;</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"make"</span>].get_string(car.make))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"model"</span>].get_string(car.model))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"year"</span>].get_int64().get(car.year))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"tire_pressure"</span>].get<std::vector<double>>().get(car.tire_pressure))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> car;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>And that is all that is needed! The following code is a complete example:</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
<div class="line"><span class="preprocessor">#include <iostream></span></div>
|
|
<div class="line"><span class="preprocessor">#include <vector></span></div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">using namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a>;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">struct </span>Car {</div>
|
|
<div class="line"> std::string make;</div>
|
|
<div class="line"> std::string model;</div>
|
|
<div class="line"> int64_t year;</div>
|
|
<div class="line"> std::vector<double> tire_pressure;</div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="preprocessor">#if !SIMDJSON_SUPPORTS_CONCEPTS</span></div>
|
|
<div class="line"><span class="comment">// This code is not necessary if you have a C++20 compliant system:</span></div>
|
|
<div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<std::vector<double></a>></div>
|
|
<div class="line">simdjson::ondemand::value::get() noexcept {</div>
|
|
<div class="line"> ondemand::array array;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_array().get(array);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> std::vector<double> vec;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> v : <a class="code hl_enumvalue" href="namespacesimdjson_1_1_s_i_m_d_j_s_o_n___i_m_p_l_e_m_e_n_t_a_t_i_o_n_1_1ondemand.html#ac915f0c06e5ab0363e09593bba651330af1f713c9e000f5d3f280adbd124df4f5">array</a>) {</div>
|
|
<div class="line"> <span class="keywordtype">double</span> val;</div>
|
|
<div class="line"> error = v.get_double().get(val);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> vec.push_back(val);</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> vec;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="preprocessor">#endif</span></div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<Car></a> simdjson::ondemand::value::get() noexcept {</div>
|
|
<div class="line"> ondemand::object obj;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_object().get(obj);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> Car car;</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"make"</span>].get_string(car.make))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"model"</span>].get_string(car.model))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"year"</span>].get_int64().get(car.year))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"tire_pressure"</span>].get<std::vector<double>>().get(car.tire_pressure))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> car;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keywordtype">int</span> main(<span class="keywordtype">void</span>) {</div>
|
|
<div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = R<span class="stringliteral">"( [ { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 30.1, 31.0 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 29.8, 30.0 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> Car c(val); <span class="comment">// an exception may be thrown</span></div>
|
|
<div class="line"> std::cout << c.make << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> EXIT_SUCCESS;</div>
|
|
<div class="line">}</div>
|
|
<div class="ttc" id="anamespacesimdjson_1_1_s_i_m_d_j_s_o_n___i_m_p_l_e_m_e_n_t_a_t_i_o_n_1_1ondemand_html_ac915f0c06e5ab0363e09593bba651330af1f713c9e000f5d3f280adbd124df4f5"><div class="ttname"><a href="namespacesimdjson_1_1_s_i_m_d_j_s_o_n___i_m_p_l_e_m_e_n_t_a_t_i_o_n_1_1ondemand.html#ac915f0c06e5ab0363e09593bba651330af1f713c9e000f5d3f280adbd124df4f5">simdjson::SIMDJSON_IMPLEMENTATION::ondemand::json_type::array</a></div><div class="ttdeci">@ array</div><div class="ttdoc">A JSON array ( [ 1, 2, 3 ... ] )</div></div>
|
|
</div><!-- fragment --><p>Observe that we require an explicit cast (<code>Car c(val)</code> instead of <code>for (Car c : doc) {</code>): it is by design. We require explicit casting.</p>
|
|
<p>If you prefer to avoid exceptions, you may modify the <code>main</code> function as follows:</p>
|
|
<div class="fragment"><div class="line"><span class="keywordtype">int</span> main(<span class="keywordtype">void</span>) {</div>
|
|
<div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = R<span class="stringliteral">"( [ { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 30.1, 31.0 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 29.8, 30.0 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc;</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keyword">auto</span> error = parser.iterate(json).get(doc);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error) << std::endl; <span class="keywordflow">return</span> EXIT_FAILURE; }</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> Car c;</div>
|
|
<div class="line"> error = val.get<Car>().get(c);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error) << std::endl; <span class="keywordflow">return</span> EXIT_FAILURE; }</div>
|
|
<div class="line"> std::cout << c.make << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> EXIT_SUCCESS;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>Our example is limited to <code>ondemand::value</code> instances. If you wish to also be able to map directly the document instance itself to a custom type, you need to provide the definitions to the <code>ondemand::document</code> type. In this instance, we must replace the function with signature <code>simdjson_result<Car> simdjson::ondemand::value::get()</code> with a function having signature <code>simdjson_result<Car> simdjson::ondemand::document::get() &</code>. The following is a complete example:</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
<div class="line"><span class="preprocessor">#include <iostream></span></div>
|
|
<div class="line"><span class="preprocessor">#include <vector></span></div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">using namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a>;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">struct </span>Car {</div>
|
|
<div class="line"> std::string make;</div>
|
|
<div class="line"> std::string model;</div>
|
|
<div class="line"> int64_t year;</div>
|
|
<div class="line"> std::vector<double> tire_pressure;</div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="preprocessor">#if !SIMDJSON_SUPPORTS_CONCEPTS</span></div>
|
|
<div class="line"><span class="comment">// This code is not necessary if you have a C++20 compliant system:</span></div>
|
|
<div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<std::vector<double></a>></div>
|
|
<div class="line">simdjson::ondemand::value::get() noexcept {</div>
|
|
<div class="line"> ondemand::array array;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_array().get(array);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> std::vector<double> vec;</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> v : <a class="code hl_enumvalue" href="namespacesimdjson_1_1_s_i_m_d_j_s_o_n___i_m_p_l_e_m_e_n_t_a_t_i_o_n_1_1ondemand.html#ac915f0c06e5ab0363e09593bba651330af1f713c9e000f5d3f280adbd124df4f5">array</a>) {</div>
|
|
<div class="line"> <span class="keywordtype">double</span> val;</div>
|
|
<div class="line"> error = v.get_double().get(val);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> vec.push_back(val);</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> vec;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="preprocessor">#endif</span></div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<Car></a> simdjson::ondemand::document::get() & <span class="keyword">noexcept</span> {</div>
|
|
<div class="line"> ondemand::object obj;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_object().get(obj);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> Car car;</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"make"</span>].get_string(car.make))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"model"</span>].get_string(car.model))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"year"</span>].get_int64().get(car.year))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"tire_pressure"</span>].get<std::vector<double>>().get(car.tire_pressure))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> car;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">template</span> <></div>
|
|
<div class="line">simdjson_inline <a class="code hl_struct" href="structsimdjson_1_1simdjson__result.html">simdjson_result<Car></a> simdjson::ondemand::document::get() noexcept {</div>
|
|
<div class="line"> ondemand::object obj;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = get_object().get(obj);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> Car car;</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"make"</span>].get_string(car.make))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"model"</span>].get_string(car.model))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"year"</span>].get_int64().get(car.year))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>((error = obj[<span class="stringliteral">"tire_pressure"</span>].get<std::vector<double>>().get(car.tire_pressure))) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> car;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keywordtype">int</span> main(<span class="keywordtype">void</span>) {</div>
|
|
<div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = R<span class="stringliteral">"( { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] } )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> Car c(doc);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::cout << c.make << std::endl;</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">return</span> EXIT_SUCCESS;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><h2><a class="anchor" id="autotoc_md14"></a>
|
|
2. Use <tt>tag_invoke</tt> for custom types (C++20)</h2>
|
|
<p>The simdjson library takes advantage of C++20. An immediate benefit is that you can deserialize JSON data directly in standard containers and other standard value types:</p>
|
|
<div class="fragment"><div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> json = R<span class="stringliteral">"({"data" : [1,2,3,4]})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document d = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral">std::vector<uint8_t> array = d[</span><span class="stringliteral">"data"</span>].get<std::vector<uint8_t>>();</div>
|
|
</div><!-- fragment --><p>Appending to an existing container is just as easy:</p>
|
|
<div class="fragment"><div class="line">std::vector<uint32_t> array = {0, 0};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> json = R<span class="stringliteral">"({"data" : [1,2,3,4]})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document d = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">d[</span><span class="stringliteral">"data"</span>].get<std::vector<uint32_t>>(array);</div>
|
|
<div class="line"><span class="comment">// array is now {0,0,1,2,3,4}</span></div>
|
|
</div><!-- fragment --><p>In C++20, the standard introduced the notion of <em>customization point</em>. A customization point is a function or function object that can be customized for different types. It allows library authors to provide default behavior while giving users the ability to override this behavior for specific types.</p>
|
|
<p>A tag_invoke function serves as a mechanism for customization points. It is not directly part of the C++ standard library but is often used in libraries that implement customization points. The tag_invoke function is typically a generic function that takes a tag type and additional arguments. The first argument is usually a tag type (often an empty struct) that uniquely identifies the customization point (e.g., deserialization of custom types in simdjson). Users or library providers can specialize tag_invoke for their types by defining it in the appropriate namespace, often inline namespace.</p>
|
|
<p>If your system supports C++20, we recommend that you adopt the <code>tag_invoke</code> approach instead to deserialize custom types. It may prove to be considerably simpler. When simdjson detects the necessary support, it sets the <code>SIMDJSON_SUPPORTS_CONCEPTS</code> macro to 1, otherwise it is set to 0.</p>
|
|
<p>Consider a custom class <code>Car</code>:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">struct </span>Car {</div>
|
|
<div class="line"> std::string make;</div>
|
|
<div class="line"> std::string model;</div>
|
|
<div class="line"> <span class="keywordtype">int</span> year;</div>
|
|
<div class="line"> std::vector<float> tire_pressure;</div>
|
|
<div class="line">};</div>
|
|
</div><!-- fragment --><p>Observe how we define the class to use types that simdjson does not directly support (<code>float</code>, <code>int</code>). With C++20 support, the library grabs from the JSON the generic type (<code>double</code>, <code>int</code>) and then it casts it automatically.</p>
|
|
<p>You may support deserializing directly from a JSON value or document to your own <code>Car</code> instance by defining a single <code>tag_invoke</code> function:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a> {</div>
|
|
<div class="line"><span class="comment">// This tag_invoke MUST be inside simdjson namespace</span></div>
|
|
<div class="line"><span class="keyword">template</span> <<span class="keyword">typename</span> simdjson_value></div>
|
|
<div class="line"><span class="keyword">auto</span> tag_invoke(deserialize_tag, simdjson_value &val, Car& car) {</div>
|
|
<div class="line"> ondemand::object obj;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = val.get_object().get(obj);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">if</span> ((error = obj[<span class="stringliteral">"make"</span>].get_string(car.make))) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">if</span> ((error = obj[<span class="stringliteral">"model"</span>].get_string(car.model))) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">if</span> ((error = obj[<span class="stringliteral">"year"</span>].get(car.year))) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">if</span> ((error = obj[<span class="stringliteral">"tire_pressure"</span>].get<std::vector<float>>().get(</div>
|
|
<div class="line"> car.tire_pressure))) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> <a class="code hl_enumvalue" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681">simdjson::SUCCESS</a>;</div>
|
|
<div class="line">}</div>
|
|
<div class="line">} <span class="comment">// namespace simdjson</span></div>
|
|
<div class="ttc" id="anamespacesimdjson_html_a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681"><div class="ttname"><a href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681">simdjson::SUCCESS</a></div><div class="ttdeci">@ SUCCESS</div><div class="ttdoc">No error.</div><div class="ttdef"><b>Definition</b> <a href="error_8h_source.html#l00020">error.h:20</a></div></div>
|
|
</div><!-- fragment --><p>Observe how we call <code>get<std::vector<float>>()</code> even though we never defined support for <code>std::vector<float></code> in the simdjson library: it is all automated thanks to C++20 concepts.</p>
|
|
<p>Importantly, the <code>tag_invoke</code> function must be inside the <code>simdjson</code> namespace. Let us explain each argument of <code>tag_invoke</code> function.</p>
|
|
<ul>
|
|
<li><code>simdjson::deserialize_tag</code>: it is the tag for Customization Point Object (CPO). You may often ignore this parameter. It is used to indicate that you mean to provide a deserialization function for simdjson.</li>
|
|
<li><code>var</code>: It receives automatically a <code>simdjson</code> value type (document, value, document_reference).</li>
|
|
<li>The third parameter is an instance of the type that you want to support.</li>
|
|
</ul>
|
|
<p>You can use it like so:</p>
|
|
<div class="fragment"><div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> json =</div>
|
|
<div class="line"> R<span class="stringliteral">"( { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] })"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral">Car c(doc);</span></div>
|
|
<div class="line"><span class="stringliteral">std::cout << c.make << std::endl;</span></div>
|
|
</div><!-- fragment --><p>Observe how we first get an instance of <code>document</code> and then we cast.</p>
|
|
<p>You can also handle errors explicitly:</p>
|
|
<div class="fragment"><div class="line">Car c;</div>
|
|
<div class="line"><span class="keyword">auto</span> error = doc.get(c);</div>
|
|
<div class="line"><span class="keywordflow">if</span>(error) { std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error); <span class="keywordflow">return</span> <span class="keyword">false</span>; }</div>
|
|
<div class="line">std::cout << c.make << std::endl;</div>
|
|
</div><!-- fragment --><p>You can also read instances of <code>Car</code> from an array or an object: </p><div class="fragment"><div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> json =</div>
|
|
<div class="line"> R<span class="stringliteral">"( [ { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 30.1, 31.0 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 29.8, 30.0 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"> simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> simdjson::ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> Car c(val); <span class="comment">// an exception may be thrown</span></div>
|
|
<div class="line"> std::cout << c.year << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
</div><!-- fragment --><p>Observe how we first get a generic (<code>val</code>) which we cast to <code>Car</code>. It is by design: we require explicit casting. The cast may throw an exception.</p>
|
|
<p>Once more, you can handle errors explicitly:</p>
|
|
<div class="fragment"><div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> Car c;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = val.get(c);</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(error) { std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error) << std::endl; <span class="keywordflow">return</span> <span class="keyword">false</span>; }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>You can also use the custom <code>Car</code> type as part of a template such as <code>std::vector</code>:</p>
|
|
<div class="fragment"><div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line">simdjson::ondemand::document doc = parser.iterate(json);</div>
|
|
<div class="line">std::vector<Car> cars(doc);</div>
|
|
<div class="line"><span class="comment">// visual studio users need an explicit call:</span></div>
|
|
<div class="line"><span class="comment">// std::vector<Car> cars = doc.get<std::vector<Car>>();</span></div>
|
|
<div class="line"><span class="comment">// because the compiler does not know whether to convert</span></div>
|
|
<div class="line"><span class="comment">// doc to an unsigned int or to a vector.</span></div>
|
|
<div class="line"><span class="keywordflow">for</span>(Car& c : cars) {</div>
|
|
<div class="line"> std::cout << c.year << std::endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>By default, we support a wide range of standard templates such as <code>std::vector</code>, <code>std::list</code>, <code>std::set</code>, <code>std::stack</code>, <code>std:queue</code>, <code>std:deque</code>, <code>std::priority_queue</code>, <code>std::unordered_set</code>, <code>std::multiset</code>, <code>std::unordered_multiset</code>, <code>std::unique_ptr</code>, <code>std::shared_ptr</code>, <code>std::optional</code>, etc. They are handled automatically.</p>
|
|
<p>E.g., you can recover an <code>std::unique_ptr<Car></code> like so: </p><div class="fragment"><div class="line"><span class="keywordtype">int</span> main() {</div>
|
|
<div class="line"> <span class="keyword">auto</span> <span class="keyword">const</span> json = R<span class="stringliteral">"( { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] })"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> simdjson::ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::unique_ptr<Car> c(doc);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::cout << c->make << std::endl;</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">return</span> EXIT_SUCCESS;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>You may also conditionally fill in <code>std::optional</code> values.</p>
|
|
<div class="fragment"><div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json =</div>
|
|
<div class="line"> R<span class="stringliteral">"( { "car1": { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::optional<Car> car;</span></div>
|
|
<div class="line"><span class="stringliteral"> error = doc[</span><span class="stringliteral">"key not found"</span>].get<std::optional<Car>>().get(car);</div>
|
|
<div class="line"> <span class="comment">// car has no value, error != simdjson::SUCCESS</span></div>
|
|
<div class="line"> error = doc[<span class="stringliteral">"car1"</span>].get<std::optional<Car>>().get(car);</div>
|
|
<div class="line"> <span class="comment">// car has value Car{"Toyota", "Camry", 2018, {40.1f, 39.9f}}</span></div>
|
|
<div class="line"> <span class="comment">// error is simdjson::SUCCESS</span></div>
|
|
</div><!-- fragment --><p>You can also deserialize to map-like types with keys that can be constructed from <code>std::string_view</code> instances:</p>
|
|
<div class="fragment"><div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json =</div>
|
|
<div class="line"> R<span class="stringliteral">"( { "car1": { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::map<std::string,Car> cars;</span></div>
|
|
<div class="line"><span class="stringliteral"> error = doc.get<std::map<std::string,Car>>().get(cars);</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="comment">// car has value car1->Car{"Toyota", "Camry", 2018, {40.1f, 39.9f}}</span></div>
|
|
<div class="line"> <span class="comment">// error is simdjson::SUCCESS</span></div>
|
|
</div><!-- fragment --><p>And so forth.</p>
|
|
<p>Advanced users may want to overwrite the defaults provided by the simdjson library. Suppose for example that you want to construct an instance of <code>std::list<Car></code>, but you also want to filter out any car made by Toyota. You may provide your own <code>tag_invoke</code> function:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a> {</div>
|
|
<div class="line"><span class="comment">// suppose we want to filter out all Toyotas</span></div>
|
|
<div class="line"><span class="keyword">template</span> <<span class="keyword">typename</span> simdjson_value></div>
|
|
<div class="line"><span class="keyword">auto</span> tag_invoke(deserialize_tag, simdjson_value &val, std::list<Car>& car) {</div>
|
|
<div class="line"> ondemand::array arr;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = val.get_array().get(arr);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> v : arr) {</div>
|
|
<div class="line"> Car c;</div>
|
|
<div class="line"> <span class="keywordflow">if</span> ((error = v.get<Car>().get(c))) {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> error;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(c.make != <span class="stringliteral">"Toyota"</span>) {</div>
|
|
<div class="line"> car.push_back(c);</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> <a class="code hl_enumvalue" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681">simdjson::SUCCESS</a>;</div>
|
|
<div class="line">}</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>With this code, deserializing an <code>std::list<Car></code> instance would capture only the cars that are not made by Toyota.</p>
|
|
<h2><a class="anchor" id="autotoc_md15"></a>
|
|
3. Using static reflection (C++26)</h2>
|
|
<p>If you have a C++26 compatible compiler, you can compile your code with the <code>SIMDJSON_STATIC_REFLECTION</code> macro set:</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#define SIMDJSON_STATIC_REFLECTION 1</span></div>
|
|
<div class="line"><span class="comment">//...</span></div>
|
|
<div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
</div><!-- fragment --><p>Then you can deserialize a type such as <code>Car</code> automatically:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">struct </span>Car {</div>
|
|
<div class="line"> std::string make;</div>
|
|
<div class="line"> std::string model;</div>
|
|
<div class="line"> <span class="keywordtype">int</span> year;</div>
|
|
<div class="line"> std::vector<float> tire_pressure;</div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line">std::string json = R<span class="stringliteral">"( { "make": "Toyota", "model": "Camry", "year": 2018,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ] } )";</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(<a class="code hl_function" href="namespacesimdjson.html#ac4131fd8cef9c8046d0bc9aee8d44d16">simdjson::pad</a>(json));</span></div>
|
|
<div class="line"><span class="stringliteral">Car c = doc.get<Car>();</span></div>
|
|
</div><!-- fragment --><p>We try to automate the parsing of any given structure or class by looking at its non-static public members. At compile-time, the library looks at a simple structure like <code>Car</code> and maps it to parsing code. We call the default constructor, and then assign values to the public members.</p>
|
|
<p>If a key is missing in the JSON document, an error is generated (<code>NO_SUCH_FIELD</code>), except if the attribute is of a type like <code>std::optional</code> (<code>simdjson::concepts::optional_type</code>).</p>
|
|
<p>Sometimes you might want to only extract some attributes from the JSON. You can achieve this result with the <code>extract_into</code> method supported by both <code>object</code> and <code>document</code> instances. It returns an error code that evaluates to false when there is no error.</p>
|
|
<p>Consider the following example where you only want to parse the make and the model from the JSON:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">struct </span>car_type {</div>
|
|
<div class="line"> std::string make;</div>
|
|
<div class="line"> std::string model;</div>
|
|
<div class="line"> uint64_t year;</div>
|
|
<div class="line"> std::vector<double> tire_pressure;</div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keywordtype">void</span> f() {</div>
|
|
<div class="line"> <span class="keyword">auto</span> json = R<span class="stringliteral">"( {</span></div>
|
|
<div class="line"><span class="stringliteral"> "make": "Toyota",</span></div>
|
|
<div class="line"><span class="stringliteral"> "model": "Camry",</span></div>
|
|
<div class="line"><span class="stringliteral"> "year": 2024,</span></div>
|
|
<div class="line"><span class="stringliteral"> "tire_pressure": [ 40.1, 39.9 ]</span></div>
|
|
<div class="line"><span class="stringliteral"> } )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> Car car{};</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keyword">auto</span> error = doc.extract_into<<span class="stringliteral">"make"</span>,<span class="stringliteral">"model"</span>>(car);</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(error) { }</div>
|
|
<div class="line"> <span class="comment">// only car.make and car.</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><h3><a class="anchor" id="autotoc_md16"></a>
|
|
Special cases</h3>
|
|
<p>However, there are instances where the construction cannot be easily automated. Let us consider a class without any public member.</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">class </span>MyDate {</div>
|
|
<div class="line"><span class="keyword">public</span>:</div>
|
|
<div class="line"> <span class="keywordtype">void</span> assign(std::string_view str) {</div>
|
|
<div class="line"> date_str = str;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keyword">const</span> std::string& to_string()<span class="keyword"> const </span>{</div>
|
|
<div class="line"> <span class="keywordflow">return</span> date_str;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"><span class="keyword">private</span>:</div>
|
|
<div class="line"> std::string date_str;</div>
|
|
<div class="line">};</div>
|
|
</div><!-- fragment --><p>This class has a default constructor, but it must be initialized with the <code>assign</code> method. We need to help the library with a <code>tag_invoke</code> function (just as in the C++20 case).</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">namespace </span><a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a> {</div>
|
|
<div class="line"><span class="keyword">template</span> <<span class="keyword">typename</span> simdjson_value></div>
|
|
<div class="line"><span class="keyword">auto</span> tag_invoke(deserialize_tag, simdjson_value &val, MyDate& date) {</div>
|
|
<div class="line"> std::string_view str;</div>
|
|
<div class="line"> <span class="keyword">auto</span> error = val.get_string().get(str);</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(error) { <span class="keywordflow">return</span> error; }</div>
|
|
<div class="line"> date.assign(str);</div>
|
|
<div class="line"> <span class="keywordflow">return</span> <a class="code hl_enumvalue" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681">simdjson::SUCCESS</a>;</div>
|
|
<div class="line">}</div>
|
|
<div class="line">} <span class="comment">// namespace simdjson</span></div>
|
|
</div><!-- fragment --><p>Once this is done, we can now automatically parse a custom type like <code>complicated_weather_data</code> containing <code>MyDate</code> values.</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">struct </span>complicated_weather_data {</div>
|
|
<div class="line"> std::vector<MyDate> time;</div>
|
|
<div class="line"> std::vector<float> temperature;</div>
|
|
<div class="line">};</div>
|
|
</div><!-- fragment --><p>The code might be as simple as the following.</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">auto</span> padded = R<span class="stringliteral">"({"time":["2023-03-15T12:00:00Z"],"temperature":[42]})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(padded);</span></div>
|
|
<div class="line"><span class="stringliteral">complicated_weather_data p = doc.get<complicated_weather_data>();</span></div>
|
|
</div><!-- fragment --><p>Thus you can combine C++26 static reflection with custom deserialization functions.</p>
|
|
<p>You can also automatically serialize the <code>Car</code> instance to a JSON string, see our <a class="el" href="md_doc_2builder.html">Builder documentation</a>.</p>
|
|
<h2><a class="anchor" id="autotoc_md17"></a>
|
|
The simdjson::from shortcut (experimental, C++20)</h2>
|
|
<p>For even more convenience, you can parse a JSON document directly to a supported type without a document instance like so:</p>
|
|
<div class="fragment"><div class="line">Car car = simdjson::from(json);</div>
|
|
</div><!-- fragment --><p>You can also use the <code>simdjson::from</code> syntax without exceptions, like so: </p><div class="fragment"><div class="line">Car car;</div>
|
|
<div class="line"><a class="code hl_enumeration" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9">simdjson::error_code</a> err = simdjson::from(json_car).get(car);</div>
|
|
<div class="ttc" id="anamespacesimdjson_html_a7b735a3a50ba79e3f7f14df5f77d8da9"><div class="ttname"><a href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9">simdjson::error_code</a></div><div class="ttdeci">error_code</div><div class="ttdoc">All possible errors returned by simdjson.</div><div class="ttdef"><b>Definition</b> <a href="error_8h_source.html#l00019">error.h:19</a></div></div>
|
|
</div><!-- fragment --><p>You can also use the <code>simdjson::from</code> syntax to iterate over an array.</p>
|
|
<div class="fragment"><div class="line"><span class="keywordflow">for</span>(<span class="keyword">auto</span> val : <a class="code hl_namespace" href="namespacesimdjson.html">simdjson</a>::from(json).array()) {</div>
|
|
<div class="line"> Car c = val.get<Car>(); <span class="comment">// ...</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p>Standard STL types are supported:</p>
|
|
<div class="fragment"><div class="line">std::map<std::string, std::string> obj =</div>
|
|
<div class="line"> simdjson::from(R<span class="stringliteral">"({"key": "value"})"_padded);</span></div>
|
|
</div><!-- fragment --><p>The <code>simdjson::from</code> construction is EXPERIMENTAL and subject to changes.</p>
|
|
<h1><a class="anchor" id="autotoc_md18"></a>
|
|
Minifying JSON strings without parsing</h1>
|
|
<p>In some cases, you may have valid JSON strings that you do not wish to parse but that you wish to minify. That is, you wish to remove all unnecessary spaces. We have a fast function for this purpose (<code><a class="el" href="namespacesimdjson.html#aeb4ef5cab43d52da3fdd99cb689aff2c" title="Minifies a JSON element or document, printing the smallest possible valid JSON.">simdjson::minify</a>(const char * input, size_t length, const char * output, size_t& new_length)</code>). This function does not validate your content, and it does not parse it. It is much faster than parsing the string and re-serializing it in minified form (<code><a class="el" href="namespacesimdjson.html#aeb4ef5cab43d52da3fdd99cb689aff2c" title="Minifies a JSON element or document, printing the smallest possible valid JSON.">simdjson::minify</a>(parser.parse())</code>). Usage is relatively simple. You must pass an input pointer with a length parameter, as well as an output pointer and an output length parameter (by reference). The output length parameter is not read, but written to. The output pointer should point to a valid memory region that is as large as the original string length. The input pointer and input length are read, but not written to.</p>
|
|
<div class="fragment"><div class="line"><span class="comment">// Starts with a valid JSON document as a string.</span></div>
|
|
<div class="line"><span class="comment">// It does not have to be null-terminated.</span></div>
|
|
<div class="line"><span class="keyword">const</span> <span class="keywordtype">char</span> * some_string = <span class="stringliteral">"[ 1, 2, 3, 4] "</span>;</div>
|
|
<div class="line"><span class="keywordtype">size_t</span> length = std::strlen(some_string);</div>
|
|
<div class="line"><span class="comment">// Create a buffer to receive the minified string. Make sure that there is enough room (length bytes).</span></div>
|
|
<div class="line">std::unique_ptr<char[]> buffer{<span class="keyword">new</span> <span class="keywordtype">char</span>[length]};</div>
|
|
<div class="line"><span class="keywordtype">size_t</span> new_length{}; <span class="comment">// It will receive the minified length.</span></div>
|
|
<div class="line"><span class="keyword">auto</span> error = <a class="code hl_function" href="namespacesimdjson.html#aeb4ef5cab43d52da3fdd99cb689aff2c">simdjson::minify</a>(some_string, length, buffer.get(), new_length);</div>
|
|
<div class="line"><span class="keywordflow">if</span>(error) { std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error); }</div>
|
|
<div class="line"><span class="comment">// The buffer variable now has "[1,2,3,4]" and new_length has value 9.</span></div>
|
|
<div class="ttc" id="anamespacesimdjson_html_aeb4ef5cab43d52da3fdd99cb689aff2c"><div class="ttname"><a href="namespacesimdjson.html#aeb4ef5cab43d52da3fdd99cb689aff2c">simdjson::minify</a></div><div class="ttdeci">std::string minify(T x)</div><div class="ttdoc">Minifies a JSON element or document, printing the smallest possible valid JSON.</div><div class="ttdef"><b>Definition</b> <a href="dom_2serialization_8h_source.html#l00274">serialization.h:274</a></div></div>
|
|
</div><!-- fragment --><p>Though it does not validate the JSON input, it will detect when the document ends with an unterminated string. E.g., it would refuse to minify the string <code>"this string is not terminated</tt> because of the missing final quote.
|
|
|
|
|
|
@section autotoc_md19 UTF-8 validation (alone)
|
|
|
|
|
|
The simdjson library has fast functions to validate UTF-8 strings. They are many times faster than most functions commonly found in libraries. You can use our fast functions, even if you do not care about JSON.
|
|
|
|
@icode{cpp}
|
|
const char * some_string = "[ 1, 2, 3, 4] ";
|
|
size_t length = std::strlen(some_string);
|
|
bool is_ok = simdjson::validate_utf8(some_string, length);
|
|
@endicode
|
|
|
|
The UTF-8 validation function merely checks that the input is valid UTF-8: it works with strings in general, not just JSON strings.
|
|
|
|
Your input string does not need any padding. Any string will do. The <tt>validate_utf8</tt> function does not do any memory allocation on the heap, and it does not throw exceptions.
|
|
|
|
If you find yourself needing only fast Unicode functions, consider using the simdutf library instead: https://github.com/simdutf/simdutf
|
|
|
|
@section autotoc_md20 JSON Pointer
|
|
|
|
|
|
The simdjson library also supports <a href="<a href="https://tools.ietf.org/html/rfc6901">https://tools.ietf.org/html/rfc6901</a>" >JSON pointer</a> through the <tt>at_pointer()</tt> method, letting you reach further down into the document in a single call. JSON Pointer is supported by both the <a href="<a href="https://github.com/simdjson/simdjson/blob/master/doc/dom.md#json-pointer">https://github.com/simdjson/simdjson/blob/master/doc/dom.md#json-pointer</a>" >DOM approach</a> as well as the On-Demand approach.
|
|
|
|
<strong>Note:</strong> When matching keys, we do a byte-by-byte comparison. We do not unescape keys when matching.
|
|
|
|
Consider the following example:
|
|
|
|
@icode{cpp}
|
|
auto cars_json = R"( [ { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] }, { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] }, { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] } ] )"_padded;
|
|
ondemand::parser parser;
|
|
auto cars = parser.iterate(cars_json);
|
|
cout << cars.at_pointer("/0/tire_pressure/1") << endl; // Prints 39.9
|
|
@endicode
|
|
|
|
A JSON Pointer path is a sequence of segments each starting with the '/' character. Within arrays, a zero-based integer
|
|
index allows you to select the indexed node. Within objects, the string value of the key allows you to
|
|
select the value. If your keys contain the characters '/' or '~', they must be escaped as '~1' and
|
|
'~0' respectively. An empty JSON Pointer Path refers to the whole document.
|
|
|
|
For multiple JSON Pointer queries on a document, one can call <tt>at_pointer</tt> multiple times.
|
|
|
|
@icode{cpp}
|
|
auto cars_json = R"( [ { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] }, { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] }, { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] } ] )"_padded;
|
|
ondemand::parser parser;
|
|
auto cars = parser.iterate(cars_json);
|
|
size_t size = cars.count_elements();
|
|
|
|
for (size_t i = 0; i < size; i++) {
|
|
std::string json_pointer = "/" + std::to_string(i) + "/tire_pressure/1";
|
|
double x = cars.at_pointer(json_pointer);
|
|
std::cout << x << std::endl; // Prints 39.9, 31 and 30
|
|
}
|
|
@endicode
|
|
|
|
In most instances, a JSON Pointer is an ASCII string and the keys in a JSON document
|
|
are ASCII strings. We support UTF-8 in JSON Pointer, but key values are matched exactly, without unescaping or Unicode normalization. We do a byte-by-byte comparison. The e acute character is
|
|
considered distinct from its escaped version <tt>\\u00E9</tt>. E.g.,
|
|
|
|
@icode{cpp}
|
|
const padded_string json = "{"\u00E9":123}"_padded;
|
|
auto doc = parser.iterate(json);
|
|
doc.at_pointer("/\u00E9") == 123; // true
|
|
doc.at_pointer((const char*)u8"/\u00E9") // returns an error (NO_SUCH_FIELD)
|
|
@endicode
|
|
|
|
Note that <tt>at_pointer</tt> calls @ref "rewind" "`rewind`" to reset the parser at the beginning of the document. Hence, it invalidates all previously parsed values, objects and arrays: make sure to consume the values between each call to <tt>at_pointer</tt>. Consider the following example where one wants to store each object from the JSON into a vector of <tt>struct car_type</tt>:
|
|
|
|
@icode{cpp}
|
|
struct car_type {
|
|
std::string make;
|
|
std::string model;
|
|
uint64_t year;
|
|
std::vector<double> tire_pressure;
|
|
car_type(std::string_view _make, std::string_view _model, uint64_t _year,
|
|
std::vector<double>&& _tire_pressure) :
|
|
make{_make}, model{_model}, year(_year), tire_pressure(_tire_pressure) {}
|
|
};
|
|
|
|
auto cars_json = R"( [ { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] }, { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] }, { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] } ] )"_padded;
|
|
|
|
ondemand::parser parser;
|
|
std::vector<double> measured;
|
|
ondemand::document cars = parser.iterate(cars_json);
|
|
std::vector<car_type> content;
|
|
for (int i = 0; i < 3; i++) {
|
|
std::string json_pointer = "/" + std::to_string(i);
|
|
// Each successive at_pointer call invalidates
|
|
// previously parsed values, strings, objects and array.
|
|
ondemand::object obj(cars.at_pointer(json_pointer).get_object());
|
|
// We materialize the object.
|
|
std::string_view make = obj["make"];
|
|
std::string_view model = obj["model"];
|
|
uint64_t year(obj["year"]);
|
|
// We materialize the array.
|
|
ondemand::array arr(obj["tire_pressure"].get_array());
|
|
std::vector<double> values;
|
|
for(auto x : arr) {
|
|
double value_double(x.get_double());
|
|
values.push_back(value_double);
|
|
}
|
|
content.emplace_back(make, model, year, std::move(values));
|
|
}
|
|
@endicode
|
|
|
|
Furthermore, <tt>at_pointer</tt> calls <tt>rewind</tt> at the beginning of the call (i.e. the document is not reset after <tt>at_pointer</tt>). Consider the following example,
|
|
|
|
@icode{cpp}
|
|
auto json = R"( { "k0": 27, "k1": [13,26], "k2": true } )"_padded;
|
|
ondemand::parser parser;
|
|
auto doc = parser.iterate(json);
|
|
std::cout << doc.at_pointer("/k1/1") << std::endl; // Prints 26
|
|
std::cout << doc.at_pointer("/k2") << std::endl; // Prints true
|
|
doc.rewind(); // Need to manually rewind to be able to use find_field properly from start of document
|
|
std::cout << doc.find_field("k0") << std::endl; // Prints 27
|
|
@endicode
|
|
|
|
When the JSON Pointer Path is the empty string (<tt>""</tt>) applied to a scalar document (lone string, number, Boolean or null), a SCALAR_DOCUMENT_AS_VALUE error is returned because scalar document cannot
|
|
be represented as <tt>value</tt> instances. You can check that a document is a scalar with the method <tt>scalar()</tt>.
|
|
|
|
@section autotoc_md21 JSONPath
|
|
|
|
|
|
The simdjson library supports a subset of <a href="<a href="https://www.rfc-editor.org/rfc/rfc9535">https://www.rfc-editor.org/rfc/rfc9535</a>" >JSONPath</a> (RFC 9535) through the <tt>at_path()</tt> method, allowing you to reach further into the document in a single call. The subset of JSONPath that is implemented is the subset that is trivially convertible into the JSON Pointer format, using <tt>.</tt> to access a field and <tt>[]</tt> to access a specific index.
|
|
|
|
This implementation relies on <tt>at_path()</tt> converting its argument to JSON Pointer and then calling <tt>at_pointer</tt>, which makes use of
|
|
@ref "rewind" "`rewind`" to reset the parser at the beginning of the document. Hence, it invalidates all previously parsed values, objects
|
|
and arrays: make sure to consume the values between each call to <tt>at_path</tt>.
|
|
|
|
Consider the following example:
|
|
|
|
@icode{cpp}
|
|
auto cars_json = R"( [ { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] }, { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] }, { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] } ] )"_padded;
|
|
ondemand::parser parser;
|
|
auto cars = parser.iterate(cars_json);
|
|
cout << cars.at_path("[0].tire_pressure[1]") << endl; // Prints 39.9
|
|
@endicode
|
|
|
|
A call to <tt>at_path(json_path)</tt> can result in any of the errors that are returned by the <tt>at_pointer</tt> method and if the conversion of <tt>json_path</tt> to JSON Pointer fails, it will lead to an <tt>simdjson::INVALID_JSON_POINTER</tt>error.
|
|
|
|
@icode{cpp}
|
|
auto cars_json = R"( [ { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] }, { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] }, { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] } ] )"_padded;
|
|
ondemand::parser parser;
|
|
auto cars = parser.iterate(cars_json);
|
|
ASSERT_ERROR(cars.at_path("[0].tire_presure[1").get(x), INVALID_JSON_POINTER); // Fails on conversion to JSON Pointer, since last square bracket was not properly closed.
|
|
ASSERT_ERROR(cars.at_path("[0].incorrect_field[1]").get(x), NO_SUCH_FIELD); // Conversion to JSON Pointer succeeds, but fails on at_pointer() since the path is invalid.
|
|
@endicode
|
|
|
|
In most instances, a JSONPath is an ASCII string and the keys in a JSON document
|
|
are ASCII strings. We support UTF-8 within a JSONPath expression, but key values are
|
|
matched exactly, without unescaping or Unicode normalization. We do a byte-by-byte comparison.
|
|
The e acute character is considered distinct from its escaped version <tt>\\u00E9</tt>. E.g.,
|
|
|
|
@icode{cpp}
|
|
const padded_string json = "{"\u00E9":123}"_padded;
|
|
auto doc = parser.iterate(json);
|
|
doc.at_path(".\u00E9") == 123; // true
|
|
doc.at_path((const char*)u8".\u00E9") // returns an error (NO_SUCH_FIELD)
|
|
@endicode
|
|
|
|
|
|
We also support the <tt>\$</tt> prefix. When you start a JSONPath expression with $, you are indicating that the path starts from the root of the JSON document. E.g.,
|
|
|
|
@icode{cpp}
|
|
auto json = R"( { "c" :{ "foo": { "a": [ 10, 20, 30 ] }}, "d": { "foo2": { "a": [ 10, 20, 30 ] }} , "e": 120 })"_padded;
|
|
ondemand::parser parser;
|
|
ondemand::document doc = parser.iterate(json);
|
|
ondemand::object obj = doc.get_object();
|
|
int64_t x = obj.at_path("$.c.foo.a[1]"); // 20
|
|
x = obj.at_path("$.d.foo2.a.2"); // 30
|
|
@endicode
|
|
|
|
@section autotoc_md22 Using <tt>for_each_at_path_with_wildcard</tt> for JSONPath Queries (On-Demand)
|
|
|
|
The <tt>for_each_at_path_with_wildcard</tt> function in simdjson extends the JSONPath querying capabilities by supporting wildcard expressions (<tt>*</tt>) in JSON paths. It calls a user-provided callback for each matching element, avoiding the need to materialize all results into a vector. For example, you can use <tt>\$.address.*</tt> to fetch all fields within the <tt>address</tt> object or <tt>\$.phoneNumbers[*].numbers[*]</tt> to retrieve all phone numbers across multiple objects in an array.
|
|
|
|
The <tt>*</tt> wildcard matches all elements at a specific level. For instance, <tt>\$.address.*</tt> retrieves all key-value pairs in the <tt>address</tt> object, while <tt>\$.*.streetAddress</tt> fetches all <tt>streetAddress</tt> fields across objects at the root level. You can combine wildcards with array indexing. For example, <tt>\$.phoneNumbers[*].numbers[1]</tt> retrieves the second number from each <tt>numbers</tt> array in the <tt>phoneNumbers</tt> array. If no elements match the wildcard query, the callback is simply never called. For instance, querying <tt>\$.empty_object.*</tt> or <tt>\$.empty_array.*</tt> will yield no callbacks.
|
|
|
|
@subsection autotoc_md23 Example Usage
|
|
|
|
Here is an example demonstrating the use of <tt>for_each_at_path_with_wildcard</tt>:
|
|
|
|
@icode{cpp}
|
|
simdjson::padded_string json_string = R"( { "firstName": "John", "lastName": "doe", "age": 26, "address": { "streetAddress": "naist street", "city": "Nara", "postalCode": "630-0192" }, "phoneNumbers": [ { "type": "iPhone", "numbers": ["0123-4567-8888", "0123-4567-8788"] }, { "type": "home", "numbers": ["0123-4567-8910"] } ] })"_padded;
|
|
|
|
ondemand::parser parser;
|
|
ondemand::document doc = parser.iterate(json_string);
|
|
|
|
// Fetch all fields in the address object
|
|
auto error = doc.for_each_at_path_with_wildcard("$.address.*",
|
|
[](ondemand::value value) {
|
|
std::string_view field;
|
|
if (value.get(field) == SUCCESS) {
|
|
std::cout << field << std::endl;
|
|
}
|
|
});
|
|
|
|
// Fetch all phone numbers
|
|
doc.for_each_at_path_with_wildcard("$.phoneNumbers[*].numbers[*]",
|
|
[](ondemand::value value) {
|
|
std::string_view number;
|
|
if (value.get(number) == SUCCESS) {
|
|
std::cout << number << std::endl;
|
|
}
|
|
});
|
|
@endicode
|
|
|
|
This function is particularly useful for extracting data from complex JSON structures with nested arrays and objects. By leveraging wildcards, you can simplify your queries and reduce the need for multiple iterations.
|
|
|
|
@section autotoc_md24 Compile-Time JSONPath and JSON Pointer (C++26 Reflection)
|
|
|
|
The simdjson library provides <strong>compile-time validated</strong> JSONPath and JSON Pointer accessors when using C++26 Static Reflection. These accessors validate paths against struct definitions at compile time and generate optimized code with zero runtime overhead. In some cases, we find that it is much faster. Furthermore, it is safer in the sense that the expression
|
|
is validated at compile-time.
|
|
|
|
<strong>Requirements:</strong> C++26 compiler with P2996 reflection support and <tt>-DSIMDJSON_STATIC_REFLECTION=ON</tt> build flag.
|
|
|
|
@icode{cpp}
|
|
ondemand::parser parser;
|
|
auto doc = parser.iterate(json);
|
|
|
|
// Without validation - path parsed at compile time only
|
|
std::string_view city;
|
|
result = ondemand::json_path::at_path_compiled<".address.city">(doc);
|
|
result.get(city);
|
|
@endicode
|
|
|
|
We further provide type-validation so that you can check that the types are as you expect.
|
|
|
|
<strong>See @ref "/home/runner/work/simdjson/simdjson/doc/compile_time_accessors.md" "Compile-Time Accessors" for complete documentation.</strong>
|
|
|
|
@section autotoc_md25 Error handling
|
|
|
|
|
|
Error handling with exception and a single try/catch clause makes the code simple, but it gives you little control over errors. For easier debugging or more robust error handling, you may want to consider our exception-free approach.
|
|
|
|
The entire simdjson API is usable with and without exceptions. All simdjson APIs that can fail return <tt>simdjson_result\<T\></tt>, which is a &lt;value, error_code&gt;
|
|
pair. You can retrieve the value with .get() without generating an exception, like so:
|
|
|
|
@icode{cpp}
|
|
ondemand::document doc;
|
|
auto error = parser.iterate(json).get(doc);
|
|
if(error) { std::cerr << simdjson::error_message(error); exit(1); }
|
|
@endicode
|
|
|
|
When there is no error, the error code <tt>simdjson::SUCCESS</tt>is returned: it evaluates as false as a Boolean.
|
|
We have several error codes to indicate errors, they all evaluate to true as a Boolean: your software should not generally not depend on exact
|
|
error codes. We may change the error codes in future releases and the exact error codes could vary depending on your system.
|
|
|
|
Some errors are recoverable:
|
|
|
|
* You may get the error <tt>simdjson::INCORRECT_TYPE</tt> after trying to convert a value to an incorrect type: e.g., you expected a number and try to convert the value to a number, but it is an array.
|
|
* You may query a key from an object, but the key is missing in which case you get the error <tt>simdjson::NO_SUCH_FIELD</tt>: e.g., you call <tt>obj["myname"]</tt> and the object does not have a key <tt>"myname"</tt>.
|
|
|
|
Other errors (<tt>simdjson::INCOMPLETE_ARRAY_OR_OBJECT</tt> and <tt>simdjson::TAPE_ERROR</tt>) indicate a fatal error and follow from the fact that the document is not valid JSON. These errors are not recoverable: you cannot continue. In which case, it is no longer safe to continue accessing the document: calling the method <tt>is_alive()</tt> on the document instance returns false. It is your responsibility as a user to stop using the simdjson
|
|
document after encountering these fatal errors. Consider the following example, after
|
|
the fatal error, the document instance cannot be used. Observe how the JSON input is invalid.
|
|
@icode{cpp}
|
|
simdjson::padded_string badjson = R"( { "make": "Toyota", "model": "Camry", "year"})"_padded;
|
|
simdjson::ondemand::parser parser;
|
|
simdjson::ondemand::document doc;
|
|
auto errordoc = parser.iterate(badjson).get(doc);
|
|
// errordoc == simdjson::SUCCESS
|
|
simdjson::ondemand::value v;
|
|
auto error = doc.get_object()["year"].get(v);
|
|
// simdjson::is_fatal(error)) is true!
|
|
// doc.is_alive() is false
|
|
@endicode
|
|
|
|
When you use the code without exceptions, it is your responsibility to check for error before using the
|
|
result: if there is an error, the result value will not be valid and using it will caused undefined behavior. Most compilers should be able to help you if you activate the right
|
|
set of warnings: they can identify variables that are written to but never otherwise accessed.
|
|
|
|
Let us illustrate with an example where we try to access a number that is not valid (<tt>3.14.1</tt>).
|
|
If we want to proceed without throwing and catching exceptions, we can do so as follows:
|
|
|
|
@icode{cpp}
|
|
bool simple_error_example() {
|
|
ondemand::parser parser;
|
|
auto json = R"({"bad number":3.14.1 })"_padded;
|
|
ondemand::document doc;
|
|
if (parser.iterate(json).get(doc) != SUCCESS) { return false; }
|
|
double x;
|
|
auto error = doc["bad number"].get_double().get(x);
|
|
// returns "<a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9adfcabbbd1ddb0843a0f320e062866ae8" title="Problem while parsing a number.">simdjson::NUMBER_ERROR</a>"
|
|
if (error != SUCCESS) {
|
|
std::cerr << simdjson::error_message(error) << std::endl;
|
|
return false;
|
|
}
|
|
std::cout << "Got " << x << std::endl;
|
|
return true;
|
|
}
|
|
@endicode
|
|
|
|
Observe how we verify the error variable before accessing the retrieved number (variable <tt>x</tt>).
|
|
|
|
The equivalent with exception handling might look as follows.
|
|
|
|
@icode{cpp}
|
|
bool simple_error_example_except() {
|
|
TEST_START();
|
|
ondemand::parser parser;
|
|
auto json = R"({"bad number":3.14.1 })"_padded;
|
|
try {
|
|
ondemand::document doc = parser.iterate(json);
|
|
double x = doc["bad number"].get_double();
|
|
std::cout << "Got " << x << std::endl;
|
|
return true;
|
|
} catch(simdjson_error& e) {
|
|
// e.error() == NUMBER_ERROR
|
|
std::cout << e.error() << std::endl;
|
|
return false;
|
|
}
|
|
}
|
|
@endicode
|
|
|
|
Notice how we can retrieve the exact error condition (in this instance <tt>simdjson::NUMBER_ERROR</tt>)
|
|
from the exception.
|
|
|
|
We can write a "quick start" example where we attempt to parse the following JSON file and access some data, without triggering exceptions:
|
|
@icode{JSON}
|
|
{
|
|
"statuses": [
|
|
{
|
|
"id": 505874924095815700
|
|
},
|
|
{
|
|
"id": 505874922023837700
|
|
}
|
|
],
|
|
"search_metadata": {
|
|
"count": 100
|
|
}
|
|
}
|
|
@endicode
|
|
|
|
Our program loads the file, selects value corresponding to key <tt>"search_metadata"</tt> which expected to be an object, and then
|
|
it selects the key <tt>"count"</tt> within that object.
|
|
|
|
@icode{cpp}
|
|
#include <iostream>
|
|
#include "<a class="el" href="simdjson_8h.html">simdjson.h</a>"
|
|
|
|
int main(void) {
|
|
simdjson::ondemand::parser parser;
|
|
auto error = padded_string::load("twitter.json").get(json);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return EXIT_FAILURE; }
|
|
simdjson::ondemand::document tweets;
|
|
error = parser.iterate(json).get(tweets);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return EXIT_FAILURE; }
|
|
simdjson::ondemand::value res;
|
|
error = tweets["search_metadata"]["count"].get(res);
|
|
if (error != SUCCESS) {
|
|
std::cerr << "could not access keys : " << error << std::endl;
|
|
return EXIT_FAILURE;
|
|
}
|
|
std::cout << res << " results." << std::endl;
|
|
return EXIT_SUCCESS;
|
|
}
|
|
@endicode
|
|
|
|
The following is a similar example where one wants to get the id of the first tweet without
|
|
triggering exceptions. To do this, we use <tt>["statuses"].at(0)["id"]</tt>. We break that expression down:
|
|
|
|
- Get the list of tweets (the <tt>"statuses"</tt> key of the document) using <tt>["statuses"]</tt>). The result is expected to be an array.
|
|
- Get the first tweet using <tt>.at(0)</tt>. The result is expected to be an object. Observe that the <tt>at</tt> method can only be called once on an array (it cannot be used for iteration).
|
|
- Get the id of the tweet using ["id"]. We expect the value to be a non-negative integer.
|
|
|
|
Observe how we use the <tt>at</tt> method when querying an index into an array, and not the bracket operator.
|
|
|
|
|
|
@icode{cpp}
|
|
#include <iostream>
|
|
#include "<a class="el" href="simdjson_8h.html">simdjson.h</a>"
|
|
|
|
int main(void) {
|
|
simdjson::ondemand::parser parser;
|
|
simdjson::ondemand::document tweets;
|
|
padded_string json;
|
|
auto error = padded_string::load("twitter.json").get(json);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return EXIT_FAILURE; }
|
|
error = parser.iterate(json).get(tweets);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return EXIT_FAILURE; }
|
|
uint64_t identifier;
|
|
error = tweets["statuses"].at(0)["id"].get(identifier);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return EXIT_FAILURE; }
|
|
std::cout << identifier << std::endl;
|
|
}
|
|
@endicode
|
|
|
|
<em>Important remark</em>: The <tt>at</tt> method can only be called once on an array. It cannot be used
|
|
to iterate through the values of an array. We deliberately forbid this usage to avoid performance antipatterns. If you need to iterate through the values of an array, you should use a <tt>for</tt> loop.
|
|
|
|
@subsection autotoc_md26 Error handling examples without exceptions
|
|
|
|
This is how the example in "Using the parsed JSON" could be written using only error code checking (without exceptions):
|
|
|
|
@icode{cpp}
|
|
bool parse() {
|
|
ondemand::parser parser;
|
|
auto cars_json = R"( [ { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] }, { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] }, { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] } ] )"_padded;
|
|
ondemand::document doc;
|
|
|
|
// Iterating through an array of objects
|
|
auto error = parser.iterate(cars_json).get(doc);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
ondemand::array cars; // invalid until the get() succeeds
|
|
error = doc.get_array().get(cars);
|
|
|
|
for (auto car_value : cars) {
|
|
ondemand::object car; // invalid until the get() succeeds
|
|
error = car_value.get_object().get(car);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
|
|
// Accessing a field by name
|
|
std::string_view make;
|
|
std::string_view model;
|
|
error = car["make"].get(make);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
error = car["model"].get(model);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
|
|
cout << "Make/Model: " << make << "/" << model << endl;
|
|
|
|
// Casting a JSON element to an integer
|
|
uint64_t year{};
|
|
error = car["year"].get(year);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
cout << "- This car is " << 2020 - year << " years old." << endl;
|
|
|
|
// Iterating through an array of floats
|
|
double total_tire_pressure = 0;
|
|
ondemand::array pressures;
|
|
error = car["tire_pressure"].get_array().get(pressures);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
for (auto tire_pressure_value : pressures) {
|
|
double tire_pressure;
|
|
error = tire_pressure_value.get_double().get(tire_pressure);
|
|
if (error) { std::cerr << simdjson::error_message(error) << std::endl; return false; }
|
|
total_tire_pressure += tire_pressure;
|
|
}
|
|
cout << "- Average tire pressure: " << (total_tire_pressure / 4) << endl;
|
|
}
|
|
return true;
|
|
}
|
|
@endicode
|
|
|
|
For safety, you should only use our ondemand instances (e.g., <tt>ondemand::object</tt>)
|
|
after you have initialized them and checked that there is no error:
|
|
|
|
@icode{cpp}
|
|
ondemand::object car; // invalid until the get() succeeds
|
|
// the `car` instance should not use used before it is initialized
|
|
error = car_value.get_object().get(car);
|
|
if (error) {
|
|
// the `car` instance should not use used
|
|
} else {
|
|
// the `car` instance can be safely used
|
|
}
|
|
@endicode
|
|
|
|
The following examples illustrates how to iterate through the content of an object without
|
|
having to handle exceptions.
|
|
@icode{cpp}
|
|
auto json = R"({"k\u0065y": 1})"_padded;
|
|
ondemand::parser parser;
|
|
ondemand::document doc;
|
|
auto error = parser.iterate(json).get(doc);
|
|
if (error) { return false; }
|
|
ondemand::object object; // invalid until the get() succeeds
|
|
error = doc.get_object().get(object);
|
|
if (error) { return false; }
|
|
for(auto field : object) {
|
|
// We could replace 'field.key() with field.unescaped_key() or field.escaped_key(),
|
|
// and ondemand::raw_json_string by std::string_view.
|
|
ondemand::raw_json_string keyv;
|
|
error = field.key().get(keyv);
|
|
if (error) { return false; }
|
|
if (keyv == "key") {
|
|
uint64_t intvalue;
|
|
error = field.value().get(intvalue);
|
|
if (error) { return false; }
|
|
std::cout << intvalue;
|
|
}
|
|
}
|
|
@endicode
|
|
|
|
@subsection autotoc_md27 Disabling exceptions
|
|
|
|
The simdjson can be build with exceptions entirely disabled. It checks the <tt>__cpp_exceptions</tt> macro at compile time. Even if exceptions are enabled in your compiler, you may still disable exceptions specifically for simdjson, by setting <tt>SIMDJSON_EXCEPTIONS</tt> to <tt>0</tt> (false) at compile-time when building the simdjson library. If you are building with CMake, to ensure you don't write any code that uses exceptions, you compile with <tt>SIMDJSON_EXCEPTIONS=OFF</tt>. For example, if including the project via cmake:
|
|
|
|
@icode{cmake}
|
|
target_compile_definitions(simdjson PUBLIC SIMDJSON_EXCEPTIONS=OFF)
|
|
@endicode
|
|
|
|
@subsection autotoc_md28 Exceptions
|
|
|
|
Users more comfortable with an exception flow may choose to directly cast the <tt>simdjson_result\<T\></tt> to the desired type:
|
|
|
|
@icode{cpp}
|
|
simdjson::ondemand::document doc = parser.iterate(json); // Throws an exception if there was an error!
|
|
@endicode
|
|
|
|
When used this way, a <tt>simdjson_error</tt> exception will be thrown if an error occurs, preventing the
|
|
program from continuing if there was an error.
|
|
|
|
|
|
If one is willing to trigger exceptions, it is possible to write simpler code:
|
|
|
|
@icode{cpp}
|
|
#include <iostream>
|
|
#include "<a class="el" href="simdjson_8h.html">simdjson.h</a>"
|
|
|
|
int main(void) {
|
|
simdjson::ondemand::parser parser;
|
|
padded_string json = padded_string::load("twitter.json");
|
|
simdjson::ondemand::document tweets = parser.iterate(json);
|
|
uint64_t identifier = tweets["statuses"].at(0)["id"];
|
|
std::cout << identifier << std::endl;
|
|
return EXIT_SUCCESS;
|
|
}
|
|
@endicode
|
|
|
|
|
|
You can do handle errors gracefully as well...
|
|
|
|
@icode{cpp}
|
|
#include <iostream>
|
|
#include "<a class="el" href="simdjson_8h.html">simdjson.h</a>"
|
|
int main(void) {
|
|
simdjson::ondemand::parser parser;
|
|
simdjson::padded_string json_string;
|
|
simdjson::ondemand::document doc;
|
|
try {
|
|
json_string = padded_string::load("twitter.json");
|
|
doc = parser.iterate(json_string);
|
|
uint64_t identifier = doc["statuses"].at(0)["id"];
|
|
std::cout << identifier << std::endl;
|
|
} catch (simdjson::simdjson_error &error) {
|
|
std::cerr << "JSON error: " << error.what() << " near "
|
|
<< doc.current_location() << " in " << json_string << std::endl;
|
|
}
|
|
}
|
|
@endicode
|
|
|
|
@subsection autotoc_md29 Current location in document
|
|
|
|
Sometimes, it might be helpful to know the current location in the document during iteration. This is especially useful when encountering errors. The <tt>current_location()</tt> method on a
|
|
<tt>document</tt> instances makes it easy to identify common JSON errors. Users can call the <tt>current_location()</tt> method on a valid document instance to retrieve a <tt>const char *</tt> pointer to the current location in the document. This method also works even after an error has invalidated the document and the parser (e.g. <tt>TAPE_ERROR</tt>, <tt>INCOMPLETE_ARRAY_OR_OBJECT</tt>).
|
|
When the input was a <tt>padding_string</tt> or another null-terminated source, then you may
|
|
use the <tt>const char *</tt> pointer as a C string. As an example, consider the following
|
|
example where we used the exception-free simdjson interface:
|
|
|
|
@icode{cpp}
|
|
auto broken_json = R"( {"double": 13.06, false, "integer": -343} )"_padded; // Missing key
|
|
ondemand::parser parser;
|
|
auto doc = parser.iterate(broken_json);
|
|
int64_t i;
|
|
auto error = doc["integer"].get_int64().get(i); // Expect to get integer from "integer" key, but get TAPE_ERROR
|
|
if (error) {
|
|
std::cerr << simdjson::error_message(error) << std::endl; // Prints TAPE_ERROR error message
|
|
// Recover a pointer to the location of the first error:
|
|
const char * ptr;
|
|
doc.current_location().get(ptr);
|
|
// ptr points at 'false, "integer": -343} " which is the location of the error // // Because we pad <a class="el" href="structsimdjson_1_1padded__string.html" title="String with extra allocation for ease of use with parser::parse()">simdjson::padded_string</a> instances with null characters, you may also do the following: std::cout<< doc.current_location() << std::endl; // Prints "false, "integer": -343} " (location of TAPE_ERROR) } </code></p>
|
|
<p><code>You may also use <code>current_location()</code> with exceptions as follows:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> broken_json = R<span class="stringliteral">"( {"double": 13.06, false, "integer": -343} )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(broken_json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">try</span> {</div>
|
|
<div class="line"> <span class="keywordflow">return</span> int64_t(doc[<span class="stringliteral">"integer"</span>]);</div>
|
|
<div class="line">} <span class="keywordflow">catch</span>(<a class="code hl_struct" href="structsimdjson_1_1simdjson__error.html">simdjson_error</a>& err) {</div>
|
|
<div class="line"> std::cerr << doc.current_location() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">return</span> -1;</div>
|
|
<div class="line">}</div>
|
|
<div class="ttc" id="astructsimdjson_1_1simdjson__error_html"><div class="ttname"><a href="structsimdjson_1_1simdjson__error.html">simdjson::simdjson_error</a></div><div class="ttdoc">Exception thrown when an exception-supporting simdjson method is called.</div><div class="ttdef"><b>Definition</b> <a href="error_8h_source.html#l00091">error.h:91</a></div></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>In these examples, we tried to access the <code>"integer"</code> key, but since the parser had to go through a value without a key before (<code>false</code>), a <code>TAPE_ERROR</code> error is thrown. The pointer returned by the <code>current_location()</code> method then points at the location of the error. The <code>current_location()</code> may also be used when the error is triggered by a user action, even if the JSON input is valid. Consider the following example:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( [1,2,3] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line">int64_t i;</div>
|
|
<div class="line"><span class="keyword">auto</span> error = doc[<span class="stringliteral">"integer"</span>].get_int64().get(i); <span class="comment">// Incorrect call on array, INCORRECT_TYPE error</span></div>
|
|
<div class="line"><span class="keywordflow">if</span> (error) {</div>
|
|
<div class="line"> std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error) << std::endl; <span class="comment">// Prints INCORRECT_TYPE error message</span></div>
|
|
<div class="line"> std::cout<< doc.current_location() << std::endl; <span class="comment">// Prints "[1,2,3] " (location of INCORRECT_TYPE error)</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>If the location is invalid (i.e. at the end of a document), the <code>current_location()</code> methods returns an <code>OUT_OF_BOUNDS</code> error. For example:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( [1,2,3] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> <span class="comment">// Do something with val</span></div>
|
|
<div class="line">}</div>
|
|
<div class="line">std::cout << doc.current_location() << std::endl; <span class="comment">// Throws OUT_OF_BOUNDS</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Conversely, if <code>doc.current_location().error() == <a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681" title="No error.">simdjson::SUCCESS</a></code>, then the document has more content.</code></p>
|
|
<p><code>Finally, the <code>current_location()</code> method may also be used even when no exceptions/errors are thrown. This can be helpful for users that want to know the current state of iteration during parsing. For example:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( [[1,2,3], -23.4, {"key": "value"}, true] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> val : doc) {</div>
|
|
<div class="line"> ondemand::object obj; <span class="comment">// invalid until the get() succeeds</span></div>
|
|
<div class="line"> <span class="keyword">auto</span> error = val.get_object().get(obj); <span class="comment">// Only get objects</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span> (!error) {</div>
|
|
<div class="line"> std::cout << doc.current_location() << std::endl; <span class="comment">// Prints ""key": "value"}, true] "</span></div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The <code>current_location()</code> method requires a valid <code>document</code> instance. If the <code>iterate</code> function fails to return a valid document, then you cannot use <code>current_location()</code> to identify the location of an error in the input string. The errors reported by <code>iterate</code> function include EMPTY if no JSON document is detected, UTF8_ERROR if the string is not a valid UTF-8 string, UNESCAPED_CHARS if a string contains control characters that must be escaped and UNCLOSED_STRING if there is an unclosed string in the document. We do not provide location information for these errors.</code></p>
|
|
<p><code></code></p>
|
|
<h2><a class="anchor" id="autotoc_md30"></a>
|
|
Checking for trailing content</h2>
|
|
<p><code></code></p>
|
|
<p><code>The parser validates all parsed content, but your code may exhaust the content while not having processed the entire document. Thus, as a final optional step, you may call <code>at_end()</code> on the document instance. If it returns <code>false</code>, then you may conclude that you have trailing content and that your document is not valid JSON. You may then use <code>doc.current_location()</code> to obtain a pointer to the start of the trailing content.</code></p>
|
|
<p><code>Example 1.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"([1, 2] foo ])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::array array = doc.get_array();</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span> (uint64_t values : array) {</div>
|
|
<div class="line"> std::cout << values << std::endl;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="keywordflow">if</span> (!doc.at_end()) {</div>
|
|
<div class="line"> <span class="comment">// In this instance, we will be left pointing at 'foo' since we have consumed the array [1,2].</span></div>
|
|
<div class="line"> std::cerr << <span class="stringliteral">"trailing content at byte index "</span> << doc.current_location() - json.data() << std::endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Example 2.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"(["extra close"]])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::array array = doc.get_array();</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span> (std::string_view values : array) {</div>
|
|
<div class="line"> std::cout << values << std::endl;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="keywordflow">if</span>(!doc.at_end()) {</div>
|
|
<div class="line"> std::cerr << <span class="stringliteral">"trailing content at byte index "</span> << doc.current_location() - json.data() << std::endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The <code>at_end()</code> method is equivalent to <code>doc.current_location().error() == <a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9aa4ffc800804829a88c5b52ac3c4c6681" title="No error.">simdjson::SUCCESS</a></code> but more convenient.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md31"></a>
|
|
Rewinding</h1>
|
|
<p><code></code></p>
|
|
<p><code>In some instances, you may need to go through a document more than once. For that purpose, you may call the <code>rewind()</code> method on the document instance. It allows you to restart processing from the beginning without rescanning all of the input data again. It invalidates all values, objects and arrays that you have created so far (including unescaped strings).</code></p>
|
|
<p><code>In the following example, we print on the screen the number of cars in the JSON input file before printout the data.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> cars_json = R<span class="stringliteral">"( [</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Camry", "year": 2018, "tire_pressure": [ 40.1, 39.9, 37.7, 40.4 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Kia", "model": "Soul", "year": 2012, "tire_pressure": [ 30.1, 31.0, 28.6, 28.7 ] },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "make": "Toyota", "model": "Tercel", "year": 1999, "tire_pressure": [ 29.8, 30.0, 30.2, 30.5 ] }</span></div>
|
|
<div class="line"><span class="stringliteral">] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">auto</span> doc = parser.iterate(cars_json);</div>
|
|
<div class="line"><span class="keywordflow">for</span> (simdjson_unused ondemand::object car : doc) {</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (car[<span class="stringliteral">"make"</span>] == <span class="stringliteral">"Toyota"</span>) { count++; }</div>
|
|
<div class="line">}</div>
|
|
<div class="line">std::cout << <span class="stringliteral">"We have "</span> << count << <span class="stringliteral">" Toyota cars.\n"</span>;</div>
|
|
<div class="line">doc.rewind(); <span class="comment">// requires simdjson 1.0 or better</span></div>
|
|
<div class="line"><span class="keywordflow">for</span> (ondemand::object car : doc) {</div>
|
|
<div class="line"> cout << <span class="stringliteral">"Make/Model: "</span> << std::string_view(car[<span class="stringliteral">"make"</span>]) << <span class="stringliteral">"/"</span> << std::string_view(car[<span class="stringliteral">"model"</span>]) << endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Performance note: the On-Demand front-end does not materialize the parsed numbers and other values. If you are accessing everything twice, you may need to parse them twice. Thus the rewind functionality is best suited for cases where the first pass only scans the structure of the document.</code></p>
|
|
<p><code>Both arrays and objects have a similar method <code>reset()</code>. It is similar to the document <code>rewind()</code> method, except that it does not rewind the internal string buffer. Thus you should consume values only once even if you can iterate through the array or object more than once. If you unescape a string within an array more than once, you have unsafe code. You must not call <code>reset()</code> on an object or an array as you are iterating through it.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md32"></a>
|
|
Newline-Delimited JSON (ndjson) and JSON lines</h1>
|
|
<p><code></code></p>
|
|
<p><code>When processing large inputs (e.g., in the context of data engineering), engineers commonly serialize data into streams of multiple JSON documents. That is, instead of one large (e.g., 2 GB) JSON document containing multiple records, it is often preferable to write out multiple records as independent JSON documents, to be read one-by-one.</code></p>
|
|
<p><code>The simdjson library also supports multithreaded JSON streaming through a large file containing many smaller JSON documents in either <a href="https://github.com/ndjson/ndjson-spec">ndjson</a> or <a href="http://jsonlines.org">JSON lines</a> format. If your JSON documents all contain arrays or objects, we even support direct file concatenation without whitespace. However, if there is content between your JSON documents, it should be exclusively ASCII white-space characters.</code></p>
|
|
<p><code>The concatenated file has no size restrictions (including larger than 4GB), though each individual document must be no larger than 4 GB.</code></p>
|
|
<p><code>Here is an example:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"({ "foo": 1 } { "foo": 2 } { "foo": 3 } )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document_stream docs = parser.iterate_many(json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span> (<span class="keyword">auto</span> doc : docs) {</div>
|
|
<div class="line"> std::cout << doc[<span class="stringliteral">"foo"</span>] << std::endl;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="comment">// Prints 1 2 3</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Unlike <code>parser.iterate</code>, <code>parser.iterate_many</code> may parse "On-Demand" (lazily). That is, no parsing may have been done before you enter the loop <code>for (auto doc : docs) {</code> and you should expect the parser to only ever fully parse one JSON document at a time.</code></p>
|
|
<p><code>As with <code>parser.iterate</code>, when calling <code>parser.iterate_many(string)</code>, no copy is made of the provided string input. The provided memory buffer may be accessed each time a JSON document is parsed. Calling <code>parser.iterate_many(string)</code> on a temporary string buffer (e.g., <code>docs = parser.parse_many("[1,2,3]"_padded)</code>) is unsafe (and will not compile) because the <code>document_stream</code> instance needs access to the buffer to return the JSON documents.</code></p>
|
|
<p><code>The <code>iterate_many</code> function can also take an optional parameter <code>size_t batch_size</code> which defines the window processing size. It is set by default to a large value (<code>1000000</code> corresponding to 1 MB). None of your JSON documents should exceed this window size, or else you will get the error <code><a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9a2d50048bbe409023df4e88f6ffe3bc71" title="This parser can't support a document that big.">simdjson::CAPACITY</a></code>. You cannot set this window size larger than 4 GB: you will get the error <code><a class="el" href="namespacesimdjson.html#a7b735a3a50ba79e3f7f14df5f77d8da9a2d50048bbe409023df4e88f6ffe3bc71" title="This parser can't support a document that big.">simdjson::CAPACITY</a></code>. The smaller the window size is, the less memory the function will use. Setting the window size too small (e.g., less than 100 kB) may also impact performance negatively. Leaving it to 1 MB is expected to be a good choice, unless you have some larger documents.</code></p>
|
|
<p><code>The following toy examples illustrates how to get capacity errors. It is an artificial example since you should never use a <code>batch_size</code> of 50 bytes (it is far too small).</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="comment">// We are going to set the capacity to 50 bytes which means that we cannot</span></div>
|
|
<div class="line"><span class="comment">// loading a document longer than 50 bytes. The first few documents are small,</span></div>
|
|
<div class="line"><span class="comment">// but the last one is large. We will get an error at the last document.</span></div>
|
|
<div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"([1,2,3,4,5] [1,2,3,4,5] [1,2,3,4,5] [1,2,3,4,5] [1,2,3,4,5] [1,2,3,4,5] [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document_stream stream;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordtype">size_t</span> counter{0};</div>
|
|
<div class="line"><span class="keyword">auto</span> error = parser.iterate_many(json, 50).get(stream);</div>
|
|
<div class="line"><span class="keywordflow">if</span> (error) { <span class="comment">/* handle the error */</span> }</div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> doc: stream) {</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (counter < 6) {</div>
|
|
<div class="line"> int64_t val;</div>
|
|
<div class="line"> error = doc.at_pointer(<span class="stringliteral">"/4"</span>).get(val);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="comment">/* handle the error */</span> }</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"5 = "</span> << val << std::endl;</div>
|
|
<div class="line"> } <span class="keywordflow">else</span> {</div>
|
|
<div class="line"> ondemand::value val;</div>
|
|
<div class="line"> error = doc.at_pointer(<span class="stringliteral">"/4"</span>).get(val);</div>
|
|
<div class="line"> <span class="comment">// error == simdjson::CAPACITY</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) {</div>
|
|
<div class="line"> std::cerr << <a class="code hl_function" href="namespacesimdjson.html#a65b00a7a7a9c8fafe9eb3b9b413a36fa">simdjson::error_message</a>(error) << std::endl;</div>
|
|
<div class="line"> <span class="comment">// We left 293 bytes unprocessed at the tail end of the input.</span></div>
|
|
<div class="line"> std::cout << <span class="stringliteral">" unprocessed bytes at the end: "</span> << stream.truncated_bytes() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> counter++;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>This example should print out:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">5 = 5</div>
|
|
<div class="line">5 = 5</div>
|
|
<div class="line">5 = 5</div>
|
|
<div class="line">5 = 5</div>
|
|
<div class="line">5 = 5</div>
|
|
<div class="line">5 = 5</div>
|
|
<div class="line">This parser can't support a document that big</div>
|
|
<div class="line"> unprocessed bytes at the end: 293</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>If your documents are large (e.g., larger than a megabyte), then the <code>iterate_many</code> function is maybe ill-suited. It is really meant to support reading efficiently streams of relatively small documents (e.g., a few kilobytes each). If you have larger documents, you should use other functions like <code>iterate</code>.</code></p>
|
|
<p><code>We also provide some support for comma-separated documents and other advanced features. See <a class="el" href="md_doc_2iterate__many.html">iterate_many.md</a> for detailed information and design.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md33"></a>
|
|
Parsing numbers inside strings</h1>
|
|
<p><code></code></p>
|
|
<p><code>Though the JSON specification allows for numbers and string values, many engineers choose to integrate the numbers inside strings, e.g., they prefer <code>{"a":"1.9"}</code> to<code>{"a":1.9}</code>. The simdjson library supports parsing valid numbers inside strings which makes it more convenient for people working with those types of documents. This feature is supported through three methods: <code>get_double_in_string</code>, <code>get_int64_in_string</code> and <code>get_uint64_in_string</code>. However, it is important to note that these methods are not substitute to the regular <code>get_double</code>, <code>get_int64</code> and <code>get_uint64</code>. The usage of the <code>get_*_in_string</code> methods is solely to parse valid JSON numbers inside strings, and so we expect users to call these methods appropriately. In particular, a valid JSON number has no leading and no trailing whitespaces, and the strings <code>"nan"</code>, <code>"1e"</code> and <code>"infinity"</code> will not be accepted as valid numbers (although you have access to the raw string with the <code>raw_json_token()</code> method, see General direct access to the raw JSON string ). As an example, suppose we have the following JSON text:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json =</div>
|
|
<div class="line">{</div>
|
|
<div class="line"> <span class="stringliteral">"ticker"</span>:{</div>
|
|
<div class="line"> <span class="stringliteral">"base"</span>:<span class="stringliteral">"BTC"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"target"</span>:<span class="stringliteral">"USD"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"price"</span>:<span class="stringliteral">"443.7807865468"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"volume"</span>:<span class="stringliteral">"31720.1493969300"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"change"</span>:<span class="stringliteral">"Infinity"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"markets"</span>:[</div>
|
|
<div class="line"> {</div>
|
|
<div class="line"> <span class="stringliteral">"market"</span>:<span class="stringliteral">"bitfinex"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"price"</span>:<span class="stringliteral">"447.5000000000"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"volume"</span>:<span class="stringliteral">"10559.5293639000"</span></div>
|
|
<div class="line"> },</div>
|
|
<div class="line"> {</div>
|
|
<div class="line"> <span class="stringliteral">"market"</span>:<span class="stringliteral">"bitstamp"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"price"</span>:<span class="stringliteral">"448.5400000000"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"volume"</span>:<span class="stringliteral">"11628.2880079300"</span></div>
|
|
<div class="line"> },</div>
|
|
<div class="line"> {</div>
|
|
<div class="line"> <span class="stringliteral">"market"</span>:<span class="stringliteral">"btce"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"price"</span>:<span class="stringliteral">"432.8900000000"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"volume"</span>:<span class="stringliteral">"8561.0563600000"</span></div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> ]</div>
|
|
<div class="line"> },</div>
|
|
<div class="line"> <span class="stringliteral">"timestamp"</span>:1399490941,</div>
|
|
<div class="line"> <span class="stringliteral">"timestampstr"</span>:<span class="stringliteral">"1399490941"</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Now, suppose that a user wants to get the time stamp from the <code>timestampstr</code> key. One could do the following:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line">uint64_t time = doc.at_pointer(<span class="stringliteral">"/timestampstr"</span>).get_uint64_in_string();</div>
|
|
<div class="line">std::cout << time << std::endl; <span class="comment">// Prints 1399490941</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Another thing a user might want to do is extract the <code>markets</code> array and get the market name, price and volume. Here is one way to do so:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="comment">// Getting markets array</span></div>
|
|
<div class="line">ondemand::array markets = doc.find_field(<span class="stringliteral">"ticker"</span>).find_field(<span class="stringliteral">"markets"</span>).get_array();</div>
|
|
<div class="line"><span class="comment">// Iterating through markets array</span></div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> value : markets) {</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"Market: "</span> << value.find_field(<span class="stringliteral">"market"</span>).get_string();</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"\tPrice: "</span> << value.find_field(<span class="stringliteral">"price"</span>).get_double_in_string();</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"\tVolume: "</span> << value.find_field(<span class="stringliteral">"volume"</span>).get_double_in_string() << std::endl;</div>
|
|
<div class="line">}</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="comment">/* The above prints</span></div>
|
|
<div class="line"><span class="comment">Market: bitfinex Price: 447.5 Volume: 10559.5</span></div>
|
|
<div class="line"><span class="comment">Market: bitstamp Price: 448.54 Volume: 11628.3</span></div>
|
|
<div class="line"><span class="comment">Market: btce Price: 432.89 Volume: 8561.06</span></div>
|
|
<div class="line"><span class="comment">*/</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Finally, here is an example dealing with errors where the user wants to convert the string <code>"Infinity"</code>(<code>"change"</code> key) to a float with infinity value.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="comment">// Get "change"/"Infinity" key/value pair</span></div>
|
|
<div class="line">ondemand::value value = doc.find_field(<span class="stringliteral">"ticker"</span>).find_field(<span class="stringliteral">"change"</span>);</div>
|
|
<div class="line"><span class="keywordtype">double</span> d;</div>
|
|
<div class="line">std::string_view view;</div>
|
|
<div class="line"><span class="keyword">auto</span> error = value.get_double_in_string().get(d);</div>
|
|
<div class="line"><span class="comment">// Check if parsed value into double successfully</span></div>
|
|
<div class="line"><span class="keywordflow">if</span> (error) {</div>
|
|
<div class="line"> error = value.get_string().get(view);</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (error) { <span class="comment">/* Handle error */</span> }</div>
|
|
<div class="line"> <span class="keywordflow">else</span> <span class="keywordflow">if</span> (view == <span class="stringliteral">"Infinity"</span>) {</div>
|
|
<div class="line"> d = std::numeric_limits::infinity();</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">else</span> { <span class="comment">/* Handle wrong value */</span> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p> It is also important to note that when dealing an invalid number inside a string, simdjson will report a <code>NUMBER_ERROR</code> error if the string begins with a number whereas simdjson will report an <code>INCORRECT_TYPE</code> error otherwise.</code></p>
|
|
<p><code>The <code>*_in_string</code> methods can also be called on a single document instance: e.g., when your document consist solely of a quoted number.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md34"></a>
|
|
Dynamic Number Types</h1>
|
|
<p><code></code></p>
|
|
<p><code>The JSON standard does not offer strongly typed numbers. It suggests that using the binary64 type (<code>double</code> in C++) is a safe choice, but little else. Given the JSON array <code>[1.0,1]</code>, it is not specified whether it is an array of two floating-point numbers, two integers, or one floating-point number followed by an integer.</code></p>
|
|
<p><code>Given an <code>ondemand::value</code> instance, you may ask whether it is a negative value with the <code>is_negative()</code> method. The function is inexpensive.</code></p>
|
|
<p><code>To occasionally distinguish between floating-point values and integers given an <code>ondemand::value</code> instance, you may call the <code>is_integer()</code> method. We recognize an integer number by the lack decimal point and the lack of exponential suffix. E.g., <code>1e1</code> is always considered to be a floating-point number. The <code>is_integer()</code> method does not consume the value, but it scans the number string. You should avoid calling it repeatedly.</code></p>
|
|
<p><code>If you need to determine both the type of the number (integer or floating-point) and its value efficiently, you may call the <code>get_number()</code> method on the <code>ondemand::value</code> instance. Upon success, it returns an <code>ondemand::number</code> instance.</code></p>
|
|
<p><code>An <code>ondemand::number</code> instance may contain an integer value or a floating-point value. Thus it is a dynamically typed number. Before accessing the value, you must determine the detected type:</code></p>
|
|
<p><code></p><ul>
|
|
<li><code>number.get_number_type()</code> has value <code>number_type::signed_integer</code> if we have a integer in [-9223372036854775808,9223372036854775808). You can recover the value by the <code>get_int64()</code> method applied on the <code>ondemand::number</code> instance. When <code>number.get_number_type()</code> has value <code>number_type::signed_integer</code>, you also have that <code>number.is_int64()</code> is true. Calling <code>get_int64()</code> on the <code>ondemand::number</code> instance when <code>number.get_number_type()</code> is not <code>number_type::signed_integer</code> is unsafe. You may replace <code>get_int64()</code> by a cast to a <code>int64_t</code> value.</li>
|
|
<li><code>number.get_number_type()</code> has value <code>number_type::unsigned_integer</code> if we have a integer in <code>[9223372036854775808,18446744073709551616)</code>. You can recover the value by the <code>get_uint64()</code> method applied on the <code>ondemand::number</code> instance. When <code>number.get_number_type()</code> has value <code>number_type::unsigned_integer</code>, you also have that <code>number.is_uint64()</code> is true. Calling <code>get_uint64()</code> on the <code>ondemand::number</code> instance when <code>number.get_number_type()</code> is not <code>number_type::unsigned_integer</code> is unsafe. You may replace <code>get_uint64()</code> by a cast to a <code>uint64_t</code> value.</li>
|
|
<li><code>number.get_number_type()</code> has value <code>number_type::floating_point_number</code> if we have and we have a floating-point (binary64) number. You can recover the value by the <code>get_double()</code> method applied on the <code>ondemand::number</code> instance. When <code>number.get_number_type()</code> has value <code>number_type::floating_point_number</code>, you also have that <code>number.is_double()</code> is true. Calling <code>get_double()</code> on the <code>ondemand::number</code> instance when <code>number.get_number_type()</code> is not <code>number_type::floating_point_number</code> is unsafe. You may replace <code>get_double()</code> by a cast to a <code>double</code> value.</li>
|
|
<li><p class="startli">When the value is an integer outside of the valid ranges for a 64-bit integers, e.g., when it is smaller than -9223372036854775808 or larger than 18446744073709551615, then <code>number.get_number_type()</code> has value <code>number_type::big_integer</code>. If you try to parse such a number of <code>get_number()</code>, you get the error <code>BIGINT_ERROR</code>. You can access the underlying string of digits with the function <code>raw_json_token()</code> which returns a <code>std::string_view</code> instance starting at the beginning of the digit. You can also call <code>get_double()</code> to get a floating-point approximation.</p>
|
|
<p class="startli">By default, the string <code>-0</code> is parsed as the integer 0 as in Python or C++. If you set the macro <code>SIMDJSON_MINUS_ZERO_AS_FLOAT</code> to <code>1</code> when building simdjson, you can get that <code>-0</code> is mapped to <code>-0.0</code> as in JavaScript. You can get the desired effect by building simdjson with cmake setting the <code>SIMDJSON_MINUS_ZERO_AS_FLOAT</code> to on: <code>cmake -B build -D SIMDJSON_MINUS_ZERO_AS_FLOAT=ON</code>.</p>
|
|
</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code>You must check the type before accessing the value: it is an error to call <code>get_int64()</code> when <code>number.get_number_type()</code> is not <code>number_type::signed_integer</code> and when <code>number.is_int64()</code> is false. You are responsible for this check as the user of the library.</code></p>
|
|
<p><code>The <code>get_number()</code> function is designed with performance in mind. When calling <code>get_number()</code>, you scan the number string only once, determining efficiently the type and storing it in an efficient manner.</code></p>
|
|
<p><code>Consider the following example: </p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> docdata = R<span class="stringliteral">"([1.0, 3, 1, 3.1415,-13231232,9999999999999999999])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::array arr = doc.get_array();</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span>(ondemand::value val : arr) {</div>
|
|
<div class="line"> std::cout << val << <span class="stringliteral">" "</span>;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"negative: "</span> << val.is_negative() << <span class="stringliteral">" "</span>;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"is_integer: "</span> << val.is_integer() << <span class="stringliteral">" "</span>;</div>
|
|
<div class="line"> ondemand::number num = val.get_number();</div>
|
|
<div class="line"> ondemand::number_type t = num.get_number_type();</div>
|
|
<div class="line"> <span class="keywordflow">switch</span>(t) {</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::number_type::signed_integer:</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"integer: "</span> << int64_t(num) << <span class="stringliteral">" "</span>;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"integer: "</span> << num.get_int64() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::number_type::unsigned_integer:</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"large 64-bit integer: "</span> << uint64_t(num) << <span class="stringliteral">" "</span>;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"large 64-bit integer: "</span> << num.get_uint64() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::number_type::floating_point_number:</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"float: "</span> << double(num) << <span class="stringliteral">" "</span>;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"float: "</span> << num.get_double() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> ondemand::number_type::big_integer:</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"big-integer: "</span> << val.raw_json_token() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>It will output:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">1.0 negative: 0 is_integer: 0 float: 1 float: 1</div>
|
|
<div class="line">3 negative: 0 is_integer: 1 integer: 3 integer: 3</div>
|
|
<div class="line">1 negative: 0 is_integer: 1 integer: 1 integer: 1</div>
|
|
<div class="line">3.1415 negative: 0 is_integer: 0 float: 3.1415 float: 3.1415</div>
|
|
<div class="line">-13231232 negative: 1 is_integer: 1 integer: -13231232 integer: -13231232</div>
|
|
<div class="line">9999999999999999999 negative: 0 is_integer: 1 large 64-bit integer: 9999999999999999999 large 64-bit integer: 9999999999999999999</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>In the following example, we have an array of integers that are outside the valid range of 64-bit signed or unsigned integers. Calling <code>get_number_type()</code> on the values returns <code>ondemand::number_type::big_integer</code>. You can try to represent these big integers as 64-bit floating-point numbers, though you typically lose precision in the process (as illustrated in the example).</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> docdata = R<span class="stringliteral">"([-9223372036854775809, 18446744073709551617, 99999999999999999999999 ])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordtype">double</span> dexpected[] = {-9223372036854775808.0, 18446744073709551616.0, 1e23};</div>
|
|
<div class="line">ondemand::document doc = parser.iterate(docdata);</div>
|
|
<div class="line">ondemand::array arr = doc.get_array();</div>
|
|
<div class="line"><span class="keywordflow">for</span>(ondemand::value val : arr) {</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(val.get_number_type() == ondemand::number_type::big_integer) {</div>
|
|
<div class="line"> std::cout << val.get_double() << std::endl;</div>
|
|
<div class="line"> <span class="comment">// might print -9.22337e+18, 1.84467e+19, 1e+23</span></div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p> This program might print: </p><div class="fragment"><div class="line">-9.22337e+18</div>
|
|
<div class="line">1.84467e+19</div>
|
|
<div class="line">1e+23</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>You may get access to the underlying string representing the big integer with <code>raw_json_token()</code> and you may parse the resulting number strings using your own parser.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> docdata = R<span class="stringliteral">"([-9223372036854775809, 18446744073709551617, 99999999999999999999999 ])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::array arr = doc.get_array();</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span>(ondemand::value val : arr) {</div>
|
|
<div class="line"> <span class="comment">// val.get_number_type() == ondemand::number_type::big_integer</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span>(val.get_number_type() == ondemand::number_type::big_integer) {</div>
|
|
<div class="line"> std::string_view token = val.raw_json_token();</div>
|
|
<div class="line"> <span class="comment">// token = "-9223372036854775809", "18446744073709551617", "99999999999999999999999 "</span></div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"'"</span> << token << <span class="stringliteral">"'"</span> << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p> This code prints the following: </p><div class="fragment"><div class="line">'-9223372036854775809'</div>
|
|
<div class="line">'18446744073709551617'</div>
|
|
<div class="line">'99999999999999999999999 '</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md35"></a>
|
|
Raw strings from keys</h1>
|
|
<p><code></code></p>
|
|
<p><code>It is sometimes useful to have access to a raw (unescaped) string: we make available a minimalist <code>raw_json_string</code> data type which contains a pointer inside the string in the original document, right after the quote. It is accessible via <code>get_raw_json_string()</code> on a string instance and returned by the <code>key()</code> method on an object's field instance. It is always optional: replacing <code>get_raw_json_string()</code> with <code>get_string()</code> and <code>key()</code> by <code>unescaped_key()</code> or <code>escaped_key()</code> returns an <code>string_view</code> instance of the unescaped/unprocessed string.</code></p>
|
|
<p><code>You can quickly compare a <code>raw_json_string</code> instance with a target string. You may also unescape the <code>raw_json_string</code> on your own string buffer: <code>parser.unescape(mystr, ptr)</code> advances the provided pointer <code>ptr</code> and returns a string_view instance on the newly serialized string upon success, otherwise it returns an error. When unescaping to your own string buffer, you should ensure that you have sufficient memory space: the total size of the strings plus <code><a class="el" href="namespacesimdjson.html#aecdd750132f0eb123a6d61113b4197bf" title="The amount of padding needed in a buffer to parse JSON.">simdjson::SIMDJSON_PADDING</a></code> bytes. The following example illustrates how we can unescape JSON string to a user-provided buffer:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( {"name": "Jack The Ripper \u0033"} )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// We create a buffer large enough to store all strings we need:</span></div>
|
|
<div class="line">std::unique_ptr<uint8_t[]> buffer(<span class="keyword">new</span> uint8_t[json.size() + <a class="code hl_variable" href="namespacesimdjson.html#aecdd750132f0eb123a6d61113b4197bf">simdjson::SIMDJSON_PADDING</a>]);</div>
|
|
<div class="line">uint8_t * ptr = buffer.get();</div>
|
|
<div class="line">ondemand::parser parser;</div>
|
|
<div class="line">ondemand::document doc = parser.iterate(json);</div>
|
|
<div class="line"><span class="comment">// We store our strings as 'string_view' instances in a vector:</span></div>
|
|
<div class="line">std::vector<std::string_view> mystrings;</div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> key_value : doc.get_object()) {</div>
|
|
<div class="line"> std::string_view keysv = parser.unescape(key_value.key(), ptr);<span class="comment">// writes 'name'</span></div>
|
|
<div class="line"> mystrings.push_back(keysv);</div>
|
|
<div class="line"> std::string_view valuesv = parser.unescape(key_value.value().get_raw_json_string(), ptr);</div>
|
|
<div class="line"> <span class="comment">// writes 'Jack The Ripper 3', escaping the \u0033</span></div>
|
|
<div class="line"> mystrings.push_back(valuesv);</div>
|
|
<div class="line">}</div>
|
|
<div class="ttc" id="anamespacesimdjson_html_aecdd750132f0eb123a6d61113b4197bf"><div class="ttname"><a href="namespacesimdjson.html#aecdd750132f0eb123a6d61113b4197bf">simdjson::SIMDJSON_PADDING</a></div><div class="ttdeci">constexpr size_t SIMDJSON_PADDING</div><div class="ttdoc">The amount of padding needed in a buffer to parse JSON.</div><div class="ttdef"><b>Definition</b> <a href="base_8h_source.html#l00033">base.h:33</a></div></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Some users might prefer to have a direct access to a <code>std::string_view</code> instance pointing inside the source document. The <code>key_raw_json_token()</code> method serves this purpose. It provides a view on the key, including the starting quote character, and everything up to the next <code>:</code> character after the final quote character. E.g., if the key is <code>"name"</code> then <code>key_raw_json_token()</code> returns a <code>std::string_view</code> which begins with <code>"name"</code> and may containing trailing white-space characters. </p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( {"name" : "Jack The Ripper \u0033"} )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span> (<span class="keyword">auto</span> key_value : doc.get_object()) {</div>
|
|
<div class="line"> std::string_view keysv = key_value.key_raw_json_token(); <span class="comment">// keysv is "\"name\" "</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md36"></a>
|
|
General direct access to the raw JSON string</h1>
|
|
<p><code></code></p>
|
|
<p><code>If your value is a string, the <code>raw_json_string</code> you get with <code>get_raw_json_string()</code> gives you direct access to the unprocessed string. But the simdjson library allows you to have access to the raw underlying JSON more generally, not just for strings.</code></p>
|
|
<p><code>The simdjson library makes explicit assumptions about types. For examples, numbers must be integers (up to 64-bit integers) or binary64 floating-point numbers. Some users have different needs. For example, some users might want to support big integers. The library makes this possible by providing a <code>raw_json_token</code> method which returns a <code>std::string_view</code> instance containing the value as a string which you may then parse as you see fit.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> docdata = R<span class="stringliteral">"({"value":12321323213213213213213213213211223})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::object obj = doc.get_object();</span></div>
|
|
<div class="line"><span class="stringliteral">std::string_view token = obj[</span><span class="stringliteral">"value"</span>].raw_json_token();</div>
|
|
<div class="line"><span class="comment">// token has value 12321323213213213213213213213211223, it points inside the input string</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The <code>raw_json_token</code> method even works when the JSON value is a string. In such cases, it will return the complete string with the quotes and with eventual escaped sequences as in the source document.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> docdata = R<span class="stringliteral">"({"value":"12321323213213213213213213213211223"})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::object obj = doc.get_object();</span></div>
|
|
<div class="line"><span class="stringliteral">string_view token = obj[</span><span class="stringliteral">"value"</span>].raw_json_token();</div>
|
|
<div class="line"><span class="comment">// token has value "12321323213213213213213213213211223", it points inside the input string</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The <code>raw_json_token()</code> should be fast and free of allocation.</code></p>
|
|
<p><code>Given a quote-deliminated string, you find the string sequence inside the quote with a single line of code:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">std::string_view noquote(std::string_view v) { <span class="keywordflow">return</span> {v.data()+1, v.find_last_of(<span class="charliteral">'"'</span>)-1}; }</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The <code>raw_json_token()</code> method can enable you to provide fallbacks when parsing fails. Consider the following example.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = <span class="stringliteral">"{\"key\": NaN}"</span>_padded;</div>
|
|
<div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line">simdjson::ondemand::document doc = parser.iterate(json);</div>
|
|
<div class="line">simdjson::ondemand::object <span class="keywordtype">object</span> = doc.get_object();</div>
|
|
<div class="line">simdjson::ondemand::value val = <span class="keywordtype">object</span>[<span class="stringliteral">"key"</span>];</div>
|
|
<div class="line">simdjson::ondemand::json_type type = val.type();</div>
|
|
<div class="line"><span class="comment">// type == simdjson::ondemand::json_type::unknown</span></div>
|
|
<div class="line"><span class="keywordflow">try</span> {</div>
|
|
<div class="line"> <span class="keywordtype">double</span> num = val.get_double();</div>
|
|
<div class="line">} <span class="keywordflow">catch</span> (<span class="keyword">const</span> <a class="code hl_struct" href="structsimdjson_1_1simdjson__error.html">simdjson::simdjson_error</a>& e) {</div>
|
|
<div class="line"> <span class="comment">// e == simdjson::error_code::INCORRECT_TYPE</span></div>
|
|
<div class="line"> std::string_view str = val.raw_json_token();</div>
|
|
<div class="line"> <span class="comment">// str == "NaN"</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The NaN is not supported in JSON. However, in the On-Demand API, you can check the string corresponding to the JSON token and determine how to handle it.</code></p>
|
|
<p><code></code></p>
|
|
<h2><a class="anchor" id="autotoc_md37"></a>
|
|
Raw JSON string for objects and arrays</h2>
|
|
<p><code></code></p>
|
|
<p><code>If your value is an array or an object, <code>raw_json_token()</code> returns effectively a single character (<code>[</code>) or (<code>}</code>) which is not very useful. For arrays and objects, we have another method called <code>raw_json()</code> which consumes (traverses) the array or the object.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> docdata = R<span class="stringliteral">"({"value":123})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::object obj = doc.get_object();</span></div>
|
|
<div class="line"><span class="stringliteral">string_view token = obj.raw_json(); </span><span class="comment">// gives you `{"value":123}`</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> docdata = R<span class="stringliteral">"([1,2,3])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::array arr = doc.get_array();</span></div>
|
|
<div class="line"><span class="stringliteral">string_view token = arr.raw_json(); </span><span class="comment">// gives you `[1,2,3]`</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>Because <code>raw_json()</code> consumes to object or the array, if you want to both have access to the raw string, and also use the array or object, you should call <code>reset()</code>.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">simdjson::ondemand::parser parser;</div>
|
|
<div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> docdata = R<span class="stringliteral">"({"value":123})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(docdata);</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::object obj = doc.get_object();</span></div>
|
|
<div class="line"><span class="stringliteral">string_view token = obj.raw_json(); </span><span class="comment">// gives you `{"value":123}`</span></div>
|
|
<div class="line">obj.reset(); <span class="comment">// revise the object</span></div>
|
|
<div class="line">uint64_t x = obj[<span class="stringliteral">"value"</span>]; <span class="comment">// gives me 123</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>You can use <code>raw_json()</code> with the values inside an array and object. When calling <code>raw_json()</code> on an untyped value, it acts as <code>raw_json()</code> when the value is an array or an object. Otherwise, it acts as <code>raw_json_token()</code>. It is useful if you do not care for the type of the value and just wants a string representation.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( [1,2,"fds", {"a":1}, [1,344]] )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordtype">size_t</span> counter = 0;</div>
|
|
<div class="line"><span class="keywordflow">for</span>(<span class="keyword">auto</span> array: doc) {</div>
|
|
<div class="line"> std::string_view raw = array.raw_json();</div>
|
|
<div class="line"> <span class="comment">// will capture "1", "2", "\"fds\"", "{\"a\":1}", "[1,344]"</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"( {"key1":1,"key2":2,"key3":"fds", "key4":{"a":1}, "key5":[1,344]} )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordtype">size_t</span> counter = 0;</div>
|
|
<div class="line"><span class="keywordflow">for</span>(<span class="keyword">auto</span> key_value: doc.get_object()) {</div>
|
|
<div class="line"> std::string_view raw = key_value.value().raw_json();</div>
|
|
<div class="line"> <span class="comment">// will capture "1", "2", "\"fds\"", "{\"a\":1}", "[1,344]"</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>You can use <code>raw_json()</code> to capture the content of some JSON values as <code>std::string_view</code> instances which can be safely used later. The <code>std::string_view</code> instances point inside the original document and do not depend in any way on simdjson. In the following example, we store the <code>std::string_view</code> instances inside a <code>std::vector<std::string_view></code> instance and print the out after the parsing is concluded:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json_padded = <span class="stringliteral">"{\"a\":[1,2,3], \"b\": 2, \"c\": \"hello\"}"</span>_padded;</div>
|
|
<div class="line">std::vector<std::string_view> fields;</div>
|
|
<div class="line"> </div>
|
|
<div class="line">ondemand::parser parser;</div>
|
|
<div class="line"><span class="keyword">auto</span> doc = parser.iterate(json_padded);</div>
|
|
<div class="line"><span class="keyword">auto</span> <span class="keywordtype">object</span> = doc.get_object();</div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> field : object) {</div>
|
|
<div class="line"> fields.push_back(field.value().raw_json());</div>
|
|
<div class="line">}</div>
|
|
<div class="line"><span class="comment">// Output the fields</span></div>
|
|
<div class="line"><span class="comment">// Expected output:</span></div>
|
|
<div class="line"><span class="comment">// [1,2,3]</span></div>
|
|
<div class="line"><span class="comment">// 2</span></div>
|
|
<div class="line"><span class="comment">// "hello"</span></div>
|
|
<div class="line"><span class="keywordflow">for</span> (std::string_view field_ref : fields) {</div>
|
|
<div class="line"> std::cout << field_ref << std::endl;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md38"></a>
|
|
Storing directly into an existing string instance</h1>
|
|
<p><code></code></p>
|
|
<p><code>The simdjson library favours the use of <code>std::string_view</code> instances because it tends to lead to better performance due to causing fewer memory allocations. However, they are cases where you need to store a string result in a <code>std::string</code> instance. You can do so with a templated version of the <code>to_string()</code> method which takes as a parameter a reference to a <code>std::string</code>.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"> <span class="keyword">auto</span> json = R<span class="stringliteral">"({</span></div>
|
|
<div class="line"><span class="stringliteral"> "name": "Daniel",</span></div>
|
|
<div class="line"><span class="stringliteral"> "age": 42</span></div>
|
|
<div class="line"><span class="stringliteral">})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> std::string name;</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keyword">auto</span> error = doc[<span class="stringliteral">"name"</span>].get_string(name);</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(error) { <span class="comment">/* handle error */</span> }</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The same routine can be written without exceptions handling:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line">std::string name;</div>
|
|
<div class="line"><span class="keyword">auto</span> error = doc[<span class="stringliteral">"name"</span>].get_string(name);</div>
|
|
<div class="line"><span class="keywordflow">if</span> (error) { <span class="comment">/* handle error */</span> }</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>The <code>std::string</code> instance, once created, is independent. Unlike our <code>std::string_view</code> instances, it does not point at data that is within our <code>parser</code> instance. The same caveat applies: you should only consume a JSON string once.</code></p>
|
|
<p><code>Because <code>get_string()</code> is a template that requires a type that can be assigned a <code>std::string</code>, you can use it with features such as <code>std::optional</code>:</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"({ "foo1": "3.1416" } )"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral">std::optional<std::string> value;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">if</span> (doc[<span class="stringliteral">"foo1"</span>].get_string(value)) { <span class="comment">/* error */</span> }</div>
|
|
<div class="line"><span class="comment">// value was populated with "3.1416"</span></div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>You can generally convert any answer that would return an <code>std::string_view</code>.</code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"({"\u0062\u0065\u0062\u0065": 2} })"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral">ondemand::object </span><span class="keywordtype">object</span> = doc.get_object();</div>
|
|
<div class="line"><span class="keywordflow">for</span> (<span class="keyword">auto</span> field : object) {</div>
|
|
<div class="line"> std::string key;</div>
|
|
<div class="line"> error = field.unescaped_key().get(key);</div>
|
|
<div class="line"> <span class="keywordflow">if</span>(error) { <span class="comment">/* */</span> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code>You should be mindful of the trade-off: allocating multiple <code>std::string</code> instances can become expensive.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md39"></a>
|
|
Thread safety</h1>
|
|
<p><code></code></p>
|
|
<p><code>We built simdjson with thread safety in mind.</code></p>
|
|
<p><code>The simdjson library is single-threaded except for <a class="el" href="md_doc_2iterate__many.html">`iterate_many`</a> and <a class="el" href="md_doc_2parse__many.html">`parse_many`</a> which may use secondary threads under their control when the library is compiled with thread support.</code></p>
|
|
<p><code>We recommend using one <code>parser</code> object per thread. When using the On-Demand front-end (our default), you should access the <code>document</code> instances in a single-threaded manner since it acts as an iterator (and is therefore not thread safe).</code></p>
|
|
<p><code>The CPU detection, which runs the first time parsing is attempted and switches to the fastest parser for your CPU, is transparent and thread-safe. Our runtime dispatching is based on global objects that are instantiated at the beginning of the main thread and may be discarded at the end of the main thread. If you have multiple threads running and some threads use the library while the main thread is cleaning up resources, you may encounter issues. If you expect such problems, you may consider using <a href="https://en.cppreference.com/w/cpp/utility/program/quick_exit">std::quick_exit</a>.</code></p>
|
|
<p><code>In a threaded environment, stack space is often limited. Running code like simdjson in debug mode may require hundreds of kilobytes of stack memory. Thus stack overflows are a possibility. We recommend you turn on optimization when working in an environment where stack space is limited. If you must run your code in debug mode, we recommend you configure your system to have more stack space. We discourage you from running production code based on a debug build.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md40"></a>
|
|
Standard compliance</h1>
|
|
<p><code></code></p>
|
|
<p><code>The simdjson library is fully compliant with the <a href="https://www.tbray.org/ongoing/When/201x/2017/12/14/rfc8259.html">RFC 8259</a> JSON specification.</code></p>
|
|
<p><code></p><ul>
|
|
<li>The only insignificant whitespace characters allowed are the space, the horizontal tab, the line feed and the carriage return. In particular, a JSON document may not contain an unescaped null character.</li>
|
|
<li>A single string or a single number is considered to be a valid JSON document.</li>
|
|
<li>We fully validate the numbers according to the JSON specification. For example, the string <code>01</code> is not valid JSON document since the specification states that <em>leading zeros are not allowed</em>.</li>
|
|
<li>The specification allows implementations to set limits on the range and precision of numbers accepted. We support 64-bit floating-point numbers as well as integer values.<ul>
|
|
<li>We parse integers and floating-point numbers as separate types which allows us to support all signed (two's complement) 64-bit integers, like a Java <code>long</code> or a C/C++ <code>long long</code> and all 64-bit unsigned integers. When we cannot represent exactly an integer as a signed or unsigned 64-bit value, we reject the JSON document.</li>
|
|
<li>We support the full range of 64-bit floating-point numbers (binary64). The values range from <code>std::numeric_limits<double>::lowest()</code> to <code>std::numeric_limits<double>::max()</code>, so from -1.7976e308 all the way to 1.7975e308. Extreme values (less or equal to -1e308, greater or equal to 1e308) are rejected: we refuse to parse the input document. Numbers are parsed with a perfect accuracy (ULP 0): the nearest floating-point value is chosen, rounding to even when needed. If you serialized your floating-point numbers with 17 significant digits in a standard compliant manner, the simdjson library is guaranteed to recover the same numbers, exactly.</li>
|
|
</ul>
|
|
</li>
|
|
<li>The specification states that JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8. The simdjson library does full UTF-8 validation as part of the parsing. The specification states that implementations MUST NOT add a byte order mark: the simdjson library rejects documents starting with a byte order mark.</li>
|
|
<li>The simdjson library validates string content for unescaped characters. Unescaped line breaks and tabs in strings are not allowed.</li>
|
|
<li>The simdjson library accepts objects with repeated keys: all of the name/value pairs, including duplicates, are reported. We do not enforce key uniqueness.</li>
|
|
<li>The specification states that an implementation may set limits on the size of texts that it accepts. The simdjson library limits single JSON documents to 4 GiB. It will refuse to parse a JSON document larger than 4294967295 bytes. (This limitation does not apply to streams of JSON documents, only to single JSON documents.)</li>
|
|
<li>The specification states that an implementation may set limits on the maximum depth of nesting. By default, the simdjson will refuse to parse documents with a depth exceeding 1024.</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md41"></a>
|
|
Backwards compatibility</h1>
|
|
<p><code></code></p>
|
|
<p><code>The only header file supported by simdjson is <code><a class="el" href="simdjson_8h.html">simdjson.h</a></code>. Older versions of simdjson published a number of other include files such as <code>document.h</code> or <code>ParsedJson.h</code> alongside <code><a class="el" href="simdjson_8h.html">simdjson.h</a></code>; these headers may be moved or removed in future versions.</code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md42"></a>
|
|
Examples</h1>
|
|
<p><code></code></p>
|
|
<p><code>Some users like to have example. The following code samples illustrate how to process specific JSON inputs. For simplicity, we do not include full error support: this code would throw exceptions on error.</code></p>
|
|
<p><code></p><ul>
|
|
<li>Example 1: ZuluBBox</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keyword">struct </span>ZuluBBox {</div>
|
|
<div class="line"> <span class="keywordtype">double</span> xmin;</div>
|
|
<div class="line"> <span class="keywordtype">double</span> ymin;</div>
|
|
<div class="line"> <span class="keywordtype">double</span> width;</div>
|
|
<div class="line"> <span class="keywordtype">double</span> height;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="keywordtype">void</span> print() {</div>
|
|
<div class="line"> std::cout << xmin << <span class="stringliteral">", "</span> << ymin << <span class="stringliteral">", "</span> << width << <span class="stringliteral">", "</span> << height</div>
|
|
<div class="line"> << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keywordtype">bool</span> example() {</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="keyword">auto</span> json = R<span class="stringliteral">"+( {</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluROI": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluBBox": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "xmin": 0,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ymin": 0,</span></div>
|
|
<div class="line"><span class="stringliteral"> "width": 1,</span></div>
|
|
<div class="line"><span class="stringliteral"> "height": 1</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "SubObjects": [</span></div>
|
|
<div class="line"><span class="stringliteral"> {</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluDetection": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "label": "car",</span></div>
|
|
<div class="line"><span class="stringliteral"> "class_id": 3,</span></div>
|
|
<div class="line"><span class="stringliteral"> "confidence": 0.7587034106254578,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluBBox": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "xmin": 0.3843536376953125,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ymin": 0.4532909393310547,</span></div>
|
|
<div class="line"><span class="stringliteral"> "width": 0.09115534275770187,</span></div>
|
|
<div class="line"><span class="stringliteral"> "height": 0.04127710685133934</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "SubObjects": []</span></div>
|
|
<div class="line"><span class="stringliteral"> }</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> {</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluDetection": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "label": "car",</span></div>
|
|
<div class="line"><span class="stringliteral"> "class_id": 3,</span></div>
|
|
<div class="line"><span class="stringliteral"> "confidence": 0.6718865633010864,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluBBox": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "xmin": 0.7500002980232239,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ymin": 0.5212296843528748,</span></div>
|
|
<div class="line"><span class="stringliteral"> "width": 0.07592231780290604,</span></div>
|
|
<div class="line"><span class="stringliteral"> "height": 0.038947589695453644</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "SubObjects": []</span></div>
|
|
<div class="line"><span class="stringliteral"> }</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> {</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluDetection": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "label": "car",</span></div>
|
|
<div class="line"><span class="stringliteral"> "class_id": 3,</span></div>
|
|
<div class="line"><span class="stringliteral"> "confidence": 0.5806200504302979,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ZuluBBox": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "xmin": 0.9025363922119141,</span></div>
|
|
<div class="line"><span class="stringliteral"> "ymin": 0.5925348401069641,</span></div>
|
|
<div class="line"><span class="stringliteral"> "width": 0.05478987470269203,</span></div>
|
|
<div class="line"><span class="stringliteral"> "height": 0.046337299048900604</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "SubObjects": []</span></div>
|
|
<div class="line"><span class="stringliteral"> }</span></div>
|
|
<div class="line"><span class="stringliteral"> }</span></div>
|
|
<div class="line"><span class="stringliteral"> ]</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "timestamp (ms)": 1677085594421,</span></div>
|
|
<div class="line"><span class="stringliteral"> "buffer_offset": 35673</span></div>
|
|
<div class="line"><span class="stringliteral">} )+"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::object root_object = doc.get_object();</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::object roi_object = root_object[</span><span class="stringliteral">"ZuluROI"</span>];</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> ondemand::object box_roi_object = roi_object[<span class="stringliteral">"ZuluBBox"</span>];</div>
|
|
<div class="line"> ZuluBBox box = {</div>
|
|
<div class="line"> double(box_roi_object[<span class="stringliteral">"xmin"</span>]), double(box_roi_object[<span class="stringliteral">"ymin"</span>]),</div>
|
|
<div class="line"> double(box_roi_object[<span class="stringliteral">"width"</span>]), double(box_roi_object[<span class="stringliteral">"height"</span>])};</div>
|
|
<div class="line"> box.print();</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> <span class="keywordflow">for</span> (ondemand::object value : roi_object[<span class="stringliteral">"SubObjects"</span>]) {</div>
|
|
<div class="line"> ondemand::object detect = value[<span class="stringliteral">"ZuluDetection"</span>];</div>
|
|
<div class="line"> std::cout << detect[<span class="stringliteral">"label"</span>].get_string() << std::endl;</div>
|
|
<div class="line"> std::cout << detect[<span class="stringliteral">"class_id"</span>].get_uint64() << std::endl;</div>
|
|
<div class="line"> std::cout << detect[<span class="stringliteral">"confidence"</span>].get_double() << std::endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> ondemand::object vbox_roi_object = detect[<span class="stringliteral">"ZuluBBox"</span>];</div>
|
|
<div class="line"> ZuluBBox vbox = {</div>
|
|
<div class="line"> double(vbox_roi_object[<span class="stringliteral">"xmin"</span>]), double(vbox_roi_object[<span class="stringliteral">"ymin"</span>]),</div>
|
|
<div class="line"> double(vbox_roi_object[<span class="stringliteral">"width"</span>]), double(vbox_roi_object[<span class="stringliteral">"height"</span>])};</div>
|
|
<div class="line"> vbox.print();</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> std::cout << root_object[<span class="stringliteral">"timestamp (ms)"</span>].get_uint64() << std::endl;</div>
|
|
<div class="line"> std::cout << root_object[<span class="stringliteral">"buffer_offset"</span>].get_uint64() << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">return</span> <span class="keyword">true</span>;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></p><ul>
|
|
<li>Example 2: Demos</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keywordtype">bool</span> example() {</div>
|
|
<div class="line"> <span class="keyword">auto</span> json = R<span class="stringliteral">"+( {</span></div>
|
|
<div class="line"><span class="stringliteral"> "5f08a730b280e54fd1e75a7046b93fdc": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "file": "/DEMOS/0-9/10_Orbyte.sid",</span></div>
|
|
<div class="line"><span class="stringliteral"> "len": [</span></div>
|
|
<div class="line"><span class="stringliteral"> "1:17"</span></div>
|
|
<div class="line"><span class="stringliteral"> ],</span></div>
|
|
<div class="line"><span class="stringliteral"> "loud": [</span></div>
|
|
<div class="line"><span class="stringliteral"> "-22.8"</span></div>
|
|
<div class="line"><span class="stringliteral"> ],</span></div>
|
|
<div class="line"><span class="stringliteral"> "name": "10 Orbyte",</span></div>
|
|
<div class="line"><span class="stringliteral"> "author": "Michael Becker (Premium)",</span></div>
|
|
<div class="line"><span class="stringliteral"> "release": "2014 Tristar & Red Sector Inc.",</span></div>
|
|
<div class="line"><span class="stringliteral"> "bits": 20</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "2727236ead44a62f0c6e01f6dd4dc484": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "file": "/DEMOS/0-9/12345.sid",</span></div>
|
|
<div class="line"><span class="stringliteral"> "len": [</span></div>
|
|
<div class="line"><span class="stringliteral"> "0:56"</span></div>
|
|
<div class="line"><span class="stringliteral"> ],</span></div>
|
|
<div class="line"><span class="stringliteral"> "loud": [</span></div>
|
|
<div class="line"><span class="stringliteral"> "-33.3"</span></div>
|
|
<div class="line"><span class="stringliteral"> ],</span></div>
|
|
<div class="line"><span class="stringliteral"> "name": "12345",</span></div>
|
|
<div class="line"><span class="stringliteral"> "author": "Beal",</span></div>
|
|
<div class="line"><span class="stringliteral"> "release": "1988 Beal",</span></div>
|
|
<div class="line"><span class="stringliteral"> "bits": 20</span></div>
|
|
<div class="line"><span class="stringliteral"> },</span></div>
|
|
<div class="line"><span class="stringliteral"> "7ea765fce6c0f92570b18adc7bf52f54": {</span></div>
|
|
<div class="line"><span class="stringliteral"> "file": "/DEMOS/0-9/128_Byte_Blues_BASIC.sid",</span></div>
|
|
<div class="line"><span class="stringliteral"> "len": [</span></div>
|
|
<div class="line"><span class="stringliteral"> "0:18"</span></div>
|
|
<div class="line"><span class="stringliteral"> ],</span></div>
|
|
<div class="line"><span class="stringliteral"> "loud": [</span></div>
|
|
<div class="line"><span class="stringliteral"> "-27.1"</span></div>
|
|
<div class="line"><span class="stringliteral"> ],</span></div>
|
|
<div class="line"><span class="stringliteral"> "name": "128 Byte Blues",</span></div>
|
|
<div class="line"><span class="stringliteral"> "author": "Leonard J. Paul (Freaky DNA)",</span></div>
|
|
<div class="line"><span class="stringliteral"> "release": "2005 Freaky DNA",</span></div>
|
|
<div class="line"><span class="stringliteral"> "bits": 62</span></div>
|
|
<div class="line"><span class="stringliteral"> }</span></div>
|
|
<div class="line"><span class="stringliteral">} )+"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> ondemand::object root_object = doc.get_object();</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">for</span>(<span class="keyword">auto</span> key_value : root_object) {</div>
|
|
<div class="line"> <span class="comment">// could get std::string_view with 'unescaped_key()' or 'escaped_key()':</span></div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"key: "</span> << key_value.key() << std::endl;</div>
|
|
<div class="line"> ondemand::object obj = key_value.value();</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"file: "</span> << std::string_view(obj[<span class="stringliteral">"file"</span>]) << std::endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"len: "</span>;</div>
|
|
<div class="line"> <span class="keywordflow">for</span>(std::string_view values : obj[<span class="stringliteral">"len"</span>]) {</div>
|
|
<div class="line"> std::cout << values << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> std::cout << std::endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"loud: "</span>;</div>
|
|
<div class="line"> <span class="keywordflow">for</span>(std::string_view values : obj[<span class="stringliteral">"loud"</span>]) {</div>
|
|
<div class="line"> std::cout << values << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> std::cout << std::endl;</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"name: "</span> << std::string_view(obj[<span class="stringliteral">"name"</span>]) << std::endl;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"author: "</span> << std::string_view(obj[<span class="stringliteral">"author"</span>]) << std::endl;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"release: "</span> << std::string_view(obj[<span class="stringliteral">"release"</span>]) << std::endl;</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"bits: "</span> << uint64_t(obj[<span class="stringliteral">"bits"</span>]) << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> <span class="keyword">true</span>;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></p><ul>
|
|
<li>Example 3: CRT</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keywordtype">bool</span> example() {</div>
|
|
<div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> padded_input_json = R<span class="stringliteral">"([</span></div>
|
|
<div class="line"><span class="stringliteral"> { "monitor": [</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "monitor", "type": "toggle", "label": "monitor" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "profile", "type": "selector", "label": "collection" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "overlay", "type": "selector", "label": "overlay" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "zoom", "type": "toggleSlider", "label": "zoom" }</span></div>
|
|
<div class="line"><span class="stringliteral"> ] },</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"> { "crt": [</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "system", "type": "multi", "label": "system", "choices": "PAL, NTSC" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "type": "spacer" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "brightness", "type": "slider", "icon": "brightness" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "contrast", "type": "slider", "icon": "contrast" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "saturation", "type": "slider", "icon": "saturation" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "type": "spacer" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "overscan", "type": "toggleSlider", "label": "overscan" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "type": "spacer" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "emulation", "type": "toggle", "label": "CRT emulation" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "type": "spacer" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "curve", "type": "toggleSlider", "label": "curve" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "bleed", "type": "toggleSlider", "label": "bleed" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "vignette", "type": "toggleSlider", "label": "vignette" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "scanlines", "type": "toggleSlider", "label": "scanlines" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "gridlines", "type": "toggleSlider", "label": "gridlines" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "glow", "type": "toggleSlider", "label": "glow" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "flicker", "type": "toggleSlider", "label": "flicker" },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "id": "noise", "type": "toggleSlider", "label": "noise" },</span></div>
|
|
<div class="line"><span class="stringliteral"> {}</span></div>
|
|
<div class="line"><span class="stringliteral"> ] }</span></div>
|
|
<div class="line"><span class="stringliteral">])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keyword">auto</span> parser = ondemand::parser{};</div>
|
|
<div class="line"> <span class="keyword">auto</span> doc = parser.iterate(padded_input_json);</div>
|
|
<div class="line"> <span class="keyword">auto</span> root_array = doc.get_array();</div>
|
|
<div class="line"> <span class="comment">// the root should be an object, not an array, but that's the JSON we are</span></div>
|
|
<div class="line"> <span class="comment">// given.</span></div>
|
|
<div class="line"> <span class="keywordflow">for</span> (ondemand::object node : root_array) {</div>
|
|
<div class="line"> <span class="comment">// We know that we are going to have just one element in the object.</span></div>
|
|
<div class="line"> <span class="keywordflow">for</span> (<span class="keyword">auto</span> field : node) {</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"\n\ntop level:"</span> << field.key() << std::endl;</div>
|
|
<div class="line"> <span class="comment">// You can get a proper std::string_view for the key with:</span></div>
|
|
<div class="line"> <span class="comment">// std::string_view key = field.unescaped_key();</span></div>
|
|
<div class="line"> <span class="comment">// or</span></div>
|
|
<div class="line"> <span class="comment">// std::string_view key = field.escaped_key();</span></div>
|
|
<div class="line"> <span class="comment">// and second for-range loop to get child-elements here</span></div>
|
|
<div class="line"> <span class="keywordflow">for</span> (ondemand::object inner_object : field.value()) {</div>
|
|
<div class="line"> <span class="keyword">auto</span> i = inner_object.begin();</div>
|
|
<div class="line"> <span class="keywordflow">if</span> (i == inner_object.end()) {</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"empty object"</span> << std::endl;</div>
|
|
<div class="line"> <span class="keywordflow">continue</span>;</div>
|
|
<div class="line"> } <span class="keywordflow">else</span> {</div>
|
|
<div class="line"> <span class="keywordflow">for</span> (; i != inner_object.end(); ++i) {</div>
|
|
<div class="line"> <span class="keyword">auto</span> inner_field = *i;</div>
|
|
<div class="line"> std::cout << <span class="charliteral">'"'</span> << inner_field.key()</div>
|
|
<div class="line"> << <span class="stringliteral">"\" : "</span> << inner_field.value() << <span class="stringliteral">", "</span>;</div>
|
|
<div class="line"> <span class="comment">// You can get proper std::string_view for the key and value with:</span></div>
|
|
<div class="line"> <span class="comment">// std::string_view inner_key = field.unescaped_key();</span></div>
|
|
<div class="line"> <span class="comment">// or</span></div>
|
|
<div class="line"> <span class="comment">// std::string_view inner_key = field.escaped_key();</span></div>
|
|
<div class="line"> <span class="comment">// and</span></div>
|
|
<div class="line"> <span class="comment">// std::string_view value_str = field.value();</span></div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> std::cout << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="comment">// You can break here if you only want just the first element.</span></div>
|
|
<div class="line"> <span class="comment">// break;</span></div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">return</span> <span class="keyword">true</span>;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></p><ul>
|
|
<li>Example 4: Passing an array to a function</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
<div class="line"><span class="preprocessor">#include <iostream></span></div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="comment">// prints the content of the array as hexadecimal 64-bit integers</span></div>
|
|
<div class="line"><span class="keywordtype">void</span> f(simdjson::ondemand::array v) {</div>
|
|
<div class="line"> <span class="keywordflow">for</span>(uint64_t val : v) {</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"0x"</span> << std::hex << val << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
<div class="line"> </div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keywordtype">int</span> main(<span class="keywordtype">void</span>) {</div>
|
|
<div class="line"> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">simdjson::padded_string</a> json = R<span class="stringliteral">"( [ 897314173811950000, 3122321 ])"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral"> simdjson::ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"> f(doc.get_array());</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keywordflow">return</span> EXIT_SUCCESS;</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></p><ul>
|
|
<li>Example 4: Value capture with <code>std::string_view</code> instances</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></p><div class="fragment"><div class="line"><span class="keywordtype">void</span> example() {</div>
|
|
<div class="line"> ondemand::parser parser;</div>
|
|
<div class="line"> <span class="keyword">const</span> <a class="code hl_struct" href="structsimdjson_1_1padded__string.html">padded_string</a> json = R<span class="stringliteral">"({ "parent": {"child1": {"name": "John"} , "child2": {"name": "Daniel"}} })"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"> </span><span class="keyword">auto</span> doc = parser.iterate(json);</div>
|
|
<div class="line"> ondemand::object parent = doc[<span class="stringliteral">"parent"</span>];</div>
|
|
<div class="line"> <span class="comment">// parent owns the focus</span></div>
|
|
<div class="line"> ondemand::object c1 = parent[<span class="stringliteral">"child1"</span>];</div>
|
|
<div class="line"> <span class="comment">// c1 owns the focus</span></div>
|
|
<div class="line"> <span class="comment">//</span></div>
|
|
<div class="line"> std::string_view as1 = c1[<span class="stringliteral">"name"</span>];</div>
|
|
<div class="line"> <span class="comment">// We have that as1 == "John", as long as 'parser' and 'json' live</span></div>
|
|
<div class="line"> <span class="comment">// c2 attempts to grab the focus from parent but fails</span></div>
|
|
<div class="line"> ondemand::object c2 = parent[<span class="stringliteral">"child2"</span>];</div>
|
|
<div class="line"> <span class="comment">// c2 owns the focus, at this point c1 is invalid</span></div>
|
|
<div class="line"> std::string_view as2 = c2[<span class="stringliteral">"name"</span>];</div>
|
|
<div class="line"> <span class="comment">// We have that as2 == "Daniel", as long as 'parser' and 'json' live</span></div>
|
|
<div class="line"> std::cout << as1 << <span class="stringliteral">" "</span> << as2 << std::endl; <span class="comment">// prints John Daniel</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p></code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md43"></a>
|
|
Performance tips</h1>
|
|
<p><code></code></p>
|
|
<p><code></p><ul>
|
|
<li>Read <a class="el" href="md_doc_2performance.html">our performance notes</a> for advanced topics.</li>
|
|
<li>To better understand the operation of your On-Demand parser, and whether it is performing as well as you think it should be, there is a logger feature built in to simdjson! To use it, define the pre-processor directive <code>SIMDJSON_VERBOSE_LOGGING</code> prior to including the <code><a class="el" href="simdjson_8h.html">simdjson.h</a></code> header, which enables logging in simdjson. Run your code. It may generate a lot of logging output; adding printouts from your application that show each section may be helpful. The log's output will show step-by-step information on state, buffer pointer position, depth, and key retrieval status. Importantly, unless <code>SIMDJSON_VERBOSE_LOGGING</code> is defined, logging is entirely disabled and thus carries no overhead.</li>
|
|
<li>The On-Demand front-end works best when doing a single pass over the input: avoid calling <code>count_elements</code>, <code>rewind</code>, <code>reset</code> and similar methods.</li>
|
|
<li>If you are familiar with assembly language, you may use the online tool godbolt to explore the compiled code. The following example may work: <a href="https://godbolt.org/z/xE4GWs573">https://godbolt.org/z/xE4GWs573</a>.</li>
|
|
<li>Given a field <code>field</code> in an object, calling <code>field.key()</code> is often faster than <code>field.unescaped_key()</code> so if you do not need an unescaped <code>std::string_view</code> instance, prefer <code>field.key()</code>. Similarly, we expect <code>field.escaped_key()</code> to be faster than <code>field.unescaped_key()</code> even though both return a <code>std::string_view</code> instance.</li>
|
|
<li>For release builds, we recommend setting the <code>NDEBUG</code> pre-processor directive when compiling the <code>simdjson</code> library. Importantly, using the optimization flags <code>-O2</code> or <code>-O3</code> under GCC and LLVM clang does not set the <code>NDEBUG</code> directive, you must set it manually (e.g., <code>-DNDEBUG</code>).</li>
|
|
<li>For long streams of JSON documents, consider <a class="el" href="md_doc_2iterate__many.html">`iterate_many`</a> and <a class="el" href="md_doc_2parse__many.html">`parse_many`</a> for better performance.</li>
|
|
<li>Never seek to access a field twice (e.g., o["data"] and later again o["data"]). Instead capture once an ondemand::value and reuse it.</li>
|
|
<li>If you must access several different keys in an object, it might be preferable to iterate through all the fields in the object instead, and branch on the field keys. Consider this example. <div class="fragment"><div class="line"><span class="keyword">auto</span> json = R<span class="stringliteral">"({"price": 123.456789, "volume": 9999,</span></div>
|
|
<div class="line"><span class="stringliteral"> "timestamp": "2025-09-04T09:45:00Z",</span></div>
|
|
<div class="line"><span class="stringliteral"> "symbol": "XYZ", "currency": "USD", "change": 1.23,</span></div>
|
|
<div class="line"><span class="stringliteral"> "isActive": true})"_padded;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::parser parser;</span></div>
|
|
<div class="line"><span class="stringliteral">simdjson::ondemand::document doc = parser.iterate(json);</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keywordflow">for</span>(<span class="keyword">auto</span> keyvalue : doc.get_object()) {</div>
|
|
<div class="line"> simdjson::ondemand::raw_json_string key = keyvalue.key();</div>
|
|
<div class="line"> <span class="keywordflow">switch</span>(key[0]) {</div>
|
|
<div class="line"> <span class="keywordflow">case</span> <span class="charliteral">'p'</span>: <span class="comment">// price</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span> (key == <span class="stringliteral">"price"</span>) {</div>
|
|
<div class="line"> std::string_view price_str = keyvalue.value().raw_json();</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"Price: "</span> << price_str << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> <span class="charliteral">'v'</span>: <span class="comment">// volume</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span> (key == <span class="stringliteral">"volume"</span>) {</div>
|
|
<div class="line"> std::string_view volume_str = keyvalue.value().raw_json();</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"Volume: "</span> << volume_str << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">case</span> <span class="charliteral">'t'</span>: <span class="comment">// timestamp</span></div>
|
|
<div class="line"> <span class="keywordflow">if</span> (key == <span class="stringliteral">"timestamp"</span>) {</div>
|
|
<div class="line"> std::string_view timestamp = keyvalue.value();</div>
|
|
<div class="line"> std::cout << <span class="stringliteral">"Timestamp: "</span> << timestamp << std::endl;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line"> <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> <span class="keywordflow">default</span>: <span class="keywordflow">break</span>;</div>
|
|
<div class="line"> }</div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --></li>
|
|
<li>If possible, refer to each object and array in your code once. For example, the following code repeatedly refers to the <code>"data"</code> key to create an object... <div class="fragment"><div class="line">std::string_view make = o[<span class="stringliteral">"data"</span>][<span class="stringliteral">"make"</span>];</div>
|
|
<div class="line">std::string_view model = o[<span class="stringliteral">"data"</span>][<span class="stringliteral">"model"</span>];</div>
|
|
<div class="line">std::string_view year = o[<span class="stringliteral">"data"</span>][<span class="stringliteral">"year"</span>];</div>
|
|
</div><!-- fragment --> We expect that it is more efficient to access the <code>"data"</code> key once: <div class="fragment"><div class="line">simdjson::ondemand::object data = o[<span class="stringliteral">"data"</span>];</div>
|
|
<div class="line">std::string_view model = data[<span class="stringliteral">"model"</span>];</div>
|
|
<div class="line">std::string_view year = data[<span class="stringliteral">"year"</span>];</div>
|
|
<div class="line">std::string_view rating = data[<span class="stringliteral">"rating"</span>];</div>
|
|
</div><!-- fragment --></li>
|
|
<li>You will get better performance if you seek the keys in the order in which they appear in the document. So if processing <code>{"a":1, "b":2, "c":3}</code>, do <code>value1 = data["a"]; value2 = data["b"]; value3 data["c"];</code> and not <code>value2 = data["b"]; value1 = data["a"]; value3 data["c"];</code>. Of course, it is not always possible to know for sure in which order the keys appear.</li>
|
|
</ul>
|
|
<p></code></p>
|
|
<p><code></code></p>
|
|
<h1><a class="anchor" id="autotoc_md44"></a>
|
|
Further reading</h1>
|
|
<p><code></code></p>
|
|
<p><code></p><ul>
|
|
<li>John Keiser, Daniel Lemire, <a href="http://arxiv.org/abs/2312.17149">On-Demand JSON: A Better Way to Parse Documents?</a>, Software: Practice and Experience 54 (6), 2024 </li>
|
|
</ul>
|
|
<p></code></p>
|
|
</div></div><!-- contents -->
|
|
</div><!-- PageDoc -->
|
|
</div><!-- doc-content -->
|
|
<!-- start footer part -->
|
|
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
|
|
<ul>
|
|
<li class="footer">Generated by <a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.8 </li>
|
|
</ul>
|
|
</div>
|
|
</body>
|
|
</html>
|