mirror of
https://github.com/simdjson/simdjson
synced 2026-06-08 17:27:07 +00:00
lemire
264 lines
18 KiB
HTML
264 lines
18 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
|
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
|
|
<meta name="generator" content="Doxygen 1.9.8"/>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
|
<title>simdjson: Parse json at compile time</title>
|
|
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="jquery.js"></script>
|
|
<script type="text/javascript" src="dynsections.js"></script>
|
|
<link href="navtree.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="resize.js"></script>
|
|
<script type="text/javascript" src="navtreedata.js"></script>
|
|
<script type="text/javascript" src="navtree.js"></script>
|
|
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="search/searchdata.js"></script>
|
|
<script type="text/javascript" src="search/search.js"></script>
|
|
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
|
<link href="doxygen-awesome.css" rel="stylesheet" type="text/css"/>
|
|
<link href="doxygen-awesome-sidebar-only.css" rel="stylesheet" type="text/css"/>
|
|
<link href="doxygen-awesome-sidebar-only-darkmode-toggle.css" rel="stylesheet" type="text/css"/>
|
|
</head>
|
|
<body>
|
|
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
|
<div id="titlearea">
|
|
<table cellspacing="0" cellpadding="0">
|
|
<tbody>
|
|
<tr id="projectrow">
|
|
<td id="projectlogo"><img alt="Logo" src="logotiny.png"/></td>
|
|
<td id="projectalign">
|
|
<div id="projectname">simdjson<span id="projectnumber"> 4.6.4</span>
|
|
</div>
|
|
<div id="projectbrief">Ridiculously Fast JSON</div>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<!-- end header part -->
|
|
<!-- Generated by Doxygen 1.9.8 -->
|
|
<script type="text/javascript">
|
|
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
|
|
var searchBox = new SearchBox("searchBox", "search/",'.html');
|
|
/* @license-end */
|
|
</script>
|
|
<script type="text/javascript" src="menudata.js"></script>
|
|
<script type="text/javascript" src="menu.js"></script>
|
|
<script type="text/javascript">
|
|
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
|
|
$(function() {
|
|
initMenu('',true,false,'search.php','Search');
|
|
$(document).ready(function() { init_search(); });
|
|
});
|
|
/* @license-end */
|
|
</script>
|
|
<div id="main-nav"></div>
|
|
</div><!-- top -->
|
|
<div id="side-nav" class="ui-resizable side-nav-resizable">
|
|
<div id="nav-tree">
|
|
<div id="nav-tree-contents">
|
|
<div id="nav-sync" class="sync"></div>
|
|
</div>
|
|
</div>
|
|
<div id="splitbar" style="-moz-user-select:none;"
|
|
class="ui-resizable-handle">
|
|
</div>
|
|
</div>
|
|
<script type="text/javascript">
|
|
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
|
|
$(document).ready(function(){initNavTree('md_doc_2compile__time.html',''); initResizable(); });
|
|
/* @license-end */
|
|
</script>
|
|
<div id="doc-content">
|
|
<!-- window showing the filter options -->
|
|
<div id="MSearchSelectWindow"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
|
</div>
|
|
|
|
<!-- iframe showing the search results (closed by default) -->
|
|
<div id="MSearchResultsWindow">
|
|
<div id="MSearchResults">
|
|
<div class="SRPage">
|
|
<div id="SRIndex">
|
|
<div id="SRResults"></div>
|
|
<div class="SRStatus" id="Loading">Loading...</div>
|
|
<div class="SRStatus" id="Searching">Searching...</div>
|
|
<div class="SRStatus" id="NoMatches">No Matches</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div><div class="header">
|
|
<div class="headertitle"><div class="title">Parse json at compile time</div></div>
|
|
</div><!--header-->
|
|
<div class="contents">
|
|
<div class="textblock"><p><a class="anchor" id="autotoc_md54"></a></p><ul>
|
|
<li>Introduction</li>
|
|
<li>Example</li>
|
|
<li>Concepts</li>
|
|
<li>Loading from disk</li>
|
|
<li>Limitations (compile-time errors)</li>
|
|
</ul>
|
|
<h1><a class="anchor" id="autotoc_md55"></a>
|
|
Introduction</h1>
|
|
<p>In some instances, you may want to configure your software at compile-time with a JSON document. Maybe you have a single code base but many different possible configurations, all resulting in different software. For example, you might be programming robots, using the same software, but different robot configurations.</p>
|
|
<p>To achieve the desired result, you have a few options. You may start the software and parser a JSON file at runtime. Or you might convert your JSON data into C++ code that you can compile with your software.</p>
|
|
<p>With C++26, there is another way: parse the JSON file along with your C++ code. In this manner, the JSON data becomes native C++ data.</p>
|
|
<p>The simdjson library supports parsing JSON documents at compile time if you have C++26 support. To activate C++26 reflection support, you can compile your code with the <code>SIMDJSON_STATIC_REFLECTION</code> macro set:</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#define SIMDJSON_STATIC_REFLECTION 1</span></div>
|
|
<div class="line"><span class="comment">//...</span></div>
|
|
<div class="line"><span class="preprocessor">#include "<a class="code" href="simdjson_8h.html">simdjson.h</a>"</span></div>
|
|
<div class="ttc" id="asimdjson_8h_html"><div class="ttname"><a href="simdjson_8h.html">simdjson.h</a></div></div>
|
|
</div><!-- fragment --><p>The <code>simdjson::compile_time::parse_json</code> function parses a JSON document at <b>compile time</b> and returns a <code>constexpr</code> structure reflecting its content. We support the full range of JSON values, which are mapped to C++ types as in the following table.</p>
|
|
<table class="markdownTable">
|
|
<tr class="markdownTableHead">
|
|
<th class="markdownTableHeadNone">JSON type </th><th class="markdownTableHeadNone">C++ type </th></tr>
|
|
<tr class="markdownTableRowOdd">
|
|
<td class="markdownTableBodyNone">object </td><td class="markdownTableBodyNone">anonymous struct </td></tr>
|
|
<tr class="markdownTableRowEven">
|
|
<td class="markdownTableBodyNone">array </td><td class="markdownTableBodyNone"><code>std::array<T, N></code> (homogeneous) </td></tr>
|
|
<tr class="markdownTableRowOdd">
|
|
<td class="markdownTableBodyNone">string </td><td class="markdownTableBodyNone"><code>const char*</code> (UTF-8) </td></tr>
|
|
<tr class="markdownTableRowEven">
|
|
<td class="markdownTableBodyNone">number </td><td class="markdownTableBodyNone"><code>int64_t</code>, <code>uint64_t</code>, <code>double</code> </td></tr>
|
|
<tr class="markdownTableRowOdd">
|
|
<td class="markdownTableBodyNone"><code>true</code>/<code>false</code> </td><td class="markdownTableBodyNone"><code>bool</code> </td></tr>
|
|
<tr class="markdownTableRowEven">
|
|
<td class="markdownTableBodyNone"><code>null</code> </td><td class="markdownTableBodyNone"><code>std::nullptr_t</code> </td></tr>
|
|
</table>
|
|
<h1><a class="anchor" id="autotoc_md56"></a>
|
|
Example</h1>
|
|
<p>Suppose you want to parse the following JSON document:</p>
|
|
<div class="fragment"><div class="line">{</div>
|
|
<div class="line"> <span class="stringliteral">"port"</span>: 8080,</div>
|
|
<div class="line"> <span class="stringliteral">"host"</span>: <span class="stringliteral">"localhost"</span>,</div>
|
|
<div class="line"> <span class="stringliteral">"debug"</span>: <span class="keyword">true</span></div>
|
|
<div class="line">}</div>
|
|
</div><!-- fragment --><p><b>Reminder</b>: In C++, <code>R"( )"</code> allows us to write multi-line strings with unescaped quotes.</p>
|
|
<p>You can do so, at compile-time, as follows:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">constexpr</span> <span class="keyword">auto</span> cfg = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">{</span></div>
|
|
<div class="line"><span class="stringliteral"> "port": 8080,</span></div>
|
|
<div class="line"><span class="stringliteral"> "host": "localhost",</span></div>
|
|
<div class="line"><span class="stringliteral"> "debug": true</span></div>
|
|
<div class="line"><span class="stringliteral">}</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">)"_json;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// cfg.port == 8080</span></div>
|
|
<div class="line"><span class="comment">// std::string_view(cfg.host) == "localhost"</span></div>
|
|
<div class="line"><span class="comment">// cfg.debug == true</span></div>
|
|
</div><!-- fragment --><p>You can nest objects and arrays:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">constexpr</span> <span class="keyword">auto</span> data = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">{</span></div>
|
|
<div class="line"><span class="stringliteral"> "servers": [</span></div>
|
|
<div class="line"><span class="stringliteral"> {"host": "s1", "port": 3000},</span></div>
|
|
<div class="line"><span class="stringliteral"> {"host": "s2", "port": 3001}</span></div>
|
|
<div class="line"><span class="stringliteral"> ]</span></div>
|
|
<div class="line"><span class="stringliteral">}</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">)"_json;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// data.servers.size() == 2</span></div>
|
|
<div class="line"><span class="comment">// std::string_view(data.servers[0].host) == "s1"</span></div>
|
|
</div><!-- fragment --><p>Top-level arrays are allowed:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">constexpr</span> <span class="keyword">auto</span> arr = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">[1, 2, 3]</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">)"_json;</span></div>
|
|
<div class="line"><span class="stringliteral"></span><span class="keyword">static_assert</span>(arr.size() == 3);</div>
|
|
<div class="line"><span class="keyword">static_assert</span>(arr[1] == 2);</div>
|
|
</div><!-- fragment --><h1><a class="anchor" id="autotoc_md57"></a>
|
|
Concepts</h1>
|
|
<p>Given that the parsed data is made of structures that depend on the JSON input, you might want to check that it conforms to your expectation. You can do so with concepts.</p>
|
|
<p>Let us consider this example:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">constexpr</span> <span class="keyword">auto</span> config = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">[</span></div>
|
|
<div class="line"><span class="stringliteral"> { "name": "Alice", "age": 30 },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "name": "Bob", "age": 25 },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "name": "Charlie", "age": 35 }</span></div>
|
|
<div class="line"><span class="stringliteral">]</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">)"_json;</span></div>
|
|
</div><!-- fragment --><p>You might want to ensure that the result is an array of persons. You can define your expectation with concepts like so:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">template</span> <<span class="keyword">typename</span> T></div>
|
|
<div class="line"><span class="keyword">concept </span>person = <span class="keyword">requires</span>(T p) {</div>
|
|
<div class="line"> std::string_view(p.name); <span class="comment">// has name field convertible to string_view</span></div>
|
|
<div class="line"> p.age; <span class="comment">// has age field</span></div>
|
|
<div class="line"> <span class="keyword">requires</span> std::is_integral_v<<span class="keyword">decltype</span>(p.age)>; <span class="comment">// age is integral</span></div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">template</span> <<span class="keyword">typename</span> T></div>
|
|
<div class="line"><span class="keyword">concept </span>array_of_person = <span class="keyword">requires</span>(T arr) {</div>
|
|
<div class="line"> arr.size(); <span class="comment">// has size method</span></div>
|
|
<div class="line"> arr[0]; <span class="comment">// can access elements with []</span></div>
|
|
<div class="line"> <span class="keyword">requires</span> person<<span class="keyword">decltype</span>(arr[0])>; <span class="comment">// elements satisfy person concept</span></div>
|
|
<div class="line">};</div>
|
|
</div><!-- fragment --><p>And then a simple static assert with <code>decltype</code> is sufficient to check that the expectation is met:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">constexpr</span> <span class="keyword">auto</span> config = R<span class="stringliteral">"(</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">[</span></div>
|
|
<div class="line"><span class="stringliteral"> { "name": "Alice", "age": 30 },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "name": "Bob", "age": 25 },</span></div>
|
|
<div class="line"><span class="stringliteral"> { "name": "Charlie", "age": 35 }</span></div>
|
|
<div class="line"><span class="stringliteral">]</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral">)"_json;</span></div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span> </div>
|
|
<div class="line"><span class="stringliteral"></span><span class="comment">// Validate that the array satisfies the array_of_person concept</span></div>
|
|
<div class="line"><span class="keyword">static_assert</span>(array_of_person<<span class="keyword">decltype</span>(config)>);</div>
|
|
</div><!-- fragment --><h1><a class="anchor" id="autotoc_md58"></a>
|
|
Loading from disk</h1>
|
|
<p>In practice, you may have a JSON file, say <code>json_data</code> that you want to parse at compile time. You may do so as follows.</p>
|
|
<div class="fragment"><div class="line"> ++</div>
|
|
<div class="line"><span class="keyword">constexpr</span> <span class="keyword">const</span> <span class="keywordtype">char</span> json_data[] = {</div>
|
|
<div class="line"><span class="preprocessor"> #embed "test.json"</span></div>
|
|
<div class="line"> , 0</div>
|
|
<div class="line">};</div>
|
|
<div class="line"> </div>
|
|
<div class="line"><span class="keyword">constexpr</span> <span class="keyword">auto</span> json = simdjson::compile_time::parse_json<json_data>();</div>
|
|
</div><!-- fragment --><h1><a class="anchor" id="autotoc_md59"></a>
|
|
Limitations (compile-time errors)</h1>
|
|
<p>We have a few limitations which trigger compile-time errors if violated.</p>
|
|
<ul>
|
|
<li>Only JSON objects and arrays are supported at the top level (no primitives). We will lift this limitation in the future.</li>
|
|
<li>Strings are represented using the <code>const char*</code> in UTF-8, but they must not contain embedded nulls. We would prefer to represent them as std::string or std::string_view, and hope to do so in the future.</li>
|
|
<li>Heterogeneous arrays are not supported yet. E.g., you need to have arrays of all integers, or all strings, all floats, all compatible objects, etc. For example, the following is accepted: <div class="fragment"><div class="line"> [</div>
|
|
<div class="line"> { "name": "Alice", "age": 30 },</div>
|
|
<div class="line"> { "name": "Bob", "age": 25 },</div>
|
|
<div class="line"> { "name": "Charlie", "age": 35 }</div>
|
|
<div class="line">]</div>
|
|
</div><!-- fragment --> but the following is not: <div class="fragment"><div class="line">[</div>
|
|
<div class="line"> { "name": "Alice", "age": 30 },</div>
|
|
<div class="line"> "Just a string",</div>
|
|
<div class="line"> 42,</div>
|
|
<div class="line"> { "name": "Charlie", "age": 35 }</div>
|
|
<div class="line">]</div>
|
|
</div><!-- fragment --> We may support heterogeneous arrays in the future with std::variant types.</li>
|
|
<li>We parse the first JSON document encountered in the string. Trailing characters are ignored. Thus if your JSON begins with {"a":1}, everything after the closing brace is ignored. This limitation will be lifted in the future, reporting an error.</li>
|
|
</ul>
|
|
<p>These limitations are safe in the sense that they result in compile-time errors. Thus you will not get truncated strings or imprecise floats silently.</p>
|
|
<p>Although we are committed to maintaining the functionality in the long run, the <code>compile_time::parse_json</code> function is subject to change. </p>
|
|
</div></div><!-- contents -->
|
|
</div><!-- PageDoc -->
|
|
</div><!-- doc-content -->
|
|
<!-- start footer part -->
|
|
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
|
|
<ul>
|
|
<li class="footer">Generated by <a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.8 </li>
|
|
</ul>
|
|
</div>
|
|
</body>
|
|
</html>
|