mirror of
https://github.com/simdjson/simdjson
synced 2026-06-08 17:27:07 +00:00
c806e955c4
* Initial work on JSON builder * moving the files back to ondemand for now. * tweak * more later * update * minor edits * dropping vs arm (missing support) * adding tests. we still specialized write_string_escaped * tweaking * fix typo * tweaking the approach * minor fix * missing store * another missing store * Attempt at fixing failing serialization tests. (#2292) * Fixing appeand_float typo (#2294) * applying a couple of fixes * updating single header * fix for pre C++17 if constexpr * Fixing unused argument problem and updating the singleheader file * various pedantic fixes * Sketch of builder * reordering. * simplify * Adding draft of static reflection based deserialization * Updating simdjson singleheader * patching the automated deserialization. * automated * Adding support for smart pointers of user defined types. * Adding specialization for smart pointers for basic types. I think it is highly likely that this can be done in a more generic way. * Referncing a later version of rapidjson that fixed the issue related with assignment attempt of a const variable for GenericStringRef class. * guarding the tests * adding documentation for string_builder * saving * rename to 'append' * saving * non-functional benchmarks (#2342) * non-functional benchmarks * Fix typo * various fixes * tweaking --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com> * tuning * various minor fixes * minor tweak * minor simplification * updating amal * adding a cast * update * fancy casting * removing dead code * Pushing latest changes. CITM benchmark is still not working. * Still not working, but now I am getting only 10 errors. * add static reflection benchmark to 'large random' benchmark and allows (#2349) deserialization (with static reflection) from objects and arrays. Co-authored-by: Daniel Lemire <dlemire@lemire.me> * Removing std::map from CitmCatalog definition, since that is not currently supported. * Added free to rust bench, segfault is still happening.. * The syntax changed: ^E became ^^E. (#2350) * The syntax changed: ^E became ^^E. * guarding --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> * Adding support for string_view_keyed_map types. * Adding concepts as a conditional include. * updating single-header * Adding concepts to ondemand deps * rust benchmark is finally working * Fixing small typo in docs. * adding docker config and instructions so that our users can test the static reflection (#2358) * adding docker config and instructions so that our users can test the static reflection * completing the instructions * pruning white spaces --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> * minor optimizations on the JSON builder branch * avoiding undef behaviour * saving * somewhat nicer builder * make it possible to run just one benchmark * adding linux perf * fixing minor issue * updating swar * Adding real world compilation benchmark (#2379) * Adding compilation benchmark for json parsing with and without reflection * Moving it to the benchmark folder, also reducing a bit the number of iterations. * Removing script from root folder. * Reducing number of iterations * Update benchmark/benchmark_reflection_usage_compilation.sh Co-authored-by: Daniel Lemire <daniel@lemire.me> * Update benchmark/benchmark_reflection_usage_compilation.sh Co-authored-by: Daniel Lemire <daniel@lemire.me> * Update benchmark/benchmark_reflection_usage_compilation.sh Co-authored-by: Daniel Lemire <daniel@lemire.me> * Making the script more customizable and also test whether the compiler being used supports reflection before actually running the benchmark --------- Co-authored-by: Daniel Lemire <daniel@lemire.me> * Using define_static_string from https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3491r2.html (#2389) * Applying changes needed after latest reflection paper updates. * Working, but no template for yet. * Updating single-header to incldue the use of define_static_string. * copying over master --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>
132 lines
5.5 KiB
Markdown
132 lines
5.5 KiB
Markdown
Builder
|
|
==========
|
|
|
|
Sometimes you want to generate JSON string outputs efficiently.
|
|
The simdjson library provides high-performance low-level facilities.
|
|
When using these low-level functionalities, you are responsible to
|
|
define the structure of your JSON document. However, string escaping
|
|
and UTF-8 validation is automated.
|
|
|
|
Overview: string_builder
|
|
---------------------------
|
|
|
|
The string_builder class is a low-level utility for constructing JSON strings representing documents. It is optimized for performance, potentially leveraging kernel-specific features like SIMD instructions for tasks such as string escaping. This class supports atomic types (e.g., booleans, numbers, strings) but does not handle composed types directly (like arrays or objects).
|
|
|
|
An `string_builder` is created with an initial buffer capacity (e.g., 1kB). The memory
|
|
is reallocated when needed. It has the following methods to add content to the string:
|
|
|
|
|
|
- `append(number_type v)`: Appends a number (including booleans) to the JSON buffer. Booleans are converted to the strings "false" or "true". Numbers are formatted according to the JSON standard, with floating-point numbers using the shortest representation that accurately reflects the value.
|
|
- `append(char c)`: Appends a single character to the JSON buffer.
|
|
- `append_null()`: Appends the string "null" to the JSON buffer.
|
|
- `clear()`: Clears the contents of the JSON buffer, resetting the position to 0 while retaining the allocated capacity.
|
|
- `escape_and_append(std::string_view input)`: Appends a string view to the JSON buffer after escaping special characters (e.g., quotes, backslashes) as required by JSON.
|
|
- `escape_and_append_with_quotes(std::string_view input)` Appends a string view surrounded by double quotes (e.g., "input") to the JSON buffer after escaping special characters.
|
|
Parameters:
|
|
- `escape_and_append_with_quotes(char input)`: Appends a single character surrounded by double quotes (e.g., "c") to the JSON buffer after escaping it if necessary.
|
|
- `append_raw(const char *c)`: Appends a null-terminated C string directly to the JSON buffer without escaping.
|
|
- `append_raw(std::string_view input)`: Appends a string view directly to the JSON buffer without escaping.
|
|
- `append_raw(const char *str, size_t len)`: Appends a specified number of characters from a C string directly to the JSON
|
|
|
|
After writting the content, if you have reasons to believe that the content might violate UTF-8 conventions, you can check it as follows:
|
|
|
|
- `validate_unicode()`: Checks if the content in the JSON buffer is valid UTF-8. Returns: true if the content is valid UTF-8, false otherwise.
|
|
|
|
Once you are satisfied, you can recover the string as follows:
|
|
|
|
- `operator std::string()`: Converts the JSON buffer to an std::string. (Might throw if an error occurred.)
|
|
- `operator std::string_view()`: Converts the JSON buffer to an std::string_view. (Might throw if an error occurred.)
|
|
- `view()`: Returns a view of the written JSON buffer as a `simdjson_result<std::string_view>`.
|
|
|
|
The later method (`view()`) is recommended.
|
|
|
|
Example: string_builder
|
|
---------------------------
|
|
|
|
```C++
|
|
|
|
void serialize_car(const Car& car, simdjson::builder::string_builder& builder) {
|
|
// start of JSON
|
|
builder.start_object();
|
|
|
|
// "make"
|
|
builder.append_key_value("make", car.make);
|
|
builder.append_comma();
|
|
|
|
// "model"
|
|
builder.append_key_value("model", car.model);
|
|
builder.append_comma();
|
|
|
|
// "year"
|
|
builder.append_key_value("year", car.year);
|
|
builder.append_comma();
|
|
|
|
// "tire_pressure"
|
|
builder.escape_and_append_with_quotes("tire_pressure");
|
|
builder.append_colon();
|
|
builder.start_array();
|
|
// vector tire_pressure
|
|
for (size_t i = 0; i < car.tire_pressure.size(); ++i) {
|
|
builder.append(car.tire_pressure[i]);
|
|
if (i < car.tire_pressure.size() - 1) {
|
|
builder.append_comma();
|
|
}
|
|
}
|
|
builder.end_array();
|
|
builder.end_object();
|
|
}
|
|
|
|
bool car_test() {
|
|
simdjson::builder::string_builder sb;
|
|
Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
|
|
serialize_car(c, sb);
|
|
std::string_view p;
|
|
if(sb.view().get(p)) {
|
|
return false; // there was an error
|
|
}
|
|
// p holds the JSON:
|
|
// "{\"make\":\"Toyota\",\"model\":\"Corolla\",\"year\":2017,\"tire_pressure\":[30.0,30.2,30.513,30.79]}"
|
|
return true;
|
|
}
|
|
```
|
|
|
|
C++26 static reflection
|
|
------------------------
|
|
|
|
If you have a compiler with support C++26 static reflection, you can compile
|
|
your code with the `SIMDJSON_STATIC_REFLECTION` macro set:
|
|
|
|
```cpp
|
|
#define SIMDJSON_STATIC_REFLECTION 1
|
|
//...
|
|
#include "simdjson.h"
|
|
```
|
|
|
|
And then you can append your data structures to a `string_builder` instance
|
|
automatically. In most cases, it should work automatically:
|
|
|
|
```cpp
|
|
bool car_test() {
|
|
simdjson::builder::string_builder sb;
|
|
Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
|
|
append(sb, c);
|
|
std::string_view p;
|
|
if(sb.view().get(p)) {
|
|
return false; // there was an error
|
|
}
|
|
// p holds the JSON:
|
|
// "{\"make\":\"Toyota\",\"model\":\"Corolla\",\"year\":2017,\"tire_pressure\":[30.0,30.2,30.513,30.79]}"
|
|
return true;
|
|
}
|
|
```
|
|
|
|
If you prefer, you can also create a string directly:
|
|
|
|
```cpp
|
|
std::string json;
|
|
if(simdjson::builder::to_json_string(c).get(json)) {
|
|
// there was an error
|
|
} else {
|
|
// json contain the serialized JSON
|
|
}
|
|
``` |