mirror of
https://github.com/simdjson/simdjson
synced 2026-06-08 17:27:07 +00:00
c806e955c4
* Initial work on JSON builder * moving the files back to ondemand for now. * tweak * more later * update * minor edits * dropping vs arm (missing support) * adding tests. we still specialized write_string_escaped * tweaking * fix typo * tweaking the approach * minor fix * missing store * another missing store * Attempt at fixing failing serialization tests. (#2292) * Fixing appeand_float typo (#2294) * applying a couple of fixes * updating single header * fix for pre C++17 if constexpr * Fixing unused argument problem and updating the singleheader file * various pedantic fixes * Sketch of builder * reordering. * simplify * Adding draft of static reflection based deserialization * Updating simdjson singleheader * patching the automated deserialization. * automated * Adding support for smart pointers of user defined types. * Adding specialization for smart pointers for basic types. I think it is highly likely that this can be done in a more generic way. * Referncing a later version of rapidjson that fixed the issue related with assignment attempt of a const variable for GenericStringRef class. * guarding the tests * adding documentation for string_builder * saving * rename to 'append' * saving * non-functional benchmarks (#2342) * non-functional benchmarks * Fix typo * various fixes * tweaking --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com> * tuning * various minor fixes * minor tweak * minor simplification * updating amal * adding a cast * update * fancy casting * removing dead code * Pushing latest changes. CITM benchmark is still not working. * Still not working, but now I am getting only 10 errors. * add static reflection benchmark to 'large random' benchmark and allows (#2349) deserialization (with static reflection) from objects and arrays. Co-authored-by: Daniel Lemire <dlemire@lemire.me> * Removing std::map from CitmCatalog definition, since that is not currently supported. * Added free to rust bench, segfault is still happening.. * The syntax changed: ^E became ^^E. (#2350) * The syntax changed: ^E became ^^E. * guarding --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> * Adding support for string_view_keyed_map types. * Adding concepts as a conditional include. * updating single-header * Adding concepts to ondemand deps * rust benchmark is finally working * Fixing small typo in docs. * adding docker config and instructions so that our users can test the static reflection (#2358) * adding docker config and instructions so that our users can test the static reflection * completing the instructions * pruning white spaces --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> * minor optimizations on the JSON builder branch * avoiding undef behaviour * saving * somewhat nicer builder * make it possible to run just one benchmark * adding linux perf * fixing minor issue * updating swar * Adding real world compilation benchmark (#2379) * Adding compilation benchmark for json parsing with and without reflection * Moving it to the benchmark folder, also reducing a bit the number of iterations. * Removing script from root folder. * Reducing number of iterations * Update benchmark/benchmark_reflection_usage_compilation.sh Co-authored-by: Daniel Lemire <daniel@lemire.me> * Update benchmark/benchmark_reflection_usage_compilation.sh Co-authored-by: Daniel Lemire <daniel@lemire.me> * Update benchmark/benchmark_reflection_usage_compilation.sh Co-authored-by: Daniel Lemire <daniel@lemire.me> * Making the script more customizable and also test whether the compiler being used supports reflection before actually running the benchmark --------- Co-authored-by: Daniel Lemire <daniel@lemire.me> * Using define_static_string from https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3491r2.html (#2389) * Applying changes needed after latest reflection paper updates. * Working, but no template for yet. * Updating single-header to incldue the use of define_static_string. * copying over master --------- Co-authored-by: Daniel Lemire <dlemire@lemire.me> Co-authored-by: Francisco Geiman Thiesen <franciscogthiesen@gmail.com>
215 lines
6.2 KiB
C++
215 lines
6.2 KiB
C++
/**
|
|
* This file is part of the builder API. It is temporarily in the ondemand directory
|
|
* but we will move it to a builder directory later.
|
|
*/
|
|
#ifndef SIMDJSON_GENERIC_STRING_BUILDER_H
|
|
|
|
#ifndef SIMDJSON_CONDITIONAL_INCLUDE
|
|
#define SIMDJSON_GENERIC_STRING_BUILDER_H
|
|
#include "simdjson/generic/implementation_simdjson_result_base.h"
|
|
#endif // SIMDJSON_CONDITIONAL_INCLUDE
|
|
|
|
namespace simdjson {
|
|
namespace SIMDJSON_IMPLEMENTATION {
|
|
namespace builder {
|
|
|
|
/**
|
|
* A builder for JSON strings representing documents. This is a low-level
|
|
* builder that is not meant to be used directly by end-users. Though it
|
|
* supports atomic types (Booleans, strings), it does not support composed
|
|
* types (arrays and objects).
|
|
*
|
|
* Ultimately, this class should support kernel-specific optimizations. E.g.,
|
|
* it may make use of SIMD instructions to escape strings faster.
|
|
*/
|
|
class string_builder {
|
|
public:
|
|
simdjson_inline string_builder(size_t initial_capacity = 1024);
|
|
|
|
/**
|
|
* Append number (includes Booleans). Booleans are mapped to the strings
|
|
* false and true. Numbers are converted to strings abiding by the JSON standard.
|
|
* Floating-point numbers are converted to the shortest string that 'correctly'
|
|
* represents the number.
|
|
*/
|
|
template<typename number_type,
|
|
typename = typename std::enable_if<std::is_arithmetic<number_type>::value>::type>
|
|
simdjson_inline void append(number_type v) noexcept;
|
|
|
|
/**
|
|
* Append character c.
|
|
*/
|
|
simdjson_inline void append(char c) noexcept;
|
|
|
|
/**
|
|
* Append the string 'null'.
|
|
*/
|
|
simdjson_inline void append_null() noexcept;
|
|
|
|
/**
|
|
* Clear the content.
|
|
*/
|
|
simdjson_inline void clear() noexcept;
|
|
|
|
/**
|
|
* Append the std::string_view, after escaping it.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void escape_and_append(std::string_view input) noexcept;
|
|
|
|
/**
|
|
* Append the std::string_view surrounded by double quotes, after escaping it.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void escape_and_append_with_quotes(std::string_view input) noexcept;
|
|
|
|
/**
|
|
* Append the character surrounded by double quotes, after escaping it.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void escape_and_append_with_quotes(char input) noexcept;
|
|
|
|
/**
|
|
* Append the character surrounded by double quotes, after escaping it.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void escape_and_append_with_quotes(const char* input) noexcept;
|
|
|
|
/**
|
|
* Append the C string directly, without escaping.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void append_raw(const char *c) noexcept;
|
|
|
|
/**
|
|
* Append "{" to the buffer.
|
|
*/
|
|
simdjson_inline void start_object() noexcept;
|
|
|
|
/**
|
|
* Append "}" to the buffer.
|
|
*/
|
|
simdjson_inline void end_object() noexcept;
|
|
|
|
/**
|
|
* Append "[" to the buffer.
|
|
*/
|
|
simdjson_inline void start_array() noexcept;
|
|
|
|
/**
|
|
* Append "]" to the buffer.
|
|
*/
|
|
simdjson_inline void end_array() noexcept;
|
|
|
|
/**
|
|
* Append "," to the buffer.
|
|
*/
|
|
simdjson_inline void append_comma() noexcept;
|
|
|
|
/**
|
|
* Append ":" to the buffer.
|
|
*/
|
|
simdjson_inline void append_colon() noexcept;
|
|
|
|
/**
|
|
* Append a key-value pair to the buffer.
|
|
* The key is escaped and surrounded by double quotes.
|
|
* The value is escaped if it is a string.
|
|
*/
|
|
template<typename key_type, typename value_type>
|
|
simdjson_inline void append_key_value(key_type key, value_type value) noexcept;
|
|
/**
|
|
* Append the std::string_view directly, without escaping.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void append_raw(std::string_view input) noexcept;
|
|
|
|
/**
|
|
* Append len characters from str.
|
|
* There is no UTF-8 validation.
|
|
*/
|
|
simdjson_inline void append_raw(const char *str, size_t len) noexcept;
|
|
#if SIMDJSON_EXCEPTIONS
|
|
/**
|
|
* Creates an std::string from the written JSON buffer.
|
|
* Throws if memory allocation failed
|
|
*
|
|
* The result may not be valid UTF-8 if some of your content was not valid UTF-8.
|
|
* Use validate_unicode() to check the content if needed.
|
|
*/
|
|
simdjson_inline operator std::string() const noexcept(false);
|
|
|
|
/**
|
|
* Creates an std::string_view from the written JSON buffer.
|
|
* Throws if memory allocation failed.
|
|
*
|
|
* The result may not be valid UTF-8 if some of your content was not valid UTF-8.
|
|
* Use validate_unicode() to check the content if needed.
|
|
*/
|
|
simdjson_inline operator std::string_view() const noexcept(false);
|
|
#endif
|
|
|
|
/**
|
|
* Returns a view on the written JSON buffer. Returns an error
|
|
* if memory allocation failed.
|
|
*
|
|
* The result may not be valid UTF-8 if some of your content was not valid UTF-8.
|
|
* Use validate_unicode() to check the content.
|
|
*/
|
|
simdjson_inline simdjson_result<std::string_view> view() const noexcept;
|
|
|
|
/**
|
|
* Appends the null character to the buffer and returns
|
|
* a pointer to the beginning of the written JSON buffer.
|
|
* Returns an error if memory allocation failed.
|
|
* The result is null-terminated.
|
|
*
|
|
* The result may not be valid UTF-8 if some of your content was not valid UTF-8.
|
|
* Use validate_unicode() to check the content.
|
|
*/
|
|
simdjson_inline simdjson_result<const char *> c_str() noexcept;
|
|
|
|
/**
|
|
* Return true if the content is valid UTF-8.
|
|
*/
|
|
simdjson_inline bool validate_unicode() const noexcept;
|
|
|
|
/**
|
|
* Returns the current size of the written JSON buffer.
|
|
* If an error occurred, returns 0.
|
|
*/
|
|
simdjson_inline size_t size() const noexcept;
|
|
|
|
private:
|
|
/**
|
|
* Returns true if we can write at least upcoming_bytes bytes.
|
|
* The underlying buffer is reallocated if needed. It is designed
|
|
* to be called before writing to the buffer. It should be fast.
|
|
*/
|
|
simdjson_inline bool capacity_check(size_t upcoming_bytes);
|
|
|
|
/**
|
|
* Grow the buffer to at least desired_capacity bytes.
|
|
* If the allocation fails, is_valid is set to false. We expect
|
|
* that this function would not be repeatedly called.
|
|
*/
|
|
simdjson_inline void grow_buffer(size_t desired_capacity);
|
|
|
|
/**
|
|
* We use this helper function to make sure that is_valid is kept consistent.
|
|
*/
|
|
simdjson_inline void set_valid(bool valid) noexcept;
|
|
|
|
std::unique_ptr<char[]> buffer{};
|
|
size_t position{0};
|
|
size_t capacity{0};
|
|
bool is_valid{true};
|
|
};
|
|
|
|
|
|
|
|
}
|
|
}
|
|
} // namespace simdjson
|
|
|
|
#endif // SIMDJSON_GENERIC_STRING_BUILDER_H
|