Compare commits

..

9 Commits

Author SHA1 Message Date
Daniel Lemire b1c31b428d update 2025-11-11 14:21:04 -05:00
Daniel Lemire 56ac56ba32 Merge branch 'master' of github.com:simdjson/simdjson 2025-11-11 14:17:08 -05:00
Muhammad Rizal Nurromdhoni 19549c60ec string_builder range-based append fix (#2544)
* Use std::ranges::range_value_t on range

* Add ranges test
2025-11-11 14:15:00 -05:00
Daniel Lemire 16e99f229b Update iterate_many.md for clarity on JSON processing
Clarified the example JSON format and emphasized the need for efficient processing.
2025-11-10 13:49:13 -05:00
Liqiang TAO 21342a4142 Fix some wrong content in doc (#2542) 2025-11-10 11:24:38 -05:00
Daniel Lemire d0e841d3e9 Release Candidate 4.2.2 (#2539)
* adding documentation.

* release candidate
2025-11-06 12:00:21 -05:00
Daniel Lemire a962652ec3 adding documentation. 2025-11-05 11:42:38 -05:00
hiteshmk05 77d73b068a add: windows wstring support for padded_str (#2537)
* add: windows wstring support for padded_str

* add: padded_string::load for wstring windows

* fix: extra space

* change: file path
2025-11-05 11:29:30 -05:00
Daniel Lemire 19ff7a572d adding concept examples to the compile-time JSON. (#2538) 2025-11-04 15:06:40 -05:00
19 changed files with 436 additions and 34 deletions
+1 -1
View File
@@ -3,7 +3,7 @@ cmake_minimum_required(VERSION 3.14)
project( project(
simdjson simdjson
# The version number is modified by tools/release.py # The version number is modified by tools/release.py
VERSION 4.2.1 VERSION 4.2.2
DESCRIPTION "Parsing gigabytes of JSON per second" DESCRIPTION "Parsing gigabytes of JSON per second"
HOMEPAGE_URL "https://simdjson.org/" HOMEPAGE_URL "https://simdjson.org/"
LANGUAGES CXX C LANGUAGES CXX C
+1 -1
View File
@@ -38,7 +38,7 @@ PROJECT_NAME = simdjson
# could be handy for archiving the generated documentation or if some version # could be handy for archiving the generated documentation or if some version
# control system is used. # control system is used.
PROJECT_NUMBER = "4.2.1" PROJECT_NUMBER = "4.2.2"
# Using the PROJECT_BRIEF tag one can provide an optional one line description # Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a # for a project that appears at the top of each page and should give viewer a
+7 -4
View File
@@ -183,6 +183,9 @@ auto json = padded_string::load("twitter.json"); // load JSON file 'twitter.json
ondemand::document doc = parser.iterate(json); // position a pointer at the beginning of the JSON data ondemand::document doc = parser.iterate(json); // position a pointer at the beginning of the JSON data
``` ```
(Windows users compiling with C++17 or better may use `wchar_t` strings to support non-ASCII
filenames: `padded_string::load(L"twitter.json")`.)
If you prefer not to create your own `ondemand::parser` instance, you can access If you prefer not to create your own `ondemand::parser` instance, you can access
a thread-local version by calling `ondemand::parser.get_parser()`. a thread-local version by calling `ondemand::parser.get_parser()`.
@@ -449,7 +452,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
When you are iterating through an object, you are advancing through its keys and values. You should not also access the object or other objects. E.g. within a loop over `myobject`, you should not be accessing `myobject`. The following is an anti-pattern: `for(auto value: myobject) {myobject["mykey"]}`. When you are iterating through an object, you are advancing through its keys and values. You should not also access the object or other objects. E.g. within a loop over `myobject`, you should not be accessing `myobject`. The following is an anti-pattern: `for(auto value: myobject) {myobject["mykey"]}`.
You should never reset an object as you are iterating through it. The following is an anti-pattern: `for(auto value: myobject) {myobject.reset()}`. You should never reset an object as you are iterating through it. The following is an anti-pattern: `for(auto value: myobject) {myobject.reset()}`.
* **Array Index:** Because it is forward-only, you cannot look up an array element by index by index. Instead, * **Array Index:** Because it is forward-only, you cannot look up an array element by index. Instead,
you should iterate through the array and keep an index yourself. Exceptionally, if need a single value you should iterate through the array and keep an index yourself. Exceptionally, if need a single value
out of the array, you may use an array access (e.g., `array[1]`). You should never reset an array as you are iterating through it. The following is an anti-pattern: `for(auto value: myarray) {myarray.reset()}`. out of the array, you may use an array access (e.g., `array[1]`). You should never reset an array as you are iterating through it. The following is an anti-pattern: `for(auto value: myarray) {myarray.reset()}`.
* **Field Access:** To get the value of the "foo" field in an object, use `object["foo"]`. This will * **Field Access:** To get the value of the "foo" field in an object, use `object["foo"]`. This will
@@ -524,13 +527,13 @@ support for users who avoid exceptions. See [the simdjson error handling documen
> auto silly_json = R"( { "test": "result" } )"_padded; > auto silly_json = R"( { "test": "result" } )"_padded;
> ondemand::document doc = parser.iterate(silly_json); > ondemand::document doc = parser.iterate(silly_json);
> std::cout << simdjson::to_json_string(doc["test"]) << std::endl; // Requires simdjson 1.0 or better > std::cout << simdjson::to_json_string(doc["test"]) << std::endl; // Requires simdjson 1.0 or better
>```` > ```
> ```cpp > ```cpp
> // retrieves an unescaped string value as a string_view instance > // retrieves an unescaped string value as a string_view instance
> auto silly_json = R"( { "test": "result" } )"_padded; > auto silly_json = R"( { "test": "result" } )"_padded;
> ondemand::document doc = parser.iterate(silly_json); > ondemand::document doc = parser.iterate(silly_json);
> std::cout << std::string_view(doc["test"]) << std::endl; > std::cout << std::string_view(doc["test"]) << std::endl;
>```` > ```
You can use `to_json_string` to efficiently extract components of a JSON document to reconstruct a new JSON document, as in the following example: You can use `to_json_string` to efficiently extract components of a JSON document to reconstruct a new JSON document, as in the following example:
> ```cpp > ```cpp
> auto cars_json = R"( [ > auto cars_json = R"( [
@@ -559,7 +562,7 @@ support for users who avoid exceptions. See [the simdjson error handling documen
> oss << "]"; > oss << "]";
> auto json_string = oss.str(); > auto json_string = oss.str();
> // json_string == "[[ 40.1, 39.9, 37.7, 40.4 ],[ 30.1, 31.0, 28.6, 28.7 ]]" > // json_string == "[[ 40.1, 39.9, 37.7, 40.4 ],[ 30.1, 31.0, 28.6, 28.7 ]]"
>```` > ```
* **Extracting Values (without exceptions):** You can use a variant usage of `get()` with error * **Extracting Values (without exceptions):** You can use a variant usage of `get()` with error
codes to avoid exceptions. You first declare the variable of the appropriate type (`double`, codes to avoid exceptions. You first declare the variable of the appropriate type (`double`,
`uint64_t`, `int64_t`, `bool`, `ondemand::object` and `ondemand::array`) and pass it by reference `uint64_t`, `int64_t`, `bool`, `ondemand::object` and `ondemand::array`) and pass it by reference
+63
View File
@@ -1,6 +1,7 @@
# Parse json at compile time # Parse json at compile time
* [Introduction](#introduction) * [Introduction](#introduction)
* [Example](#example) * [Example](#example)
* [Concepts](#concepts)
* [Loading from disk](#loading-from-disk) * [Loading from disk](#loading-from-disk)
* [Limitations (compile-time errors)](#limitations-compile-time-errors) * [Limitations (compile-time errors)](#limitations-compile-time-errors)
@@ -106,6 +107,68 @@ static_assert(arr.size() == 3);
static_assert(arr[1] == 2); static_assert(arr[1] == 2);
``` ```
## Concepts
Given that the parsed data is made of structures that depend on the JSON input, you might
want to check that it conforms to your expectation. You can do so with concepts.
Let us consider this example:
```cpp
constexpr auto config = R"(
[
{ "name": "Alice", "age": 30 },
{ "name": "Bob", "age": 25 },
{ "name": "Charlie", "age": 35 }
]
)"_json;
```
You might want to ensure that the result is an array of persons. You can define your
expection with concepts like so:
```cpp
template <typename T>
concept person = requires(T p) {
std::string_view(p.name); // has name field convertible to string_view
p.age; // has age field
requires std::is_integral_v<decltype(p.age)>; // age is integral
};
/**
* Concept to validate that a type is an array of person objects
*/
template <typename T>
concept array_of_person = requires(T arr) {
arr.size(); // has size method
arr[0]; // can access elements with []
requires person<decltype(arr[0])>; // elements satisfy person concept
};
```
And then a simple static assert with `decltype` is sufficient to check that the expectation is met:
```cpp
constexpr auto config = R"(
[
{ "name": "Alice", "age": 30 },
{ "name": "Bob", "age": 25 },
{ "name": "Charlie", "age": 35 }
]
)"_json;
// Validate that the array satisfies the array_of_person concept
static_assert(array_of_person<decltype(config)>);
```
## Loading from disk ## Loading from disk
In practice, you may have a JSON file, say `json_data` that you want to parse In practice, you may have a JSON file, say `json_data` that you want to parse
+17 -1
View File
@@ -54,6 +54,22 @@ dom::parser parser;
dom::element doc = parser.parse("[1,2,3]"_padded); // parse a string, the _padded suffix creates a simdjson::padded_string instance dom::element doc = parser.parse("[1,2,3]"_padded); // parse a string, the _padded suffix creates a simdjson::padded_string instance
``` ```
You can also load a `padded_string` from a file.
```cpp
auto json = padded_string::load("twitter.json"); // load JSON file 'twitter.json'.
dom::element doc = parser.parse(json);
```
(Windows users compiling with C++17 or better may use `wchar_t` strings to support non-ASCII
filenames: `padded_string::load(L"twitter.json")`.)
(Windows users compiling with C++17 or better may use `wchar_t` strings to support non-ASCII
filenames: `padded_string::load(L"twitter.json")`.)
You can copy your data directly on a `simdjson::padded_string` as follows: You can copy your data directly on a `simdjson::padded_string` as follows:
```cpp ```cpp
@@ -827,7 +843,7 @@ memcpy(padded_json_copy.get(), json, json_len);
memset(padded_json_copy.get() + json_len, 0, SIMDJSON_PADDING); memset(padded_json_copy.get() + json_len, 0, SIMDJSON_PADDING);
simdjson::dom::parser parser; simdjson::dom::parser parser;
simdjson::dom::element element = parser.parse(padded_json_copy.get(), json_len, false); simdjson::dom::element element = parser.parse(padded_json_copy.get(), json_len, false);
```` ```
Setting the `realloc_if_needed` parameter `false` in this manner may lead to better performance since copies are avoided, but it requires that the user takes more responsibilities: the simdjson library cannot verify that the input buffer was padded with SIMDJSON_PADDING extra bytes. Setting the `realloc_if_needed` parameter `false` in this manner may lead to better performance since copies are avoided, but it requires that the user takes more responsibilities: the simdjson library cannot verify that the input buffer was padded with SIMDJSON_PADDING extra bytes.
+1 -1
View File
@@ -8,7 +8,7 @@ library provides high-speed access to files or streams containing multiple small
{"text":"a"} {"text":"a"}
{"text":"b"} {"text":"b"}
{"text":"c"} {"text":"c"}
... "..."
``` ```
... you want to read the entries (individual JSON documents) as quickly and as conveniently as possible. Importantly, the input might span several gigabytes, but you want to use a small (fixed) amount of memory. Ideally, you'd also like the parallelize the processing (using more than one core) to speed up the process. ... you want to read the entries (individual JSON documents) as quickly and as conveniently as possible. Importantly, the input might span several gigabytes, but you want to use a small (fixed) amount of memory. Ideally, you'd also like the parallelize the processing (using more than one core) to speed up the process.
+1 -1
View File
@@ -545,7 +545,7 @@ To help visualize the algorithm, we'll walk through the example C++ given at the
"statuses": [ "statuses": [
{ "id": 1, "text": "first!", "user": { "screen_name": "lemire", "name": "Daniel" }, "retweet_count": 40 }, { "id": 1, "text": "first!", "user": { "screen_name": "lemire", "name": "Daniel" }, "retweet_count": 40 },
{ "id": 2, "text": "second!", "user": { "screen_name": "jkeiser2", "name": "John" }, "retweet_count": 3 } { "id": 2, "text": "second!", "user": { "screen_name": "jkeiser2", "name": "John" }, "retweet_count": 3 }
^ (depth 3 - root > statuses > tweet) ^ (depth 4 - root > statuses > tweet > field)
], ],
"search_metadata": { "count": 2 } "search_metadata": { "count": 2 }
} }
+4
View File
@@ -265,6 +265,8 @@ struct simdjson_result_base : protected std::pair<T, error_code> {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
}; // struct simdjson_result_base }; // struct simdjson_result_base
} // namespace internal } // namespace internal
@@ -376,6 +378,8 @@ struct simdjson_result : public internal::simdjson_result_base<T> {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
}; // struct simdjson_result }; // struct simdjson_result
#if SIMDJSON_EXCEPTIONS #if SIMDJSON_EXCEPTIONS
@@ -138,6 +138,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -538,7 +538,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
+57
View File
@@ -8,6 +8,7 @@
#include "simdjson/padded_string_view-inl.h" #include "simdjson/padded_string_view-inl.h"
#include <climits> #include <climits>
#include <cwchar>
namespace simdjson { namespace simdjson {
namespace internal { namespace internal {
@@ -185,6 +186,62 @@ inline simdjson_result<padded_string> padded_string::load(std::string_view filen
return s; return s;
} }
#if defined(_WIN32) && SIMDJSON_CPLUSPLUS17
inline simdjson_result<padded_string> padded_string::load(std::wstring_view filename) noexcept {
// Open the file using the wide characters
SIMDJSON_PUSH_DISABLE_WARNINGS
SIMDJSON_DISABLE_DEPRECATED_WARNING // Disable CRT_SECURE warning on MSVC: manually verified this is safe
std::FILE *fp = _wfopen(filename.data(), L"rb");
SIMDJSON_POP_DISABLE_WARNINGS
if (fp == nullptr) {
return IO_ERROR;
}
// Get the file size
int ret;
#if SIMDJSON_VISUAL_STUDIO && !SIMDJSON_IS_32BITS
ret = _fseeki64(fp, 0, SEEK_END);
#else
ret = std::fseek(fp, 0, SEEK_END);
#endif // _WIN64
if(ret < 0) {
std::fclose(fp);
return IO_ERROR;
}
#if SIMDJSON_VISUAL_STUDIO && !SIMDJSON_IS_32BITS
__int64 llen = _ftelli64(fp);
if(llen == -1L) {
std::fclose(fp);
return IO_ERROR;
}
#else
long llen = std::ftell(fp);
if((llen < 0) || (llen == LONG_MAX)) {
std::fclose(fp);
return IO_ERROR;
}
#endif
// Allocate the padded_string
size_t len = static_cast<size_t>(llen);
padded_string s(len);
if (s.data() == nullptr) {
std::fclose(fp);
return MEMALLOC;
}
// Read the padded_string
std::rewind(fp);
size_t bytes_read = std::fread(s.data(), 1, len, fp);
if (std::fclose(fp) != 0 || bytes_read != len) {
return IO_ERROR;
}
return s;
}
#endif
} // namespace simdjson } // namespace simdjson
inline simdjson::padded_string operator ""_padded(const char *str, size_t len) { inline simdjson::padded_string operator ""_padded(const char *str, size_t len) {
+13
View File
@@ -127,6 +127,19 @@ struct padded_string final {
**/ **/
inline static simdjson_result<padded_string> load(std::string_view path) noexcept; inline static simdjson_result<padded_string> load(std::string_view path) noexcept;
#if defined(_WIN32) && SIMDJSON_CPLUSPLUS17
/**
* This function accepts a wide string path (UTF-16) and converts it to
* UTF-8 before loading the file. This allows windows users to work
* with unicode file paths without manually converting the paths everytime.
*
* @return IO_ERROR on error, including conversion failures.
*
* @param path the path to the file as a wide string.
**/
inline static simdjson_result<padded_string> load(std::wstring_view path) noexcept;
#endif
private: private:
padded_string &operator=(const padded_string &o) = delete; padded_string &operator=(const padded_string &o) = delete;
padded_string(const padded_string &o) = delete; padded_string(const padded_string &o) = delete;
+2 -2
View File
@@ -4,7 +4,7 @@
#define SIMDJSON_SIMDJSON_VERSION_H #define SIMDJSON_SIMDJSON_VERSION_H
/** The version of simdjson being used (major.minor.revision) */ /** The version of simdjson being used (major.minor.revision) */
#define SIMDJSON_VERSION "4.2.1" #define SIMDJSON_VERSION "4.2.2"
namespace simdjson { namespace simdjson {
enum { enum {
@@ -19,7 +19,7 @@ enum {
/** /**
* The revision (major.minor.REVISION) of simdjson being used. * The revision (major.minor.REVISION) of simdjson being used.
*/ */
SIMDJSON_VERSION_REVISION = 1 SIMDJSON_VERSION_REVISION = 2
}; };
} // namespace simdjson } // namespace simdjson
+29 -1
View File
@@ -1,4 +1,4 @@
/* auto-generated on 2025-11-03 11:03:21 -0500. version 4.2.1 Do not edit! */ /* auto-generated on 2025-11-11 14:17:08 -0500. version 4.2.2 Do not edit! */
/* including simdjson.cpp: */ /* including simdjson.cpp: */
/* begin file simdjson.cpp */ /* begin file simdjson.cpp */
#define SIMDJSON_SRC_SIMDJSON_CPP #define SIMDJSON_SRC_SIMDJSON_CPP
@@ -2752,6 +2752,8 @@ struct simdjson_result_base : protected std::pair<T, error_code> {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
}; // struct simdjson_result_base }; // struct simdjson_result_base
} // namespace internal } // namespace internal
@@ -2863,6 +2865,8 @@ struct simdjson_result : public internal::simdjson_result_base<T> {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
}; // struct simdjson_result }; // struct simdjson_result
#if SIMDJSON_EXCEPTIONS #if SIMDJSON_EXCEPTIONS
@@ -10105,6 +10109,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -16600,6 +16607,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -22950,6 +22960,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -29457,6 +29470,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -36323,6 +36339,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -43011,6 +43030,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -49145,6 +49167,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -54871,6 +54896,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
+109 -11
View File
@@ -1,4 +1,4 @@
/* auto-generated on 2025-11-03 11:03:21 -0500. version 4.2.1 Do not edit! */ /* auto-generated on 2025-11-11 14:17:08 -0500. version 4.2.2 Do not edit! */
/* including simdjson.h: */ /* including simdjson.h: */
/* begin file simdjson.h */ /* begin file simdjson.h */
#ifndef SIMDJSON_H #ifndef SIMDJSON_H
@@ -2513,7 +2513,7 @@ namespace std {
#define SIMDJSON_SIMDJSON_VERSION_H #define SIMDJSON_SIMDJSON_VERSION_H
/** The version of simdjson being used (major.minor.revision) */ /** The version of simdjson being used (major.minor.revision) */
#define SIMDJSON_VERSION "4.2.1" #define SIMDJSON_VERSION "4.2.2"
namespace simdjson { namespace simdjson {
enum { enum {
@@ -2528,7 +2528,7 @@ enum {
/** /**
* The revision (major.minor.REVISION) of simdjson being used. * The revision (major.minor.REVISION) of simdjson being used.
*/ */
SIMDJSON_VERSION_REVISION = 1 SIMDJSON_VERSION_REVISION = 2
}; };
} // namespace simdjson } // namespace simdjson
@@ -2815,6 +2815,8 @@ struct simdjson_result_base : protected std::pair<T, error_code> {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
}; // struct simdjson_result_base }; // struct simdjson_result_base
} // namespace internal } // namespace internal
@@ -2926,6 +2928,8 @@ struct simdjson_result : public internal::simdjson_result_base<T> {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
}; // struct simdjson_result }; // struct simdjson_result
#if SIMDJSON_EXCEPTIONS #if SIMDJSON_EXCEPTIONS
@@ -4165,6 +4169,19 @@ struct padded_string final {
**/ **/
inline static simdjson_result<padded_string> load(std::string_view path) noexcept; inline static simdjson_result<padded_string> load(std::string_view path) noexcept;
#if defined(_WIN32) && SIMDJSON_CPLUSPLUS17
/**
* This function accepts a wide string path (UTF-16) and converts it to
* UTF-8 before loading the file. This allows windows users to work
* with unicode file paths without manually converting the paths everytime.
*
* @return IO_ERROR on error, including conversion failures.
*
* @param path the path to the file as a wide string.
**/
inline static simdjson_result<padded_string> load(std::wstring_view path) noexcept;
#endif
private: private:
padded_string &operator=(const padded_string &o) = delete; padded_string &operator=(const padded_string &o) = delete;
padded_string(const padded_string &o) = delete; padded_string(const padded_string &o) = delete;
@@ -4476,6 +4493,7 @@ inline padded_string_view pad_with_reserve(std::string& s) noexcept {
/* end file simdjson/padded_string_view-inl.h */ /* end file simdjson/padded_string_view-inl.h */
#include <climits> #include <climits>
#include <cwchar>
namespace simdjson { namespace simdjson {
namespace internal { namespace internal {
@@ -4653,6 +4671,62 @@ inline simdjson_result<padded_string> padded_string::load(std::string_view filen
return s; return s;
} }
#if defined(_WIN32) && SIMDJSON_CPLUSPLUS17
inline simdjson_result<padded_string> padded_string::load(std::wstring_view filename) noexcept {
// Open the file using the wide characters
SIMDJSON_PUSH_DISABLE_WARNINGS
SIMDJSON_DISABLE_DEPRECATED_WARNING // Disable CRT_SECURE warning on MSVC: manually verified this is safe
std::FILE *fp = _wfopen(filename.data(), L"rb");
SIMDJSON_POP_DISABLE_WARNINGS
if (fp == nullptr) {
return IO_ERROR;
}
// Get the file size
int ret;
#if SIMDJSON_VISUAL_STUDIO && !SIMDJSON_IS_32BITS
ret = _fseeki64(fp, 0, SEEK_END);
#else
ret = std::fseek(fp, 0, SEEK_END);
#endif // _WIN64
if(ret < 0) {
std::fclose(fp);
return IO_ERROR;
}
#if SIMDJSON_VISUAL_STUDIO && !SIMDJSON_IS_32BITS
__int64 llen = _ftelli64(fp);
if(llen == -1L) {
std::fclose(fp);
return IO_ERROR;
}
#else
long llen = std::ftell(fp);
if((llen < 0) || (llen == LONG_MAX)) {
std::fclose(fp);
return IO_ERROR;
}
#endif
// Allocate the padded_string
size_t len = static_cast<size_t>(llen);
padded_string s(len);
if (s.data() == nullptr) {
std::fclose(fp);
return MEMALLOC;
}
// Read the padded_string
std::rewind(fp);
size_t bytes_read = std::fread(s.data(), 1, len, fp);
if (std::fclose(fp) != 0 || bytes_read != len) {
return IO_ERROR;
}
return s;
}
#endif
} // namespace simdjson } // namespace simdjson
inline simdjson::padded_string operator ""_padded(const char *str, size_t len) { inline simdjson::padded_string operator ""_padded(const char *str, size_t len) {
@@ -12996,6 +13070,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -15191,6 +15268,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -17885,6 +17965,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -20579,6 +20662,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -23388,6 +23474,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -26513,6 +26602,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -29115,6 +29207,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -31730,6 +31825,9 @@ struct implementation_simdjson_result_base {
*/ */
simdjson_inline T&& value_unsafe() && noexcept; simdjson_inline T&& value_unsafe() && noexcept;
using value_type = T;
using error_type = error_code;
protected: protected:
/** users should never directly access first and second. **/ /** users should never directly access first and second. **/
T first{}; /** Users should never directly access 'first'. **/ T first{}; /** Users should never directly access 'first'. **/
@@ -46859,7 +46957,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -61324,7 +61422,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -76288,7 +76386,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -91252,7 +91350,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -106331,7 +106429,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -121726,7 +121824,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -136598,7 +136696,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
@@ -151483,7 +151581,7 @@ template <std::ranges::range R>
simdjson_inline void string_builder::append(const R &range) noexcept { simdjson_inline void string_builder::append(const R &range) noexcept {
auto it = std::ranges::begin(range); auto it = std::ranges::begin(range);
auto end = std::ranges::end(range); auto end = std::ranges::end(range);
if constexpr (concepts::is_pair<typename R::value_type>) { if constexpr (concepts::is_pair<std::ranges::range_value_t<R>>) {
start_object(); start_object();
if (it == end) { if (it == end) {
Binary file not shown.
+25 -2
View File
@@ -4,6 +4,10 @@
#include <string> #include <string>
#include <string_view> #include <string_view>
#if SIMDJSON_SUPPORTS_RANGES
#include <ranges>
#endif
using namespace simdjson; using namespace simdjson;
struct Car { struct Car {
@@ -517,6 +521,24 @@ bool map_test() {
ASSERT_EQUAL(s, "{\"key1\":1.0,\"key2\":1.0}"); ASSERT_EQUAL(s, "{\"key1\":1.0,\"key2\":1.0}");
TEST_SUCCEED(); TEST_SUCCEED();
} }
bool ranges_test() {
TEST_START();
struct Foo {
int a;
float b;
};
std::vector<Foo> c = {{1, 2.0f}, {3, 4.0f}, {5, 6.0f}, {7, 8.0f}};
simdjson::builder::string_builder sb;
sb.append(c | std::views::transform(&Foo::b));
std::string_view p;
auto result = sb.view().get(p);
ASSERT_SUCCESS(result);
ASSERT_EQUAL(p, "[2.0,4.0,6.0,8.0]");
std::string s;
ASSERT_SUCCESS(simdjson::to_json(c | std::views::transform(&Foo::b)).get(s));
ASSERT_EQUAL(s, "[2.0,4.0,6.0,8.0]");
TEST_SUCCEED();
}
bool double_double_test() { bool double_double_test() {
TEST_START(); TEST_START();
std::vector<std::vector<double>> c = {{1.0, 2.0}, {3.0, 4.0}}; std::vector<std::vector<double>> c = {{1.0, 2.0}, {3.0, 4.0}};
@@ -586,8 +608,9 @@ bool run() {
car_test_exception() && string_convertion_except() && car_test_exception() && string_convertion_except() &&
#endif #endif
#if SIMDJSON_SUPPORTS_RANGES && SIMDJSON_SUPPORTS_CONCEPTS #if SIMDJSON_SUPPORTS_RANGES && SIMDJSON_SUPPORTS_CONCEPTS
map_test() && double_double_test() && double_double_test_to_string() && map_test() && ranges_test() && double_double_test() &&
car_test_simple() && car_test_simple_complete() && double_double_test_to_string() && car_test_simple() &&
car_test_simple_complete() &&
#if SIMDJSON_EXCEPTIONS #if SIMDJSON_EXCEPTIONS
car_test_simple_complete_exceptions() && car_test_simple_complete_exceptions() &&
#endif #endif
+6 -6
View File
@@ -87,16 +87,16 @@ bool cast_tester<T>::test_get_error(simdjson_result<element> element, error_code
template<typename T> template<typename T>
bool cast_tester<T>::test_get_t(element element, T expected) { bool cast_tester<T>::test_get_t(element element, T expected) {
auto actual = element.get<T>(); T value;
ASSERT_SUCCESS(actual.error()); ASSERT_SUCCESS(element.get<T>().get(value));
return assert_equal(actual.value_unsafe(), expected); return assert_equal(value, expected);
} }
template<typename T> template<typename T>
bool cast_tester<T>::test_get_t(simdjson_result<element> element, T expected) { bool cast_tester<T>::test_get_t(simdjson_result<element> element, T expected) {
auto actual = element.get<T>(); T value;
ASSERT_SUCCESS(actual.error()); ASSERT_SUCCESS(element.get<T>().get(value));
return assert_equal(actual.value_unsafe(), expected); return assert_equal(value, expected);
} }
template<typename T> template<typename T>
+95 -1
View File
@@ -597,6 +597,98 @@ bool test_top_level_array_example() {
TEST_SUCCEED(); TEST_SUCCEED();
} }
/**
* Concept to validate that a type represents a person with name and age
*/
template <typename T>
concept person = requires(T p) {
std::string_view(p.name); // has name field convertible to string_view
p.age; // has age field
requires std::is_integral_v<decltype(p.age)>; // age is integral
};
/**
* Concept to validate that a type is an array of person objects
*/
template <typename T>
concept array_of_person = requires(T arr) {
arr.size(); // has size method
arr[0]; // can access elements with []
requires person<decltype(arr[0])>; // elements satisfy person concept
};
/**
* Test: Array of objects with concept validation
*/
bool test_array_of_objects_with_concept() {
TEST_START();
constexpr auto config = R"(
[
{ "name": "Alice", "age": 30 },
{ "name": "Bob", "age": 25 },
{ "name": "Charlie", "age": 35 }
]
)"_json;
std::print("Array size: {}\n", config.size());
static_assert(config.size() == 3);
// Validate that the array satisfies the array_of_person concept
static_assert(array_of_person<decltype(config)>);
// Test the actual values
static_assert(std::string_view(config[0].name) == "Alice");
static_assert(config[0].age == 30);
static_assert(std::string_view(config[1].name) == "Bob");
static_assert(config[1].age == 25);
static_assert(std::string_view(config[2].name) == "Charlie");
static_assert(config[2].age == 35);
// Runtime assertions
ASSERT_EQUAL(config.size(), 3);
ASSERT_EQUAL(std::string_view(config[0].name), "Alice"sv);
ASSERT_EQUAL(config[0].age, 30);
ASSERT_EQUAL(std::string_view(config[1].name), "Bob"sv);
ASSERT_EQUAL(config[1].age, 25);
ASSERT_EQUAL(std::string_view(config[2].name), "Charlie"sv);
ASSERT_EQUAL(config[2].age, 35);
TEST_SUCCEED();
}
/**
* Test: #embed support for external JSON files (C++26)
*/
bool test_embed_twitter_json() {
TEST_START();
// C++26 #embed allows embedding files directly into the binary at compile time
// This creates a const char array with the file contents plus null terminator
constexpr const char twitter_json[] = {
#embed TWITTER_JSON
, 0
};
// Parse the embedded JSON at compile time
constexpr auto parsed_twitter = simdjson::compile_time::parse_json<twitter_json>();
// Verify the structure - twitter.json should have a "statuses" array
static_assert(parsed_twitter.statuses.size() > 0);
// Runtime verification
ASSERT_TRUE(parsed_twitter.statuses.size() > 0);
std::cout << "Successfully parsed embedded twitter.json with "
<< parsed_twitter.statuses.size() << " statuses" << std::endl;
TEST_SUCCEED();
}
bool run() { bool run() {
return test_basic_object() && return test_basic_object() &&
test_nested_objects() && test_nested_objects() &&
@@ -620,7 +712,9 @@ bool run() {
array_of_objects() && array_of_objects() &&
test_user_config_example() && test_user_config_example() &&
test_nested_servers_example() && test_nested_servers_example() &&
test_top_level_array_example(); test_top_level_array_example() &&
test_array_of_objects_with_concept() &&
test_embed_twitter_json();
} }
} // namespace compile_time_json_tests } // namespace compile_time_json_tests