admin/simdjson-simdjson
1
0
Fork 0
mirror of https://github.com/simdjson/simdjson synced 2026-06-08 17:27:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Documenting a specific use case where you need a value if and only if it is another key is not present (#1865)

Browse Source 
* Documenting a specific use case.

* Adding more comments and documentation.
This commit is contained in:
Daniel Lemire
2022-07-09 10:54:28 -04:00
committed by GitHub
parent a89d57d0d4
commit bd3d67e889
4 changed files with 121 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/basics.md
+12 -6
View File
@@ -261,11 +261,15 @@ idiomatic C++ iterators, operators and casts. Besides the document instances and
native types (`double`, `uint64_t`, `int64_t`, `bool`), we also access
Unicode (UTF-8) strings (`std::string_view`), objects (`simdjson::ondemand::object`)
and arrays (`simdjson::ondemand::array`).
We also have a generic type (`simdjson::ondemand::value`) which represents a potential
We also have a generic ephemeral type (`simdjson::ondemand::value`) which represents a potential
array or object, or scalar type (`double`, `uint64_t`, `int64_t`, `bool`, `null`, string) inside
an array or an object. Both generic types (`simdjson::ondemand::document` and
`simdjson::ondemand::value`) have a `type()` method returning a `json_type` value describing the
value (`json_type::array`, `json_type::object`, `json_type::number`, `json_type::string`, `json_type::boolean`, `json_type::null`).
value (`json_type::array`, `json_type::object`, `json_type::number`, `json_type::string`,
`json_type::boolean`, `json_type::null`). A generic value (`simdjson::ondemand::value`)
is only valid temporarily, as soon as you access other values, other keys in objects, etc.
it becomes invalid: you should therefore consume the value immediately by converting it to a
scalar type, an array or an object.
Advanced users who need to determine the number types (integer or float) dynamically,
should review our section [dynamic number types](#dynamic-number-types). Indeed,
@@ -300,10 +304,12 @@ support for users who avoid exceptions. See [the simdjson error handling documen
comparison. For efficiency reason, you should avoid looking up the same field repeatedly: e.g., do
not do `object["foo"]` followed by `object["foo"]` with the same `object` instance. If you consume an
object twice: `std::string_view(object["foo"]` followed by `std::string_view(object["foo"]`, your code
is in error. Furthermore, you can only consume one field at a time, on the same object. Thus
if you have retrieved `content["bids"].get_array()` and you later call `content["asks"].get_array()`, then the
first array should no longer be accessed: it would be unsafe to do so. You can detect such mistakes by first
compiling and running the code in Debug mode: an OUT_OF_ORDER_ITERATION error is generated.
is in error. Furthermore, you can only consume one field at a time, on the same object. The
value instance you get from `content["bids"]` becomes invalid when you call `content["asks"]`.
If you have retrieved `content["bids"].get_array()` and you later call
`content["asks"].get_array()`, then the first array should no longer be accessed: it would be
unsafe to do so. You can detect such mistakes by first compiling and running the code in
Debug mode: an OUT_OF_ORDER_ITERATION error is generated.
> NOTE: JSON allows you to escape characters in keys. E.g., the key `"date"` may be written as
> `"\u0064\u0061\u0074\u0065"`. By default, simdjson does *not* unescape keys when matching by default.
include/simdjson/generic/ondemand/object.h
+4 -2
View File
@@ -38,7 +38,8 @@ public:
* e.g. `object["a"]` will match `{ "a": 1 }`, but will *not* match `{ "\u0061": 1 }`.
*
* You must consume the fields on an object one at a time. A request for a new key
* invalidates previous field values: it makes them unsafe. E.g., the array
* invalidates previous field values: it makes them unsafe. The value instance you get
* from `content["bids"]` becomes invalid when you call `content["asks"]`. The array
* given by content["bids"].get_array() should not be accessed after you have called
* content["asks"].get_array(). You can detect such mistakes by first compiling and running
* the code in Debug mode (or with the macro `SIMDJSON_DEVELOPMENT_CHECKS` set to 1): an
@@ -75,7 +76,8 @@ public:
* that only one field is returned.
*
* You must consume the fields on an object one at a time. A request for a new key
* invalidates previous field values: it makes them unsafe. E.g., the array
* invalidates previous field values: it makes them unsafe. The value instance you get
* from `content["bids"]` becomes invalid when you call `content["asks"]`. The array
* given by content["bids"].get_array() should not be accessed after you have called
* content["asks"].get_array(). You can detect such mistakes by first compiling and running
* the code in Debug mode (or with the macro `SIMDJSON_DEVELOPMENT_CHECKS` set to 1): an
include/simdjson/generic/ondemand/value.h
+2 -1
View File
@@ -11,7 +11,8 @@ class object;
class raw_json_string;
/**
* An ephemeral JSON value returned during iteration.
* An ephemeral JSON value returned during iteration. It is only valid for as long as you do
* not access more data in the JSON document.
*/
class value {
public:
tests/ondemand/ondemand_object_tests.cpp
+103
View File
@@ -1074,8 +1074,111 @@ namespace object_tests {
TEST_SUCCEED();
}
bool test_strager() {
// Find exactly one of a/myint or b/myint, if both or none are present,
// return false.
auto process = [](ondemand::object root) -> std::pair<bool,uint64_t> {
///////////////////////////////////////////////////////////
// The core idea with On-Demand is that you should visit everything only
// once.
//
// Thus, with On-Demand, you must consume a field once you have accessed it.
// Thus if you do root["a"], you must full consume the corresponding
// value (if present) before you move to another field (e.g., root["a"]).
//
// If, somehow, you only want the value corresponding to "a" if some
// other key is not present, then you must still parse/consume the value
// corresponding to "a", and later throw the result away.
///////////////////////////////////////////////////////////
uint64_t myint;
// Notice in the line below that I use chaining... I do not
// first convert root["a"] to an object and then seek
// "myint" into. Chaining works in this instance because
// I only need to access the object 'root["a"]' for a single
// key, if I needed to do more work on root["a"], then I should
// create an object.
if(root["a"]["myint"].get(myint) == SUCCESS) {
// We want myint to be discarded if "b" is also present.
if(! root["b"].error() ) {
// Oops, the other key is also present!!! We must throw
// away 'myint' and return false.
// Notice how I do not parse the value corresponding to
// '"b"', as it is unneeded.
return std::make_pair(false, 0);
}
// All is good,
return std::make_pair(true,myint);
}
// Now we know that "a" is not present, so seek "b".
if(root["b"]["myint"].get(myint) == SUCCESS) {
return std::make_pair(true, myint);
}
// None are present.
return std::make_pair(false, 0);
};
TEST_START();
{
ondemand::parser parser;
padded_string json = R"({ "a": { "myint": 42 } })"_padded;
std::cout << " - Subtest 'a' - JSON: " << (json) << " ..." << std::endl;
ondemand::document doc;
ASSERT_SUCCESS(parser.iterate(json).get(doc));
ondemand::object obj;
ASSERT_SUCCESS(doc.get_object().get(obj));
std::pair<bool,uint64_t> ret = process(obj);
ASSERT_TRUE(ret.first);
ASSERT_EQUAL(ret.second, 42);
}
{
ondemand::parser parser;
padded_string json = R"({ "b": { "myint": 42 } })"_padded;
std::cout << " - Subtest 'b' - JSON: " << (json) << " ..." << std::endl;
ondemand::document doc;
ASSERT_SUCCESS(parser.iterate(json).get(doc));
ondemand::object obj;
ASSERT_SUCCESS(doc.get_object().get(obj));
std::pair<bool,uint64_t> ret = process(obj);
ASSERT_TRUE(ret.first);
ASSERT_EQUAL(ret.second, 42);
}
{
ondemand::parser parser;
padded_string json = R"({ "b": { "myint": 42 }, "a": { "myint": 42 } })"_padded;
std::cout << " - Subtest 'b&a' - JSON: " << (json) << " ..." << std::endl;
ondemand::document doc;
ASSERT_SUCCESS(parser.iterate(json).get(doc));
ondemand::object obj;
ASSERT_SUCCESS(doc.get_object().get(obj));
std::pair<bool,uint64_t> ret = process(obj);
ASSERT_FALSE(ret.first);
ASSERT_EQUAL(ret.second, 0);
}
{
ondemand::parser parser;
padded_string json = R"({ "z": { "myint": 42 }, "y": { "myint": 42 } })"_padded;
std::cout << " - Subtest 'none' - JSON: " << (json) << " ..." << std::endl;
ondemand::document doc;
ASSERT_SUCCESS(parser.iterate(json).get(doc));
ondemand::object obj;
ASSERT_SUCCESS(doc.get_object().get(obj));
std::pair<bool,uint64_t> ret = process(obj);
ASSERT_FALSE(ret.first);
ASSERT_EQUAL(ret.second, 0);
}
TEST_SUCCEED();
}
bool run() {
return
test_strager() &&
issue1745() &&
issue1742() &&
issue1742_value() &&