When find_field_unordered is used on an empty object, it calls
json_iterator::reenter_child. reenter_child asserts that it doesn't
rewind too far back by consulting parser->start_positions.
When the On Demand parser sees an empty object, it fails to update
parser->start_positions. This means that the assertion in
json_iterator::reenter_child reads stale data, or potentially
uninitialized memory. Reading uninitialized memory can cause spurious
assertion failures and Valgrind memcheck reports:
Running missing_keys_for_empty_top_level_object ...
==170679== Conditional jump or move depends on uninitialised value(s)
==170679== at 0x4943D7: reenter_child (json_iterator-inl.h:208)
==170679== by 0x4943D7: find_field_unordered_raw (value_iterator-inl.h:197)
==170679== by 0x4943D7: find_field_unordered (object-inl.h:13)
==170679== by 0x4943D7: find_field_unordered (object-inl.h:96)
==170679== by 0x4943D7: find_field_unordered (value-inl.h:110)
==170679== by 0x4943D7: find_field_unordered (document-inl.h:105)
==170679== by 0x4943D7: object_tests::missing_keys_for_empty_top_level_object() (ondemand_object_tests.cpp:117)
==170679== by 0x4CA761: object_tests::run() (ondemand_object_tests.cpp:1085)
==170679== by 0x8BA314: int test_main<bool ()>(int, char**, bool ( const&)()) (test_ondemand.h:81)
==170679== by 0x4CA9C8: main (ondemand_object_tests.cpp:1119)
==170679==
Fix the read of uninitialized or stale memory by updating
parser->start_positions regardless of whether we see an empty object or
an object with some keys.
This commit only affects builds where development checks
(SIMDJSON_DEVELOPMENT_CHECKS) are enabled. Builds where development
checks are disabled are unaffected by this bug.
* Verifies bug with missing keys.
* Allowing search from any key.
* Workaround for buggy msys
* Restricting how we can end key searches.
* Adding a few tests.
* Under ARM, it is slightly better to reverse the word once and then extract the bits.
* Guarding the zero_leading_bit call to avoid sanitizer warnings.
* Truncate final unclosed string.
* Adding more precise remarks.
* Better documentation and more robust code.
* ARM + PPC corrections.
* Patching ARM implementation with new stage1_mode parameter.
* Fixed most problems.
* Correcting white spaces and adding a remark.
* This adds the truncated_bytes() method to the stream instances.
* This implementations string serialization for On Demand instances.
* Adding more documentation.
* Another remark.
* Marking the new functions as inline.
* casts apparently do not work.
* Upgrading the API.
* Making the code really free from exceptions.
* At another fix for exceptionless.
* Modify to_chars so that it does not pad integers with '.0'.
* Negative 0 cannot be expressed as an integer.
* Again, accomodating exceptionless usage.
* Using x <= -0 does not allow you to determine the sign since 0 <= -0. I am not sure where
this bug comes from.
This eliminates the possibility of inlining target failures for ondemand
Also makes it so we always compile common architectures needed by simdjson.cpp in simdjson.h, since amalgamation has no way to reason about whether to include / exclude it.
John Keiser