When find_field_unordered is used on an empty object, it calls
json_iterator::reenter_child. reenter_child asserts that it doesn't
rewind too far back by consulting parser->start_positions.
When the On Demand parser sees an empty object, it fails to update
parser->start_positions. This means that the assertion in
json_iterator::reenter_child reads stale data, or potentially
uninitialized memory. Reading uninitialized memory can cause spurious
assertion failures and Valgrind memcheck reports:
Running missing_keys_for_empty_top_level_object ...
==170679== Conditional jump or move depends on uninitialised value(s)
==170679== at 0x4943D7: reenter_child (json_iterator-inl.h:208)
==170679== by 0x4943D7: find_field_unordered_raw (value_iterator-inl.h:197)
==170679== by 0x4943D7: find_field_unordered (object-inl.h:13)
==170679== by 0x4943D7: find_field_unordered (object-inl.h:96)
==170679== by 0x4943D7: find_field_unordered (value-inl.h:110)
==170679== by 0x4943D7: find_field_unordered (document-inl.h:105)
==170679== by 0x4943D7: object_tests::missing_keys_for_empty_top_level_object() (ondemand_object_tests.cpp:117)
==170679== by 0x4CA761: object_tests::run() (ondemand_object_tests.cpp:1085)
==170679== by 0x8BA314: int test_main<bool ()>(int, char**, bool ( const&)()) (test_ondemand.h:81)
==170679== by 0x4CA9C8: main (ondemand_object_tests.cpp:1119)
==170679==
Fix the read of uninitialized or stale memory by updating
parser->start_positions regardless of whether we see an empty object or
an object with some keys.
This commit only affects builds where development checks
(SIMDJSON_DEVELOPMENT_CHECKS) are enabled. Builds where development
checks are disabled are unaffected by this bug.
* Verifies bug with missing keys.
* Allowing search from any key.
* Workaround for buggy msys
* Restricting how we can end key searches.
* Adding a few tests.
* Truncate final unclosed string.
* Adding more precise remarks.
* Better documentation and more robust code.
* ARM + PPC corrections.
* Patching ARM implementation with new stage1_mode parameter.
* Fixed most problems.
* Correcting white spaces and adding a remark.
* This adds the truncated_bytes() method to the stream instances.
* This implementations string serialization for On Demand instances.
* Adding more documentation.
* Another remark.
* Marking the new functions as inline.
* casts apparently do not work.
* Upgrading the API.
* Making the code really free from exceptions.
* At another fix for exceptionless.
* Modify to_chars so that it does not pad integers with '.0'.
* Negative 0 cannot be expressed as an integer.
* Again, accomodating exceptionless usage.
* Using x <= -0 does not allow you to determine the sign since 0 <= -0. I am not sure where
this bug comes from.
This avoids a very unlikely buffer overrun that can occur in a particular kind of invalid JSON:
- the document is invalid with an unclosed top level array or object
- the last thing in the document is a number that ends at EOF
- the padding is filled entirely with numeric digits
John Keiser