admin/simdjson-simdjson
1
0
Fork 0
mirror of https://github.com/simdjson/simdjson synced 2026-06-08 17:27:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fixing UTF-8 validation under PPC64 (#1346)

Browse Source 
* Entering a new UTF-8 test

* Maybe *I* had a bug in the tests.

* Replacing nulls with 1s.

* Let us try to be more verbose.

* Return 0.

* Fixing issue.

* Adding puzzler scenario.

* Fixing PPC64

Co-authored-by: Daniel Lemire <dlemire@rcs-power9-talos>
This commit is contained in:
Daniel Lemire
2020-12-19 10:42:27 -05:00
committed by GitHub
parent f785f76d98
commit 85001c55fb
4 changed files with 51 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ppc64/dom_parser_implementation.cpp
+2 -1
View File
@@ -49,7 +49,8 @@ simdjson_really_inline json_character_block json_character_block::classify(const
}
simdjson_really_inline bool is_ascii(const simd8x64<uint8_t>& input) {
return input.reduce_or().saturating_sub(0b10000000u).bits_not_set_anywhere();
// careful: 0x80 is not ascii.
return input.reduce_or().saturating_sub(0b01111111u).bits_not_set_anywhere();
}
simdjson_unused simdjson_really_inline simd8<bool> must_be_continuation(const simd8<uint8_t> prev1, const simd8<uint8_t> prev2, const simd8<uint8_t> prev3) {