Fix broken UTF-8 characters by IO#getc

Character (multi-byte UTF-8) is destroyed when character spanning
`IO::BUF_SIZE` (4096 bytes) exist.

- Prepare file:

  ```ruby
  File.open("sample", "wb") { |f| f << "●" * 1370 }
  ```

- Before patched:

  ```ruby
  File.open("sample") { |f| a = []; while ch = f.getc; a << ch; end; p a }
  # => ["●", "●", ..., "●", "\xe2", "\x97", "\x8f", "●", "●", "●", "●"]

- After patched:

  ```ruby
  File.open("sample") { |f| a = []; while ch = f.getc; a << ch; end; p a }
  # => ["●", "●", ..., "●", "●", "●", "●", "●", "●"]
This commit is contained in:
dearblue
2019-09-15 23:50:24 +09:00
parent 7cc8c7d2ff
commit 992ba476a9
+8 -2
View File
@@ -170,8 +170,14 @@ class IO
end
def _read_buf
return @buf if @buf && @buf.bytesize > 0
@buf = sysread(BUF_SIZE)
return @buf if @buf && @buf.bytesize >= 4 # maximum UTF-8 character is 4 bytes
@buf ||= ""
begin
@buf += sysread(BUF_SIZE)
rescue EOFError => e
raise e if @buf.empty?
end
@buf
end
def ungetc(substr)