readfloat.c: correctly round fraction via division by exact 10^n

The previous code computed `frac_part * pow10_negative[n]`, where
pow10_negative[n] is already a rounded approximation of 10^-n (since
10^-n is not exactly representable in binary). The multiplication then
adds another rounding step, leaving up to ~1 ulp of error.

Dividing `frac_part` by `pow10_positive[n]` is exact for n <= 22 (the
range where 10^n fits exactly in a double), so the division is the
only rounding and the result is correctly rounded. For example,
"0.3".to_f now matches the 0.3 literal's bit pattern (and libc strtod).
This commit is contained in:
Yasuhiro Matsumoto
2026-04-16 14:41:31 +09:00
parent 2f5a24ed9b
commit 10c9e83128
+3 -2
View File
@@ -168,10 +168,11 @@ mrb_read_float(const char *str, char **endp, double *fp)
res = (double)int_part;
}
else {
// Fast path: combine integer and fractional parts
// Divide by the exact 10^n (exact for n <= 22) rather than multiplying
// by the inexact 10^-n, so the fraction is correctly rounded.
res = (double)int_part;
if (frac_digits > 0) {
res += (double)frac_part * mrb_pow10(-frac_digits);
res += (double)frac_part / mrb_pow10(frac_digits);
}
}