mirror of
https://github.com/mruby/mruby
synced 2026-06-08 16:11:16 +00:00
readfloat.c: correctly round fraction via division by exact 10^n
The previous code computed `frac_part * pow10_negative[n]`, where pow10_negative[n] is already a rounded approximation of 10^-n (since 10^-n is not exactly representable in binary). The multiplication then adds another rounding step, leaving up to ~1 ulp of error. Dividing `frac_part` by `pow10_positive[n]` is exact for n <= 22 (the range where 10^n fits exactly in a double), so the division is the only rounding and the result is correctly rounded. For example, "0.3".to_f now matches the 0.3 literal's bit pattern (and libc strtod).
This commit is contained in:
+3
-2
@@ -168,10 +168,11 @@ mrb_read_float(const char *str, char **endp, double *fp)
|
||||
res = (double)int_part;
|
||||
}
|
||||
else {
|
||||
// Fast path: combine integer and fractional parts
|
||||
// Divide by the exact 10^n (exact for n <= 22) rather than multiplying
|
||||
// by the inexact 10^-n, so the fraction is correctly rounded.
|
||||
res = (double)int_part;
|
||||
if (frac_digits > 0) {
|
||||
res += (double)frac_part * mrb_pow10(-frac_digits);
|
||||
res += (double)frac_part / mrb_pow10(frac_digits);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user