bigint.c: increase karatsuba scratch allocation safety margin

the previous fixed safety margin of 8 limbs was insufficient for certain
edge cases involving deep recursion levels in karatsuba multiplication,
as discovered by oss-fuzz. changed to proportional margin (~12.5% plus
fixed overhead of 16) that scales with input size.

this prevents potential buffer overruns in deeply nested karatsuba
multiplications while maintaining efficiency for typical cases.

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Yukihiro "Matz" Matsumoto
2025-11-24 09:35:43 +09:00
parent 857de45036
commit 5ed0cd4ecb
+5 -2
View File
@@ -1112,8 +1112,11 @@ mpz_mul(mpz_ctx_t *ctx, mpz_t *ww, mpz_t *u, mpz_t *v)
mpz_realloc(ctx, ww, result_size);
size_t scratch_size = karatsuba_scratch_size(u->sz, v->sz);
/* Add safety margin to account for rounding in recursive partitioning */
scratch_size += 8;
/* Add safety margin proportional to scratch size.
* While the calculation is mathematically exact, empirical testing
* (valgrind, oss-fuzz) reveals edge cases requiring extra space.
* Proportional margin scales better than fixed for large inputs. */
scratch_size += (scratch_size >> 3) + 16;
size_t pool_state = pool_save(ctx);
mp_limb *scratch = NULL;