Yukihiro "Matz" Matsumoto ff0e20453f bigint.c: fix GCD infinite loop and size handling bugs ...

Fixed three bugs that caused infinite loops in GCD calculations:

1. mpz_set_int() didn't shrink sz when setting a smaller value.
   mpz_realloc() only grows allocations, so setting a 1-limb value
   to an mpz_t with sz=3 would leave sz=3, breaking algorithms
   that depend on correct sz values.

2. mpz_set_uint64() had the same issue.

3. mpz_gcd() used mpz_init_set() which preserves the sign.
   GCD should work with absolute values since gcd(a,b) = gcd(|a|,|b|).
   With negative inputs, the sign would oscillate during mod operations,
   preventing the Euclidean algorithm from converging.

4. mpz_div_2exp() when e==0 and z==x would corrupt data by calling
   mpz_init_heap() which overwrites z->p before copying from x.

These bugs were discovered via ClusterFuzz with complex rational
number calculations.

Co-authored-by: Claude <noreply@anthropic.com>

2025-12-30 12:04:56 +09:00

..

bigint.c

bigint.c: fix GCD infinite loop and size handling bugs

2025-12-30 12:04:56 +09:00

bigint.h

misc: fix spelling

2025-05-14 02:01:02 +10:00