pytest-randomly passes an unclamped seed to entry-point reseeders, breaking thinc

[helmutholzler]

Python Version

3.11.15

pytest Version

8.4.2

Package Version

4.1.0

Description

_reseed passes an unclamped seed to entry-point reseeders, breaking thinc

Summary

pytest-randomly clamps the per-test seed for its own numpy call but passes the
raw, unclamped value to third-party pytest_randomly.random_seeder entry points.
thinc registers such a hook, which calls numpy.random.seed() without clamping, so
numpy rejects the value and every affected test errors at setup/teardown.

Environment

python           3.11.15
pytest           8.4.2
pytest-randomly  4.1.0
thinc            8.3.10
numpy            1.26.4

thinc arrives transitively (via spaCy); no thinc code is used by the tests themselves.

Reproduction

No project code required:

pip install pytest pytest-randomly thinc
printf 'def test_trivial():\n    assert True\n' > test_trivial.py
pytest test_trivial.py --randomly-seed=4000000000

Actual

2 errors (setup + teardown):

E   ValueError: Seed must be between 0 and 2**32 - 1
_mt19937.pyx:182: ValueError

Expected

1 passed.

The failure is seed-dependent — the same file passes with --randomly-seed=1.

Root cause

pytest_randomly/__init__.py::_reseed:

seed: int = config.getoption("randomly_seed") + offset   # can reach ~2**33
...
if have_numpy:
    np_random.seed(seed % 2**32)   # clamped for its OWN numpy handle
...
for reseed in entrypoint_reseeds:
    reseed(seed)                   # raw, UNCLAMPED, handed to third parties

randomly_seed is random.Random().getrandbits(32) and offset is crc32(nodeid),
each in [0, 2**32), so the sum reaches 2**33 - 2.

thinc.api:fix_random_seed then does:

def fix_random_seed(seed: int = 0) -> None:
    random.seed(seed)
    numpy.random.seed(seed)   # no clamping -> ValueError
    ...

Why it looks intermittent

A given test errors iff crc32(nodeid) > 2**32 - 1 - session_seed, so the share of
failing tests scales with the randomly-chosen session seed — anywhere from zero to the
entire suite, varying run to run. In a ~300-test suite this produced ~390 errors on one
run and none on another, which makes it hard to attribute.

Worked example:

seed=1           + crc32 offset=3719205183 = 3719205184   exceeds 2**32-1? False
seed=4000000000  + crc32 offset=3719205183 = 7719205183   exceeds 2**32-1? True

Suggested fix

One line, and consistent with the np_random.seed() call five lines above:

    reseed(seed % 2**32)

thinc could also clamp defensively inside fix_random_seed, but pytest-randomly
seems the better place: it already encodes numpy's constraint for its own call, and
fixing it here covers every registered seeder rather than just thinc's.

If the intent is instead that entry-point seeders may receive arbitrarily large ints,
documenting that contract on the pytest_randomly.random_seeder hook would help — the
current asymmetry (clamped internally, unclamped externally) reads as an oversight.

Workaround

In conftest.py:

def pytest_configure(config):
    try:
        import pytest_randomly
    except ImportError:
        return
    # [] rather than None: pytest-randomly skips its lazy entry-point load.
    pytest_randomly.entrypoint_reseeds = []

This drops the third-party reseeders while leaving pytest-randomly's own seeding of
random and numpy (correctly clamped) intact.