Skip to content

The coverage gate runs an unpinned coverage version #3402

Description

@adamtheturtle

What happens

The coverage gate installs coverage unpinned:

      - name: Require 100% Coverage
        id: coverage
        run: |
          uv tool install 'coverage[toml]'

          coverage combine
          coverage html --skip-covered --skip-empty
          ...

.github/workflows/test.yml:330. uv tool install resolves the latest release at run time, ignoring the project's own pin of coverage==7.15.3 in optional-dependencies.dev.

It is the only unpinned tool install across all seven workflows — everything else runs through uv run --extra=dev, which uses the pinned version.

Why it matters

Combine & check coverage is one of the five checks the require-builds ruleset requires on the default branch, alongside completion-ci, completion-lint, pages and autofix. It is also what enforces report.fail_under = 100.

So the behaviour of a required merge gate is determined by whatever coverage released most recently, not by anything in the repository. A change to coverage combine semantics, to how fail_under rounds, or to report.exclude_also handling would change what merges — with no commit and nothing to bisect. Dependabot bumps coverage in pyproject.toml and CI runs the tests against that bump, but the gate itself is not the version being bumped.

The matrix jobs that produce the data use the pinned version via uv run --extra=dev coverage run, so the producing and consuming sides of the coverage data can be different versions of coverage. That normally works, since the data format is stable across minor releases, but it is not guaranteed and it is not what the pin is there to express.

This is a low-probability problem. It is worth fixing because it is a one-line change and because everything else in this project is pinned exactly, which suggests the intent is already there.

Also in that step, minor

          # Report and write to summary.
          coverage report --format=markdown >> "$GITHUB_STEP_SUMMARY"

          # Report again and fail if under 100%.
          coverage report

GitHub runs run: blocks under bash -e, so the first coverage report already fails the step when coverage is under 100 — the second call is never reached in that case. The gate works; the comment describes an intent the shell does not implement. Either || true on the first call or a reworded comment would make it say what it does.

Suggested resolution

Use the pinned version, matching every other tool invocation in the workflows:

          uv run --extra=dev coverage combine
          uv run --extra=dev coverage html --skip-covered --skip-empty

That also removes the uv tool install step entirely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions