Skip to content

Add dry_run option to the accessibility scanner#229

Open
taarikashenafi wants to merge 6 commits into
github:mainfrom
taarikashenafi:dry-run-option
Open

Add dry_run option to the accessibility scanner#229
taarikashenafi wants to merge 6 commits into
github:mainfrom
taarikashenafi:dry-run-option

Conversation

@taarikashenafi

Copy link
Copy Markdown

Tracking issue with context: https://github.com/github/accessibility/issues/10757

Based on feedback from @joseinthearena: with many open scanner issues there was no way to preview what a scan would do without actually filing, closing, or reopening issues — and no way to do so without mutating the gh-cache branch. This adds a dry_run input that runs a normal scan and logs the issues that would be filed, while making no changes: no issues are opened, closed, or reopened, no Copilot assignment, and the cache is not written. Because dry runs don't update the cache, the next real run behaves exactly as if the dry run never happened.

Changes

  • action.yml (root) — New dry_run input (default false); forwarded to the file step; skips the Fix, Save screenshots, Copy results to cache path, and Save cached results steps when dry_run is true
  • .github/actions/file/action.yml — New dry_run input declared
  • file/src/index.ts — When dry_run is set, each filing is classified via the existing isNewFiling / isRepeatedFiling / isResolvedFiling guards and the intended action is logged (early continue, so no Octokit calls are made); grouped/tracking-issue creation is skipped; a summary count is logged; filings_file is still written so the output contract is unchanged
  • README.mddry_run input documented in the inputs table and getting-started example
  • FAQ.md — New entry explaining how to preview a scan and noting that dry runs don't write the cache

Test Updates

  • Added file/tests/dryRun.test.ts — asserts no issues are opened/closed/reopened in dry run, the correct [dry run] Would … lines and summary are logged, and filings_file is still written; includes a regression guard that mutations still occur when dry_run is false

Usage

- uses: github/accessibility-scanner@v3
  with:
    urls: |
      https://example.com
    repository: owner/repo
    token: ${{ secrets.GH_TOKEN }}
    cache_key: cached_results.json
    dry_run: true # Scan and log what would be filed without creating/closing issues or writing the cache

When dry_run is not provided, the scanner behaves exactly as before.

Notes

  • Design decision: cache restore is kept (read-only) so dry-run logs can accurately distinguish "would open (new)" from "would reopen (existing)"; only the cache save is skipped.
  • The file-action logic is covered by the new tests. The action.yml step gating was verified by inspection but not run end-to-end — flagging for confirmation in CI.
  • Per the tracking issue's second acceptance criterion, this will need a new minor release (and the v3 tag re-pointed) cut by a maintainer.

Copilot AI review requested due to automatic review settings June 16, 2026 20:48
@taarikashenafi taarikashenafi requested a review from a team as a code owner June 16, 2026 20:48
@taarikashenafi taarikashenafi self-assigned this Jun 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a dry_run mode to the accessibility-scanner GitHub Action so users can preview what a scan would do (open/reopen/close) via logs without mutating issues or writing to the gh-cache branch, while keeping the existing outputs wired up.

Changes:

  • Introduces a new dry_run input in the root action and the nested file action, and forwards it through.
  • Updates the file action implementation to log intended filing actions and skip Octokit mutations when dry_run is enabled.
  • Documents the new option and adds a dedicated dry-run test suite.
Show a summary per file
File Description
README.md Documents dry_run in the inputs table and usage example.
FAQ.md Adds an FAQ entry explaining how to preview scanner behavior with dry_run.
action.yml Adds dry_run input and gates cache writes / mutation steps when enabled.
.github/actions/file/action.yml Declares dry_run input for the file sub-action.
.github/actions/file/src/index.ts Implements dry-run behavior (log-only, no issue mutations).
.github/actions/file/tests/dryRun.test.ts Adds tests validating dry-run does not mutate issues and logs expected output.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 6/6 changed files
  • Comments generated: 3

Comment on lines +159 to +163
if (dryRun) {
core.info(
`[dry run] ${filings.length} findings: ${dryRunCounts.open} would open, ${dryRunCounts.reopen} would reopen, ${dryRunCounts.close} would close.`,
)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ useful feedback from the bot in this case.

in addition, some minor formatting feedback -- if we could use console.table here, it'd be a bit quicker to visually scan the different numbers (as well as scroll right to the bottom of the Action output and verify a dry run occurred)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to console.table and dropped the "findings" wording, much easier to scan now. Thanks!

Comment thread .github/actions/file/tests/dryRun.test.ts Outdated
Comment thread .github/actions/file/tests/dryRun.test.ts Outdated
@taarikashenafi taarikashenafi removed their assignment Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants