Add dry_run option to the accessibility scanner#229
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a dry_run mode to the accessibility-scanner GitHub Action so users can preview what a scan would do (open/reopen/close) via logs without mutating issues or writing to the gh-cache branch, while keeping the existing outputs wired up.
Changes:
- Introduces a new
dry_runinput in the root action and the nestedfileaction, and forwards it through. - Updates the
fileaction implementation to log intended filing actions and skip Octokit mutations whendry_runis enabled. - Documents the new option and adds a dedicated dry-run test suite.
Show a summary per file
| File | Description |
|---|---|
README.md |
Documents dry_run in the inputs table and usage example. |
FAQ.md |
Adds an FAQ entry explaining how to preview scanner behavior with dry_run. |
action.yml |
Adds dry_run input and gates cache writes / mutation steps when enabled. |
.github/actions/file/action.yml |
Declares dry_run input for the file sub-action. |
.github/actions/file/src/index.ts |
Implements dry-run behavior (log-only, no issue mutations). |
.github/actions/file/tests/dryRun.test.ts |
Adds tests validating dry-run does not mutate issues and logs expected output. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 6/6 changed files
- Comments generated: 3
| if (dryRun) { | ||
| core.info( | ||
| `[dry run] ${filings.length} findings: ${dryRunCounts.open} would open, ${dryRunCounts.reopen} would reopen, ${dryRunCounts.close} would close.`, | ||
| ) | ||
| } |
There was a problem hiding this comment.
^ useful feedback from the bot in this case.
in addition, some minor formatting feedback -- if we could use console.table here, it'd be a bit quicker to visually scan the different numbers (as well as scroll right to the bottom of the Action output and verify a dry run occurred)
There was a problem hiding this comment.
Switched to console.table and dropped the "findings" wording, much easier to scan now. Thanks!
Tracking issue with context: https://github.com/github/accessibility/issues/10757
Based on feedback from
@joseinthearena: with many open scanner issues there was no way to preview what a scan would do without actually filing, closing, or reopening issues — and no way to do so without mutating thegh-cachebranch. This adds adry_runinput that runs a normal scan and logs the issues that would be filed, while making no changes: no issues are opened, closed, or reopened, no Copilot assignment, and the cache is not written. Because dry runs don't update the cache, the next real run behaves exactly as if the dry run never happened.Changes
action.yml(root) — Newdry_runinput (defaultfalse); forwarded to thefilestep; skips theFix,Save screenshots,Copy results to cache path, andSave cached resultssteps whendry_runistrue.github/actions/file/action.yml— Newdry_runinput declaredfile/src/index.ts— Whendry_runis set, each filing is classified via the existingisNewFiling/isRepeatedFiling/isResolvedFilingguards and the intended action is logged (earlycontinue, so no Octokit calls are made); grouped/tracking-issue creation is skipped; a summary count is logged;filings_fileis still written so the output contract is unchangedREADME.md—dry_runinput documented in the inputs table and getting-started exampleFAQ.md— New entry explaining how to preview a scan and noting that dry runs don't write the cacheTest Updates
file/tests/dryRun.test.ts— asserts no issues are opened/closed/reopened in dry run, the correct[dry run] Would …lines and summary are logged, andfilings_fileis still written; includes a regression guard that mutations still occur whendry_runisfalseUsage
When
dry_runis not provided, the scanner behaves exactly as before.Notes
file-action logic is covered by the new tests. Theaction.ymlstep gating was verified by inspection but not run end-to-end — flagging for confirmation in CI.v3tag re-pointed) cut by a maintainer.