feat: improve agent/CI ergonomics — capabilities command, non-interactive mode, JSON error envelopes, exit-code contract#8300
Open
DavidWells wants to merge 11 commits into
Open
feat: improve agent/CI ergonomics — capabilities command, non-interactive mode, JSON error envelopes, exit-code contract#8300DavidWells wants to merge 11 commits into
DavidWells wants to merge 11 commits into
Conversation
…ocumented --site flag
Agent-ergonomics audit R-002 + R-010 (pass 1).
- env:* with --json in an unlinked dir now emits {"error":{"code":"NOT_LINKED",...}} on stdout and exits 1 (was: exit 0, empty output)
- prose errors go to stderr
- --site now resolves via the existing base-command siteInfo path (was: silently ignored)
…tate Agent-ergonomics audit R-004 (pass 1). - unlinked / logged-out / expired-token paths emit JSON envelopes with NOT_LINKED / NOT_LOGGED_IN codes (was: exit 1 with zero stdout) - non-JSON mode semantics unchanged; happy-path JSON keys additive only
Agent-ergonomics audit R-011 (pass 1). - JSON.parse failures name --data, echo truncated input, show a concrete example, note key=value is not accepted - missing-path-variable errors list the method's required variables
…piped --help
Agent-ergonomics audit R-001 + R-005 + R-008 (pass 1).
- R-001: namespace parents (sites/env/functions/blobs/teams/agents/completion/open) now exit 1 with a colon-form suggestion on stderr instead of printing parent help with exit 0 ('netlify sites delete x' no longer fake-succeeds)
- R-005: unknown flags get levenshtein did-you-mean (distance<=2); root-level errors name owning commands; unknown-command suggestions moved stdout->stderr
- R-008: help formatter pads term/description with real whitespace in non-TTY output (was ANSI-color-only separator producing '--dryDry run:')
…orruption guard, capabilities command Agent-ergonomics audit R-003 + R-006 + R-007/R-009 + R-014 + R-027 (pass 1). - R-006: src/utils/exit-codes.ts documents 0/1/2/4; unknown command/option paths now exit 2; dictionary in root help epilogue - R-014: NO_COLOR (no-color.org) disables colors regardless of TTY - R-027: global --non-interactive fails would-be prompts with exit 4 naming the missing flag/env var (extends CI=true gate) - R-007/R-009: corrupt .netlify/state.json / global config backed up to <file>.corrupt.<mtime> with stderr warning before dev-utils would silently reset it - R-003: netlify capabilities [--json] emits a deterministic machine-readable manifest (71 commands, flags, json support, exit codes, env vars, config paths)
…ge errors exit 2, capabilities full-path names, colon-form hints keep flags
Agent-ergonomics audit phase 7 round 1.
- prettyjson render sites (status, status:hooks, deploy, sites:create, watch) honor NO_COLOR/--json
- USAGE_ERROR_CODES covers missingArgument/optionMissingArgument/invalidArgument/etc -> exit 2
- capabilities manifest uses full command paths (database status), deduped, deterministic
- space-form suggestions preserve trailing flags ('netlify env list --json' -> suggests 'netlify env:list --json')
… stop reading early Agent-ergonomics audit phase 7 round 2. 'netlify capabilities | head' hit the uncaughtException crash banner; stdout error listener now treats EPIPE as success.
Agent-ergonomics audit phase 7 round 3. localeCompare collation varies with LANG/LC_ALL; contract-versioned manifest now sorts by code units and the determinism test asserts the same comparator.
Agent-ergonomics audit phase 7 round 5. capabilities needs no project context; in a multi-package workspace under CI/--non-interactive the preAction workspace prompt blocked it with exit 4 and empty stdout.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (8)
🔗 Linked repositories identifiedCodeRabbit considers these linked repositories for cross-repo context during reviews:
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis PR introduces structured exit code handling (0, 1, 2, 4), non-interactive mode support via Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed due to a network error. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📊 Benchmark resultsComparing with 392af83
|
DavidWells
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/unit/commands/env/utils.test.ts (1)
1-87:⚠️ Potential issue | 🟠 Major | ⚡ Quick winFix CI formatting failure in this test file.
Format check is failing for this file in CI (
Prettier --check). Please run Prettier on this file and commit the result so the format job passes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/unit/commands/env/utils.test.ts` around lines 1 - 87, The file tests/unit/commands/env/utils.test.ts is failing Prettier --check; reformat it with your project's formatter (e.g. run prettier --write tests/unit/commands/env/utils.test.ts or the repo's format script) and commit the changes so CI passes; you can verify the sections around the getEnvSiteId and failNotLinked tests (and the vi.mock block at the top) are properly spaced/quoted and end-of-line/newline rules conform to Prettier.Source: Pipeline failures
🧹 Nitpick comments (2)
src/commands/main.ts (1)
214-216: ⚡ Quick winConsolidated: remove behavior-describing comments across
src/commands/main.tsandsrc/commands/capabilities/capabilities.ts.Both files introduce inline comments that explain runtime behavior instead of making intent self-evident through naming/structure, which conflicts with the same repository rule.
As per coding guidelines, "
**/*.{ts,tsx,js,jsx}: Never write comments on what the code does; make the code clean and self-explanatory instead".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/commands/main.ts` around lines 214 - 216, Remove the inline behavior-describing comments in src/commands/main.ts (around the isInteractive() check) and src/commands/capabilities/capabilities.ts and instead make the intent explicit via clearer names or small helper functions; for example, replace a naked if (!isInteractive()) comment block by extracting a well-named function like handleNonInteractiveOutput or use a constant named DIAGNOSTICS_TO_STDERR to express why stderr is used, and in capabilities.ts rename or split ambiguous code into self-describing functions/variables so the code reads without explanatory comments.Source: Coding guidelines
src/utils/command-helpers.ts (1)
36-42: ⚡ Quick winConsolidated: remove newly added behavior-describing comments across
src/utils/command-helpers.ts,src/utils/scripted-commands.ts, andsrc/utils/command-error-handler.ts.The shared root cause is the same style-rule breach: new TS/JS comments that narrate behavior rather than letting names/structure carry intent.
As per coding guidelines,
**/*.{ts,tsx,js,jsx}: Never write comments on what the code does; make the code clean and self-explanatory instead.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/utils/command-helpers.ts` around lines 36 - 42, Remove the behavior-describing comments that narrate what the code does and instead rely on clear names and structure: delete the comment above shouldDisableColors and the one above chalk/prettyJsonRenderOptions in src/utils/command-helpers.ts, and likewise remove similar explanatory comments in src/utils/scripted-commands.ts and src/utils/command-error-handler.ts; leave function/constant names (shouldDisableColors, chalk, prettyJsonRenderOptions and any exported handlers/commands) intact and ensure any remaining comments are focused on why (intent) or non-obvious implementation details only.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/commands/api/api.ts`:
- Around line 65-75: The current cast of methods to a specific shape is unsafe
and can throw when building the missing-path error message; add a runtime type
guard (e.g., isApiMethod(obj): obj has operationId:string and optional
parameters.path:Record<string,unknown>) and use it to filter/map methods before
calling find by apiMethodName, then safely read parameters?.path and guard
Object.keys with a default empty object; update the block that computes
apiMethods, pathVariables and requiredNames (references: methods, apiMethods,
apiMethodName, parameters.path, logAndThrowError) to use these checks so the
error path never assumes the shape.
In `@src/commands/main.ts`:
- Line 207: Run Prettier to format src/commands/main.ts and commit the changes
so CI passes; specifically reformat the offending write call that uses
process.stderr.write with chalk and BANG (the expression containing
`process.stderr.write(\` ${chalk.yellow(BANG)} Warning:
${chalk.yellow(command.args[0])} is not a ${command.name()} command.\n\`)`) and
the similar hunk around `command.args[0]`/`command.name()` so spacing, quotes,
and line breaks match project Prettier rules; after running Prettier verify the
file is saved and staged, then push the formatted file.
In `@src/utils/config-guard.ts`:
- Around line 38-41: Wrap the synchronous filesystem operations that build and
write the backup (the statSync(...) call used to compute backupPath and the
copyFileSync(...) call) in try/catch so any EIO/ENOENT/EPERM errors do not
propagate and abort initialization; on statSync failure fall back to a safe
timestamp (e.g., Date.now()) when composing backupPath, and on copyFileSync
failure catch and log a non-fatal warning (using the module/logger available)
and continue without throwing. Ensure you keep the existsSync(...) check but
still guard copyFileSync with try/catch so removal/permission changes between
existsSync and copyFileSync won’t crash the process.
---
Outside diff comments:
In `@tests/unit/commands/env/utils.test.ts`:
- Around line 1-87: The file tests/unit/commands/env/utils.test.ts is failing
Prettier --check; reformat it with your project's formatter (e.g. run prettier
--write tests/unit/commands/env/utils.test.ts or the repo's format script) and
commit the changes so CI passes; you can verify the sections around the
getEnvSiteId and failNotLinked tests (and the vi.mock block at the top) are
properly spaced/quoted and end-of-line/newline rules conform to Prettier.
---
Nitpick comments:
In `@src/commands/main.ts`:
- Around line 214-216: Remove the inline behavior-describing comments in
src/commands/main.ts (around the isInteractive() check) and
src/commands/capabilities/capabilities.ts and instead make the intent explicit
via clearer names or small helper functions; for example, replace a naked if
(!isInteractive()) comment block by extracting a well-named function like
handleNonInteractiveOutput or use a constant named DIAGNOSTICS_TO_STDERR to
express why stderr is used, and in capabilities.ts rename or split ambiguous
code into self-describing functions/variables so the code reads without
explanatory comments.
In `@src/utils/command-helpers.ts`:
- Around line 36-42: Remove the behavior-describing comments that narrate what
the code does and instead rely on clear names and structure: delete the comment
above shouldDisableColors and the one above chalk/prettyJsonRenderOptions in
src/utils/command-helpers.ts, and likewise remove similar explanatory comments
in src/utils/scripted-commands.ts and src/utils/command-error-handler.ts; leave
function/constant names (shouldDisableColors, chalk, prettyJsonRenderOptions and
any exported handlers/commands) intact and ensure any remaining comments are
focused on why (intent) or non-obvious implementation details only.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 618cda5f-b77a-4e1d-aba8-efd4d34481cd
📒 Files selected for processing (44)
src/commands/agents/agents.tssrc/commands/api/api.tssrc/commands/base-command.tssrc/commands/blobs/blobs.tssrc/commands/capabilities/capabilities.tssrc/commands/capabilities/index.tssrc/commands/completion/index.tssrc/commands/deploy/deploy.tssrc/commands/env/env-clone.tssrc/commands/env/env-get.tssrc/commands/env/env-import.tssrc/commands/env/env-list.tssrc/commands/env/env-set.tssrc/commands/env/env-unset.tssrc/commands/env/env.tssrc/commands/env/utils.tssrc/commands/functions/functions.tssrc/commands/link/link.tssrc/commands/main.tssrc/commands/open/index.tssrc/commands/sites/sites-create.tssrc/commands/sites/sites.tssrc/commands/status/status-hooks.tssrc/commands/status/status.tssrc/commands/teams/teams.tssrc/commands/watch/watch.tssrc/utils/command-error-handler.tssrc/utils/command-helpers.tssrc/utils/config-guard.tssrc/utils/exit-codes.tssrc/utils/run-program.tssrc/utils/scripted-commands.tstests/integration/commands/dev/dev-miscellaneous.test.tstests/unit/commands/api/api.test.tstests/unit/commands/capabilities/capabilities.test.tstests/unit/commands/env/env-get.test.tstests/unit/commands/env/utils.test.tstests/unit/commands/help-format.test.tstests/unit/commands/main-suggestions.test.tstests/unit/commands/status/status.test.tstests/unit/utils/config-guard.test.tstests/unit/utils/exit-codes.test.tstests/unit/utils/no-color.test.tstests/unit/utils/scripted-commands.test.ts
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
netlify/blueprints(manual)
Comment on lines
+65
to
+75
| if (error_ instanceof Error && error_.message.includes('Missing required path variable')) { | ||
| const apiMethods = methods as { operationId: string; parameters: { path?: Record<string, unknown> } }[] | ||
| const pathVariables = apiMethods.find((method) => method.operationId === apiMethodName)?.parameters.path ?? {} | ||
| const requiredNames = Object.keys(pathVariables).join(', ') | ||
| return logAndThrowError( | ||
| `${error_.message} | ||
| The ${chalk.cyanBright('--data')} flag must include the path variable(s) required by ${apiMethodName}${ | ||
| requiredNames ? `: ${requiredNames}` : '' | ||
| }, e.g. --data '{"site_id":"123456"}'`, | ||
| ) | ||
| } |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Verify the type definition of methods from `@netlify/api`
# Check if `@netlify/api` has type definitions
rg -n "export.*methods" node_modules/@netlify/api --type=ts -A 5
# Also check for the type definition in .d.ts files
fd -e d.ts . node_modules/@netlify/api --exec rg -n "methods" -A 10Repository: netlify/cli
Length of output: 245
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the relevant section around the snippet
sed -n '1,140p' src/commands/api/api.ts | nl -ba | sed -n '40,120p'
# Find how `methods` is imported/typed in this file
rg -n "import .*methods|from ['\"]`@netlify/api`['\"]" src/commands/api/api.ts
# Check other usages of `methods` in the repo (to understand assumed runtime shape)
rg -n "\bmethods\b" src | head -n 200Repository: netlify/cli
Length of output: 98
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show relevant section around the snippet without `nl`
sed -n '40,120p' src/commands/api/api.ts | sed -n '1,80p'
# Find how `methods` is imported/typed in this file
rg -n "import .*methods|from ['\"]`@netlify/api`['\"]" src/commands/api/api.ts
# Check other usages of `methods` in this repo (limits output)
rg -n "\bmethods\b" src | head -n 200Repository: netlify/cli
Length of output: 2933
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the code around the snippet and the earlier usage of `methods`
sed -n '1,120p' src/commands/api/api.ts
sed -n '120,220p' src/commands/api/api.ts
# Inspect `@netlify/api` typing for `methods` and related types
sed -n '1,120p' node_modules/@netlify/api/lib/index.d.ts
# Search for more specific types/shapes around methods/operationId/parameters.path
rg -n "operationId|parameters|path\\??" node_modules/@netlify/api/lib/index.d.ts
rg -n "interface .*method|type .*method|declare const methods" node_modules/@netlify/api/lib -SRepository: netlify/cli
Length of output: 5169
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "\bmethods\.|methods\b.*operationId|as \{ operationId" src
rg -n "from '`@netlify/api`'" srcRepository: netlify/cli
Length of output: 2267
Guard against unchecked methods shape when building the “missing path variable” message
@netlify/api types methods as any[], so the cast to { operationId: string; parameters: { path?: Record<string, unknown> } }[] is unchecked and can fail at runtime if the element shape differs. Add runtime guards/safe access before reading operationId and parameters.path (or extract via a small helper type-guard) so this error-path doesn’t crash.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/commands/api/api.ts` around lines 65 - 75, The current cast of methods to
a specific shape is unsafe and can throw when building the missing-path error
message; add a runtime type guard (e.g., isApiMethod(obj): obj has
operationId:string and optional parameters.path:Record<string,unknown>) and use
it to filter/map methods before calling find by apiMethodName, then safely read
parameters?.path and guard Object.keys with a default empty object; update the
block that computes apiMethods, pathVariables and requiredNames (references:
methods, apiMethods, apiMethodName, parameters.path, logAndThrowError) to use
these checks so the error path never assumes the shape.
Comment on lines
+38
to
+41
| const backupPath = `${filePath}.corrupt.${String(Math.round(statSync(filePath).mtimeMs))}` | ||
| if (!existsSync(backupPath)) { | ||
| copyFileSync(filePath, backupPath) | ||
| } |
Contributor
There was a problem hiding this comment.
Handle backup I/O failures without aborting command initialization.
On Line 38 through Line 41, statSync()/copyFileSync() can throw in the corrupt-file path (e.g., readonly home/config dir or file removed between read/stat). That exception escapes and can crash command startup before recovery logic runs.
Suggested fix
} catch {
- const backupPath = `${filePath}.corrupt.${String(Math.round(statSync(filePath).mtimeMs))}`
- if (!existsSync(backupPath)) {
- copyFileSync(filePath, backupPath)
- }
- writeCorruptFileWarning(filePath, backupPath, recoveryHint)
- return backupPath
+ try {
+ const backupPath = `${filePath}.corrupt.${String(Math.round(statSync(filePath).mtimeMs))}`
+ if (!existsSync(backupPath)) {
+ copyFileSync(filePath, backupPath)
+ }
+ writeCorruptFileWarning(filePath, backupPath, recoveryHint)
+ return backupPath
+ } catch {
+ process.stderr.write(` ${chalk.yellow(BANG)} Warning: ${filePath} contains malformed JSON and could not be backed up.\n`)
+ process.stderr.write(` ${chalk.yellow(BANG)} Repair the file manually, or ${recoveryHint}.\n`)
+ return undefined
+ }
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/utils/config-guard.ts` around lines 38 - 41, Wrap the synchronous
filesystem operations that build and write the backup (the statSync(...) call
used to compute backupPath and the copyFileSync(...) call) in try/catch so any
EIO/ENOENT/EPERM errors do not propagate and abort initialization; on statSync
failure fall back to a safe timestamp (e.g., Date.now()) when composing
backupPath, and on copyFileSync failure catch and log a non-fatal warning (using
the module/logger available) and continue without throwing. Ensure you keep the
existsSync(...) check but still guard copyFileSync with try/catch so
removal/permission changes between existsSync and copyFileSync won’t crash the
process.
- env:clone unlinked now exits 1 with teaching error (was exit 0 + snapshot) - didyoumean suggestions moved to stderr; assert reason.stderr - help snapshot includes the new exit-codes epilogue
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
AI agents are now heavy users of
netlify— and they drive it through pipes, parse its output, and branch on its exit codes. We ran a structured agent-ergonomics audit of every command, flag, and error path, then fixed the 13 highest-impact issues. Everything is additive: no existing flag, output field, or success path changes behavior.netlify capabilities --json— machine-readable manifest of every command, flag, exit code, and env var--non-interactiveglobal flag — prompts fail fast with a clear message instead of hanging headless sessionsnetlify sites delete x(space instead of colon) no longer prints help and exits 0 — it errors with the correctsites:deletesuggestionstatus --jsonandenv:* --jsonalways emit JSON, even on auth/link errors — no more zero-byte stdout0ok ·1error ·2usage error ·4input needed--helpis parseable again,NO_COLORis honored, typos get better suggestions, corrupt state files are backed up instead of silently resetMeasured with a fresh-context agent on 8 realistic tasks before/after: 5/8 → 8/8 task success, and the most common workflow (reading env vars) went from 5 round-trips to 1. Each change ships with unit tests; validation: typecheck ✓ lint ✓ 434 unit tests ✓.
The changes
1. Space-form subcommands no longer fake-succeed
Why:
netlify sites delete my-siteused to print thesiteshelp text and exit 0. A script — or an agent doing cleanup — reads that as "site deleted." Nothing happened. This was the single most dangerous false-success in the CLI.Better for agents: exit 0 now always means the command actually ran. The error names the exact colon-form command (flags preserved), so an agent self-corrects in one step.
Before / after
Before — exit 0, parent help on stdout:
After — exit 2, stderr:
Also covers
netlify env list,netlify functions invoke hello, etc. Trailing flags survive the suggestion (netlify env list --json→netlify env:list --json).2.
--jsonalways means JSON — even on errorsWhy:
netlify status --jsonin an unlinked folder emitted zero bytes of stdout.netlify env:get FOO --jsonemitted zero bytes with exit 0 — indistinguishable from "the variable is empty," which is how empty secrets end up in configs.Better for agents: any consumer of
--jsoncan now alwaysJSON.parsestdout and branch on a stableerror.code(NOT_LINKED,NOT_LOGGED_IN) with a suggested fix, instead of guessing from silence.Before / after
Before — exit 0, no output at all.
After — exit 1, stdout:
{ "error": { "code": "NOT_LINKED", "message": "No project id found, please run inside a project folder or `netlify link`", "fix": "netlify link" } }Before — exit 1, zero bytes of stdout (can't tell "not logged in" from "not linked").
After — stdout always carries the answer:
{ "loggedIn": true, "linked": false, "account": { "Name": "…", "Email": "…" }, ... }Non-JSON output and exit semantics are unchanged. The
env:*commands also now honor their documented--siteflag (it was silently ignored).3. New:
netlify capabilities --jsonWhy: the only way to learn what this CLI can do programmatically was ~56 separate
--helpinvocations at ~2.5s each — and the help text isn't machine-readable.Better for agents: one call returns the full contract: all 71 commands with their flags, which support
--json, the exit-code dictionary, relevant env vars, and config file locations. Output is deterministic (byte-identical across runs and locales), pipe-safe, and works in monorepos and CI.Output shape
{ "contract_version": "1", "cli_version": "26.1.0", "commands": [ { "name": "env:list", "description": "…", "flags": [ … ], "json_output": true, "mutates": null }, … ], "global_flags": [ … ], "exit_codes": { "0": "success", "1": "general error", "2": "usage error", "4": "non-interactive prompt blocked" }, "env_vars": [ { "name": "NETLIFY_AUTH_TOKEN", "description": "…" }, … ], "config_files": [ { "path": "netlify.toml", "scope": "project" }, … ] }4. New:
--non-interactive— prompts fail fast instead of hangingWhy: a hidden interactive prompt in a headless session means an agent or CI job hangs forever with no diagnostic.
Better for agents: with
--non-interactive(orCI=true, which already gated some prompts), any would-be prompt exits immediately with code4and a message naming the exact flags that would answer it.Before / after
Before — exit 1:
unknown option '--non-interactive'(and without it, CI hangs on the prompt).After — exit 4, stderr:
5. Documented exit codes; usage errors are distinguishable
Why: every failure exited
1— a typo'd flag looked identical to a failed deploy. Nothing documented the contract.Better for agents:
0ok /1error /2usage error /4input needed, documented in the--helpfooter and incapabilities --json. Retry logic can finally distinguish "I made a mistake" (fix the command) from "the operation failed" (maybe retry). New codes only appear on paths that already failed — nothing that exited 0 changes.Before / after
Before: all exit 1.
After: all exit 2; messages unchanged. Unknown-command suggestions also moved from stdout to stderr, keeping stdout data-clean.
6. Errors teach the fix
Why:
netlify api getSite --data 'site_id=123'leaked a rawSyntaxError: Unexpected token 's'…— we found a real agent session with 18 consecutive failures on exactly this confusion.Better for agents: the error names the flag, echoes the bad input, and shows a correct example — one failure instead of eighteen.
Before / after
Before:
After:
Related: root-level unknown flags now name the commands that own the flag (
netlify --jsno→'--json' is a flag of: agents:create, agents:list, …). Subcommand-level suggestions already existed and are unchanged.7. Piped
--helpis parseable againWhy: the help renderer separated flag names from descriptions with only an ANSI color code. Pipe it (which is how every agent reads help) and you get
--dryDry run: show instructions…— corrupted data on the CLI's primary self-documentation surface.Better for agents: real whitespace padding on every row, in every terminal.
Before / after
Before:
--dryDry run: show instructions without running them (default: false)After:
--dry Dry run: show instructions without running them8.
NO_COLORhonoredWhy: the informal standard (no-color.org) wasn't checked anywhere; agents that set it still got ANSI noise to strip.
Better for agents:
NO_COLOR=1produces zero escape sequences across help, status, and all prettyjson output (29 → 0 inenv --helpunder a TTY).9. Corrupt state files are preserved, not silently reset
Why: a malformed
.netlify/state.json(or global config) was silently ignored — and could be silently wiped, losing the project link with no trace.Better for agents: the corrupt file is backed up to
<file>.corrupt.<timestamp>with a stderr warning naming both paths, then the command proceeds. Nothing is lost, and the failure is diagnosable.Compatibility
capabilitiescommand,--non-interactiveglobal flag, JSON error envelopes (additive fields), exit codes 2/4 on previously-exit-1 error paths.Testing
bin/run.jsexist in the audit workspace and can be ported intotests/integrationif wanted.How this was produced
A structured agent-ergonomics audit: 295 CLI surfaces inventoried and scored on an 11-dimension rubric (two independent scorers + tiebreaks), a 510-case wrong-invocation corpus run against the binary, mining of real agent session logs for failure frequency, recommendations cross-reviewed by two other models, and before/after validation by a fresh-context agent attempting 8 canonical tasks (5/8 → 8/8). Full artifacts (scorecards, corpus, transcripts, playbook) are available outside this PR.