Skip to content

fix(scripts): tolerate an unusable integration.json in the Python helper - #3785

Merged
mnriem merged 2 commits into
github:mainfrom
jawwad-ali:fix/python-helper-integration-json-guards
Jul 30, 2026
Merged

fix(scripts): tolerate an unusable integration.json in the Python helper#3785
mnriem merged 2 commits into
github:mainfrom
jawwad-ali:fix/python-helper-integration-json-guards

Conversation

@jawwad-ali

Copy link
Copy Markdown
Contributor

Problem

get_invoke_separator() in scripts/python/common.py indexes the parsed JSON directly:

state = json.loads(integration_json.read_text(encoding="utf-8"))
key = state.get("default_integration") or state.get("integration") or ""   # AttributeError
...
except (OSError, json.JSONDecodeError):    # covers neither escape below
    pass

Two shapes escape that tuple, while both of its twins fall back to ".":

  1. A non-mapping top level ([], "forge", 42, true, null) is valid JSON, so json.JSONDecodeError never fires and .get() raises AttributeError.
  2. A non-UTF-8 file raises UnicodeDecodeError — a ValueError, not an OSError. Realistic on Windows, where PowerShell 5.1's Out-File / > default to UTF-16.

Measured on main — the Python helper crashed on 6 of 7 inputs while bash and PowerShell 5.1 (tested natively) returned "." for every one:

.specify/integration.json python bash pwsh 5.1
{"default_integration":"forge"} '.' . .
[] AttributeError . .
[{"a":1}] AttributeError . .
"forge" AttributeError . .
42 AttributeError . .
null AttributeError . .
UTF-16 file UnicodeDecodeError . .

The bash twin keeps its separator="." default when jq → python3 → awk all fail to parse; the PowerShell twin likewise returns ".".

Fix

Split the parse out of the lookup, complete the exception tuple, and guard the top-level shape — matching read_feature_json_feature_directory in this same module, which already does exactly this:

try:
    state = json.loads(integration_json.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError):
    return "."
if not isinstance(state, dict):
    return "."

So this makes get_invoke_separator consistent both with its cross-language twins and with its neighbour in the same file — it was the odd one out.

Verification

  • New TestGetInvokeSeparatorTolerance: 6 parametrized non-mapping shapes + the UTF-16 case → 7 fail before, pass after.
  • test_hyphen_separator_is_still_honoured — regression guard that the real feature (invoke_separator: "-" for e.g. droid/forge) keeps working; passes before and after.
  • tests/test_check_prerequisites_python_parity.py: 19 passed, 8 skipped (the skips are the pre-existing bash/pwsh legs gated off on this machine; baseline was 11 passed, 8 skipped).
  • Tests are plain unit tests (no bash/pwsh gating), so they run on every CI platform.
  • uvx ruff@0.15.0 check src tests scripts clean.

No behaviour change for a well-formed integration.json: every path that previously returned a value is byte-for-byte identical.


AI-assisted: authored with Claude Code. I measured all three implementations side by side (PowerShell 5.1 natively on Windows) before writing the fix, and matched the guard style already present in the same module.

@jawwad-ali
jawwad-ali requested a review from mnriem as a code owner July 28, 2026 11:27
`get_invoke_separator()` in scripts/python/common.py indexed the parsed JSON
directly, so two shapes escaped its `except (OSError, json.JSONDecodeError)`
while BOTH of its twins fall back to "." for them:

  * A non-mapping top level is valid JSON, so JSONDecodeError never fires and
    `state.get(...)` raised AttributeError.
  * A non-UTF-8 file raises UnicodeDecodeError -- a ValueError, not an OSError.
    Realistic on Windows, where PowerShell 5.1's Out-File/`>` default to UTF-16.

Measured on main -- 6 of 7 inputs crashed the Python helper while bash and
PowerShell 5.1 returned "." for every one:

    input                             python        bash   pwsh 5.1
    {"default_integration":"forge"}   '.'           .      .
    []                                AttributeError .      .
    "forge"                           AttributeError .      .
    42                                AttributeError .      .
    null                              AttributeError .      .
    UTF-16 file                       UnicodeDecodeError .  .

Split the parse out of the lookup, complete the exception tuple, and guard the
top-level shape -- matching `read_feature_json_feature_directory` in this same
module, which already does exactly this. The hyphen-separator feature is
unchanged (regression test included).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jawwad-ali
jawwad-ali force-pushed the fix/python-helper-integration-json-guards branch from ef83ff3 to 3c1fa08 Compare July 28, 2026 14:56
@mnriem
mnriem requested a review from Copilot July 28, 2026 20:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves Python helper parity by safely handling unusable integration.json files.

Changes:

  • Handles decoding errors and non-object JSON inputs.
  • Adds regression tests for malformed shapes, UTF-16 input, and valid separators.
Show a summary per file
File Description
scripts/python/common.py Adds safe parsing and fallback behavior.
tests/test_check_prerequisites_python_parity.py Covers invalid inputs and valid separator preservation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment thread scripts/python/common.py Outdated

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

read_feature_json_feature_directory is defined at line 81, above
get_invoke_separator, so "below" sent maintainers the wrong way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (1)

tests/test_check_prerequisites_python_parity.py:404

  • spec_from_file_location(...) can return None or a spec with loader=None (e.g., if COMMON_PY is wrong/missing). As written, this would fail with a less actionable AttributeError at spec.name or spec.loader.exec_module(...). Consider adding explicit guards (e.g., assert spec is not None and spec.loader is not None) with a clear message so test failures point directly to the root cause.
        spec = importlib.util.spec_from_file_location("_speckit_common_py", COMMON_PY)
        module = importlib.util.module_from_spec(spec)
        # Register before exec: the module defines @dataclass types, and
        # dataclasses resolves cls.__module__ through sys.modules.
        sys.modules[spec.name] = module
        try:
            spec.loader.exec_module(module)
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Low

@mnriem
mnriem merged commit 5e2f9bc into github:main Jul 30, 2026
14 checks passed
@mnriem

mnriem commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants