fix(check): keep evaluating a motion spec past an ambiguous selector - #2805
Draft
xuanruli wants to merge 1 commit into
Draft
fix(check): keep evaluating a motion spec past an ambiguous selector#2805xuanruli wants to merge 1 commit into
xuanruli wants to merge 1 commit into
Conversation
One selector matching more than one element used to disable the whole motion audit. `planMotionSampling` set `times = []`, and the evaluate step was skipped a second time by a `motionIssues.length === 0` guard, so every other assertion in the sidecar went unevaluated while the report still read `enabled: true`. Measured on two fixtures with identical HTML: the spec without an ambiguous selector reports `samples=121` and its missing-selector finding; adding one ambiguous selector drops it to `samples=0` and that finding disappears. Only the assertions naming an ambiguous selector are dropped now. The rest are sampled and evaluated, and their findings are appended to the preflight ones rather than replacing them. Filtering a spec down must not be silent either — `samples === 0` was the only signal that part of a spec went unevaluated, and sampling again removes it. The ambiguity finding now names what it cost, so the human report says so too. The selector-to-assertion mapping lives in `assertionTargets` in motionAudit.ts, where `collectSamplingTargets` already folds over it, behind an exhaustive switch: a new assertion kind that forgets to declare its targets fails to compile instead of having its ambiguous selector reported and then evaluated against an arbitrary first match. This is strictly louder and never newly-breaking: reaching this path means a hardcoded-error `motion_selector_ambiguous` was already present, so the run had already failed. All 12 motion fixtures keep their verdicts; 13 registry examples are byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
One selector matching more than one element disabled the whole motion audit.
planMotionSamplingsettimes = [], and the evaluate step was skipped a second time by amotionIssues.length === 0guard — so every other assertion in the sidecar went unevaluated whilereport.motion.enabledstill readtrue.Two fixtures with identical HTML:
mainm-selectorsamples=121, reportsmotion_selector_missing #typo-does-not-existm-ambig.itemsamples=0, reports onlymotion_selector_ambiguous— the missing-selector finding disappearsA single typo silently switched off the rest of the checks.
Change
Only the assertions naming an ambiguous selector are dropped. The rest are sampled and evaluated, and their findings are appended to the preflight ones instead of replacing them.
Filtering must not be silent either.
samples === 0was the only machine-readable signal that part of a spec went unevaluated, and sampling again removes it — that would have traded one silent skip for another, one level down. The ambiguity finding now names what it cost:The selector-to-assertion mapping is single-sourced.
assertionTargetslives inmotionAudit.tsnext tocollectSamplingTargets, which now folds over it, behind an exhaustiveswitch. Verified: adding a fifthMotionAssertionkind produces a compile error rather than a silent gap — without that, a new kind would have its ambiguous selector reported as ambiguous and still evaluated against an arbitrary first match, resurrecting exactly this bug for the new kind.Verification
m-ambignow reportssamples=121with both findings. The other 11 motion fixtures keep their verdicts byte-for-byte; 13 registry examples are identical (they carry no sidecar, so the motion section is disabled there). 2150 tests pass — the one failure (src/telemetry/agent_runtime.test.ts, agent-env detection) reproduces on cleanHEADand is environment-dependent. oxlint / oxfmt /tsc --noEmitclean.Reviewers independently confirmed the parts I was least sure of:
preflightIssues[].selectoris the verbatim spec string (a deliberately double-spaced#list .itemround-trips unchanged), so the filter keys correctly; and a dropped assertion is genuinely not evaluated — a spec with an ambiguous selector plus abySecthat would fail produced no finding for it.plan.selectorsis always a superset of what they need.plan.selectors, soevaluateMotionnever sees them.Strictly louder, never newly-breaking
Reaching this path means a hardcoded-
errormotion_selector_ambiguouswas already present, so the run had already failed — the change cannot flip a passingcheckto failing. What it does is front-load errors the author would have hit on the next run anyway.Worth knowing: users with an ambiguous selector will now see additional real errors they had never been shown, and the motion grid is sampled again in that case (up to
MOTION_MAX_SAMPLESseeks;--caption-zone/--frame-checkfindings can also increase, since those run at motion sample times). Layout findings cannot change —collectLayoutis gated on the layout sample set.Related, not fixed here
Eight other defects in this family are confirmed and left for follow-ups. Two are worth naming because they interact with this one:
0 errors, 0 warningsfor Layout, Runtime and Contrast — an affirmatively false clean bill of health for four sections (measured: 8 layout findings on the same composition become 0). Same principle as this PR, bigger blast radius; a reviewer argued it is the more urgent next PR and I agree.isVisibleElementhas no viewport test — so any slide-in entrance breaks bothbeforeandstaysInFrame. This PR increases how often that is seen (it was accidentally masked whenever the grid was zeroed), which raises its priority; it shares a root cause withmotion_off_frameflagging full-bleed backgrounds and the two should land together.Also pre-existing and untouched: an ambiguous
keepsMovingscope is never preflighted (scopes do not go throughfindAmbiguousSelectors) and reports a factually falsemotion_selector_missing. Confirmed identical onmainwhen a scope is the only ambiguity, so this change does not widen it — but it becomes reachable in one more configuration.🤖 Generated with Claude Code