Skip to content

fix(mcp): stabilize authenticated tool discovery#5754

Closed
j15z wants to merge 27 commits into
stagingfrom
fix/mcp-discovery-races
Closed

fix(mcp): stabilize authenticated tool discovery#5754
j15z wants to merge 27 commits into
stagingfrom
fix/mcp-discovery-races

Conversation

@j15z

@j15z j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Authenticated MCP servers that advertise OAuth can now be added and refreshed with an explicitly configured static Authorization: Bearer ... credential without being diverted into OAuth. The previous fix made the pre-save connection test try the configured header, but runtime discovery still mapped every SDK UnauthorizedError to “authorization required.” This PR makes that classification depend on the server's configured auth type, so static-header servers stay on the normal credential path and only OAuth-configured servers enter OAuth.

OAuth-pending connections now remain actionable in Workspace Settings: the persistent server status says OAuth authorization required, and a refresh that cleanly returns to that state says Authorization required instead of the generic Failed / Not Connected. Concrete connection errors still take precedence, so timeouts and other failures are not mislabeled as an OAuth prompt.

Discovery publication is ordered end to end. Redis and memory cache adapters allocate a monotonic millisecond mutation token; conditional cache publication and the database lastToolsRefresh compare-and-swap use that same token, so retries, failures, successes, and invalidations choose one winner across both layers. A credential-safe discovery revision covers every connection-affecting field, preserving valid tools across metadata-only edits while failing closed after URL, credential, auth-type, or transport changes.

Each retry acquires fresh mutation ownership. Successful, failed, and OAuth-pending status publication share a bounded retry for metadata-only edit races: every attempt rechecks cache ownership and the discovery revision before publishing against the latest row token. If ordering or an atomic cache transition is unavailable, freshly fetched tools may be returned to the current caller but unordered cache/database state is not persisted. Successful discovery returns its exact publication token, so workflow schemas sync only while the post-discovery reread still belongs to that winning published refresh; later success, failure, or invalidation barriers take precedence, and winner-cache, superseded, and degraded unavailable results cannot propagate as winners. Cache invalidation retries reacquire fresh ownership, publish a database barrier only after an atomic delete wins, and never fall back to an unordered delete that could erase a newer discovery.

Refresh responses now prove that a newer successful discovery actually won before preserving connected state after a failure: both lastToolsRefresh and lastConnected must advance beyond the request's original snapshot. Invalidation advances the publication token but not lastConnected, so it can no longer hide a real discovery failure behind a stale connected row. Failure counters also recompute through an optimistic JSONB compare-and-swap when success races them.

Security-sensitive paths remain bounded: MCP targets still pass domain and SSRF validation, public targets are DNS-pinned when policy permits, and HTTP/2 negotiation does not relax that pinning. Logs contain header names and structural diagnostics but never values, session IDs, or raw upstream bodies, while UI/API errors come from an allowlist. OAuth servers without dynamic client registration return an actionable 422 that directs users to configure a token. The settings copy change consumes only the existing auth type and sanitized error state; it does not alter or expose credentials, headers, tokens, or upstream responses.

Related: #5665 and #5595.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: ___________

Testing

  • bun run --cwd apps/sim test -- lib/mcp app/api/mcp 'app/workspace/[workspaceId]/settings/components/mcp' — 454 tests passed across all 29 MCP suites, including static Bearer classification, concurrent publication, repeated metadata-only races, ownership-safe invalidation retries, degraded-cache behavior, publication-token-gated workflow sync, secret-safe diagnostics, SSRF pinning, invalidation-only refresh races, and OAuth-specific settings status behavior.
  • bun run check:api-validation:strict — passed with all 964 routes Zod-backed and no boundary-policy drift.
  • Biome on the final review files and git diff --check — passed.
  • Direct GitHub Copilot MCP probes completed initialize and tools/list with 47 tools using both normal and SSRF-pinned transports; the SDK transport carried the session ID without logging it.
  • A full app-suite baseline recorded 10,715 passing tests and 41 known failures in unrelated workflow, webhook, chat, and UI tests. The final review changes were revalidated in the focused suite above.
  • bun run --cwd apps/sim type-check reaches only unrelated shared-worktree dependency errors in fork audit events, recordAuditBatch, and @google-cloud/storage; no MCP TypeScript errors are reported.

Reviewer focus: auth-type-aware handling of typed 401s, OAuth-pending UI status with concrete-error precedence, unified cache/database mutation ownership, bounded same-revision status retries, fresh-token invalidation retries, Redis conditional-mutation scripts, winner-only workflow synchronization, fail-closed degraded-cache behavior, and the two-timestamp proof for a racing successful refresh.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

N/A — status-copy-only UI change with no layout change.

@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 18, 2026 6:54pm

Request Review

@cursor

cursor Bot commented Jul 18, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches MCP auth classification, OAuth API responses, and outbound pinned HTTP behavior; changes are scoped with tests but affect connection and credential error paths users rely on.

Overview
Improves MCP connection and OAuth UX while tightening logging and transport behavior.

Runtime auth: UnauthorizedError is only treated as “authorization required” when the server is configured for OAuth; header/token servers get unauthorized / “Authentication failed” instead of being pushed into an OAuth flow.

Settings copy: Disconnected OAuth servers show OAuth authorization required; a refresh that ends disconnected with no concrete error shows Authorization required instead of generic Failed / Not Connected (persisted errors still win).

OAuth start API: Servers that don’t support dynamic client registration return 422 with guidance to configure a token instead of a generic 500.

Logging: New getMcpSafeErrorDiagnostics redacts structural error fields for connect, transport, and tools/list logs (no raw messages, session IDs, or secrets).

Networking: createPinnedFetch gains optional HTTP/2 (allowH2); MCP paths use createPinnedMcpFetch (h2 on, pinning unchanged). OAuth probe and SSRF-guarded fetch route through that wrapper.

Lifecycle: Updating a server to OAuth (or flipping auth type) resets connectionStatus to disconnected and clears workspace cache when transport or auth type changes.

Reviewed by Cursor Bugbot for commit ea7a148. Configure here.

Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/client.ts
@greptile-apps

greptile-apps Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stabilizes authenticated MCP discovery and refresh behavior. The main changes are:

  • Distinguishes static-header authentication failures from OAuth authorization prompts.
  • Shows actionable OAuth-pending states while preserving concrete connection errors.
  • Resets connection state and clears cached discovery data after authentication or transport changes.
  • Adds credential-safe MCP error diagnostics.
  • Enables HTTP/2 negotiation for DNS-pinned MCP traffic.
  • Returns an actionable response when OAuth client registration is unsupported.

Confidence Score: 5/5

This looks safe to merge.

  • The timestamp publication fix has no remaining reachable failure identified in the updated code.
  • Authentication classification and OAuth state changes have focused coverage.
  • DNS pinning and secret-safe diagnostics remain intact.
  • No blocking issues were found in the changed code.

Important Files Changed

Filename Overview
apps/sim/lib/mcp/client.ts Classifies unauthorized responses by configured authentication type and emits sanitized diagnostics.
apps/sim/lib/mcp/orchestration/server-lifecycle.ts Resets OAuth state and invalidates discovery cache when connection settings change.
apps/sim/lib/core/security/input-validation.server.ts Adds optional HTTP/2 negotiation while retaining pinned DNS resolution.
apps/sim/lib/mcp/pinned-fetch.ts Centralizes HTTP/2-enabled pinned fetch construction for MCP traffic.
apps/sim/app/api/mcp/oauth/start/route.ts Returns an actionable response when dynamic OAuth client registration is unsupported.
apps/sim/app/workspace/[workspaceId]/settings/components/mcp/refresh-action-state.ts Separates OAuth authorization prompts from refresh failures with concrete errors.

Reviews (6): Last reviewed commit: "fix(mcp): reset connectionStatus when a ..." | Re-trigger Greptile

Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
- preserve typed static-header authentication failures\n- order discovery publication by start time\n- atomically publish tools and failure cache state
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts
j15z added a commit that referenced this pull request Jul 18, 2026
Match configuration generations within the JavaScript millisecond window so PostgreSQL microseconds do not suppress valid publication.
@j15z

j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

The database publication predicate needs a precision-safe configuration generation.

Addressed in 89dce25: both status-publication predicates now match the JavaScript millisecond window, accepting PostgreSQL sub-millisecond precision while still rejecting the next generation. A boundary regression covers the behavior.

Comment thread apps/sim/lib/mcp/service.ts
j15z added a commit that referenced this pull request Jul 18, 2026
- fall back to best-effort cache writes when mutation ordering is unavailable\n- share bounded redacted MCP error diagnostics across client and service
Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Retry cache mutation ownership before publishing discovery state, avoid unordered publication that older writers can supersede, and keep explicit invalidation best effort.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/storage/redis-cache.ts
j15z added a commit that referenced this pull request Jul 18, 2026
Use timestamp-based mutation tokens for both cache ownership and database CAS ordering, and invalidate Redis mutation owners before deleting entries during a full clear.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Recompute consecutive failures after status CAS conflicts, reload winning cached tools for superseded discoveries, and return live tools without unordered publication when cache ordering is unavailable.
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Outdated
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Expose discovery publication metadata to refresh callers, report live tools during cache degradation, and compare lastToolsRefresh mutation tokens instead of wall clocks when preserving a newer success.
Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Probe mutation ownership after database CAS misses, keep valid live tools for metadata-only edits, and invalidate ownership for transport and auth-type changes.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Outdated
- Preserve OAuth-pending state when a status reread is stale
- Keep concrete refresh errors sanitized and visible
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts Outdated
- Make cache invalidation retries ownership-safe
- Preserve proven newer successes for superseded refreshes
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Outdated
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
@j15z
j15z requested a review from waleedlatif1 July 18, 2026 11:35
@waleedlatif1

Copy link
Copy Markdown
Collaborator

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@cursor review

Comment thread apps/sim/lib/mcp/orchestration/server-lifecycle.ts
… minimal fix set

A four-angle LOC review (correctness, overengineering, dead-code, intent) found
the fencing-token / discoveryRevision / conditional-publication apparatus and its
5 bounded-retry loops (~700 lines) to be correct but disproportionate to a
low-contention, self-healing per-workspace-per-server discovery path — and it
introduced two regressions: failing closed on a transient Redis blip (blocking DB
status writes) and clearCache writing the DB for every server in the workspace.

Revert service.ts, the storage adapters, the refresh route, and types to the
production-tested base (which keeps the simple lastConnected<=discoveryStartedAt
ordering guard and plain cache set/delete), and keep only the genuinely-good,
decoupled fixes: authType-aware UnauthorizedError classification, OAuth-pending
UX labels, DCR->422, credential-safe error diagnostics, and HTTP/2 negotiation.
@waleedlatif1
waleedlatif1 force-pushed the fix/mcp-discovery-races branch from bb5e320 to 99e235a Compare July 18, 2026 18:45
An authType flip to oauth (via a plain authType change or an implicit
oauthClientId) cleared the cache but left connectionStatus='connected', so a
server that had not completed its OAuth flow falsely read as connected and the
OAuth-authorization-required prompt never showed. Reset to disconnected on any
switch to oauth, matching the create path. Adds the missing test assertion.
@waleedlatif1

Copy link
Copy Markdown
Collaborator

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit ea7a148. Configure here.

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Superseded by #5757. After a four-angle LOC review found the ordered-publication consistency layer disproportionate (correct but ~700 lines of fencing/CAS/retry machinery for a low-contention, self-healing path, plus two regressions), we carved the genuinely-good fixes — auth-type classification, OAuth-pending UX, DCR→422, safe error diagnostics, and HTTP/2 — into a clean, minimal PR (#5757) off staging. The original commits here remain in git history. Closing this in favor of the simpler replacement.

@waleedlatif1
waleedlatif1 deleted the fix/mcp-discovery-races branch July 18, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants