feat(providersv2): inject static auth headers from v2 provider profiles

[Cali0707]

Summary

This PR enables injection of static provider credentials that are auth headers when providers_v2_enabled is set. It extends the existing token grant injection path to resolve and inject bearer/header credentials from provider profiles, without requiring child-env placeholder resolution.

Related Issue

Part of #896

Changes

• Gate static credential inclusion in dynamic_credentials behind providers_v2_enabled in the server
• Extend sandbox side inject_if_needed to handle static credentials as well as the existing token grant path

Testing

• mise run pre-commit passes
• Unit tests added/updated
• E2E tests added/updated (if applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[johntmyers]

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements profile-driven credential injection work called out in roadmap issue #896, and the operator confirmed profile injection plus placeholder rewrite may be handled in the same request when they do not operate on the same credential.
Head SHA: 00f1bfdabf11d8cbf3ea8a3d79b45480164c29e9

Review findings:

• Blocking: crates/openshell-server/src/grpc/provider.rs currently includes every non-token-grant credential in static proxy injection when providers_v2_enabled is set, while crates/openshell-sandbox/src/l7/token_grant_injection.rs treats empty auth_style as bearer. Built-in profiles such as Codex have multiple credentials without explicit header placement, so the proxy can inject the wrong secret as Authorization: Bearer .... Please restrict static proxy injection to credentials that explicitly declare supported header placement, for example auth_style: bearer or auth_style: header.
• Blocking: static endpoint-bound credentials do not appear to have the ambiguity protection that token grants have. Equal host/port/path static bindings can silently select one credential by lexicographic key order. Please reject equal-specificity overlapping static proxy-injectable bindings, or otherwise make selection deterministic by explicit profile semantics that cannot mix credentials.
• Blocking: provider_env_revision does not appear to include providers_v2_enabled, but running sandboxes refresh provider credentials only when that revision changes. Toggling the setting can leave running sandboxes with stale static injection metadata, including after disabling provider-v2 policy layers. Please include the setting in the provider environment revision or force a provider environment refresh when it changes.
• Warning: static bearer/header profile validation should reject invalid/framing header names before runtime, reusing the existing token-grant header validation rules with static-credential wording.
• Warning: Fern docs still describe static credentials as placeholder-only and not endpoint-scoped. Because this changes direct provider-v2 behavior, please update docs/sandboxes/providers-v2.mdx; no new docs/index.yml navigation entry appears necessary.

Suggested tests:

• Built-in multi-credential profile behavior, for example Codex.
• Equal-specificity static credential ambiguity.
• providers_v2_enabled toggling and provider environment refresh behavior.
• Static invalid/framing header profile validation.

Docs: missing for a direct provider-v2 behavior change.

Next state: gator:in-review

[johntmyers]

gator-agent

Blocked

Gator is blocked because this PR currently has merge conflicts with main (mergeable_state: dirty) at head SHA 00f1bfdabf11d8cbf3ea8a3d79b45480164c29e9.

Next action: @Cali0707, please rebase or merge the latest main and resolve the conflicts. After the branch is mergeable again, gator will re-check the outstanding provider-v2 review findings and docs requirement from the previous review comment.

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head b96f329f2cef7849946f897aaacd9601e4315dac after @Cali0707 pushed an update on 2026-06-15. The previous merge-conflict blocker is resolved: GitHub now reports mergeable: true and mergeable_state: blocked, with required checks waiting for the copy-pr /ok to test mirror.

Disposition: partially resolved.

Resolved from the previous gator review:

• Static proxy injection is now limited to credentials with explicit supported header placement, aside from the case-sensitivity gap below.
• Equal-specificity static credential ambiguity protection is now present.
• provider_env_revision now includes providers_v2_enabled.
• Static bearer/header header-name validation now rejects invalid/framing header names.
• docs/sandboxes/providers-v2.mdx now documents the provider-v2 behavior change.

Remaining items:

• Blocking: profile injection currently removes an existing same-name header before placeholder rewrite runs. In the L7 path, injection happens before the normal placeholder rewrite; the forward proxy path has the same order. A request such as Authorization: Bearer openshell:resolve:env:OTHER_TOKEN to an endpoint whose profile also injects Authorization would have the explicit placeholder header dropped and replaced by the profile credential. That violates the gate constraint that profile injection and placeholder rewrite may operate on the same request only when they are not trying to operate on the same credential/header. Please fail closed on same-header placeholder/profile-injection collisions or otherwise reject this combination, and add regression coverage for the selected L7 and forward proxy paths.
• Warning: static auth_style: header reuses token-grant access-token validation, which enforces token68 syntax. Static custom-header credentials can be valid HTTP field values without being token68 values, so this can reject valid API keys. Please split static header value validation from OAuth token-grant access-token validation.
• Warning: can_inject_credential checks auth_style case-sensitively while profile validation treats it case-insensitively. A valid auth_style: Bearer or Header profile would pass validation but not inject.

I am moving this back to gator:in-review. I am not posting /ok to test yet because blocking review feedback remains.

Next state: gator:in-review

[johntmyers]

gator-agent

PR Review Status

Validation: This PR remains project-valid because it implements profile-driven credential injection work from roadmap issue #896 under the operator constraint that profile injection and placeholder rewrite may operate on the same request only when they are not targeting the same credential/header.
Head SHA: b96f329f2cef7849946f897aaacd9601e4315dac

Review findings:

• Blocking: same-header placeholder/profile-injection collisions remain unresolved. The shared injection helper still removes an existing same-name header before the normal placeholder rewrite path sees it, so a request such as Authorization: Bearer openshell:resolve:env:OTHER_TOKEN can be replaced by the profile-injected credential instead of failing closed. Please reject same-header placeholder/profile-injection collisions in the shared injector and add regression coverage for both the L7 and forward proxy paths.
• Warning: static auth_style: header still reuses OAuth token68 access-token validation. Static custom-header values should reject header injection characters such as CR/LF/NUL, but should not need to be token68 values.
• Warning: can_inject_credential still checks auth_style case-sensitively while profile validation treats it case-insensitively. Valid Bearer or Header profiles can pass validation without producing injection metadata.
• Warning: static bearer handling is internally inconsistent: the runtime defaults a blank bearer header_name to Authorization, but profile validation still requires header_name for auth_style: bearer. Please either require explicit Authorization consistently or allow blank bearer header_name as the runtime default implies.

Docs: docs/sandboxes/providers-v2.mdx has been updated for this provider-v2 behavior change; no new docs/index.yml navigation entry appears necessary.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head 033803bef0ca7db6c27ad8e777d6dd16cbbcd145 after @Cali0707 pushed an update on 2026-06-16. The independent code review focused on the provider credential injection and placeholder rewrite paths under the operator constraint that both mechanisms may operate on the same request only when they are not targeting the same credential/header.

Disposition: partially resolved.

Resolved from the previous gator review:

• can_inject_credential now treats auth_style case-insensitively.
• Static bearer blank header_name handling now defaults to Authorization, matching runtime behavior.
• provider_env_revision still includes providers_v2_enabled.
• Fern docs remain updated for the provider-v2 behavior change; no new docs/index.yml navigation entry appears necessary.

Remaining items:

• Blocking: same-header placeholder/profile-injection collisions still do not fail closed. The current L7 path skips profile injection and preserves the placeholder-bearing header for the legacy rewrite path, so a request can still choose placeholder resolution for the same header that profile injection would target. Please reject this collision instead of returning the original request, and ensure the same fail-closed behavior covers every request path that can rewrite credentials, including the forward proxy path if it does not share the exact helper.
• Warning: static auth_style: header still appears to reuse token-grant/Bearer-oriented value handling. Static custom-header values should reject unsafe HTTP header bytes such as CR, LF, and NUL at the injection sink, but should not require OAuth token68 syntax or rely only on SecretResolver side effects.
• Warning: profile-side injection currently selects only one matching credential at equal specificity. If single-credential-per-endpoint injection is intentional for this PR, please document that limitation in the Providers v2 docs.

I am keeping this in gator:in-review and am not posting /ok to test while blocking review feedback remains.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head d18c6f0d5b911f8d2fc32f1930c6b67a9e846d2f after @Cali0707 pushed an update on 2026-06-16. The independent code review focused on provider credential injection, placeholder rewrite ordering, and the provider-v2 credential/header constraint.

Disposition: partially resolved.

Resolved from the previous gator review:

• Static auth_style: header values now reject unsafe HTTP header bytes without requiring OAuth token68 syntax.
• Static same-header placeholder/profile-injection collisions now fail closed in the shared static injection helper.
• The Providers v2 docs document static proxy injection and same-header placeholder/profile collision behavior.

Remaining items:

• Blocking: static injectable credential bindings are generated from profile metadata even when the attached provider does not currently have any active env var for that credential. This can reject valid providers as ambiguous, including the built-in google-vertex-ai profile where service_account_token and gcloud_adc_token both target authorization on the same endpoints. Please emit static injection metadata/bindings only for static credentials that have at least one active provider env key, while keeping token grants unconditional, and add a regression test for a Vertex provider with only one active token source.
• Blocking: the same-header placeholder collision check only covers static credential injection. Token-grant profile injection still calls the shared header replacement path directly, so an existing Authorization: Bearer openshell:resolve:env:... header can be removed before placeholder rewrite sees it. Please factor the collision check into a shared helper and apply it before both token-grant and static profile injection in the L7 and forward proxy paths.
• Blocking: route-selected L7 traffic does not call the profile injection helper before forwarding. Built-in profiles such as GitHub have multiple L7 configs on the same host/port (api.github.com REST plus /graphql), which uses relay_with_route_selection; that path currently forwards with placeholder rewrite only and skips profile injection and its collision checks. Please route allowed requests through the same injection helper used by the single-config L7 and passthrough paths, with regression coverage.

I am keeping this in gator:in-review and am not posting /ok to test while blocking review feedback remains.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head 043990788a22e2a6f088cca1c074784650842570 after @Cali0707 pushed an update on 2026-06-17. The independent code review focused on provider credential injection, placeholder rewrite ordering, route-selected L7 handling, and the provider-v2 credential/header constraint.

Disposition: partially resolved.

Resolved from the previous gator review:

• Static injectable credential bindings are now gated on active provider env keys for normal non-expired credentials, and the Vertex single-active-token-source case has regression coverage.
• Same-header placeholder/profile-injection collisions now fail closed through the shared injector for both static and token-grant profile injection.
• Route-selected L7 and GraphQL forwarding now call the shared injection helper before forwarding, so profile injection and collision checks are no longer skipped on that path.
• Static custom-header values no longer require OAuth token68 syntax.
• Fern docs document the new static profile injection and same-header placeholder collision behavior; no new docs/index.yml navigation entry appears necessary.

Remaining items:

• Blocking: static injection metadata still treats any stored provider credential key as active, but the provider environment resolver filters expired, invalid, and non-injectable keys before exposing values. An expired static credential can therefore still produce static injection metadata or ambiguity bindings, causing false ambiguity rejections or request-time 502s when the proxy selects metadata whose resolver has no value. Please make the static binding/dynamic metadata eligibility check reuse the same active-key semantics as provider environment resolution, including expiry and key validity, while keeping token grants unconditional. Please add regression coverage with one expired Vertex token source and one active source.
• Warning: docs/sandboxes/providers-v2.mdx still has a profile example comment saying static runtime injection uses env placeholder resolution. Please update that example so it does not contradict the new proxy-side static header injection behavior.
• Warning: the forward proxy now uses the shared injection helper, but there is still no forward-proxy regression test for same-header placeholder/profile collisions. The shared helper unit tests are useful, but the prior gate explicitly covered forward proxy ordering.
• Warning: several operator-facing errors/logs still say "Token grant failed" for static credential injection failures and placeholder collisions. Please use "credential injection failed" or equivalent wording at the shared call sites so provider-v2 static credential failures are debuggable.

I am keeping this in gator:in-review and am not posting /ok to test while blocking review feedback remains.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head 3c91a31e3324c0a4cac02018a2562ddca6225330 after @Cali0707 pushed an update on 2026-06-18. The independent code review focused on provider credential injection, placeholder rewrite ordering, active static credential eligibility, forward proxy coverage, and the provider-v2 credential/header constraint.

Disposition: resolved.

Resolved from the previous gator review:

• Static injection metadata now reuses the same active-key semantics as provider environment resolution for stored provider credentials, including expiry, invalid key names, and non-injectable special cases, while token grants remain unconditional.
• Regression coverage now includes the Vertex case with one expired token source and one active token source.
• The forward proxy has same-header placeholder/profile collision regression coverage.
• Static credential injection and placeholder collision failures now report as credential injection failures at the shared relay/proxy call sites.
• The Providers v2 docs no longer describe static proxy-side profile injection as placeholder-only behavior.

Remaining items:

• No blocking items remain.
• Non-blocking cleanup noted by the independent reviewer: some ambiguous credential-binding diagnostics and internal helper names still use token-grant-specific wording even though the path now covers static credentials too.

Docs: docs/sandboxes/providers-v2.mdx is updated for this provider-v2 behavior change; no docs/index.yml navigation update appears necessary.

E2E: applying test:e2e because this changes provider credential flow and proxy behavior.

Next state: gator:watch-pipeline

[johntmyers]

/ok to test 3c91a31

[github-actions]

Label test:e2e applied for 3c91a31. Open Branch E2E Checks, find the run for commit 3c91a31, and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[johntmyers]

gator-agent

Re-check After CI Failure

I re-checked head 3c91a31e3324c0a4cac02018a2562ddca6225330 after the Branch Checks failure.

Disposition: author action required.

Findings:

• OpenShell / Branch Checks failed in both Rust jobs during mise run rust:format:check.
• The failed log shows cargo fmt --all -- --check wants to collapse the miette! call at crates/openshell-supervisor-network/src/l7/token_grant_injection.rs:449 to return Err(miette!("credential auth_style header requires header_name"));.
• OpenShell / E2E has started and is still running, but the required Branch Checks gate is already red.

Next action: @Cali0707, please run Rust formatting and push the formatted update. Gator will re-check review status and CI on the new head.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head 2ab0da73f9df5bfdcd76fbfb51a5cb2efcf2759a after @Cali0707 pushed the 2026-06-23 update. The independent code review focused on provider-v2 static credential injection, byte-oriented header handling, active credential binding behavior, and the provider-v2 docs.

Disposition: resolved for pipeline watch.

Resolved from the previous gator review:

• inject_header is now byte-oriented and preserves unrelated non-UTF-8 header bytes during successful credential injection.
• Static provider credential values still reject unsafe CR/LF/NUL and empty values at the injection sink.
• Active and expired credential filtering remains shared between environment exposure and static binding generation.
• Import diagnostics now use neutral provider credential binding wording instead of token-grant-specific wording.

Remaining items:

• No blocking review items remain.
• Non-blocking warnings remain: the forward-proxy pre-injection body-length parse can still reject unrelated non-UTF-8 obs-text before the byte-oriented injector runs, and providers_v2_enabled=true still does not preflight existing attached provider sets before enabling static bindings. These fail closed and can be handled as follow-up or maintainer-requested cleanup.
• Non-blocking docs suggestion: the Providers v2 docs now describe static proxy injection, but the feature-flag intro still emphasizes policy composition more than credential injection behavior.

Docs: docs/sandboxes/providers-v2.mdx is updated for this provider-v2 behavior change; no docs/index.yml navigation update appears necessary.

E2E: test:e2e is already applied because this changes provider credential flow and proxy behavior.

Next state: gator:watch-pipeline

[johntmyers]

/ok to test 2ab0da7

[Cali0707]

Looks like the e2e flake is still present - I think #1968 may be what is needed

[johntmyers]

#1968

Yes that PR looks good although I left one piece of feedback.

[johntmyers]

gator-agent

PR Review Status

Validation: This PR remains project-valid because it implements provider-v2 profile-driven static credential injection work from roadmap issue #896, under the gate constraint that profile injection and placeholder rewrite must fail closed when they target the same credential/header.
Head SHA: 7f6decf67b49fb88869688e483d6d1c476a62e58

Review findings:

• Blocking: this security-sensitive provider credential flow still lacks end-to-end coverage for the new server-to-sandbox-to-proxy static credential injection path. The current unit tests cover the local mechanics well, and the independent review found the byte-oriented header collision/rewrite mechanics sound, but the PR has no E2E test change that proves providers_v2_enabled static bearer/header injection works through a real sandbox/proxy path. Please add E2E coverage that enables providers_v2_enabled, creates or imports a static bearer or header provider profile, attaches it to a sandbox, sends outbound HTTP to a test upstream, and asserts the upstream receives the injected header while the sandbox process sees only placeholders. Please also include a same-header placeholder/profile collision case that fails closed.

Resolved from the previous gator review:

• inject_header and placeholder collision checks now operate byte-wise and preserve unrelated non-UTF-8 header bytes.
• Static credential values reject CR, LF, and NUL at the injection sink.
• Provider-v2 static injection docs remain updated.

Docs: docs/sandboxes/providers-v2.mdx is updated for this provider-v2 behavior change; no docs/index.yml navigation update appears necessary.

E2E: test:e2e is already applied because this changes provider credential flow and proxy behavior. I am not posting /ok to test while blocking review feedback remains.

Next state: gator:in-review

[johntmyers]

/ok to test 515c495

[johntmyers]

Addressed the latest Gator finding in 515c495 after rebasing onto current main.

The provider-v2 E2E coverage now verifies:

• static bearer injection reaches the upstream while the sandbox sees only the revision-aware placeholder
• static custom-header injection has the same secret-isolation behavior
• a same-header placeholder/profile collision fails closed with 502

Local pre-commit and focused Docker E2E passed, and the full required Branch Checks and E2E workflows are green.

[Cali0707]

@johntmyers thanks for the rebase!

Sorry this PR is taking so long 😓

[johntmyers]

gator-agent

Re-check After Maintainer Update

Thanks @johntmyers. I checked your July 9 update for head 515c4953540a11d5e0447f942192127f908ac794: the new E2E cases do cover static bearer injection, static custom-header injection, secret isolation via revision-aware placeholders, and same-header collision failure, and the required Branch Checks, Helm Lint, E2E, GPU E2E, and DCO gates are green.

Disposition: partially resolved. The prior missing-E2E blocker is resolved, but the independent code review found two new blocking credential-routing issues:

• Blocking: crates/openshell-server/src/grpc/provider.rs currently emits the Cartesian product of every injectable credential and every profile endpoint. That sends the Claude Code Anthropic API key to statsig.anthropic.com and sentry.io, and similarly sends the Copilot token to telemetry/configuration endpoints. This crosses credential trust boundaries (CWE-200). Please add an explicit, default-deny credential-to-endpoint association, emit static bindings only for explicitly associated endpoints, and add regression coverage proving auxiliary endpoints receive no credential.
• Blocking: inject_static_credential() inserts the resolved static value before the relay runs generic placeholder rewriting. If credential A's value is openshell:resolve:env:B, the upstream can receive credential B, violating endpoint binding (CWE-20/CWE-441). Please reject reserved credential markers in resolved static values or prevent profile-injected headers from being rewritten again, with a complete injection-plus-relay regression test proving B is never forwarded.

Non-blocking follow-up from the review:

• Established keep-alive proxy connections snapshot credential state and can continue using a stale non-expiring value after rotation; either resolve current state per request or close tunnels when the provider environment revision changes, with rotation coverage.
• The new E2E context manager changes a gateway-global setting while the Python suite runs across xdist workers and deletes rather than restores the prior value; serialize against the whole suite or use a dedicated gateway and restore the original setting.
• Fern docs should also add static injection to the Providers v2 feature list and update the placeholder-only explanation in docs/sandboxes/manage-providers.mdx. Existing folder navigation means docs/index.yml does not need a change.

Docs: docs/sandboxes/providers-v2.mdx covers the core behavior, but the related provider guide still needs the consistency update above.

E2E: test:e2e remains appropriate and is green for this SHA. No local tests were run as part of this code-only review.

Next state: gator:in-review

[johntmyers]

Given the findings above, I worry that the overall approach that was outlined in #896 is a bit too fragile and risky. I'd like to first get Providers v2 as the default Provider system and complete the Proxy refactor then re-visit this approach, if we still even need it. While the env placeholders introduce other issues (such as not being able to dynamically update the env of an already running process), it at least gets us a better guarantee about which credentials get injected when. Relying only on the provider profile bypasses that and there are clearly some risks to it. I'm going to close this PR for now and we can re-visit later on.

[johntmyers]

gator-agent

Monitoring Complete

Monitoring is complete because this PR has been closed without merge.

Final status: the PR closed at head 515c4953540a11d5e0447f942192127f908ac794 while in gator:in-review.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.