Skip to content

Move from legacy proxy to gateway#191

Merged
Erin McNulty (erin2722) merged 6 commits into
mainfrom
move-to-gateway
Jun 8, 2026
Merged

Move from legacy proxy to gateway#191
Erin McNulty (erin2722) merged 6 commits into
mainfrom
move-to-gateway

Conversation

@erin2722

Copy link
Copy Markdown
Contributor

The AI Proxy will be deprecated soon, and the AI Gateway is being continually updated with bug fixes and new features, and so we should move this repository to use that.

@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

Braintrust eval report

Evals failed: Error: Command failed with exit code 1

Comment thread py/autoevals/oai.py

self._is_wrapped = isinstance(self.openai, NamedWrapper)
self._is_wrapped = isinstance(self.openai, NamedWrapper) or (
not has_customization and wrap_openai.__module__.startswith("braintrust.")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive-by, the test was broken

@ibolmo Olmo Maldonado (ibolmo) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just various nits, but none blocking. i wonder if you'd want to. bump the version as either minor or major. i'd ask in sdk channel.

Comment thread js/oai.ts Outdated
}

const PROXY_URL = "https://api.braintrust.dev/v1/proxy";
const GATEWAY_URL = "https://gateway.braintrust.dev";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i know it's outside of your change, but) we should move to an env var, and default to this.

Comment thread js/oai.ts
*
* @example
* // Using with Anthropic via Braintrust proxy
* // Using with Anthropic via Braintrust Gateway

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#nit Braintrust AI Gateway? what's the official product name? 🤔

@erin2722 Erin McNulty (erin2722) Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread py/autoevals/__init__.py Outdated
# Configure client to use the Braintrust Gateway with Claude
client = AsyncOpenAI(
base_url="https://api.braintrust.dev/v1/proxy",
base_url="https://gateway.braintrust.dev",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto os.getenv('BRAINTRUST_AI_GATEWAY_URL') or 'https://...'

Comment thread py/autoevals/oai.py
from typing import Any, Optional, Protocol, TypedDict, TypeVar, Union, cast, runtime_checkable

PROXY_URL = "https://api.braintrust.dev/v1/proxy"
GATEWAY_URL = "https://gateway.braintrust.dev"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ibolmo Olmo Maldonado (ibolmo) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nothing blocking, ty!

Comment thread js/oai.ts Outdated
const DEFAULT_GATEWAY_URL = "https://gateway.braintrust.dev";

const getGatewayURL = (): string =>
process.env.BRAINTRUST_AI_GATEWAY_URL || DEFAULT_GATEWAY_URL;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd encourage a sanitizing like:

(process.env.BRAINTRUST_AI_GATEWAY_URL ?? '').trim() || DEFAULT_GATEWAY_URL

Comment thread py/autoevals/test_oai.py


def test_prepare_openai_defaults_to_gateway(monkeypatch: pytest.MonkeyPatch):
monkeypatch.delenv("OPENAI_BASE_URL", raising=False)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(fyi) i like to use getEnv() and similar helpers to avoid monkey patching. you can then just set/unset env vars more easily.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense!

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (HEAD-1780953779)

Score Average Improvements Regressions
NumericDiff 78.8% (-1pp) 10 🟢 10 🔴
Time_to_first_token 11.55tok (+1tok) 117 🟢 102 🔴
Llm_calls 1.55 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 528.42tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 461.89tok (-19tok) 110 🟢 90 🔴
Completion_reasoning_tokens 351.71tok (-16.29tok) 95 🟢 79 🔴
Completion_accepted_prediction_tokens 0tok (+0tok) - -
Completion_rejected_prediction_tokens 0tok (+0tok) - -
Completion_audio_tokens 0tok (+0tok) - -
Total_tokens 990.31tok (-19tok) 110 🟢 90 🔴
Estimated_cost 0$ (0$) 99 🟢 75 🔴
Duration 11.55s (+1s) 117 🟢 102 🔴
Llm_duration 12.28s (+1.03s) 117 🟢 102 🔴

@erin2722 Erin McNulty (erin2722) merged commit 59778b9 into main Jun 8, 2026
15 checks passed
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

Braintrust eval report

Autoevals (main-1780954899)

Score Average Improvements Regressions
NumericDiff 79.2% (+0pp) 8 🟢 14 🔴
Time_to_first_token 9.1tok (-2.45tok) 150 🟢 67 🔴
Llm_calls 1.55 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 524.29tok (-4.14tok) 2 🟢 -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Prompt_cache_creation_5m_tokens 0tok (+0tok) - -
Prompt_cache_creation_1h_tokens 0tok (+0tok) - -
Completion_tokens 461tok (-0.9tok) 104 🟢 102 🔴
Completion_reasoning_tokens 350.55tok (-1.16tok) 86 🟢 93 🔴
Completion_accepted_prediction_tokens 0tok (+0tok) - -
Completion_rejected_prediction_tokens 0tok (+0tok) - -
Completion_audio_tokens 0tok (+0tok) - -
Total_tokens 985.28tok (-5.03tok) 104 🟢 102 🔴
Estimated_cost 0$ (+0$) 90 🟢 95 🔴
Duration 9.11s (-2.45s) 150 🟢 68 🔴
Llm_duration 9.8s (-2.48s) 150 🟢 67 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants