Skip to content

feat : add headroom proxy for intercept request llm for good compression then save credits#320867

Draft
damnthonyy wants to merge 2 commits into
microsoft:mainfrom
damnthonyy:feat/headroom-proxy-for-chat-copilot
Draft

feat : add headroom proxy for intercept request llm for good compression then save credits#320867
damnthonyy wants to merge 2 commits into
microsoft:mainfrom
damnthonyy:feat/headroom-proxy-for-chat-copilot

Conversation

@damnthonyy

@damnthonyy damnthonyy commented Jun 11, 2026

Copy link
Copy Markdown

Origin

This PR was initiated from the following prompt:

can you dig deep into the headroom codebase and understand some issues there? also - do a research online for competitors and what in headroom we can enhance to make it better. You can use sub agents.


What this PR resolves

Routes Copilot LLM requests through a local proxy (e.g. Headroom) for token compression, monitoring, and quota management — without modifying the core request logic.


What changed

New: extensions/copilot/src/platform/networking/common/proxyUtils.ts

  • isLLMEndpoint(url) — restricts interception to known Copilot LLM hostnames (api.githubcopilot.com and variants)
  • getConfiguredProxyUrl() — reads the COPILOT_PROXY_URL env var
  • maybeInterceptUrlThroughProxy() — rewrites the request URL to route through the proxy and injects X-Original-Url / X-Original-Host headers so the proxy knows where to forward

Modified: extensions/copilot/src/platform/endpoint/common/capiClient.ts

  • Added ProxyInterceptingFetcherService — wraps the internal fetcher to intercept CAPI requests (standard Copilot chat completions) when COPILOT_PROXY_URL is set and the resolved URL is an LLM endpoint

Modified: extensions/copilot/src/platform/networking/common/networking.ts

  • Wired getConfiguredProxyUrl() and maybeInterceptUrlThroughProxy() into networkRequest() for direct URL / BYOK requests
  • Extracted inline proxy functions to proxyUtils.ts

Coverage summary

Request path Intercepted
CAPI (standard Copilot chat) Yes — via ProxyInterceptingFetcherService
Direct URL / BYOK endpoints Yes — via networkRequest()
Non-LLM CAPI requests No — filtered by isLLMEndpoint()

How to test

COPILOT_PROXY_URL=http://localhost:4017 ./scripts/code.sh

Make a chat request in Copilot. The POST to api.githubcopilot.com/chat/completions will be intercepted and routed through the proxy.


Evidence — real token compression with Headroom

Tested end-to-end with Headroom v0.24.0:

  • 2.2k tokens saved in a single session
  • $0.0044 lifetime savings (proxy compression only)
  • Checkpoint recorded at 14:59 on 11 June 2026
  • Status: Healthy — no request failures observed
Capture d’écran 2026-06-11 à 15 02 07
Enregistrement.de.l.ecran.2026-06-11.a.15.00.13.mov

Todo

  • Intercept direct URL / BYOK requests
  • Add support for CAPI requests (via ProxyInterceptingFetcherService)
  • Test end-to-end with real Headroom instance (token compression confirmed)
  • Measure token reduction impact and latency overhead at scale
  • Handle streaming responses correctly through the proxy
  • Fix remaining build issues (@parcel/watcher native module, mermaid types)

Copilot AI review requested due to automatic review settings June 11, 2026 00:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@damnthonyy

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants