Skip to content

Conversation

@KrishnaShuk
Copy link
Contributor

Description

#868
This PR implements the FallbackAdapter for LLMs, bringing feature parity with the Python SDK. This component allows agents to automatically switch to backup LLM providers if the primary provider fails, times out, or becomes unavailable, ensuring high availability for production agents.

Python(agents) SDK code for fallback adapter: https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/llm/fallback_adapter.py

Changes Made

  • Created packages/agents/src/llm/FallbackAdapter.ts implementing the FallbackAdapter and FallbackLLMStream classes.
  • Implemented Priority Failover: Accepts a list of LLMs and attempts them in order.
  • Implemented Active Health Check: Background recovery logic that uses the actual ChatContext to verify when a failed provider has recovered (instead of a simple timer).
  • Implemented State Tracking: Emits llm_availability_changed events when providers go down or recover.
  • Implemented Glitch Protection: Respects retryOnChunkSent to prevent the agent from repeating partial sentences if a stream fails mid-generation.
  • Exported the new adapter in packages/agents/src/llm/index.ts.

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
  • Video demo: N/A (Backend logic change, verified via Unit Tests)

Testing

  • Automated tests added/updated
  • All tests pass (8 tests covering initialization, failover, total failure, events, and stream delegation)
  • Make sure both restaurant_agent.ts and realtime_agent.ts work properly (This change is additive/opt-in and does not affect existing agents unless configured to use the adapter)

@changeset-bot
Copy link

changeset-bot bot commented Dec 16, 2025

🦋 Changeset detected

Latest commit: e8dc5c5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@KrishnaShuk KrishnaShuk marked this pull request as draft December 17, 2025 05:10
@KrishnaShuk KrishnaShuk marked this pull request as ready for review December 17, 2025 05:16
@toubatbrian toubatbrian self-requested a review January 5, 2026 09:26
Copy link
Contributor

@toubatbrian toubatbrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested fallback yet? Could you add a simple example ts file under examples showcasing the basic usage of this class. Thanks!

@KrishnaShuk
Copy link
Contributor Author

I won’t be able to do this today. I’ll test the fallback and add a simple example TS file under examples tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants