Give Your AI Agent Local Speech: OpenAI-Compatible API + MCP

[LauraGPT]

Give Your AI Agent Local Speech

FunASR provides two self-hosted ways to add speech transcription to agents and applications:

• OpenAI-compatible API for OpenAI SDKs, LangChain, LlamaIndex, AutoGen, CrewAI, Dify, n8n, HTTP nodes, and existing apps.
• MCP server for Claude Code, Claude Desktop, Cursor, and other MCP clients.

Both paths run locally. Start with CPU for the simplest smoke test, then switch to CUDA when your environment is ready.

Option 1: OpenAI-compatible API

Install and start the server:

pip install -U funasr fastapi uvicorn python-multipart
funasr-server --model sensevoice --device cpu

Transcribe a file:

curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
  -F file=@audio.wav \
  -F model=sensevoice

Or reuse the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="not-needed",
)

with open("audio.wav", "rb") as audio:
    result = client.audio.transcriptions.create(
        model="sensevoice",
        file=audio,
    )

print(result.text)

The example server supports model aliases including sensevoice, paraformer, paraformer-en, and fun-asr-nano. Query GET /v1/models for the source of truth on a running server. Use response_format=verbose_json when your client needs returned segments.

Option 2: MCP for Claude and Cursor

Clone FunASR and install the Python package:

git clone https://github.com/modelscope/FunASR.git
cd FunASR
pip install -U funasr

Add the server to your MCP configuration. Replace the path below with the absolute path to your clone:

{
  "mcpServers": {
    "funasr": {
      "command": "python",
      "args": [
        "/absolute/path/FunASR/examples/mcp_server/funasr_mcp.py"
      ],
      "env": {
        "FUNASR_DEVICE": "cpu",
        "FUNASR_MODEL": "iic/SenseVoiceSmall"
      }
    }
  }
}

Then ask your assistant to transcribe a local file, for example:

Transcribe /absolute/path/to/meeting.wav and summarize the decisions.

What the default MCP path provides

• Local transcription with SenseVoiceSmall.
• Mandarin, Cantonese, English, Japanese, and Korean recognition.
• Automatic language detection or explicit zh, yue, en, ja, and ko hints.
• VAD segmentation for longer audio.
• Segment timestamps when the configured model returns them.
• A configurable model and device through FUNASR_MODEL and FUNASR_DEVICE.

Capabilities are model- and interface-specific. The default MCP path focuses on clean transcription: it does not advertise speaker diarization, and its cleaned text removes SenseVoice special tags. For emotion, audio-event, or speaker metadata, use a model and interface that explicitly returns those fields.

Documentation

• OpenAI-compatible API quickstart
• MCP server setup
• Client recipes
• JavaScript and TypeScript recipes
• Dify, n8n, and workflow recipes
• Security and gateway guide
• Deployment matrix

If you try either path, share the client or framework, audio language, and CPU/GPU environment you used. Reproducible integration feedback is especially useful.