feat: add langfuse plugin for LLM observability by sihyeonn · Pull Request #13095 · apache/apisix

sihyeonn · 2026-03-18T07:29:47Z

Description

Add a new langfuse plugin that sends LLM request traces to Langfuse for AI observability. The plugin uses the batch-processor-manager pattern (same as http-logger) to send trace data via Langfuse's ingestion API.

Key features:

Auto-detect AI endpoints via configurable URI suffix patterns
Generate trace-create and generation-create batch items per request
W3C traceparent header parsing and propagation for distributed tracing
Token usage extraction with priority: ctx.ai_token_usage (ai-proxy) > nginx vars > response body parsing
Support X-Langfuse-Tags and X-Langfuse-Metadata custom headers
encrypt_fields for secret key protection in etcd
Connection keepalive for Langfuse API calls

Which issue(s) this PR fixes:

Fixes #

Checklist

I have explained the need for this PR and the problem it solves
I have explained the changes or the new features added to this PR
I have added tests corresponding to this change
I have updated the documentation to reflect this change
I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Baoyuantop · 2026-03-18T11:05:08Z

This PR contains three fundamentally different features:

Add a Langfuse logging plugin (apisix/plugins/langfuse.lua, line 687)
Consumer label encryption/decryption (apisix/admin/consumers.lua + apisix/consumer.lua)
ai-proxy consumer_label authentication source (apisix/plugins/ai-proxy/base.lua + schema.lua)

Apache APISIX community practice requires each PR to focus on a single feature. These three features have different review dimensions, testing requirements, and regression risks; please separate them into independent PRs.

Baoyuantop · 2026-04-01T03:59:02Z

Hi @sihyeonn, following up on the previous review comments. Please let us know if you have any updates. Thank you.

Baoyuantop · 2026-04-03T03:45:14Z

Hi @sihyeonn, my suggestion is to only keep the feature-related code in the current PR, and submit other changes in separate PRs.

Add a new langfuse plugin that sends AI request traces to Langfuse via its ingestion API using batch-processor-manager pattern. Features: - Auto-detect AI endpoints (configurable patterns) - Generate trace-create and generation-create batch items per request - W3C traceparent header parsing and propagation - Token usage extraction (ai-proxy ctx > nginx vars > response body) - Support for X-Langfuse-Tags and X-Langfuse-Metadata headers - Basic auth with encrypt_fields for secret key protection - Connection keepalive for Langfuse API calls Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

Move langfuse_host, langfuse_public_key, langfuse_secret_key, ssl_verify, timeout, detect_ai_requests, and ai_endpoints from per-route schema to metadata_schema (plugin_attr), following the opentelemetry plugin pattern. Per-route schema now only contains include_metadata. Remove encrypt_fields (custom feature, not in OSS). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

Remove /v1-prefixed entries since has_suffix matching already covers them via shorter suffixes (e.g. /chat/completions matches both /chat/completions and /v1/chat/completions). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

Baoyuantop

Please add plugin doc

Baoyuantop · 2026-04-13T13:50:16Z

+
+local _M = {
+    version = 0.1,
+    priority = 398,


clickhouse-logger has same priority with langfuse

Baoyuantop · 2026-04-13T13:51:31Z

+    if not trace_id then
+        local request_id = core.request.header(ctx, "X-Request-Id")
+        if request_id then
+            trace_id = request_id


According to the W3C traceparent specification, the trace_id must consist of 32 hex characters. The X-Request-Id is typically in UUID format (with hyphens, 36 characters); using it directly as the trace_id will result in a traceparent header that does not comply with the specification. Recommendation: Either remove the hyphens during conversion, or do not use the X-Request-Id as the trace_id.

Add English documentation for the langfuse plugin covering: - Plugin description and purpose - Plugin metadata attributes (global config via plugin_attr) - Per-route schema attributes - Enable/disable instructions with examples - Register in docs config.json under Loggers Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

- Change priority from 398 to 396 to avoid conflict with clickhouse-logger - Strip hyphens from X-Request-Id and UUID-generated IDs when used as trace_id, ensuring compliance with W3C traceparent spec which requires trace_id to be exactly 32 lowercase hex characters - Validate that the resulting trace_id matches the expected format, falling back to a freshly generated UUID if it does not Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

content_by_lua blocks do not inherit globals — add local core require in TEST 3 and TEST 4 to fix 'attempt to index global core (a nil value)'. Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

Baoyuantop · 2026-04-17T03:25:12Z

+local plugin_name = "langfuse"
+local batch_processor_manager = bp_manager_mod.new("langfuse logger")
+
+local metadata_schema = {


Missing encrypt_fields: langfuse_secret_key in plugin_metadata will be stored in plaintext in etcd. Please either add encrypt_fields support or document this risk clearly. See lago.lua L111 for reference.

Baoyuantop · 2026-04-17T03:26:57Z

+    if schema_type == core.schema.TYPE_METADATA then
+        return core.schema.check(metadata_schema, conf)
+    end
+    return core.schema.check(schema, conf)


check_schema uses unwrapped schema: _M.schema is wrapped via batch_processor_manager:wrap_schema(schema), but check_schema validates against the raw schema for non-metadata types. This means batch processor config fields (e.g., batch_max_size) will fail validation. Please change to:

return core.schema.check(_M.schema, conf)

Baoyuantop · 2026-04-17T03:28:04Z

  - udp-logger                     # priority: 400
  - file-logger                    # priority: 399
  - clickhouse-logger              # priority: 398
+  - langfuse                       # priority: 396


langfuse (priority 396) should be listed after tencent-cloud-cls (priority 397) to maintain descending priority order

- Add encrypt_fields for langfuse_secret_key in metadata_schema to prevent plaintext storage in etcd - Fix check_schema to validate against _M.schema (batch-processor-wrapped) instead of raw schema, so batch_max_size and other batch fields pass validation - Fix plugin list order in config.yaml.example to maintain descending priority order (tencent-cloud-cls:397 before langfuse:396) Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. enhancement New feature or request plugin labels Mar 18, 2026

sihyeonn force-pushed the feat/langfuse-plugin branch from 187a719 to d3003c0 Compare April 2, 2026 23:48

sihyeonn added 3 commits April 10, 2026 08:08

chore: deduplicate ai_endpoints default list

9c4b25a

Remove /v1-prefixed entries since has_suffix matching already covers them via shorter suffixes (e.g. /chat/completions matches both /chat/completions and /v1/chat/completions). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

sihyeonn force-pushed the feat/langfuse-plugin branch from d3003c0 to 9c4b25a Compare April 9, 2026 23:08

Baoyuantop reviewed Apr 13, 2026

View reviewed changes

sihyeonn added 2 commits April 14, 2026 07:28

sihyeonn requested a review from Baoyuantop April 16, 2026 22:46

fix(langfuse): require apisix.core in test schema validation blocks

9c450fa

content_by_lua blocks do not inherit globals — add local core require in TEST 3 and TEST 4 to fix 'attempt to index global core (a nil value)'. Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>

Baoyuantop reviewed Apr 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add langfuse plugin for LLM observability#13095

feat: add langfuse plugin for LLM observability#13095
sihyeonn wants to merge 7 commits intoapache:masterfrom
sihyeonn:feat/langfuse-plugin

sihyeonn commented Mar 18, 2026 •

edited

Loading

Uh oh!

Baoyuantop commented Mar 18, 2026

Uh oh!

Baoyuantop commented Apr 1, 2026

Uh oh!

Baoyuantop commented Apr 3, 2026

Uh oh!

Baoyuantop left a comment

Uh oh!

Baoyuantop Apr 13, 2026

Uh oh!

Baoyuantop Apr 13, 2026

Uh oh!

Baoyuantop Apr 17, 2026

Uh oh!

Baoyuantop Apr 17, 2026

Uh oh!

Baoyuantop Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sihyeonn commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Which issue(s) this PR fixes:

Checklist

Uh oh!

Baoyuantop commented Mar 18, 2026

Uh oh!

Baoyuantop commented Apr 1, 2026

Uh oh!

Baoyuantop commented Apr 3, 2026

Uh oh!

Baoyuantop left a comment

Choose a reason for hiding this comment

Uh oh!

Baoyuantop Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Baoyuantop Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Baoyuantop Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Baoyuantop Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Baoyuantop Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sihyeonn commented Mar 18, 2026 •

edited

Loading