Skip to content

🤖 fix: show unsupported attach_file outputs#3242

Open
ThomasK33 wants to merge 5 commits intomainfrom
file-attach-4z46
Open

🤖 fix: show unsupported attach_file outputs#3242
ThomasK33 wants to merge 5 commits intomainfrom
file-attach-4z46

Conversation

@ThomasK33
Copy link
Copy Markdown
Member

Summary

Shows unsupported attach_file outputs (for example .webm) in the chat UI instead of failing the tool outright, while preventing those unsupported file bytes from being sent to providers as model attachments.

Background

attach_file previously returned an error when a readable file was not one of the model-supported attachment types. This made files such as videos invisible to the user even though Mux could still display them usefully in the transcript.

Implementation

Unsupported readable files now fall back to a display-only file-data result tagged with providerOptions.mux.displayOnly. The attach-file tool renderer shows display-only files, including inline video/audio playback when the browser supports the media type, and request-time tool-result rewriting strips the bytes before provider calls while leaving a textual notice for the model.

Validation

  • make static-check
  • make typecheck && make fmt-check && make lint
  • Display-only attach_file smoke test for .webm fallback and provider-rewrite byte stripping
  • Attempted targeted Bun tests for affected files, but this environment cannot load sharp because libstdc++.so.6 is missing.

Risks

Low-to-moderate risk in attach-file rendering and request rewriting paths. Supported attachments keep the existing media path; unsupported display-only files are explicitly marked and stripped before provider requests.


Generated with mux • Model: openai:gpt-5.5 • Thinking: high • Cost: $14.15

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@coder-agents-review coder-agents-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-pass review (Netero only). The full review panel has not yet reviewed this PR; these are mechanical findings to address before the panel spends parallel review time.

The display-only attach_file fallback is a clean design: unsupported files get shown to the user with inline playback, and the bytes are stripped before provider requests. The UnsupportedAttachmentTypeError subclass with instanceof gating, the inner catch fallback, and the Zod schema union are all well-structured.

1 P2, 2 P3, 1 Nit.

"base64 of a non-empty buffer is always non-empty, so displayFile.data.length > 0 is always true when the function returns. The assert on line 64 never fires." (Netero, on the redundant guard)

🤖 This review was automatically generated with Coder Agents.

Comment thread src/browser/features/Tools/AttachFileToolCall.tsx Outdated
Comment thread src/browser/features/Tools/AttachFileToolCall.tsx Outdated
Comment thread src/node/utils/messages/extractToolMediaAsUserMessages.test.ts Outdated
Comment thread src/node/utils/attachments/readAttachmentFromPath.ts Outdated
Comment thread src/node/services/tools/attach_file.ts Outdated
@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@coder-agents-review coder-agents-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panel review (12 reviewers + Netero). All five R1 findings (DEREM-1 through DEREM-5) verified fixed; clean work.

The display-only fallback design is sound: the UnsupportedAttachmentTypeError gating, the shared displayOnlyFileParts.ts module, the request-time stripping, and the Zod schema union are well-structured. The fix earns its complexity. Pariston tried to build a case against the approach and could not: "the problem is correctly understood, the display-only result + request-time stripping is proportional to it."

1 P2, 9 P3, 5 Nit, 2 Note.

The P2 (DEREM-9) is a type-safety gap: DisplayOnlyFilePart reuses the AI SDK's own file-data type, so the only barrier between 10MB of video bytes and the provider is remembering to call extractAttachmentsFromToolOutput. The defense is procedural, not structural.

DEREM-12 deserves explicit attention: adding txt, csv, json to the display-only map silently changes the attach_file contract for text files. Before this PR, attach_file("config.json") errored and directed the model to file_read. Now it "succeeds" with a display-only card. This needs a human decision.

"Any new code path that constructs provider messages without routing through extractAttachmentsFromToolOutput would silently forward up to 10 MB of video or audio to the LLM provider, despite the UI displaying 'not sent to the model as a file attachment.'" (Mafuuu, on the type-safety gap)

🤖 This review was automatically generated with Coder Agents.

Comment thread src/common/utils/attachments/displayOnlyFileParts.ts Outdated
Comment thread src/node/utils/attachments/readAttachmentFromPath.ts Outdated
Comment thread src/browser/features/Tools/AttachFileToolCall.tsx Outdated
Comment thread src/node/utils/attachments/readAttachmentFromPath.ts
Comment thread src/node/services/tools/attach_file.ts Outdated
Comment thread src/browser/features/Tools/AttachFileToolCall.tsx Outdated
Comment thread src/browser/features/Tools/AttachFileToolCall.tsx Outdated
Comment thread src/browser/features/Tools/AttachFileToolCall.tsx Outdated
Comment thread src/node/utils/messages/toolResultAttachments.ts
Comment thread src/browser/features/Tools/AttachFileToolCall.tsx
@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

Copy link
Copy Markdown

@coder-agents-review coder-agents-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

17 of 18 R2 findings addressed in 2257aca. Excellent turnaround, including the discriminated union refactor (DEREM-15) and the display_file type rename (DEREM-9).

Further review is blocked until one remaining finding is acknowledged:

DEREM-25 (Note, toolResultAttachments.ts:105): The model receives two overlapping text notices for every display-only file. The original tool handler text and the rewriter's byte-replacement placeholder both reach the provider. This is informational, not a bug. A brief acknowledgment or "accepted" is sufficient to unblock.

🤖 This review was automatically generated with Coder Agents.

@chatgpt-codex-connector

This comment has been minimized.

@ThomasK33
Copy link
Copy Markdown
Member Author

Acknowledged DEREM-25: accepted. The duplicate model-visible notices are intentional for now: the tool result records the user-visible outcome and the request-time replacement documents that bytes were stripped before provider submission.

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

This comment has been minimized.

@ThomasK33
Copy link
Copy Markdown
Member Author

Addressed Codex source-file fallback feedback in fdfc06e: unknown extensions now use the octet-stream display fallback only after binary-content sniffing, and common source/text files (including extensionless text like Makefile) continue to reject so the model can use file_read.

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

This comment has been minimized.

@ThomasK33
Copy link
Copy Markdown
Member Author

Addressed the Codex exact-prose test feedback in 86bd853: the attach_file test now asserts behavioral text properties instead of pinning the full notice copy.

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

DEREM-25 has been acknowledged as accepted in #3242 (comment) and again in the resolved DEREM-25 thread. Please re-check for any remaining open issues.

@ThomasK33
Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

1 similar comment
@ThomasK33
Copy link
Copy Markdown
Member Author

/coder-agents-review

Copy link
Copy Markdown

@coder-agents-review coder-agents-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combined R4/R5 review. The R4 panel review (7 reviewers + Netero) could not be posted due to persistent GitHub API 422 errors (the PR head advanced while retrying, invalidating diff positions). These findings are from R4's panel and are posted here for the first time.

All prior fixes verified (DEREM-1 through DEREM-25). DEREM-26 (P2, missing text extensions) addressed in fdfc06e with expanded TEXT_EXTENSIONS_REQUIRING_FILE_READ and binary-content sniffing. CI is green.

1 P3, 3 Nit, 1 Note remaining.

DEREM-27 is the only substantive open finding: three R4 reviewers independently proposed the same defensive fallback for malformed display_file parts. The Nits and Note are cleanup items.

Pariston (R4): "I tried to build a case against this and could not. The problem is correctly understood. The solution is proportional."

🤖 This review was automatically generated with Coder Agents.

continue;
}

if (isDisplayOnlyFilePart(item)) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [DEREM-27] isDisplayOnlyFilePart guard requires full metadata validity (providerOptions.mux.displayOnly === true and typeof size === "number"). A display_file part from chat.jsonl with corrupted or schema-drifted providerOptions fails the guard and falls through to newValue.push(item) with up to ~13MB of base64.

Three R4 reviewers converged on the same fix: after the isDisplayOnlyFilePart block, add:

const itemRecord = item as Record<string, unknown>;
if (itemRecord.type === "display_file" && typeof itemRecord.data === "string") {
  didChange = true;
  newValue.push({ type: "text", text: "[Display-only file removed before provider request.]" });
  continue;
}

(Mafuuu P3, Meruem P3, Kite P3)

🤖

// chat history never persists unexpectedly large base64 payloads.
export const MAX_ATTACH_FILE_SIZE_BYTES = 10 * 1024 * 1024; // 10MB

class UnsupportedAttachmentTypeError extends Error {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [DEREM-28] UnsupportedAttachmentTypeError is vestigial after the DEREM-15 discriminated union fix. No code checks instanceof; the class provides no behavioral differentiation. Replace with plain Error in createUnsupportedAttachmentError.

(Meruem)

🤖

function filterResultForDisplay(result: unknown): unknown {
if (!isToolContentResult(result)) {
return result;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [DEREM-29] filterResultForDisplay duplicates the type === "media" stripping from GenericToolCall.tsx:36. A shared utility would eliminate the copy.

(Zoro)

🤖

}

return {
type: "display",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit [DEREM-30] readAttachmentFromPath wrapper calls resolvePathWithinCwd redundantly; readAttachFileFromPath already resolved the path and result.file.resolvedPath is available.

(Zoro)

🤖

};
}

export function isDisplayOnlyFilePart(value: unknown): value is DisplayOnlyFilePart {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [DEREM-31] isDisplayOnlyFilePart rejection path has no direct unit tests. Tested indirectly via stripping tests; risk is low. Worth covering cheaply.

(Bisky)

🤖

Copy link
Copy Markdown

@coder-agents-review coder-agents-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKED. The 5 open findings from R4 (posted in R5 after GitHub API recovery) have no author response yet. Further review is blocked until the author responds or pushes fixes.

Open findings:

  • DEREM-27 (P3) toolResultAttachments.ts:146: Defensive fallback for malformed display_file parts.
  • DEREM-28 (Nit) readAttachmentFromPath.ts:20: Vestigial UnsupportedAttachmentTypeError.
  • DEREM-29 (Nit) AttachFileToolCall.tsx:55: filterResultForDisplay duplication.
  • DEREM-30 (Nit) readAttachmentFromPath.ts:297: Redundant path resolution.
  • DEREM-31 (Note) displayOnlyFileParts.ts:55: isDisplayOnlyFilePart rejection untested.

For the 3 Nits and 1 Note, a brief acknowledgment ("accepted" / "will fix" / "deferred") is sufficient.

🤖 This review was automatically generated with Coder Agents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant