Skip to content

feat(fs_write): fuzzy str_replace with 3-strategy fallback chain#3725

Open
sandikodev wants to merge 1 commit intoaws:mainfrom
sandikodev:feat/fs-write-fuzzy-str-replace
Open

feat(fs_write): fuzzy str_replace with 3-strategy fallback chain#3725
sandikodev wants to merge 1 commit intoaws:mainfrom
sandikodev:feat/fs-write-fuzzy-str-replace

Conversation

@sandikodev
Copy link
Copy Markdown

Problem

The current str_replace implementation uses exact byte matching only. When the model's old_str has minor differences from the actual file content — indentation drift, whitespace normalization, or small edits in surrounding context — the match fails with:

no occurrences of "..." were found

The model then either:

  1. Retries with the same wrong old_str (wastes tokens)
  2. Falls back to sed -i or other shell commands (dangerous — no preview, no validation, can corrupt files)

This is a systematic productivity killer: every failed str_replace costs tokens, breaks flow, and risks file corruption.

Root Cause Analysis

Compared to opencode and cline, Kiro CLI's str_replace has no fallback strategies. Both tools implement multi-strategy matching to handle the inevitable gap between model output and exact file content.

Solution

Implement str_replace_fuzzy() with a 3-strategy fallback chain:

Strategy 1: Exact match (unchanged)

Fast path — if old_str matches exactly, use it directly.

Strategy 2: Line-trimmed match

Compare lines after trim(), then replace the original (indented) text. Handles the most common failure mode: indentation drift between model output and actual file.

File:     "    let x = 1;\n    let y = 2;"
old_str:  "let x = 1;\nlet y = 2;"   ← no indentation
Result:   matches, replaces original indented text ✓

Strategy 3: Block-anchor match (Levenshtein similarity)

Use first+last line as anchors, score middle lines with Levenshtein distance, pick the best candidate above a 0.6 similarity threshold. Handles minor edits in surrounding context lines.

File:     "fn calculate() {\n    let result = a + b;\n    return result;\n}"
old_str:  "fn calculate() {\n    let result = a + b; // sum\n    return result;\n}"
Result:   anchors match, middle line similarity ~0.7 > 0.6 threshold ✓

All strategies are unambiguous-only: if multiple candidates score equally, the match is rejected and a clear error is returned.

Testing

# Run the targeted tests
cargo test -p chat_cli 'fs_write::tests::fuzzy'
cargo test -p chat_cli 'fs_write::tests::levenshtein'
cargo test -p chat_cli 'fs_write::tests::line_trimmed'

# Run the full test suite
cargo test -p chat_cli

8 new tests cover:

  • Exact match (happy path)
  • Ambiguous exact match rejection
  • Indentation drift via line-trimmed strategy
  • Minor middle-line edits via block-anchor strategy
  • No-match error message (mentions fs_read, warns against sed)
  • Levenshtein correctness
  • Line-trimmed match with original indentation preserved
  • Line-trimmed ambiguity rejection

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@sandikodev sandikodev force-pushed the feat/fs-write-fuzzy-str-replace branch 4 times, most recently from a6fe223 to cff84ae Compare April 4, 2026 22:02
@sandikodev sandikodev force-pushed the feat/fs-write-fuzzy-str-replace branch from cff84ae to e7a39c2 Compare April 4, 2026 22:31
@sandikodev
Copy link
Copy Markdown
Author

Analysis & Evolution of This PR

This PR went through several rounds of analysis and refinement. Here is the full chronological record.


Root Cause Analysis

The failure mode we investigated:

Execution failed:
no occurrences of "..." were found

The pipeline from model to old_str is:

LLM response (streaming JSON)
  → parser.rs: accumulates JSON chunks into AssistantToolUse.args (serde_json::Value)
  → tool_manager.rs:866: serde_json::from_value::<FsWrite>(value.args)
  → FsWrite::StrReplace { old_str: String, ... }

No preprocessing anywhere in this pipeline. old_str is byte-for-byte what the model sends in the JSON response.

The model builds old_str from its conversation context — not from reading the file. When code is displayed in the conversation (e.g. in a diff preview or a previous message), it may use different whitespace than the actual file (tabs vs spaces, different indentation levels). The model uses that visual representation as old_str, which then fails exact matching against the file.


What Was Implemented

Strategy 1: Exact match — unchanged, fast path.

Strategy 2: Line-trimmed match — compares lines after trim(), then replaces using byte offsets computed from a prefix-sum table. Handles the most common failure mode: indentation drift between model output and actual file.

Strategy 3: Block-anchor match — uses first+last line as anchors, scores middle lines with Levenshtein similarity (O(n) space), picks the best candidate above a 0.6 threshold. Handles minor edits in surrounding context lines.

File freshness checkfs_read now records the file mtime into FileLineTracker.last_read_mtime. fs_write str_replace checks the current mtime before writing; if the file was modified externally it returns a clear error asking the model to re-read before retrying.

Empty old_str validation — rejected in validate() before reaching fuzzy matching.

Tool description updatedtool_index.json now reflects fuzzy tolerance and reinforces read-before-write / no-sed-fallback guidance, keeping description and implementation consistent.


Bugs Found and Fixed During Code Review

Bug Impact Fix
replacen(&matched, ...) replaces first occurrence, not matched position Wrong location modified if matched text appears elsewhere Return (start, end) byte range, splice directly
first == last in block anchor → false positive Symmetric blocks (e.g. }/}) could match incorrectly Skip candidates where first == last
similarity_score loop not bounded to actual window Could score lines outside the candidate window Add ci < end guard
Levenshtein O(m×n) space Memory waste for long lines Rolling-row O(n) space
Denominator used .len() (bytes) not char count Wrong score for multi-byte UTF-8 Use .chars().count()
strip_trailing_empty didn't strip leading empty lines Model-sent old_str with leading blank lines would fail Renamed to strip_empty_boundary_lines, strips both ends
line_start_offset was O(n) per call Redundant iteration Replaced with build_line_offsets prefix-sum, O(1) lookup
block_anchor_match computed lengths twice Minor inefficiency Compute once, reuse for both scoring and byte range
tool_index.json description didn't mention fuzzy tolerance Split brain between description and implementation Updated to say 'minor indentation differences are tolerated'

Relationship to PR #3724

PR #3724 added read-before-write guidance to the tool description and improved the error message. This PR supersedes it — all of that is included here plus the actual fuzzy implementation that makes the guidance accurate.

…le freshness check

The current str_replace implementation uses exact byte matching only.
When the model's old_str has minor differences from the file (indentation
drift, whitespace, or small context edits), the match fails and the model
either retries wastefully or falls back to destructive shell commands.

Implement str_replace_fuzzy() with a 3-strategy fallback chain inspired
by opencode and cline's diff-apply approaches:

1. Exact match — unchanged behaviour for the common case
2. Line-trimmed match — compares lines after trim(), then replaces using
   byte offsets (prefix-sum table) into the original content. Handles
   indentation drift (tab vs spaces, different indent levels).
3. Block-anchor match — uses first+last line as anchors, scores middle
   lines with Levenshtein similarity, picks the best candidate above a
   0.6 threshold. Handles minor edits in surrounding context lines.

Also adds file freshness checking:
- fs_read now records the file mtime into FileLineTracker.last_read_mtime
  whenever it reads a file
- fs_write str_replace checks the current mtime against the recorded value
  before writing; if the file was modified externally it returns a clear
  error asking the model to re-read before retrying

Also:
- validate() rejects empty old_str before reaching fuzzy matching
- tool_index.json description updated to reflect fuzzy tolerance and
  reinforce read-before-write / no-sed-fallback guidance

Key correctness properties:
- Strategies 2 and 3 return byte ranges — replacement is always at the
  correct position even if matched text appears elsewhere in the file
- block_anchor_match skips first==last anchors (false positive guard)
- similarity_score respects actual content window bounds
- levenshtein uses O(n) rolling-row space, char count for denominator
- build_line_offsets prefix-sum gives O(1) offset lookup
- strip_empty_boundary_lines handles both leading and trailing empty lines

11 tests cover all strategies, edge cases, and error messages.
@sandikodev sandikodev force-pushed the feat/fs-write-fuzzy-str-replace branch from e7a39c2 to fc482ca Compare April 4, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant