fix(destination): recreate an index with the source's own prefix lengths - #215
Conversation
validateFieldsForIndexes() derived every length from the destination column instead of the index it was recreating: null, or the array cap for an array column. The source's recorded prefix lengths were read into the Index resource and then never used — the class had no getter for them at all. An index only fits under an adapter's byte limit BECAUSE of that prefix. Two large string columns capped at 100 and 20 bytes are a valid index on the source and a 767-byte violation on the destination, so restoring a healthy database failed to recreate its own indexes with 'Invalid index: Index length is longer than the maximum: 767'. Measured on production: 3 customers across 3 regions hit exactly this, most recently four days ago, and one retried nine times in eighty minutes before giving up (DAT-2113). Lengths now come from the resource, with zero — how a source records 'no prefix' for a position — staying no-prefix rather than becoming a zero-length one. Array columns keep their MAX_ARRAY_INDEX_LENGTH override. DAT-2113 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Greptile SummaryThe PR now preserves source index prefix lengths during destination validation and compares those lengths when reconciling an existing index.
Confidence Score: 4/5The PR is not yet safe to merge because unchanged array-column indexes can still be deleted and recreated on every eligible overwrite migration. Index creation replaces an absent source prefix with Files Needing Attention: src/Migration/Destinations/Appwrite.php Important Files Changed
Reviews (3): Last reviewed commit: "fix(destination): compare index prefix l..." | Re-trigger Greptile |
…isting index Greptile on this PR, and correct: indexSpecMatches() omitted lengths, so an Overwrite migration onto an index whose ONLY difference was its prefix treated it as already matching and skipped it, silently keeping the destination's lengths -- reintroducing the very defect this PR fixes, on the overwrite path instead of the create path. The exclusion was reasonable when it was written: lengths were derived per-column from the adapter and not carried on the resource, so comparing them compared nothing. The source's own lengths drive them now, so a length-only difference is a genuine mismatch. Positions are compared as integers so 'no prefix' matches however it is spelled -- the resource records 0, the metadata collection reads back 0 or null. The test asserts the decision rather than the recreate: the in-memory adapter cannot drop an index the destination believes it holds, and that is an adapter limit rather than the behaviour under test. DAT-2113 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Greptile's finding is correct and now fixed.
The exclusion was reasonable when it was written, and its comment said so: lengths were derived per-column from the adapter and not carried on the resource, so comparing them compared nothing. That is no longer true — the source's own lengths drive them now — so the comment and the behaviour both change. Positions compare as integers so "no prefix" matches however it is spelled (the resource records On the test: it asserts the decision — that a length-only difference is no longer reported as Full unit suite green (68 tests), Pint and PHPStan clean. |
|
@greptileai review |
The defect
validateFieldsForIndexes()derives every index length from the destination column —null, orMAX_ARRAY_INDEX_LENGTHfor an array column. The source's recorded prefix lengths are parsed into theIndexresource and then never used: the class had nogetLengths()at all.An index often only fits under an adapter's byte limit because of that prefix. Two large string columns capped at 100 and 20 bytes are a perfectly valid index on the source and a 767-byte violation on the destination — so restoring a healthy database fails to recreate its own indexes:
Why this matters
Measured against Appwrite Cloud production across all six regions: 3 separate customers in 3 regions hit exactly this signature, the most recent four days ago. One of them issued nine in-place restore attempts in eighty minutes, every one failing, before working around it (DAT-2113).
The fix
Lengths come from the resource being recreated. A recorded
0— how a source records "no prefix" for a position, e.g. a short column that needs none — stays no-prefix rather than becoming a zero-length one. Array columns keep theirMAX_ARRAY_INDEX_LENGTHoverride, which is a destination-side constraint rather than a source fact.Tests
AppwriteIndexLengthsTestdrives the realAppwritedestination throughTransfer:[100, 20]is recreated carrying them;0stays no-prefix (asserted in the shape a live production row for this exact case carries,[100, 0], since the metadata collection typeslengthsas an integer array).Verified red against
main: 2 of the 3 fail without the change. Full unit suite green (67 tests), Pint and PHPStan clean.One note for reviewers: the test transfers the schema and the index in two
run()passes, becauseTransfer::GROUP_DATABASES_RESOURCESordersTYPE_INDEXbeforeTYPE_COLUMN— the reverse of what a real source emits. A single pass would offer the index against a table with no columns yet and test the mock's ordering rather than production's.🤖 Generated with Claude Code