[bug] Unable to search after upgrade to 4.10.2

[gaeljw]

Describe the bug

Hello folks,

We just upgraded from 4.9.2 to 4.10.2.

In our DEV environment, it worked fine but in our PROD environment, the search doesn't work anymore. We get the following errors.

---

Query: junit (case insensitive, regex disabled)

In the logs:

{"level":"error","message":"Error processing chunk: Repository not found for file: build/defaults.yaml","service":"zoekt-searcher","stack":"Error: Repository not found for file: build/defaults.yaml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at h.<anonymous> (/app/packages/web/.next/server/chunks/4176.js:1:20059)","status":"error","timestamp":"2025-12-10T17:10:25.164Z"}

In the UI:

Failed to search.
Unexpected error: Repository not found for file: build/defaults.yaml

(This file build/defaults.yaml is present in several of our repositories, I don't know why it blocks on it specifically).

---

Another query: file:pom.xml junit

Similar errors in the UI:

Failed to search.
Unexpected error: Repository not found for file: pom.xml

And in the logs:

{"level":"error","message":"{\"statusCode\":400,\"errorCode\":\"FAILED_TO_PARSE_QUERY\",\"message\":\"Failed to parse query \\\"file:\\\" with message: No parse at 5\"}","service":"web-actions","serviceError":{"errorCode":"FAILED_TO_PARSE_QUERY","message":"Failed to parse query \"file:\" with message: No parse at 5","statusCode":400},"stack":"Error: {\"statusCode\":400,\"errorCode\":\"FAILED_TO_PARSE_QUERY\",\"message\":\"Failed to parse query \\\"file:\\\" with message: No parse at 5\"}\n    at j (/app/packages/web/.next/server/chunks/4176.js:1:14163)\n    at /app/packages/web/.next/server/chunks/4176.js:1:12483","status":"error","timestamp":"2025-12-10T17:12:04.776Z"}
{"level":"error","message":"Repository not found for file: pom.xml","service":"web-actions","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at /app/packages/web/.next/server/chunks/4176.js:1:18936","status":"error","timestamp":"2025-12-10T17:12:06.378Z"}
{"level":"error","message":"Repository not found for file: pom.xml","service":"web-actions","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at /app/packages/web/.next/server/chunks/4176.js:1:18936","status":"error","timestamp":"2025-12-10T17:12:06.635Z"}
{"level":"error","message":"Repository not found for file: pom.xml","service":"web-actions","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at /app/packages/web/.next/server/chunks/4176.js:1:18936","status":"error","timestamp":"2025-12-10T17:12:06.939Z"}
{"level":"error","message":"Repository not found for file: pom.xml","service":"web-actions","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at /app/packages/web/.next/server/chunks/4176.js:1:18936","status":"error","timestamp":"2025-12-10T17:12:07.985Z"}
{"level":"error","message":"Repository not found for file: pom.xml","service":"web-actions","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at /app/packages/web/.next/server/chunks/4176.js:1:18936","status":"error","timestamp":"2025-12-10T17:12:08.591Z"}
{"level":"error","message":"Repository not found for file: pom.xml","service":"web-actions","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at /app/packages/web/.next/server/chunks/4176.js:1:18936","status":"error","timestamp":"2025-12-10T17:12:08.635Z"}
{"level":"error","message":"Error processing chunk: Repository not found for file: pom.xml","service":"zoekt-searcher","stack":"Error: Repository not found for file: pom.xml\n    at /app/packages/web/.next/server/chunks/4176.js:3:406\n    at Array.map (<anonymous>)\n    at v (/app/packages/web/.next/server/chunks/4176.js:3:323)\n    at h.<anonymous> (/app/packages/web/.next/server/chunks/4176.js:1:20059)","status":"error","timestamp":"2025-12-10T17:12:11.759Z"}

To reproduce

No reproduction scenario yet

Sourcebot deployment information

Sourcebot version (e.g. v3.0.1): 4.10.2

Additional information

I'm waiting for a complete reindexation happening to see if it helps. I will report back if it does.

[gaeljw]

Additional information: If I restrict the query to a repository that has been indexed after the upgrade, it seems to work fine. I guess after a full reindexation, it'll work fine. I keep you posted :)

[gaeljw]

Update: without intervention and letting the automatic reindexation happen didn't help. I've now deleted all index files and waiting for indexation to complete. No error so far.

[gaeljw]

I confirm: no more issues after deleting all index files and indexing from scratch again.

[brendan-kellam]

Hmm this is suboptimal.. thanks for raising. Will look into this now.

[brendan-kellam]

Context: This error occurs when we cannot find the Repo record in the database for a search result we get back from zoekt. As of #307 (v3.2.1), we bake the Repo's record id in the zoekt shard so we can exactly correlate things. For back compat, we fallback to using the repository's name (see here).

All this to say: I was able to get a repro for this when upgrading from v3.2.0 -> v4.10.3, so the back compat logic we have is broken. However, when I force a re-index for the repository, I no longer get a repro (since the repository id is set).

What's strange is that you still hit this even after the shards were reindex (and probably had been reindexed many many times since March). I'll do some more digging here, very strange. At minimum, I'll fix the back compat logic so we don't fall into this.

[brendan-kellam]

Additional context: prior to v4.10.0, we would just fail silently on this case, so this issue might have been introduced in some other version. Failing silently causes search results that otherwise should be included to not be, so I would like to avoid having that behaviour and instead get to the bottom of this error.