Results cache with redis cluster is having the "EXECABORT Transaction discarded because of previous errors." #7255
Description
Problem
Redis Cluster Constraint: Transactions (MULTI/EXEC) can only operate on keys in the same hash slot (same node).
When cache keys hash to different cluster nodes:
- TxPipeline() starts a MULTI transaction
- Multiple SET commands are queued for keys on different nodes
- Exec() attempts to execute → EXECABORT error
Example Scenario:
Key: "frontend:cache:query1" → hashes to node 1 (slot 5234)
Key: "frontend:cache:query2" → hashes to node 2 (slot 10482)
Key: "frontend:cache:query3" → hashes to node 3 (slot 14301)
Transaction tries to SET all 3 keys → EXECABORT (cross-node transaction)
File: pkg/chunk/cache/redis_client.go
The result cache uses Cortex's own Redis client, which has a critical bug in cluster mode:
The Buggy Code
Lines 91-105 - MSet function:
func (c *RedisClient) MSet(ctx context.Context, keys []string, values [][]byte) error {
var cancel context.CancelFunc
if c.timeout > 0 {
ctx, cancel = context.WithTimeout(ctx, c.timeout)
defer cancel()
}
if len(keys) != len(values) {
return fmt.Errorf("MSet the length of keys and values not equal, len(keys)=%d, len(values)=%d", len(keys), len(values))
}
// 🔴 BUG: Always uses transaction regardless of cluster mode
pipe := c.rdb.TxPipeline()
for i := range keys {
pipe.Set(ctx, keys[i], values[i], c.expiration)
}
_, err := pipe.Exec(ctx) // ❌ Fails if keys are on different cluster nodes
return err
}