Skip to content

Commit 898f6a2

Browse files
authored
Merge pull request #3364 from liyun95/v2.6.x
Revert "update json shredding configs"
2 parents 22beef9 + 6a08d25 commit 898f6a2

File tree

1 file changed

+12
-14
lines changed

1 file changed

+12
-14
lines changed

site/en/userGuide/schema/json-shredding.md

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -88,13 +88,13 @@ The final phase leverages the optimized storage layout to intelligently select t
8888

8989
## Enable JSON shredding
9090

91-
To activate the feature, set `common.enabledJSONShredding` to `true` in your `milvus.yaml` configuration file. New data will automatically trigger the shredding process.
91+
To activate the feature, set `common.enabledJSONKeyStats` to `true` in your `milvus.yaml` configuration file. New data will automatically trigger the shredding process.
9292

9393
```yaml
9494
# milvus.yaml
9595
...
9696
common:
97-
enabledJSONShredding: true # Indicates whether to enable JSON key stats build and load processes
97+
enabledJSONKeyStats: true # Indicates whether to enable JSON key stats build and load processes
9898
...
9999
```
100100

@@ -112,34 +112,34 @@ For most users, once JSON shredding is enabled, the default settings for other p
112112
<th><p>Tuning Advice</p></th>
113113
</tr>
114114
<tr>
115-
<td><p><code>common.enabledJSONShredding</code></p></td>
115+
<td><p><code>common.enabledJSONKeyStats</code></p></td>
116116
<td><p>Controls whether the JSON shredding build and load processes are enabled.</p></td>
117117
<td><p>false</p></td>
118118
<td><p>Must be set to <strong>true</strong> to activate the feature.</p></td>
119119
</tr>
120120
<tr>
121-
<td><p><code>common.usingjsonShreddingForQuery</code></p></td>
121+
<td><p><code>common.usingJsonStatsForQuery</code></p></td>
122122
<td><p>Controls whether Milvus uses shredded data for acceleration.</p></td>
123123
<td><p>true</p></td>
124124
<td><p>Set to <strong>false</strong> as a recovery measure if queries fail, reverting to the original query path.</p></td>
125125
</tr>
126126
<tr>
127-
<td><p><code>queryNode.mmap.jsonShredding</code></p></td>
128-
<td><p>Determines whether Milvus uses mmap when loading shredding data.</p><p>For details, refer to <a href="https://zilliverse.feishu.cn/wiki/P3wrwSMNNihy8Vkf9p6cTsWYnTb">Use mmap</a>.</p></td>
127+
<td><p><code>queryNode.mmap.jsonStats</code></p></td>
128+
<td><p>Determines whether Milvus uses mmap when loading shredding data.</p><p>For details, refer to <a href="mmap.md">Use mmap</a>.</p></td>
129129
<td><p>true</p></td>
130130
<td><p>This setting is generally optimized for performance. Only adjust it if you have specific memory management needs or constraints on your system.</p></td>
131131
</tr>
132132
<tr>
133-
<td><p><code>dataCoord.jsonShreddingMaxColumns</code></p></td>
133+
<td><p><code>dataCoord.jsonStatsMaxShreddingColumns</code></p></td>
134134
<td><p>The maximum number of JSON keys that will be stored in shredded columns. </p><p>If the number of frequently appearing keys exceeds this limit, Milvus will prioritize the most frequent ones for shredding, and the remaining keys will be stored in the shared column.</p></td>
135135
<td><p>1024</p></td>
136136
<td><p>This is sufficient for most scenarios. For JSON with thousands of frequently appearing keys, you may need to increase this, but monitor storage usage.</p></td>
137137
</tr>
138138
<tr>
139-
<td><p><code>dataCoord.jsonShreddingRatioThreshold</code></p></td>
139+
<td><p><code>dataCoord.jsonStatsShreddingRatioThreshold</code></p></td>
140140
<td><p>The minimum occurrence ratio a JSON key must have to be considered for shredding into a shredded column.</p><p>A key is considered frequently appearing if its ratio is above this threshold.</p></td>
141141
<td><p>0.3</p></td>
142-
<td><p><strong>Increase</strong> (e.g., to 0.5) if the number of keys that meet the shredding criteria exceeds the <code>dataCoord.jsonShreddingMaxColumns</code> limit. This makes the threshold stricter, reducing the number of keys that qualify for shredding.</p><p><strong>Decrease</strong> (e.g., to 0.1) if you want to shred more keys that appear less frequently than the default 30% threshold.</p></td>
142+
<td><p><strong>Increase</strong> (e.g., to 0.5) if the number of keys that meet the shredding criteria exceeds the <code>dataCoord.jsonStatsMaxShreddingColumns</code> limit. This makes the threshold stricter, reducing the number of keys that qualify for shredding.</p><p><strong>Decrease</strong> (e.g., to 0.1) if you want to shred more keys that appear less frequently than the default 30% threshold.</p></td>
143143
</tr>
144144
</table>
145145

@@ -231,15 +231,13 @@ This test focused on querying sparse, nested keys that fall into the "shared" ca
231231

232232
1. Next, verify that the data has been loaded by running `show loaded-json-stats` on the query node. The output will display details about the loaded shredded data for each query node.
233233

234-
- **What if I encounter an error?**
235-
236-
If the build or load process fails, you can quickly disable the feature by setting `common.enabledJSONShredding=false`. To clear any remaining tasks, use the `remove stats-task <task_id>` command in [Birdwatcher](birdwatcher_usage_guides.md). If a query fails, set `common.usingjsonShreddingForQuery=false` to revert to the original query path, bypassing the shredded data.
237-
238234
- **How do I select between JSON shredding and JSON indexing?**
239235

240236
- **JSON shredding** is ideal for keys that appear frequently in your documents, especially for complex JSON structures. It combines the benefits of columnar storage and inverted indexing, making it well-suited for read-heavy scenarios where you query many different keys. However, it is not recommended for very small JSON documents as the performance gain is minimal. The smaller the proportion of the key's value to the total size of the JSON document, the better the performance optimization from shredding.
241237

242238
- **JSON indexing** is better for targeted optimization of specific key-based queries and has lower storage overhead. It's suitable for simpler JSON structures. Note that JSON shredding does not cover queries on keys inside arrays, so you need a JSON index to accelerate those.
243239

244-
For details, refer to [JSON Field Overview](json-field-overview.md#Next-Accelerate-JSON-queries).
240+
- **What if I encounter an error?**
241+
242+
If the build or load process fails, you can quickly disable the feature by setting `common.enabledJSONKeyStats=false`. To clear any remaining tasks, use the `remove stats-task <task_id>` command in [Birdwatcher](birdwatcher_usage_guides.md). If a query fails, set `common.usingJsonStatsForQuery=false` to revert to the original query path, bypassing the shredded data.
245243

0 commit comments

Comments
 (0)