Skip to content

Doc 1602: Cloud Topics GA#1469

Open
Feediver1 wants to merge 97 commits intov-WIP/26.1from
DOC-1602
Open

Doc 1602: Cloud Topics GA#1469
Feediver1 wants to merge 97 commits intov-WIP/26.1from
DOC-1602

Conversation

@Feediver1
Copy link
Contributor

@Feediver1 Feediver1 commented Nov 19, 2025

Description

Resolves
https://redpandadata.atlassian.net/browse/DOC-1911 (ticket for Cloud Topics GA in 26.1)
Review deadline:

Page previews

Manage Cloud Topics **Self-Managed version

Manage Cloud Topics RP Cloud - this is the Redpanda Cloud version of Cloud Topics, which you should review here.

redpanda.storage.mode topic property

Choose a storage mode This is in Manage Topics.

Enable Tiered Storage for a cluster

Enable Tiered Storage for specific topics

Configure Tiered Storage using legacy topic properties

Topic property mappings

Cloud Topics glossary entry

Checks

  • New feature
  • Content gap
  • Support Follow-up
  • Small fix (typos, links, copyedits, etc)

@Feediver1 Feediver1 requested a review from a team as a code owner November 19, 2025 01:24
@netlify
Copy link

netlify bot commented Nov 19, 2025

Deploy Preview for redpanda-docs-preview ready!

Name Link
🔨 Latest commit 424c5ba
🔍 Latest deploy log https://app.netlify.com/projects/redpanda-docs-preview/deploys/69bc39aeb401d100081cd30b
😎 Deploy Preview https://deploy-preview-1469--redpanda-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 19, 2025

📝 Walkthrough

Walkthrough

This PR reorganizes Redpanda's topic-related documentation by creating a new manage-topics directory structure under modules/develop/pages/. The previous config-topics.adoc file is relocated to manage-topics/config-topics.adoc with a backwards-compatible page alias. A new index page and Cloud Topics documentation are added. Internal cross-references across approximately 15 documentation files are updated to point to the new location, maintaining link consistency throughout the site.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

  • Most changes follow a consistent, repetitive pattern (cross-reference path updates across multiple files)
  • No functional code logic or control flow changes; purely documentation restructuring
  • New documentation additions are straightforward content additions

Areas to review:

  • Verify all cross-reference path updates are consistently applied across all files
  • Confirm the page alias in modules/develop/pages/manage-topics/config-topics.adoc preserves backwards compatibility
  • Validate the new Cloud Topics documentation content is complete and accurate
  • Check that navigation file updates render the hierarchy correctly

Possibly related PRs

Suggested reviewers

  • paulohtb6
  • micheleRP
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Title check ⚠️ Warning The title 'Doc 1602: Cloud Topics GA' is misleading—it claims 'GA' (General Availability) but the PR objectives and file summaries clearly indicate the feature is in Public Beta, not GA. Update the title to accurately reflect the actual status: 'Doc 1602: Cloud Topics Public Beta' or 'Add Cloud Topics documentation for public beta feature'.
Out of Scope Changes check ❓ Inconclusive The PR includes documentation restructuring (reorganizing topic-related docs into manage-topics folder) and cross-reference updates, which are necessary supporting changes for the Cloud Topics documentation but extend beyond the core feature documentation. Clarify whether the comprehensive reorganization of topic documentation under manage-topics folder was part of the original DOC-1602 scope or represents broader refactoring beyond documenting Cloud Topics.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR successfully documents the Cloud Topics feature as required by DOC-1602, adding a new cloud-topics.adoc page with metadata, prerequisites, limitations, and workflow examples.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check ✅ Passed The PR description is well-structured with the required sections including JIRA ticket reference (DOC-1911), comprehensive page previews with links, and a checked checkbox indicating this is a new feature.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch DOC-1602

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can enforce grammar and style rules using `languagetool`.

Configure the reviews.tools.languagetool setting to enable/disable rules and categories. Refer to the LanguageTool Community to learn more.

@Feediver1 Feediver1 marked this pull request as draft November 19, 2025 01:30
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
modules/develop/pages/manage-topics/config-topics.adoc (1)

92-116: Fix CLI flag typo in alter-config example.
The example uses a long dash before set. It should be --set (two hyphens) or the command fails.

Apply:

- rpk topic alter-config [TOPICS…] —-set cleanup.policy=compact
+ rpk topic alter-config [TOPICS...] --set cleanup.policy=compact

Also applies to: 105-111

🧹 Nitpick comments (2)
modules/develop/pages/manage-topics/index.adoc (1)

1-4: Nice addition. Consider brief “What’s here” bullets.
Add 2–3 bullets linking to child pages (config-topics, cloud-topics) to help orientation.

modules/develop/pages/manage-topics/cloud-topics.adoc (1)

17-24: Beta feature guardrails look good; add cluster-level enable note if required.
If Cloud Topics require a cluster flag (for dev/testing), add a note referencing the relevant cluster property (for example, if any) or document that none is needed.

Also applies to: 32-40

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Jira integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between fad42f5 and d10c4fc.

📒 Files selected for processing (16)
  • modules/ROOT/nav.adoc (1 hunks)
  • modules/console/pages/ui/edit-topic-configuration.adoc (1 hunks)
  • modules/deploy/pages/redpanda/manual/production/dev-deployment.adoc (1 hunks)
  • modules/develop/pages/kafka-clients.adoc (1 hunks)
  • modules/develop/pages/manage-topics/cloud-topics.adoc (1 hunks)
  • modules/develop/pages/manage-topics/config-topics.adoc (2 hunks)
  • modules/develop/pages/manage-topics/index.adoc (1 hunks)
  • modules/develop/pages/produce-data/configure-producers.adoc (1 hunks)
  • modules/get-started/pages/quick-start.adoc (1 hunks)
  • modules/manage/pages/cluster-maintenance/compaction-settings.adoc (1 hunks)
  • modules/manage/pages/cluster-maintenance/disk-utilization.adoc (1 hunks)
  • modules/manage/pages/cluster-maintenance/topic-property-configuration.adoc (1 hunks)
  • modules/manage/partials/audit-logging.adoc (1 hunks)
  • modules/reference/pages/properties/cluster-properties.adoc (1 hunks)
  • modules/reference/pages/properties/topic-properties.adoc (2 hunks)
  • modules/reference/pages/rpk/rpk-redpanda/rpk-redpanda-mode.adoc (1 hunks)
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: Feediver1
Repo: redpanda-data/docs PR: 1153
File: modules/reference/pages/properties/topic-properties.adoc:45-50
Timestamp: 2025-07-16T19:33:20.420Z
Learning: In the Redpanda documentation, topic property cross-references like <<max.compaction.lag.ms>> and <<min.compaction.lag.ms>> require corresponding property definition sections with anchors like [[maxcompactionlagms]] and [[mincompactionlagms]] to prevent broken links.
📚 Learning: 2025-07-16T19:33:20.420Z
Learnt from: Feediver1
Repo: redpanda-data/docs PR: 1153
File: modules/reference/pages/properties/topic-properties.adoc:45-50
Timestamp: 2025-07-16T19:33:20.420Z
Learning: In the Redpanda documentation, topic property cross-references like <<max.compaction.lag.ms>> and <<min.compaction.lag.ms>> require corresponding property definition sections with anchors like [[maxcompactionlagms]] and [[mincompactionlagms]] to prevent broken links.

Applied to files:

  • modules/manage/pages/cluster-maintenance/compaction-settings.adoc
  • modules/reference/pages/properties/cluster-properties.adoc
  • modules/reference/pages/rpk/rpk-redpanda/rpk-redpanda-mode.adoc
  • modules/reference/pages/properties/topic-properties.adoc
  • modules/manage/pages/cluster-maintenance/topic-property-configuration.adoc
  • modules/console/pages/ui/edit-topic-configuration.adoc
  • modules/get-started/pages/quick-start.adoc
  • modules/develop/pages/kafka-clients.adoc
  • modules/deploy/pages/redpanda/manual/production/dev-deployment.adoc
📚 Learning: 2025-05-07T01:06:00.937Z
Learnt from: kbatuigas
Repo: redpanda-data/docs PR: 1113
File: modules/manage/partials/iceberg/use-iceberg-catalogs.adoc:100-107
Timestamp: 2025-05-07T01:06:00.937Z
Learning: In AsciiDoc documentation for Redpanda, the syntax `+` and `--` around content blocks within a `[tabs]` section are valid AsciiDoc formatting elements for tabbed content. The `+` after a tab name (like `rpk::`) indicates that the following block belongs to that tab, and the `--` markers enclose the content for that tab. These are not diff artifacts and should not be removed.

Applied to files:

  • modules/reference/pages/rpk/rpk-redpanda/rpk-redpanda-mode.adoc
📚 Learning: 2025-08-25T21:00:26.626Z
Learnt from: micheleRP
Repo: redpanda-data/docs PR: 1334
File: modules/manage/partials/rbac-dp.adoc:93-98
Timestamp: 2025-08-25T21:00:26.626Z
Learning: In cloud documentation (env-cloud), Security is at the top level navigation, so ACL references should use `security:authorization/rbac/acl.adoc`. In self-managed documentation, Security is nested under Manage, so ACL references use `manage:security/authorization/acl.adoc`. The different xref paths in conditional blocks reflect these different navigation structures.

Applied to files:

  • modules/develop/pages/manage-topics/cloud-topics.adoc
  • modules/manage/pages/cluster-maintenance/topic-property-configuration.adoc
  • modules/ROOT/nav.adoc
📚 Learning: 2025-08-19T19:26:57.364Z
Learnt from: paulohtb6
Repo: redpanda-data/docs PR: 1324
File: modules/deploy/partials/high-availability.adoc:537-537
Timestamp: 2025-08-19T19:26:57.364Z
Learning: In AsciiDoc documentation, page aliases (`:page-aliases:`) should never be removed as they serve as old redirects to maintain backward compatibility when pages are moved or renamed. They preserve SEO and prevent broken links for users.

Applied to files:

  • modules/develop/pages/manage-topics/config-topics.adoc
🔇 Additional comments (16)
modules/manage/pages/cluster-maintenance/compaction-settings.adoc (1)

15-15: Verify the anchor #change-the-cleanup-policy exists in the reorganized target.

This line references an anchor that should exist in the moved config-topics.adoc file. Based on the retrieved learnings, anchors for topic configuration properties and sections must be explicitly defined or auto-generated from section headers.

modules/deploy/pages/redpanda/manual/production/dev-deployment.adoc (1)

11-11: LGTM.

The cross-reference update correctly points to the reorganized manage-topics/config-topics.adoc path and maintains the same anchor reference for write caching configuration. The context and surrounding text remain appropriate.

modules/manage/partials/audit-logging.adoc (1)

436-436: LGTM.

The cross-reference update correctly reflects the new manage-topics directory structure. Without an anchor fragment, this reference will point to the top-level page, which is appropriate for a general "Related topics" reference.

modules/manage/pages/cluster-maintenance/disk-utilization.adoc (1)

88-88: Verify anchor #delete-records-from-a-topic exists in target file.

This anchor reference should correspond to a section in manage-topics/config-topics.adoc covering how to delete records from topics. Based on the PR's learnings about anchors, confirm this section (or an explicit anchor) exists in the reorganized target file.

modules/develop/pages/kafka-clients.adoc (1)

63-63: Verify module prefix change is intentional.

This cross-reference is unique among the PR changes: it shifts the module prefix from get-started: to develop: in addition to restructuring the path to manage-topics/config-topics.adoc. This suggests config-topics.adoc has been moved to the develop module (rather than just reorganized within it).

Confirm this module relocation aligns with the PR's overall reorganization strategy and that no residual reference in the get-started module should exist.

modules/get-started/pages/quick-start.adoc (1)

457-457: Verify anchor #choose-the-replication-factor exists in target file.

This anchor reference is expected to correspond to a section in manage-topics/config-topics.adoc that explains replication factor selection. Confirm this section or an explicit anchor with this ID exists in the reorganized target file to prevent broken links.

modules/manage/pages/cluster-maintenance/topic-property-configuration.adoc (1)

223-223: LGTM.

The cross-reference update correctly points to the new manage-topics/config-topics.adoc path. This is a straightforward top-level page reference without an anchor, appropriate for the "Related topics" section context.

modules/develop/pages/produce-data/configure-producers.adoc (1)

77-77: Anchor verification complete—xref is valid.

The xref fragment #configure-write-caching will be auto-generated from the section heading "=== Configure write caching" at line 117 of the target file. AsciiDoc/Antora automatically generates anchors from headings using lowercase text with hyphens replacing spaces. The reorganized cross-reference is valid and will not produce broken links.

modules/console/pages/ui/edit-topic-configuration.adoc (1)

22-22: LGTM — updated xref path is correct.

modules/reference/pages/rpk/rpk-redpanda/rpk-redpanda-mode.adoc (1)

13-13: LGTM — write-caching xref now targets the new location.

modules/develop/pages/manage-topics/config-topics.adoc (1)

4-4: Good — alias preserves old URL (back-compat).

modules/ROOT/nav.adoc (1)

39-41: LGTM — nav reflects new Manage topics hub.

modules/reference/pages/properties/topic-properties.adoc (1)

488-489: LGTM — write-caching cross-ref updated to new section.

modules/develop/pages/manage-topics/cloud-topics.adoc (2)

41-42: Strong GA migration statement — confirm with PM.
“Throw away any cluster…” is severe. If true for Public Beta, add context (applies only to beta clusters) and guidance on migration; otherwise soften.


14-14: Avoid exact latency claim unless documented.
“The difference in response time is typically 500ms” — either add a “typical, workload-dependent” qualifier with a citation or remove the number.

modules/reference/pages/properties/cluster-properties.adoc (1)

6549-6549: LGTM — related topics link updated to the new anchor. Verification complete.

Verification confirms:

  • The xref at line 6549 in cluster-properties.adoc correctly points to develop:manage-topics/config-topics.adoc#configure-write-caching
  • The target anchor exists in config-topics.adoc as an auto-generated anchor from the section heading "=== Configure write caching" (line 117)
  • No lingering old-style references to develop:config-topics.adoc (without the manage-topics path) remain in the codebase

@paulohtb6 paulohtb6 marked this pull request as ready for review November 20, 2025 18:57
@paulohtb6 paulohtb6 marked this pull request as draft November 20, 2025 18:57
@Feediver1 Feediver1 force-pushed the DOC-1602 branch 2 times, most recently from 4645ef8 to 90dc1e8 Compare November 20, 2025 20:21
@Feediver1 Feediver1 marked this pull request as ready for review November 21, 2025 00:00
@Feediver1 Feediver1 changed the title Doc 1602: Cloud Topics Doc 1602: Cloud Topics (Beta) Nov 21, 2025
====
endif::[]

Starting in v25.3, Redpanda provides Cloud Topics to support multi-modal streaming workloads in the most cost-effective way possible. While standard Kafka xref:config-topics.adoc[topics] are ideal for latency-sensitive, high-throughput workloads (for example, for audit logs), Cloud Topics are optimized for high-throughput, cost-sensitive workloads that can tolerate higher latencies. Instead of replicating every byte across expensive network links, Cloud Topics leverage durable, inexpensive Cloud object storage (S3, GCS, MinIO) as the primary mechanism for backing up data. The difference in response time is typically 500ms, which is only impactful for latency-sensitive applications.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Starting in v25.3, Redpanda provides Cloud Topics to support multi-modal streaming workloads in the most cost-effective way possible: as a configurable option on a single cluster. While standard Redpanda xref:config-topics.adoc[topics] using local storage or tiered storage are ideal for latency-sensitive, lower throughput workloads, Cloud Topics are optimized for latency-tolerant, high-throughput workloads where cloud provider infrastructure costs (especially cross-zone networking charges) are a major consideration and can become the dominant cost driver at high throughput. These workloads can include observability streams, offline analytics, AI/ML model training data feeds or just development environments where streaming latency isn't critical. Instead of replicating every byte across expensive network links, Cloud Topics leverage durable, inexpensive Cloud object storage (S3, , ADLS, GCS, MinIO etc) as the primary mechanism for storing topic data, eliminating over 90% of the cost of replicating data over network links in multi-AZ or multi-region clusters. The difference in end-to-end latency is typically between 500ms-1s, which is often acceptable for many streaming workloads, and can unlock new streaming use cases that were previously not cost effective. Redpanda also allows you to mix and match cloud topics workloads with low latency workloads in the same cluster, and will (at GA time) allow you to migrate existing local or tiered storage topics to become cloud topics.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Feediver1 re-watching Noah's demo, I think it does make sense to include some of these rough cost savings numbers somewhere (though we should be clear this is just replication costs), e.g. something like this:

"The infrastructure cost savings with Cloud Topics can be quite dramatic. By way of example, in one cloud environment tested by Redpanda, the networking costs of data replication to run a single 100MBps workload were reduced from over $10,000 per month to ~$80 per month. While this doesn't represent 100% of total operating costs, and noting that kafka batch sizes can affect this cost reduction slightly, at high throughputs this savings still accounts for the vast majority of cloud provider expense, reducing the total infrastructure cost of running Kafka workloads by an order of magnitude or more."

Copy link
Contributor Author

@Feediver1 Feediver1 Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No can do on "Redpanda also allows you to mix and match cloud topics workloads with low latency workloads in the same cluster, and will (at GA time) allow you to migrate existing local or tiered storage topics to become cloud topics."
We don't ever pre-announce feature updates in the docs. This point can be added when it is in there at GA time.

Copy link
Contributor Author

@Feediver1 Feediver1 Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This intro is now quite verbose and overly complex...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it does make sense to include some of these rough cost savings numbers somewhere (though we should be clear this is just replication costs)

I had advised against this as it felt like technical documentation not marketing material, so that's on me. I don't actually care either way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allow you to migrate existing local or tiered storage topics to become cloud topics.

Also I don't think this has been discussed explicitly as something we will support in GA.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dotnwat definitely that's been my understanding with Piyush. But wrt these docs we can just leave it out

@JakeSCahill
Copy link
Contributor

JakeSCahill commented Nov 25, 2025

@Feediver1 I added docs for the new cluster configs. The descriptions come from either the code or the overrides file:

"cloud_topics_enabled": {

@mattschumpert
Copy link

mattschumpert commented Mar 12, 2026

@Feediver1 want to make sure we also update Tiered Storage docs here as well (wrt storage mode). Can you add the other page previews for TS, and the property reference here?

Feediver1 and others added 3 commits March 17, 2026 10:59
- Add migrate-to-iceberg-topics.adoc from main (referenced by cloud-docs include)
- Add explicit [[choose-the-replication-factor]] and [[change-the-replication-factor]] anchors in config-topics.adoc
- Update stale xrefs in topic-properties partial from manage-topics.adoc to manage-topics/config-topics.adoc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e fix

- Update antora.yml to point to v26.1.1-rc2
- Add category "tiered-storage" override for redpanda.storage.mode so it
  appears under the Tiered Storage Properties section (was missing from the
  page because the default category "other" has no include tag)
- Fix stale xrefs in property-overrides.json: update manage-topics.adoc
  references to manage-topics/config-topics.adoc after page restructuring
- Regenerate all property partials from v26.1.1-rc2 JSON
- Replace v25.3.7 JSON attachment with v26.1.1-rc2

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e (v26.1)

- Add "Choose a storage mode" section to config-topics.adoc explaining the
  four values (unset, local, tiered, cloud) and default_redpanda_storage_mode
  cluster property
- Add "Change the storage mode" section with allowed/disallowed transitions table
- Update rpk topic describe example output to include redpanda.storage.mode
- Update tiered-storage partial: lead "Enable Tiered Storage for specific topics"
  with the modern redpanda.storage.mode=tiered approach
- Add "Configure Tiered Storage using legacy topic properties" subsection
  clarifying that redpanda.remote.read/write only control behavior for
  redpanda.storage.mode=unset topics
- Update cluster-level NOTE: cloud_storage_enable_remote_write/read act as
  creation-time defaults only for unset topics; reference default_redpanda_storage_mode
- Update redpanda.remote.delete description to note it applies to both
  Tiered Storage and Cloud Topics

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
----

Topic-level properties override cluster-level properties. For example, for new topics, if `cloud_storage_enable_remote_write` is set to `true`, you can set `redpanda.remote.write` to `false` to turn it off for a particular topic.
For `unset` topics, topic-level properties override the cluster-level `cloud_storage_enable_remote_write` and `cloud_storage_enable_remote_read` defaults. For example, if `cloud_storage_enable_remote_write` is `true`, you can set `redpanda.remote.write=false` to disable uploads for a specific topic.
Copy link
Contributor

@WillemKauf WillemKauf Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous section here referred to "new topics", because cloud_storage_enable_remote_write and cloud_storage_enable_remote_read only dictate the default value for new topics of redpanda.remote.write/read at topic creation time. Changing them after topics are created does not affect existing topics. Therefore, I would rewrite this like,

For newly created unset topics, the cluster-level cloud_storage_enable_remote_write and cloud_storage_enable_remote_read dictate the topic-level properties redpanda.remote.write and redpanda.remote.read at topic creation time respectively. Altering the cluster properties will have no effect on existing topics, only newly created ones. To alter the permissions for existing topics, you can set these topic properties directly, e.g. redpanda.remote.write=false to disable uploads for a specific topic.

I think its HEAVILY worth noting here that toggling (enabling and disabling) tiered storage write perms is something we really don't want to encourage anymore- so we should either drop the example of disabling remote writes, or link users to the safe pause-resume functionality, or both.

cc: @mattschumpert

| When set to `true`, deleting a topic also deletes its objects in object storage. Both `redpanda.remote.write` and `redpanda.remote.read` must be enabled, and the topic must not be a Remote Read Replica topic.
| When set to `true`, deleting a topic also deletes its objects in object storage. Applies to both Tiered Storage topics and Cloud Topics.

For Tiered Storage topics with `redpanda.storage.mode=unset` or `tiered`, the topic must not be a Remote Read Replica topic.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is now a bit awkward to grok, maybe

For Tiered Storage topics with redpanda.storage.mode=unset or tiered, the topic must not be a Remote Read Replica topic to enable redpanda.remote.delete.

|===

The following tables list outcomes for combinations of cluster-level and topic-level configurations:
The following tables show outcomes for combinations of cluster-level and topic-level configurations for topics with `redpanda.storage.mode=unset`:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following tables show outcomes for combinations of cluster-level and topic-level configurations for topics with redpanda.storage.mode=unset at topic creation time:

I think is necessary to be clear here about the cluster-level properties only dictating the values new topics are created with.

* `local`: Topic data is stored only on the broker's local disk. Object storage upload is disabled for the topic, regardless of cluster-level Tiered Storage settings.
* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables xref:manage:tiered-storage.adoc[Tiered Storage] for the topic.
* `cloud`: Topic data is stored in object storage using the Cloud Topics architecture. Local storage is used only as a write buffer.
* `unset`: Uses the cluster-level config_ref:default_redpanda_storage_mode,true,properties/cluster-properties[] setting, or falls back to legacy `redpanda.remote.read` and `redpanda.remote.write` topic property behavior for backwards compatibility.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite. Explicitly specifying unset means the topic will be unset. Only if no value is provided for redpanda.storage.mode to the topic creation command is the cluster default default_redpanda_storage_mode used.

Concrete examples to explain the point:

$ rpk cluster config set default_redpanda_storage_mode=local
$ rpk topic create topic_a -c redpanda.storage.mode=unset
[ topic_a is created with redpanda.storage.mode=unset]

$ rpk topic create topic_b 
[ topic_b is created with redpanda.storage.mode=default_redpanda_storage_mode=local


== Limitations

- The Remote Read Replicas feature is not supported for Cloud Topics.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Remote Read Replicas feature is not yet supported for Cloud Topics.

Is this softer language better or worse for our messaging? cc: @mattschumpert

Copy link

@andrwng andrwng Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read replica support is merged now! We can remove this limitation. Mount/unmount isn't supported though.

endif::[]

| Related topics
|xref:manage:tiered-storage.adoc[Tiered Storage]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe cloud topics page as a related topic well?

| Legacy behavior. Tiered Storage is controlled by the `redpanda.remote.read` and `redpanda.remote.write` topic properties and their cluster-level defaults (`cloud_storage_enable_remote_read` and `cloud_storage_enable_remote_write`).

| `local`
| Data is stored only on broker-local storage. Object storage upload is disabled for the topic, regardless of cluster-level Tiered Storage settings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local: Topic data is stored only on the broker's local disk. Object storage upload is disabled for the topic, regardless of cluster-level Tiered Storage settings

tiered: Data is stored on local disk and uploaded to object storage. Enables Tiered Storage for the topic regardless of redpanda.remote.read and redpanda.remote.write values.

Why the asymmetry in description here? local could also be written as

local: Topic data is stored only on the broker's local disk. Object storage upload is disabled for the topic, regardless of redpanda.remote.read and redpanda.remote.write values

@WillemKauf
Copy link
Contributor

I think those are most of the remaining comments I have- thank you very much for the arduous efforts here 👍

If there is anything else you want my input specifically on please tag me.

Feediver1 and others added 2 commits March 19, 2026 13:04
Uses glossterm macro on first occurrence of "Cloud Topics" in body
text, referencing the new Cloud Topic glossary entry from PR #1621.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Feediver1 added a commit that referenced this pull request Mar 19, 2026
Updates hover-text per editorial review. Removes the "For more
information" link until the Cloud Topics page (PR #1469) has merged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
----

Topic-level properties override cluster-level properties. For example, for new topics, if `cloud_storage_enable_remote_write` is set to `true`, you can set `redpanda.remote.write` to `false` to turn it off for a particular topic.
For `unset` topics, topic-level properties override the cluster-level `cloud_storage_enable_remote_write` and `cloud_storage_enable_remote_read` defaults. For example, if `cloud_storage_enable_remote_write` is `true`, you can set `redpanda.remote.write=false` to disable uploads for a specific topic.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For `unset` topics, topic-level properties override the cluster-level `cloud_storage_enable_remote_write` and `cloud_storage_enable_remote_read` defaults. For example, if `cloud_storage_enable_remote_write` is `true`, you can set `redpanda.remote.write=false` to disable uploads for a specific topic.
For newly created `unset` topics, the cluster-level `cloud_storage_enable_remote_write` and `cloud_storage_enable_remote_read` dictate the topic-level properties `redpanda.remote.write` and `redpanda.remote.read` at topic creation time respectively. Altering the cluster properties will have no effect on existing topics, only newly created ones. To alter the permissions for existing topics, you can set these topic properties directly. For example, specify `redpanda.remote.write=false` to disable uploads for a specific topic.


== Limitations

- The Remote Read Replicas feature is not supported for Cloud Topics.
Copy link

@andrwng andrwng Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read replica support is merged now! We can remove this limitation. Mount/unmount isn't supported though.

ifdef::env-cloud[]
+
* The `delete.retention.ms` topic configuration in Kafka is not supported. Tombstone markers are not removed for topics with a `compact` xref:get-started:config-topics.adoc#change-the-cleanup-policy[cleanup policy]. Redpanda only deletes tombstone markers when topics with a cleanup policy of `compact,delete` have reached their xref:get-started:create-topic.adoc[retention limits].
* The `delete.retention.ms` topic configuration in Kafka is not supported. Tombstone markers are not removed for topics with a `compact` cleanup policy. Redpanda only deletes tombstone markers when topics with a cleanup policy of `compact,delete` have reached their retention limits.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't true for Cloud Topics or local topics. Tombstone removal and delete.retention.ms are supported for those topic types

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants