From ad361ab05d81287bdf187ff474297198bed033e8 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Mon, 20 Jul 2026 17:22:40 -0600 Subject: [PATCH 1/2] DOC-1860: Cluster sizing, tier migration, and user scaling GA docs Consolidated doc update for the cluster sizing and scaling initiative (DOC-1860, DOC-1375, DOC-940, DOC-1861 provisioning half; DOC-1133 API section stubbed for separate ownership): - manage/maintenance.adoc: new Tier migrations section (DOC-1375) - reference/partials/tiers.adoc: guidance (up to) framing and machine types/RPU concept for BYOC tiers (DOC-940) - get-started/.../byoc/size-byoc-cluster.adoc: new page for the workload-based sizing and pricing calculator, UI + Control Plane API (DOC-1861) - manage/cluster-maintenance/scale-cluster.adoc: new page for self-service horizontal/vertical scaling, prepared for user scaling GA (DOC-1860, ENG-928); merge with the GA release - BYOC create pages: pointer to workload-based sizing - terraform-provider.adoc: tier-migration cross-link and GA revision note for the throughput_tier warning - whats-new-cloud.adoc: scaling GA entry under a placeholder month; sizing calculator entry drafted in a comment pending rollout confirmation TODO(eng)/TODO(ux)/TODO(api) comments mark items to verify before merge. Keep the PR in draft until ENG-928 ships. Co-Authored-By: Claude Fable 5 --- modules/ROOT/nav.adoc | 2 + .../byoc/aws/create-byoc-cluster-aws.adoc | 4 +- .../byoc/azure/create-byoc-cluster-azure.adoc | 2 +- .../byoc/gcp/create-byoc-cluster-gcp.adoc | 2 +- .../cluster-types/byoc/size-byoc-cluster.adoc | 76 +++++++++++++++++ .../get-started/pages/whats-new-cloud.adoc | 17 ++++ .../cluster-maintenance/scale-cluster.adoc | 82 +++++++++++++++++++ modules/manage/pages/maintenance.adoc | 21 ++++- modules/manage/pages/terraform-provider.adoc | 10 ++- modules/reference/partials/tiers.adoc | 16 +++- 10 files changed, 224 insertions(+), 8 deletions(-) create mode 100644 modules/get-started/pages/cluster-types/byoc/size-byoc-cluster.adoc create mode 100644 modules/manage/pages/cluster-maintenance/scale-cluster.adoc diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 78d6a9346..df4f06d50 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -9,6 +9,7 @@ * xref:get-started:cluster-types/index.adoc[Deploy] ** xref:get-started:cluster-types/serverless.adoc[Serverless] ** xref:get-started:cluster-types/byoc/index.adoc[BYOC] +*** xref:get-started:cluster-types/byoc/size-byoc-cluster.adoc[] *** xref:get-started:cluster-types/byoc/aws/index.adoc[AWS] **** xref:get-started:cluster-types/byoc/aws/create-byoc-cluster-aws.adoc[] **** xref:get-started:cluster-types/byoc/aws/vpc-byo-aws.adoc[] @@ -416,6 +417,7 @@ ** xref:manage:cluster-maintenance/index.adoc[Cluster Maintenance] *** xref:manage:cluster-maintenance/cluster-state.adoc[] *** xref:manage:maintenance.adoc[] +*** xref:manage:cluster-maintenance/scale-cluster.adoc[] *** xref:manage:cluster-maintenance/config-cluster.adoc[] *** xref:manage:audit-logging.adoc[] **** xref:manage:audit-logging/audit-log-samples.adoc[] diff --git a/modules/get-started/pages/cluster-types/byoc/aws/create-byoc-cluster-aws.adoc b/modules/get-started/pages/cluster-types/byoc/aws/create-byoc-cluster-aws.adoc index adf9ab19e..605370917 100644 --- a/modules/get-started/pages/cluster-types/byoc/aws/create-byoc-cluster-aws.adoc +++ b/modules/get-started/pages/cluster-types/byoc/aws/create-byoc-cluster-aws.adoc @@ -4,7 +4,7 @@ To create a Redpanda cluster in your virtual private cloud (VPC), follow the instructions in the Redpanda Cloud UI. The UI contains the parameters necessary to successfully run `rpk cloud byoc apply`. See also: xref:get-started:byoc-arch.adoc[BYOC architecture]. -NOTE: With standard BYOC clusters, Redpanda manages security policies and resources for your VPC, including subnetworks, service accounts, IAM roles, firewall rules, and storage buckets. For the highest level of security, you can manage these resources yourself with a xref:./vpc-byo-aws.adoc[BYOVPC cluster on AWS]. +NOTE: With standard BYOC clusters, Redpanda manages security policies and resources for your VPC, including subnetworks, service accounts, IAM roles, firewall rules, and storage buckets. For the highest level of security, you can manage these resources yourself with a xref:get-started:cluster-types/byoc/aws/vpc-byo-aws.adoc[BYOVPC cluster on AWS]. == Prerequisites @@ -26,7 +26,7 @@ To verify access, you should be able to successfully run `aws sts get-caller-ide . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. . On the Clusters page, click *Create cluster*, then click *Create* for BYOC. -. Enter a cluster name, then select the resource group, provider (AWS), xref:reference:tiers/byoc-tiers.adoc[region, tier], availability, and Redpanda version. +. Enter a cluster name, then select the resource group, provider (AWS), xref:reference:tiers/byoc-tiers.adoc[region, tier], availability, and Redpanda version. If your organization uses usage-based billing, you can describe your workload to get recommended configurations and price estimates instead of selecting a tier directly. See xref:get-started:cluster-types/byoc/size-byoc-cluster.adoc[Size and price a BYOC cluster]. + [NOTE] ==== diff --git a/modules/get-started/pages/cluster-types/byoc/azure/create-byoc-cluster-azure.adoc b/modules/get-started/pages/cluster-types/byoc/azure/create-byoc-cluster-azure.adoc index d229b5055..e33bca7c0 100644 --- a/modules/get-started/pages/cluster-types/byoc/azure/create-byoc-cluster-azure.adoc +++ b/modules/get-started/pages/cluster-types/byoc/azure/create-byoc-cluster-azure.adoc @@ -150,7 +150,7 @@ To create a Redpanda cluster in your Azure VNet, follow the < Date: Mon, 20 Jul 2026 18:30:57 -0600 Subject: [PATCH 2/2] Incorporate PM messaging: calculator-first sizing, de-emphasize RPUs Per Denis Coady (Cloud PM, Slack 2026-07-20): customers should size and resize clusters with the workload calculator rather than reasoning in tiers or RPUs. - tiers partial: lead with workload-based sizing for usage-based-billing orgs; reframe RPU as an internal unit resolved by the calculator; TODO(pm) on whether docs may state the tier phase-out - size-byoc-cluster: state workload-based sizing as the recommended path; scope manual RPU mode to users who already know their capacity - scale-cluster: TODO(eng) for the upcoming single load metric as the when-to-scale signal (cross-ref DOC-1381) Co-Authored-By: Claude Fable 5 --- .../pages/cluster-types/byoc/size-byoc-cluster.adoc | 4 ++-- .../manage/pages/cluster-maintenance/scale-cluster.adoc | 2 ++ modules/reference/partials/tiers.adoc | 8 +++++--- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/modules/get-started/pages/cluster-types/byoc/size-byoc-cluster.adoc b/modules/get-started/pages/cluster-types/byoc/size-byoc-cluster.adoc index 121c99ac3..72eea71bb 100644 --- a/modules/get-started/pages/cluster-types/byoc/size-byoc-cluster.adoc +++ b/modules/get-started/pages/cluster-types/byoc/size-byoc-cluster.adoc @@ -6,7 +6,7 @@ :learning-objective-2: Compare sizing options by RPU rate, machine type, and node count :learning-objective-3: Request sizing estimates programmatically with the Control Plane API -When you create a BYOC cluster, you can size it from your workload requirements instead of translating them into Redpanda tier terminology. Describe your workload, such as throughput, glossterm:partition[,partitions], and retention, and Redpanda recommends machine configurations with itemized price estimates. +When you create a BYOC cluster, size it from your workload requirements instead of translating them into Redpanda tier terminology. Describe your workload, such as throughput, glossterm:partition[,partitions], and retention, and Redpanda recommends machine configurations with itemized price estimates. Workload-based sizing is the recommended way to size a cluster: it applies the same logic Redpanda's performance team uses, and it stays accurate as performance improvements change what a given configuration can handle. After reading this page, you will be able to: @@ -25,7 +25,7 @@ Sizing options appear only for organizations on usage-based billing. If you don' You can size a cluster in two ways: * *Describe your workload.* Provide workload parameters, such as partition count, ingress and egress throughput, retention, average message size, replication factor, producer count, and target latency. You can also indicate whether you plan to use cloud topics or Iceberg topics. Redpanda resolves the compute capacity your workload needs and recommends machine configurations that provide it. -* *Specify capacity directly.* Provide a rate of Redpanda Processing Units (RPUs), from 1 to 384. One RPU represents two vCPUs of glossterm:broker[] compute. Redpanda resolves the machine configurations that deliver that capacity. +* *Specify capacity directly.* Provide a rate of Redpanda Processing Units (RPUs), from 1 to 384. One RPU represents two vCPUs of glossterm:broker[] compute. Redpanda resolves the machine configurations that deliver that capacity. Use this mode only if you already know the capacity you need; describing your workload is the recommended path. Both ways return a ranked list of viable machine configurations. Each option includes the machine type, node count, RPU rate, and the xref:reference:tiers/byoc-tiers.adoc[usage tier] it maps to, along with an itemized price estimate covering compute, ingress, egress, storage, and Iceberg, with hourly and monthly totals. diff --git a/modules/manage/pages/cluster-maintenance/scale-cluster.adoc b/modules/manage/pages/cluster-maintenance/scale-cluster.adoc index a3f9fdd51..ec335229b 100644 --- a/modules/manage/pages/cluster-maintenance/scale-cluster.adoc +++ b/modules/manage/pages/cluster-maintenance/scale-cluster.adoc @@ -43,6 +43,8 @@ You can scale a cluster in two ways: If you are not sure how much capacity your workload needs, describe your workload in the sizing calculator to get recommended configurations and price estimates. See xref:get-started:cluster-types/byoc/size-byoc-cluster.adoc[Size and price a BYOC cluster]. +// TODO(eng): DOC-1860 - Core is releasing a single load metric for judging whether a cluster is over- or under-provisioned, and PM wants it exposed to users (Denis Coady, Slack 2026-07-20; Piyush Agarwal owns exposure timing; may not be RPU-aware, check Tyson/Matt). When it ships, document it here as the signal for deciding when to scale. Related: DOC-1381 tracks the Self-Managed load factor dashboard. + == Scale a cluster in the Cloud UI // TODO(ux): DOC-1860 - replace with the shipped UI flow at GA. The steps below are a placeholder drafted from the ENG-928 design intent (scaling UX integrates the sizing calculator so you scale in workload terms). diff --git a/modules/reference/partials/tiers.adoc b/modules/reference/partials/tiers.adoc index 24d759276..d679f10ae 100644 --- a/modules/reference/partials/tiers.adoc +++ b/modules/reference/partials/tiers.adoc @@ -1,7 +1,9 @@ ifdef::env-byoc[] == BYOC usage tiers -When you create a BYOC cluster, you select your usage tier. Each tier defines a tested workload configuration with maximum (up to) values for throughput, partitions, and connections. +When you create a BYOC cluster, you choose its capacity. If your organization uses usage-based billing, describe your workload and let Redpanda recommend a configuration, rather than choosing a tier directly. See xref:get-started:cluster-types/byoc/size-byoc-cluster.adoc[Size and Price a BYOC Cluster]. Otherwise, you select a usage tier. Each tier defines a tested workload configuration with maximum (up to) values for throughput, partitions, and connections. + +// TODO(pm): DOC-940/DOC-1861 - Denis Coady (Slack, 2026-07-20) says tiers are being phased out in favor of calculator-based sizing and customers shouldn't reason in tiers or RPUs. Asked on PR #642 whether docs should state a phase-out or keep "recommended" framing. Do not state a phase-out without official deprecation comms. Treat tier values as guidance rather than enforced caps. Redpanda does not throttle your workload at the advertised values. You can push one dimension beyond its advertised value, for example ingress throughput, by accepting trade-offs in other dimensions, such as egress capacity or latency. @@ -33,9 +35,9 @@ Treat tier values as guidance rather than enforced caps. Redpanda does not throt === Machine types and RPUs -Each tier maps to a machine type from your cloud provider, sized in Redpanda Processing Units (RPUs). One RPU represents two vCPUs of broker compute. Redpanda uses RPUs for cluster sizing and usage-based pricing. +Each tier maps to a machine type from your cloud provider, with capacity measured in Redpanda Processing Units (RPUs), the internal unit that workload-based sizing and usage-based pricing use. You don't need to reason in RPUs: describe your workload in the xref:get-started:cluster-types/byoc/size-byoc-cluster.adoc[sizing calculator], and Redpanda resolves the machine configuration for you. -To see the machine type and RPU count for each tier, use the link:/api/doc/cloud-controlplane[Control Plane API^] (`GET /v1beta2/throughput-tiers`). To get a machine configuration and price estimate matched to your workload instead of choosing a tier directly, see xref:get-started:cluster-types/byoc/size-byoc-cluster.adoc[Size and price a BYOC cluster]. +To see the machine type and RPU count for each tier, use the link:/api/doc/cloud-controlplane[Control Plane API^] (`GET /v1beta2/throughput-tiers`). // TODO(eng): DOC-940/DOC-1861 - verify the public tiers endpoint path in the published Control Plane API reference (GET /v1beta2/throughput-tiers vs /v1beta2/tiers) before publishing.