The CockroachDB operator is in .
Compatibility
Before starting migration, verify your cluster configuration is supported by the migration controller:Before You Begin
Verify the following before starting migration:- CockroachDB cluster is managed by a Helm StatefulSet (not the public operator).
- All StatefulSet pods are Running and Ready with no pending rolling updates, scale operations, or evictions.
kubectlandhelmare installed and configured with access to the target cluster.- For multi-region clusters: cloud region and provider labels are applied to K8s nodes.
- For multi-region clusters:
migration-regionsannotation is prepared with the full topology. - You have reviewed the Compatibility table and confirmed your configuration is supported, or otherwise are prepared to manually recreate unsupported feature configurations.
- If using
tls.certs.provided(user-provided certs): Your certificates include join service DNS SANs ({cluster-name}-join,{cluster-name}-join.{namespace},{cluster-name}-join.{namespace}.svc.cluster.local). The controller cannot regenerate certificates when the CA private key is not available. - If using
tls.certs.providedwithtls.certs.tlsSecret: false: Secrets are re-keyed to use Kubernetes TLS convention (tls.crt,tls.key,ca.crt) instead of cockroach-native key names (node.crt,node.key). The operator expects TLS secrets in the standard Kubernetes format. This is not needed for self-signer or cert-manager modes.
Migration controller overview
The following sections describe how the automated migration controller handles a migration from Helm StatefulSets to the CockroachDB operator:What the migration controller does automatically
- Detects cert-manager, self-signer, or user-provided certificates. Self-signer and cert-manager certs are regenerated with join service DNS SANs automatically. User-provided certs are mounted directly (regeneration skipped).
- Creates RBAC resources (ServiceAccount, Role, RoleBinding, ClusterRole, ClusterRoleBinding) with namespace-qualified names for cluster-scoped resources.
- Deletes old PDB and creates a new one during finalization (no protection gap because the new PDB is created immediately after old PDB deletion).
- Updates service selectors during migration and restores them on rollback.
- Migrates nodes one at a time (highest index first), verifying cluster health between each node.
- Detects and preserves Ingress configuration as an annotation on the CrdbCluster.
- Migrates
logsdir/logs-dirVolumeClaimTemplate to CrdbNode LogsStore. - Converts log-config Secret to ConfigMap (Helm uses Secrets, operator uses ConfigMaps).
- Detects configuration from init jobs.
- Detects configuration from VolumeClaimTemplates and
--wal-failoverflag.
What requires Manual Action
- Cloud region and provider node labels must be applied before deploying the operator.
- The StatefulSet must be manually deleted after finalization to complete migration.
- Helm adoption annotations must be applied to all resources post-migration.
- Old bare-named ClusterRole/ClusterRoleBinding must be deleted post-migration.
- Ingress resources are not annotated for Helm adoption (manual step required).
- ServiceMonitor / PodMonitor resources must be recreated with updated pod labels.
Migration Phases
A migration using the controller is performed in the following phases:Controlling migration (Stop / Resume / Rollback)
The following section describe how to safely manage an in-progress migration.Stop (Pause)
Pauses migration at the current phase. No resources are deleted. CrdbNodes already created remain running and serving traffic.Resume
There is no separate “resume” label value. Usestart again to resume from the paused phase. The controller detects the PhaseStopped state internally and resumes at the correct phase based on how many nodes have already been migrated.
Stop during rollback
If rollback is in progress and you apply the stop label, the controller pauses the rollback. The status message indicates that migration was stopped while inRollbackInProgress.
To resume the rollback:
start resumes forward migration, while rollback resumes rollback.
Gap detection on stop
If stop is requested between scaling down the StatefulSet and creating the replacement CrdbNode, the controller detects the gap and scales the StatefulSet back up before enteringPhaseStopped. This keeps every ordinal represented by either a StatefulSet pod or a CrdbNode pod while migration is paused.
Rollback
Rollback is safe at any phase before Complete. Removing the label triggers automatic rollback. The controller automatically:- Deletes all CrdbNodes and waits for their pods to terminate.
- Restores StatefulSet to original replica count.
- Removes migration labels (
crdb.cockroachlabs.com/cluster,svc) from pods. - Deletes CA ConfigMap (
{name}-ca-crt). - Deletes the ConfigMap created from the log-config Secret.
- Deletes all PDB variants (
{name},{name}-budget,{name}-pdb). - Restores original service selectors (
app.kubernetes.io/instance,app.kubernetes.io/name). - Deletes namespace-qualified ClusterRole and ClusterRoleBinding (
{namespace}-{name}). - Removes the migration label from the StatefulSet.
Rollback safety by phase
Label values reference
Rollback after complete
OncePhase=Complete, automated rollback is not possible. The StatefulSet has been deleted.
Manual recovery steps:
- Delete all CrdbNodes
- Manually recreate the StatefulSet with original spec
- Scale up and verify cluster health
- Remove the migration label
Certificate detection
The controller automatically detects your certificate method. No manual configuration is needed.- cert-manager: If cert-manager Certificate CRs exist for the cluster (
{name}-node), the controller updates the Certificate CR with join service DNS SANs and preserves the issuer references. cert-manager then regenerates the secrets automatically. CrdbNode pods mount from the cert-manager-managed secret (the name fromspec.secretNamein the Certificate CR), not{name}-node-secret. This keeps cert-manager’s automatic rotation working after migration. - Self-signer (Helm built-in): The controller loads the existing CA from cluster secrets, regenerates node and client certificates with join service DNS SANs, and writes them to the destination secrets (
{name}-node-secret,{name}-client-secret). CrdbNode pods mount from{name}-node-secret. The regenerated certificates have a 1-year TTL and are stored asExternalCertificatesin the v1beta1 spec. The operator does not auto-rotateExternalCertificates, so after migration you should switch tocockroachdb.tls.selfSigner.enabled: truevia Helm adoption to enable automatic rotation. If you delay this step, the certs will expire silently after one year. - Provided certs (
tls.certs.provided): The CA private key is not available, so the controller skips cert regeneration. CrdbNode pods mount the user’s existing secrets directly (the secret name parsed from the StatefulSet volumes). The user must update their certificates to include join service DNS SANs before starting the migration (see before you begin). - Insecure clusters: Detected from the
--insecurestart flag. The entire cert migration phase is skipped. AnInsecureClusterMigrationwarning event is emitted.
Regardless of the original cert type, after migration all certificate secret names are stored as
ExternalCertificates on the CrdbNode spec. This is the internal representation used by the operator to mount the correct secrets into pods.Step 1: Export environment variables
Export the following environment variables based on your existing deployment:Step 2: Apply cloud region and provider labels (multi-region only)
CockroachDB uses K8s node labels for locality-based pod placement. These labels must exist on the nodes before the operator starts scheduling pods.CrdbNode pods will remain Pending with scheduling errors.
For multi-region clusters, annotate the StatefulSet with the full topology before starting migration:
migration-regions annotation, (2) regionCode and cloudProvider annotations, (3) operator environment configuration, (4) default us-east1. For single-region clusters, the default is usually acceptable.
Step 3: Verify StatefulSet Is Stable
All replicas must be ready before migration begins. Run the followingkubectl commands to validate:
Step 4: Install CockroachDB Operator with Migration Enabled
Run the following command to install the operator with migration enabled. This registers the migration controller which watches for thecrdb.io/migrate label.
migration.enabled=trueon the operator chart enables the migration controller and registers the conversion webhook.- Helm StatefulSet migrations do not have a pre-existing public operator, so the default
appLabel=cockroach-operatorworks without modification.
appLabel to avoid selector conflicts. Without watchNamespaces, the CockroachDB operator’s cluster controller will attempt to reconcile all CrdbClusters cluster-wide, interfering with clusters managed by the public operator.
watchNamespaces (or use a comma-separated list) to include additional namespaces only after the previous migration is complete.
Step 5: Start Migration
Initiate the migration by labeling the StatefulSet:Step 6: Monitor Migration
The following sections describe how to monitor the ongoing migration.Phase-Specific Checks
The following table describes the high-level events to monitor for progress during each migration phase:DB Console
The DB Console is accessible via the public service throughout migration.- Node List: Watch for nodes joining and leaving as migration progresses. Total node count stays constant (new CrdbNode created before old STS pod is removed).
- Replication Status: Under-replicated and unavailable ranges should stay at zero. The migration controller waits for under-replicated ranges to clear before proceeding to the next node.
- Ranges Dashboard: Under-replicated ranges counter should not increase during migration.
Prometheus Metrics
CockroachDB health checks
The controller performs these checks automatically between each node migration, but you can verify manually:Check migration phase
Monitor Kubernetes events
Watch CrdbNode creation during the PodMigration phase
Troubleshooting pauses during PodMigration
During PodMigration, if a newly created CrdbNode does not pass health checks within 10 minutes, the controller automatically pauses migration by setting the phase toPhaseStopped. Health checks run every 10 seconds and verify:
- CrdbNode has
PodReady=Truecondition - Pod is in
Runningphase withReady=Truecondition - SQL health check passes (zero under-replicated ranges and correct live node count)
- Pod is registered in the headless service endpoints
kubectl label sts $STS_NAME crdb.io/migrate=start --overwrite.
PDB Behavior During Migration
- Init through PodMigration: The original Helm PDB remains active. Pod disruption protection is maintained throughout.
- Finalization: The controller deletes the old PDB (
{name}or{name}-budget) and immediately creates the new PDB ({name}-pdb). There is no gap in protection. - Rollback: All PDB variants are deleted (
{name},{name}-budget,{name}-pdb). The original PDB is not recreated by the migration controller. It is recreated by the Helm chart on the nexthelm upgradeor by the Helm release’s existing resources.
Step 7: Delete StatefulSet
After the finalization phase is complete, the controller stops processing and waits for the user to manually delete the StatefulSet. The StatefulSet is intentionally left intact. Once deleted, the controller setsspec.mode=MutableOnly and records Phase=Complete.
replicas=0 by the time Finalization runs (scaled down one by one during PodMigration). Deleting the StatefulSet object does not evict any running pods.
Step 8: Verify cluster health post-migration
Run the followingkubectl commands to validate that the new cluster is online and healthy:
Verify preserved configurations
Step 9: Prepare values.yaml for CockroachDB Helm Chart
Before runninghelm upgrade, create a values.yaml that matches the migrated CrdbCluster spec. The CockroachDB Helm chart generates a CrdbCluster from these values, so they must align with what migration produced.
values.yaml file automatically with a migration helper, and/or build the file manually by mapping existing values.
Option A: Generate values.yaml automatically
Build and run the migration helper binary in the root of the cockroachdb/helm-charts repository:cockroachdb.crdbCluster.rbac.serviceAccount.name, keep that value as is. It preserves the service account used by the migrated cluster.
Option B: Build values.yaml manually
Key values to set (map each to your migrated spec):
If the source Helm release used custom values (affinity, tolerations, nodeSelector, etc.), carry those forward into the new values.yaml as well.
Field Mapping Reference
Step 10: Annotate Resources for Helm Adoption
The migration controller creates resources but does not annotate them for Helm ownership. Forhelm upgrade to manage these resources, they must carry Helm ownership annotations:
helm upgrade will fail with “resource already exists” errors.
Do not annotate the migration-created ClusterRole/ClusterRoleBinding for Helm adoption. The CockroachDB Helm chart creates its own cluster-scoped RBAC with different names (
{fullname}-{namespace}-node-reader). The migration-created ones are stale and should be deleted in the next step.Step 11: Delete Stale Resources
The migration controller creates some resources that do not match what the CockroachDB Helm chart produces. These can be deleted before or after runninghelm upgrade:
Step 12: Configure LocalityMappings
The migration controller preserves the--locality flag tier keys (e.g. region, zone) as localityLabels on the CrdbNodeSpec. localityLabels is deprecated in favor of localityMappings, which maps K8s node labels to CockroachDB locality tiers. The default mapping covers standard K8s topology labels (topology.kubernetes.io/region → region, topology.kubernetes.io/zone → zone).
If your cluster uses custom K8s node labels for locality, update localityMappings to match. Each entry maps a K8s node label key to a CockroachDB locality tier name.
topology.kubernetes.io/* labels, the kubebuilder defaults are correct and no action is needed.
Step 13: Run Helm Upgrade with CockroachDB Chart
Before upgrading, verify the operator has fully reconciled the migrated cluster. Do not proceed untilgeneration and observedGeneration match and all pods are running.
helm upgrade with an updated timestamp:
Update Certificate Mode After Adoption
After migration, all certificate references are stored asexternalCertificates on the CrdbCluster spec, regardless of the original cert type. The export-values tool outputs these as cockroachdb.tls.externalCertificates. After Helm adoption, update your values.yaml to use the cert mode that matches your intended ongoing management:
- Self-signer: Switch to
cockroachdb.tls.selfSigner.enabled: trueand remove theexternalCertificatesblock. The operator’s self-signer will manage cert rotation going forward. The migration-created{name}-node-secretand{name}-client-secretsecrets become stale once the self-signer creates its own. Delete them after verifying the new certs are working. - cert-manager: Switch to
cockroachdb.tls.certManager.enabled: truewith the appropriate issuer reference. The migration preserved the Certificate CR and itsspec.secretName, so cert-manager continues to manage rotation. Remove theexternalCertificatesblock from values. - User-provided / External: Keep
cockroachdb.tls.externalCertificatesas exported. You are responsible for cert rotation.
helm upgrade again and verify pods are healthy. Clean up any stale secrets that are no longer referenced:
Troubleshooting
Follow these troubleshooting steps if there are issues with the migration:Migration stuck at PodMigration phase
If a node fails health checks for more than 10 minutes, migration auto-pauses toPhaseStopped.
Insecure cluster migration
The controller detects insecure clusters (--insecure flag) and skips certificate migration. An InsecureClusterMigration warning event is emitted. No action is needed unless you want to enable TLS post-migration.

