Secure network egress with private CockroachDB clusters

As part of zero-trust focus, InfoSec and Risk teams pay extra attention to data exfiltration threat vectors, including both when it comes to how service providers manage their data, and how to control & manage insider risk exposure through their employees. 

Solutions to a number of those requirements manifest in the form of network security controls, especially for egress. With regard to database clusters, restricting clusters to access only specific resources for things like backup-restore, publishing real-time change events, or sending observability data can be challenging.

It is common for people who manage the database infrastructure in their own environment to employ one or a mix of transparent and explicit proxies / firewalls to restrict where the data could be sent to, making that a costly & complex solution. 

On the other hand, when it comes to Database as a Service (DBaaS) offerings where the infrastructure is managed by the service provider, the options are either non-existent or they’re tied to a specific cloud provider service (which makes them tough to implement).

To help with the above challenge and to help organizations securely store their data in a DBaaS, we are excited to share that you can now create Private CockroachDB Dedicated clusters, and configure Egress perimeter controls. There is no need to install, configure, and manage a firewall / proxy appliance with a self-managed database, or to take half or no measures with a DBaaS.

Why use private clusters and egress perimeter controls?

Network security controls in the context of a database cluster require a holistic approach across the “ingress to the cluster” and “egress from the cluster” use cases. When it comes to CockroachDB Dedicated, we already have a set of strong capabilities to secure the “ingress to the database” - IP allowlisting, AWS PrivateLink & GCP VPC Peering.

For “egress from the database”, following questions often come up from database users:

Can we restrict access to a particular cloud bucket to take all our backups, and not allow access to any other cloud bucket from the database cluster?

Is there a way to limit our database cluster’s access to only a particular Kafka cluster to export real-time changefeeds, and prevent developers from sending such data to their own Kafka clusters or Webhook resources?

Is it possible to remove all public IPs from the database cluster nodes and have all data be exported to external resources through a NAT device?

Can we ensure that the backup or changefeed traffic to the cloud buckets only takes the cloud provider private network and not the public network / internet?

Private clusters and Egress perimeter controls have been designed to address all of the above requirements within CockroachDB Dedicated.

How does a private cluster work?

A private cluster is a cluster without any public IPs on the individual cluster nodes, and it employs a cloud provider-specific NAT gateway (AWS NAT Gateway / GCP Cloud NAT). The NAT Gateway has a static public IP and all egress traffic from cluster nodes to publicly-accessible external resources is routed through the NAT Gateway. For clusters on AWS a per-AZ NAT gateway is utilized to ensure that egress traffic has reliable networking and any relevant traffic can transit out even if an AZ is not available.

Another important aspect of private clusters is that the cloud buckets are accessed over the cloud-provider private network, using S3 Gateway Endpoints for clusters on AWS and Private Google Access for clusters on GCP.

This is relevant for capabilities like Backup / Restore or Changefeeds to the cloud buckets. If you already use AWS PrivateLink or GCP VPC Peering for “ingress to the cluster”, a private cluster can enable end-to-end private connectivity as it uses private connectivity for egress traffic to cloud buckets. That is akin to deploying a DBaaS in your own infrastructure, but while still handing over the entire management to CockroachDB Dedicated. It’s the best kind of Enterprise SaaS imaginable for security-conscious organizations.

Private clusters on GCP will be default for all new CockroachDB Dedicated clusters if an organization chooses to use the capability. On AWS, it’s a per-cluster opt-in where an organization can choose to create it as private or not.

How do egress perimeter controls work?

Egress perimeter controls enable a cloud-agnostic virtual firewall for your CockroachDB Dedicated clusters. Without having to manage the license, cost, and complexity for a transparent or explicit proxy / firewall, you can seamlessly enable egress rules for your database cluster using the CockroachDB Cloud API.

The rules are configured as part of a perimeter control policy that covers each node of the cluster, and are evaluated at runtime for all egress traffic - whether it’s for backup [restore] to [from] cloud buckets, changefeed streaming to a Kafka cluster or a Webhook endpoint, and log export to a cloud-native or third-party observability service.

When a CockroachDB Dedicated cluster is created, the default perimeter control policy operates in ALLOW-ALL mode i.e. it allows egress traffic to all external resources. That is done to allow for easier bootstrapping and prototyping with the cluster. But before an application team is ready to start using the cluster with real / production data, an admin or privileged user could switch it to the DENY-ALL mode before configuring the allowed egress resources for the cluster. The Cloud API provides a real-time status for if the rule enablement is in progress or is complete.

Each egress rule builds upon any of the existing ones, and could be configured for a target CIDR range or for a FQDN (Fully-qualified domain name) assigned to the external resource. Additionally, you could configure path based filtering to a resource by specifying particular endpoints off of its base URL.

One thing to note is that in addition to the egress rules you manage, CockroachDB Dedicated comes with some default rules to enable seamless cluster operations and help comply with the service SLA.

How to get started with private clusters and egress controls?

Please reach out to your Cockroach Labs team to enable Private clusters and Egress perimeter controls for your CockroachDB Cloud organization. Once you’ve created a private cluster, use these instructions to configure egress rules for your cluster. What you need is a list of resources that you would like to allow to be accessible from your cluster. For more general questions you can reach out through our community slack.