This page shows you the easiest way to test an insecure, multi-node CockroachDB cluster, using CockroachDB's AWS CloudFormation template to simplify setup and Kubernetes to automate deployment, maintenance, and load balancing of client workloads.
Before You Begin
Before getting started, it's important to review some limitations and requirements.
You can scale the cluster to a maximum of 15 nodes.
While the AWS region for your deployment is configurable, the cluster runs in a single AWS availability zone within that region. It will easily survive and recover from node failures as long as you deploy at least 3 nodes, but it will not survive an availability zone outage.
- For production resiliency, the recommendation would be to span 3 or more availability zones in a single region or 3 or more regions.
The cluster is completely insecure, which comes with risks:
- There is no network encryption or authentication, and thus no confidentiality.
- The cluster is open to any client by default, although you have the option to restrict client access to a specific range of IP addresses.
- Any user, even
root, can log in without providing a password.
- Any user, connecting as
root, can read or write any data in your cluster.
- You must have an AWS account.
- You must have SSH access in the AWS region where the cluster is deployed.
Step 1. Start CockroachDB
In the CloudFormation UI, review and customize the settings for the cluster. Most of the defaults are sufficient for testing scenarios. However, it's important to select your SSH Key so you'll be able to connect to the Kubernetes master node later, and to set CockroachDB Version to the v2.0 option.
You may also want to:
- Change the AWS region where the cluster will run. The default region is US West. Note that some instance types may not be available in some regions.
- Add an IP Address Whitelist to restrict user access to the CockroachDB admin UI and SQL client access to the cluster. By default, all locations have access.
- Increase the initial Cluster Size. The default is 3 nodes.
In the Load Generators section, select the type of Workload you would like to run against the cluster.
When you're ready to start the cluster, click Create.
The launch process generally takes 10 to 15 minutes. Once you see the
CREATE_COMPLETEstatus in the CloudFormation UI, the cluster is ready for testing.
Step 2. Test the cluster
Install CockroachDB on your local machine, if you haven't already.
In the Outputs section of the CloudFormation UI, note the Connection String.
In a terminal, start the SQL shell built into the
cockroachbinary, using the Connection String as the
$ cockroach sql \ --insecure \ --url="postgresql://root@Cockroach-ApiLoadB-LVZZ3VVHMIDA-1266691548.us-west-2.elb.amazonaws.com:26257?sslmode=disable"
# Welcome to the cockroach SQL interface. # All statements must be terminated by a semicolon. # To exit: CTRL + D. # # Server version: CockroachDB CCL v1.1.4 (linux amd64, built 2018/01/08 17:32:42, go1.8.3) (same version as client) # Cluster ID: bc181e48-da0b-4336-a7a2-a041e195ab5c # # Enter \? for a brief introduction. # root@Cockroach-ApiLoadB-LVZZ3VVHMIDA-1266691548.us-west-2.elb.amazonaws.com:26257/>
Run some basic CockroachDB SQL statements:copy
> CREATE DATABASE bank;copy
> CREATE TABLE bank.accounts (id INT PRIMARY KEY, balance DECIMAL);copy
> INSERT INTO bank.accounts VALUES (1, 1000.50);copy
> SELECT * FROM bank.accounts;
+----+---------+ | id | balance | +----+---------+ | 1 | 1000.5 | +----+---------+ (1 row)
cockroachbinary on your local machine, other client
cockroachcommands can be run in the same way.
Step 3. Monitor the cluster
You can use the cluster's Admin UI to monitor the workload and overall cluster behavior.
In the Outputs section of the CloudFormation UI, click the Web UI link. Then click Metrics on the left-hand navigation bar.
On the Overview dashboard, hover over the SQL Queries graph to see the proportion of reads and writes coming from the load generator.
Scroll down and hover over the Replicas per Node graph to see how CockroachDB automatically replicates your data behind-the-scenes.
Explore other areas of the Admin UI.
Learn more about production monitoring and alerting.
Step 4. Simulate node failure
Kubernetes ensures that the cluster always has the number of nodes you specified during initial configuration (3 by default). When a node fails, Kubernetes automatically creates another node with the same network identity and persistent storage.
To see this in action:
In the Outputs section of the CloudFormation UI, note the SSHProxyCommand.
In a new terminal, run the SSHProxyCommand to SSH into the Kubernetes master node. Be sure to update the
SSH_KEYenvironment variable definition to point to the location of your
List the Kubernetes pods that map to CockroachDB nodes:copy
$ kubectl get pods
NAME READY STATUS RESTARTS AGE cockroachdb-0 1/1 Running 0 1h cockroachdb-1 1/1 Running 0 1h cockroachdb-2 1/1 Running 0 1h
Kill one of CockroachDB nodes:copy
$ kubectl delete pod cockroachdb-2
pod "cockroachdb-2" deleted
In the Admin UI, the Cluster Overview panel may show one node as Suspect. As Kubernetes auto-restarts the node, watch how the node once again becomes healthy.
You can also select the Runtime dashboard and see the restarting of the node in the Live Node Count graph.
Step 6. Stop the cluster
In the CloudFormation UI, select Other Actions > Delete Stack. This is essential for deleting all AWS resources tied to your cluster. If you do not delete these resources, AWS will continue to charge you for them.
- Production Checklist
- Manual Deployment
- Orchestrated Deployment
- Monitoring and Alerting
- Performance Benchmarking
- Performance Tuning
- Test Deployment
- Local Deployment