Warning:
This version of CockroachDB is no longer supported. Cockroach Labs supports the current stable release and two releases prior. Please use one of these supported versions.

This page walks you through a simple demonstration of how CockroachDB replicates and distributes data. Starting with a 1-node local cluster, you'll write some data, add 2 nodes, and watch how the data is replicated automatically. You'll then update the cluster to replicate 5 ways, add 2 more nodes, and again watch how all existing replicas are re-replicated to the new nodes.

Before You Begin

Make sure you have already installed CockroachDB.

Step 1. Start a 1-node cluster

copy
icon/buttons/copy
$ cockroach start \
--insecure \
--store=repdemo-node1 \
--host=localhost

Step 2. Write data

In a new terminal, use the cockroach gen command to generate an example intro database:

copy
icon/buttons/copy
$ cockroach gen example-data intro | cockroach sql --insecure

In the same terminal, open the built-in SQL shell and verify that the new intro database was added with one table, mytable:

copy
icon/buttons/copy
$ cockroach sql --insecure
copy
icon/buttons/copy
> SHOW DATABASES;
+--------------------+
|      Database      |
+--------------------+
| intro              |
| system             |
+--------------------+
(2 rows)
copy
icon/buttons/copy
> SHOW TABLES FROM intro;
+---------+
|  Table  |
+---------+
| mytable |
+---------+
(1 row)
copy
icon/buttons/copy
> SELECT * FROM intro.mytable WHERE (l % 2) = 0;
+----+-----------------------------------------------------+
| l  |                          v                          |
+----+-----------------------------------------------------+
|  0 | !__aaawwmqmqmwwwaas,,_        .__aaawwwmqmqmwwaaa,, |
|  2 | !"VT?!"""^~~^"""??T$Wmqaa,_auqmWBT?!"""^~~^^""??YV^ |
|  4 | !                    "?##mW##?"-                    |
|  6 | !  C O N G R A T S  _am#Z??A#ma,           Y        |
|  8 | !                 _ummY"    "9#ma,       A          |
| 10 | !                vm#Z(        )Xmms    Y            |
| 12 | !              .j####mmm#####mm#m##6.               |
| 14 | !   W O W !    jmm###mm######m#mmm##6               |
| 16 | !             ]#me*Xm#m#mm##m#m##SX##c              |
| 18 | !             dm#||+*$##m#mm#m#Svvn##m              |
| 20 | !            :mmE=|+||S##m##m#1nvnnX##;     A       |
| 22 | !            :m#h+|+++=Xmm#m#1nvnnvdmm;     M       |
| 24 | ! Y           $#m>+|+|||##m#1nvnnnnmm#      A       |
| 26 | !  O          ]##z+|+|+|3#mEnnnnvnd##f      Z       |
| 28 | !   U  D       4##c|+|+|]m#kvnvnno##P       E       |
| 30 | !       I       4#ma+|++]mmhvnnvq##P`       !       |
| 32 | !        D I     ?$#q%+|dmmmvnnm##!                 |
| 34 | !           T     -4##wu#mm#pw##7'                  |
| 36 | !                   -?$##m####Y'                    |
| 38 | !             !!       "Y##Y"-                      |
| 40 | !                                                   |
+----+-----------------------------------------------------+
(21 rows)

Exit the SQL shell:

copy
icon/buttons/copy
> \q

Step 3. Add two nodes

In a new terminal, add node 2:

copy
icon/buttons/copy
$ cockroach start \
--insecure \
--store=repdemo-node2 \
--host=localhost \
--port=26258 \
--http-port=8081 \
--join=localhost:26257

In a new terminal, add node 3:

copy
icon/buttons/copy
$ cockroach start \
--insecure \
--store=repdemo-node3 \
--host=localhost \
--port=26259 \
--http-port=8082 \
--join=localhost:26257

Step 4. Watch data replicate to the new nodes

Open the Admin UI at http://localhost:8080 to see that all three nodes are listed. At first, the replica count will be lower for nodes 2 and 3. Very soon, the replica count will be identical across all three nodes, indicating that all data in the cluster has been replicated 3 times; there's a copy of every piece of data on each node.

CockroachDB Admin UI

Step 5. Increase the replication factor

As you just saw, CockroachDB replicates data 3 times by default. Now, in the terminal you used for the built-in SQL shell or in a new terminal, use the cockroach zone command change the cluster's .default replication factor to 5:

copy
icon/buttons/copy
$ echo 'num_replicas: 5' | cockroach zone set .default --insecure -f -
range_min_bytes: 1048576
range_max_bytes: 67108864
gc:
  ttlseconds: 90000
num_replicas: 5
constraints: []

In addition to the .default replication zone for database and table data, CockroachDB comes with pre-configured replication zones for important internal data. To list these pre-configured zones, use the cockroach zone ls subcommand:

copy
icon/buttons/copy
$ cockroach zone ls --insecure
.default
.liveness
.meta
system.jobs

For the cluster as a whole to remain available, the "system ranges" for this internal data must always retain a majority of their replicas. Therefore, if you increase the default replication factor, be sure to also increase the replication factor for these replication zones as well:

copy
icon/buttons/copy
$ echo 'num_replicas: 5' | cockroach zone set .liveness --insecure -f -
range_min_bytes: 1048576
range_max_bytes: 67108864
gc:
  ttlseconds: 600
num_replicas: 5
constraints: []
copy
icon/buttons/copy
$ echo 'num_replicas: 5' | cockroach zone set .meta --insecure -f -
range_min_bytes: 1048576
range_max_bytes: 67108864
gc:
  ttlseconds: 3600
num_replicas: 5
constraints: []
copy
icon/buttons/copy
$ echo 'num_replicas: 5' | cockroach zone set system.jobs --insecure -f -
range_min_bytes: 1048576
range_max_bytes: 67108864
gc:
  ttlseconds: 600
num_replicas: 5
constraints: []

Step 6. Add two more nodes

In a new terminal, add node 4:

copy
icon/buttons/copy
$ cockroach start --insecure \
--host=localhost \
--store=repdemo-node4 \
--port=26260 \
--http-port=8083 \
--join=localhost:26257

In a new terminal, add node 5:

copy
icon/buttons/copy
$ cockroach start \
--insecure \
--host=localhost \
--store=repdemo-node5 \
--port=26261 \
--http-port=8084 \
--join=localhost:26257

Step 7. Watch data replicate to the new nodes

Back in the Admin UI, you'll see that there are now 5 nodes listed. Again, at first, the replica count will be lower for nodes 4 and 5. But because you changed the default replication factor to 5, very soon, the replica count will be identical across all 5 nodes, indicating that all data in the cluster has been replicated 5 times.

CockroachDB Admin UI

Step 8. Stop the cluster

Once you're done with your test cluster, stop each node by switching to its terminal and pressing CTRL-C.

Tip:

For the last 2 nodes, the shutdown process will take longer (about a minute) and will eventually force kill the nodes. This is because, with only 2 nodes still online, a majority of replicas are no longer available (3 of 5), and so the cluster is not operational. To speed up the process, press CTRL-C a second time in the nodes' terminals.

If you do not plan to restart the cluster, you may want to remove the nodes' data stores:

copy
icon/buttons/copy
$ rm -rf repdemo-node1 repdemo-node2 repdemo-node3 repdemo-node4 repdemo-node5

What's Next?

Use a local cluster to explore these other core CockroachDB features:



Yes No