Follower reads are a mechanism that CockroachDB uses to provide faster reads in situations where you can afford to read data that may be slightly less than current (using
AS OF SYSTEM TIME). Normally, reads have to be serviced by a replica's leaseholder. This can be slow, since the leaseholder may be geographically distant from the gateway node that is issuing the query.
A follower read is a read taken from the closest replica, regardless of the replica's leaseholder status. This can result in much better latency in geo-distributed, multi-region deployments.
The shortest interval at which
AS OF SYSTEM TIME can serve follower reads is 4.8 seconds. In prior versions of CockroachDB, the interval was 48 seconds.
For instructions showing how to use follower reads to get low latency, historical reads in multi-region deployments, see the Follower Reads Topology Pattern.
This is an Enterprise feature.
Watch the demo
How follower reads work
Each CockroachDB range tracks a property called its closed timestamp, which means that no new writes can ever be introduced below that timestamp. The closed timestamp advances forward by some target interval behind the current time. If the range receives a write at a timestamp less than its closed timestamp, it rejects the write.
With follower reads enabled, any replica on a node can serve a read for a key as long as the time at which the operation is performed (i.e., the
AS OF SYSTEM TIME value) is less or equal to the node's closed timestamp.
When a gateway node in a cluster with follower reads enabled receives a request to read a key with a sufficiently old
AS OF SYSTEM TIME value, it forwards the request to the closest node that contains a replica of the data–– whether it be a follower or the leaseholder.
When to use follower reads
As long as your
SELECT operations can tolerate reading data that was current as of the follower read timestamp, follower reads can reduce read latencies and increase throughput.
You should not use follower reads when your application cannot tolerate reading data that was current as of the follower read timestamp, since the results of follower reads may not reflect the latest writes against the tables you are querying.
In addition, follower reads are "read-only" operations; they cannot be used in any way in read-write transactions.
Using follower reads
Enable/disable follower reads
SET CLUSTER SETTING to set
trueto enable follower reads (default)
falseto disable follower reads
> SET CLUSTER SETTING kv.closed_timestamp.follower_reads_enabled = false;
If you have follower reads enabled, you may want to verify that follower reads are happening.
If follower reads are enabled, but the time-travel query is not using
AS OF SYSTEM TIME far enough in the past (as defined by the follower read timestamp), CockroachDB does not perform a follower read. Instead, the read accesses the leaseholder replica. This adds network latency if the leaseholder is not the closest replica to the gateway node.
Verify that follower reads are happening
To verify that your cluster is performing follower reads:
- Make sure that follower reads are enabled.
- Go to the Custom Chart Debug Page in the DB Console and add the metric
follower_read.success_countto the time series graph you see there. The number of follower reads performed by your cluster will be shown.
Run queries that use follower reads
SELECT statement with an
AS OF SYSTEM TIME value at least 4.8 seconds in the past can be a follower read (i.e., served by any replica).
To simplify this calculation, we've added the convenience function
follower_read_timestamp() returns the
statement_timestamp() - 4.8s (known as the follower read timestamp). Using this function in an
AS OF SYSTEM TIME statement will set the time as close as possible to the present time while remaining safe for follower reads:
SELECT ... FROM ... AS OF SYSTEM TIME follower_read_timestamp();
Run read-only transactions that use follower reads
You can set the
AS OF SYSTEM TIME value for all operations in a read-only transaction:
BEGIN; SET TRANSACTION AS OF SYSTEM TIME follower_read_timestamp(); SELECT ... SELECT ... COMMIT;
Note that follower reads are "read-only" operations; they cannot be used in any way in read-write transactions.
SET TRANSACTION statement as shown in the example above will make it easier to use the follower reads feature from drivers and ORMs.
New in v21.1:
AS OF SYSTEM TIME follower_read_timestamp() on all implicit and explicit read-only transactions by default, set the
default_transaction_use_follower_reads session variable to
default_transaction_use_follower_reads=on and follower reads are enabled, all read-only transactions use follower reads.
Follower reads and long-running writes
Long-running write transactions will create write intents with a timestamp near when the transaction began. When a follower read encounters a write intent, it will often end up in a "transaction wait queue", waiting for the operation to complete; however, this runs counter to the benefit follower reads provides.
To counteract this, you can issue all follower reads in explicit transactions set with
BEGIN PRIORITY HIGH AS OF SYSTEM TIME follower_read_timestamp(); SELECT ... SELECT ... COMMIT;