EXPERIMENTAL CHANGEFEED FOR

​

Required privileges

GRANT SELECT ON TABLE example_table TO user;

​

Legacy privilege model

​

Considerations

• Because sinkless changefeeds return results differently than other SQL statements, they require a dedicated database connection with specific settings around result buffering. In normal operation, CockroachDB improves performance by buffering results server-side before returning them to a client; however, result buffering is automatically turned off for sinkless changefeeds. Also, sinkless changefeeds have different cancellation behavior than other queries: they can only be canceled by closing the underlying connection or issuing a statement on a separate connection. Combined, these attributes of changefeeds mean that applications should explicitly create dedicated connections to consume changefeed data, instead of using a connection pool as most client drivers do by default. This cancellation behavior (i.e., close the underlying connection to cancel the changefeed) also extends to client driver usage; in particular, when a client driver calls Rows.Close() after encountering errors for a stream of rows. The pgwire protocol requires that the rows be consumed before the connection is again usable, but in the case of a sinkless changefeed, the rows are never consumed. It is therefore critical that you close the connection, otherwise the application will be blocked forever on Rows.Close().
• In most cases, each version of a row will be emitted once. However, some infrequent conditions (e.g., node failures, network partitions) will cause them to be repeated. This gives our changefeeds an at-least-once delivery guarantee. For more information, see .
• As of v22.1, changefeeds filter out from events by default. This is a . To maintain the changefeed behavior in previous versions where values are emitted for virtual computed columns, see the option for more detail.

​

Synopsis

> EXPERIMENTAL CHANGEFEED FOR table_name [ WITH (option [= value] [, ...]) ];

​

Parameters

​

Options

​

Avro limitations

• must have precision specified.
• (or its aliases BYTEA and BLOB ) are often used to store machine-readable data. When you stream these types through a changefeed with , CockroachDB does not encode or change the data. However, Avro clients can often include escape sequences to present the data in a printable format, which can interfere with deserialization. A potential solution is to hex-encode BYTES values when initially inserting them into CockroachDB. This will ensure that Avro clients can consistently decode the hexadecimal. Note that hex-encoding values at insertion will increase record size.
• and types are encoded as arrays of 64-bit integers. For efficiency, CockroachDB encodes BIT and VARBIT bitfield types as arrays of 64-bit integers. That is, base-2 (binary format) BIT and VARBIT data types are converted to base 10 and stored in arrays. Encoding in CockroachDB is big-endian, therefore the last value may have many trailing zeroes. For this reason, the first value of each array is the number of bits that are used in the last value of the array. For instance, if the bitfield is 129 bits long, there will be 4 integers in the array. The first integer will be 1; representing the number of bits in the last value, the second integer will be the first 64 bits, the third integer will be bits 65–128, and the last integer will either be 0 or 9223372036854775808 (i.e., the integer with only the first bit set, or 1000000000000000000000000000000000000000000000000000000000000000 when base 2). This example is base-10 encoded into an array as follows:
  {"array": [1, <first 64 bits>, <second 64 bits>, 0 or 9223372036854775808]}
  
  For downstream processing, it is necessary to base-2 encode every element in the array (except for the first element). The first number in the array gives you the number of bits to take from the last base-2 number — that is, the most significant bits. So, in the example above this would be 1. Finally, all the base-2 numbers can be appended together, which will result in the original number of bits, 129. In a different example of this process where the bitfield is 136 bits long, the array would be similar to the following when base-10 encoded:
  {"array": [8, 18293058736425533439, 18446744073709551615, 13690942867206307840]}
  
  To then work with this data, you would convert each of the elements in the array to base-2 numbers, besides the first element. For the above array, this would convert to:
  [8, 1111110111011011111111111111111111111111111111111111111111111111, 1111111111111111111111111111111111111111111111111111111111111111, 1011111000000000000000000000000000000000000000000000000000000000]
  
  Next, you use the first element in the array to take the number of bits from the last base-2 element, 10111110. Finally, you append each of the base-2 numbers together — in the above array, the second, third, and truncated last element. This results in 136 bits, the original number of bits.
• A changefeed in will not be able to serialize .

​

Examples

​

Create a changefeed

EXPERIMENTAL CHANGEFEED FOR cdc_test;

table,key,value
cdc_test,[0],"{""after"": {""a"": 0}}"

​

Create a changefeed with Avro

EXPERIMENTAL CHANGEFEED FOR cdc_test WITH format = avro, confluent_schema_registry = 'http://localhost:8081';

table,key,value
cdc_test,\000\000\000\000\001\002\000,\000\000\000\000\002\002\002\000

​

Create a changefeed on a table with column families

EXPERIMENTAL CHANGEFEED FOR TABLE cdc_test FAMILY f1;

EXPERIMENTAL CHANGEFEED FOR TABLE cdc_test WITH split_column_families;

​

See also