Publication date: September 22, 2023
Queries reading a table that uses at least one explicit column family in its definition and was restored from a backup using the following versions may return errors or incorrect results:
- v22.2.7 to v22.2.13
- v23.1.0 to v23.1.8
Restored tables that do not use explicit column families, or that were restored using versions prior to v22.2.7, v22.2.14 or later, or v23.1.9 or later, are unaffected.
RESTORE process splits the key-value data into ranges. If the data being restored contains a table whose definition includes an explicit column family, the affected versions of
RESTORE may incorrectly choose a split boundary. If those ranges are subsequently served by separate leaseholders, queries may fail to read expected data. These queries may then return errors or incorrect results.
The incorrect results or errors can persist even after the cluster is upgraded to an unaffected version. They may require manual intervention by our support team to be permanently resolved. Observation of incorrect results or errors may be intermittent due to the movement of leaseholders or changes in query plans. Customers with tables that use at least one explicit column family and were restored using an affected version should run the following query:
SELECT start_pretty FROM crdb_internal.ranges_no_leases WHERE table_id IN (SELECT id FROM system.descriptor WHERE jsonb_array_length(crdb_internal.pb_to_json('desc', descriptor)->'table'->'families') > 1 AND id > 50) AND start_pretty ~ '.*/[0-9]+/NULL';
If the result set is empty, no further action is required. If the result set is non-empty, contact our support team.
This is resolved in CockroachDB by PR #109713 for v23.1 and PR #109750 for v22.2. The PRs update the method used to select the boundaries at which ranges of data are split during
RESTORE to avoid splitting between two column families of the same row.
These fixes are available in CockroachDB maintenance releases v23.1.9 and v22.2.14.
This public issue is tracked in 109483.
Users of CockroachDB v23.1.0 through v23.1.8 and v22.2.7 through v22.2.13 are encouraged to upgrade to v23.1.10 or v22.2.14 before performing a
RESTORE, especially if they are restoring tables which are defined with multiple explicit column families.
Users who are operating a cluster that contains potentially affected tables should contact our support team to determine if they are affected.
Queries following the restoration of some types of backups, specifically those that include larger tables which are defined with multiple explicit column families, can produce errors or incorrect results. While the behavior in
RESTORE which leads to this condition has since been fixed in v22.2.14 and v23.1.10, clusters which performed a potentially affected restore while running an affected version prior to upgrading may continue to be affected. Customers with such clusters should contact support to determine if they require remediation. Clusters which do not contain tables which utilize multiple column families and were restored on an affected version are unaffected.
Questions about any technical alert can be directed to our support team.