What's New in v21.2.4

January 10, 2022

Get future release notes emailed to you:


The CockroachDB executable for Windows is experimental and not suitable for production deployments. Windows 8 or higher is required.

Docker image

$ docker pull cockroachdb/cockroach:v21.2.4

Security updates

  • It is now possible to pre-compute the hash of the password credentials of a SQL user client-side, and set the SQL user's password using the hash, so that CockroachDB never sees the password string in the clear in the SQL session. This auto-detection is subject to the new cluster setting server.user_login.store_client_pre_hashed_passwords.enabled. This setting defaults to true (i.e. feature enabled). This feature is meant for use in automation/orchestration, when the control plane constructs passwords for users outside of CockroachDB, and there is an architectural desire to ensure that cleartext passwords are not transmitted/stored in-clear. Note: when the client provides the password hash, CockroachDB cannot carry any checks on the internal structure of the password, such as minimum length, special characters, etc. Should a deployment require such checks to be performed database-side, the operator would need to disable the mechanism via the cluster setting named above. When upgrading a cluster from a previous version, to ensure that the feature remains disabled throughout the upgrade, use the following statement prior to the upgrade: INSERT INTO system.settings(name, value, "valueType") VALUES('server.user_login.store_client_pre_hashed_passwords.enabled', 'false', 'b');. (We do not recommend relying on the database to perform password checks. Our recommended deployment best practice is to implement credential definitions in a control plane / identity provider that is separate from the database.) #73855
  • The server.identity_map.configuration cluster setting allows a pg_ident.conf file to be uploaded to support dynamically remapping system identities (e.g. Kerberos or X.509 principals) to database usernames. This supports use cases where X.509 certificates must conform to organizational standards that mandate the use of Common Names that are not valid SQL usernames (e.g. CN=carl@example.com => carl). Mapping rules that result in the root, node, or other reserved usernames will result in an error when the client attempts to connect. #74459
  • The client_authentication_info structured log message provides a new "SystemIdentity" field with the client-provided system identity. The existing "User" field will be populated after any Host-Based Authentication (HBA) rules have been selected and applied, which may include a system-identity to database-username mapping. #74459
  • GSSAPI-based authentication can now use either the HBA "map" option or "include_realm=0" to map the incoming principal to a database username. Existing configurations will operate unchanged, however operators are encouraged to migrate from "include_realm=0" to "map" to avoid ambiguity in deployments where multiple realms are present. #74459
  • Incoming system identities are normalized to lower-case before they are evaluated against any active identity-mapping HBA configuration. For example, an incoming GSSAPI principal "carl@EXAMPLE.COM" would only be matched by rules such as "example carl@example.com carl" or "example /^(.*)@example.com$ \1". #74459

Enterprise edition changes

  • Changefeeds can be created with a new option called metrics_label which lets operators configure changefeeds to use a dedicated set of metrics for those changefeed(s) so that they can be monitored independently of other changefeed(s) in the system. #73014

SQL language changes

  • The create_type_statements table now has an index on descriptor_id. #73669
  • Added the new column stmt to the crdb_internal.(cluster|node)_distsql_flows virtual table. It is populated on a best effort basis. #73581
  • Table backups of REGIONAL BY ROW, REGIONAL BY TABLE, and GLOBAL tables are now supported. #73087
  • The cluster setting called sql.defaults.reorder_joins_limit that controls the default for the session setting reorder_joins_limit is now public and included in the cluster setting docs. #73889
  • The RULE privilege was added for compatibility with Postgres. It is impossible to grant it, but it is supported as a parameter of the has_table_privilege function. #74065
  • The CREATE ROLE and ALTER ROLE statements now accept password hashes computed by the client app. For example: CREATE USER foo WITH PASSWORD 'CRDB-BCRYPT$2a$10$.....'. This feature is not meant for use by human users / in interactive sessions; it is meant for use in programs, using the computation algorithm described below. This auto-detection can be disabled by changing the cluster setting server.user_login.store_client_pre_hashed_passwords.enabled to false. This design mimics the behavior of PostgreSQL, which recognizes pre-computed password hashes when presented to the regular PASSWORD option. The password hashes are auto-detected based on their lexical structure. For example, any password that starts with the prefix CRDB-BCRYPT$, followed by a valid encoding of a bcrypt hash (as detailed below), is considered a candidate password hash. To ascertain whether a password hash will be recognized as such, orchestration code can use the new built-in function crdb_internal.check_password_hash_format(). #73855

    • CockroachDB only recognizes password hashes computed using bcrypt, as follows (we detail this algorithm so that orchestration software can implement their own password hash computation, separate from the database):

      1. Take the cleartext password string.
      2. Append the following byte array to the password: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 (These are 32 hex-encoded bytes.)
      3. Choose a bcrypt cost. (CockroachDB servers use cost 10 by default.)
      4. Generate a bcrypt hash of the string generated at step 2 with the cost chosen at step 3, as per https://en.wikipedia.org/wiki/Bcrypt or https://bcrypt.online/ Note that CockroachDB only supports hashes computed using bcrypt version 2a.
      5. Encode the hash into the format recognized by CockroachDB: the string CRDB-BCRYPT, followed by the standard bcrypt hash encoding ($2a$...).


        Hash method Recognized by check_password_hash_format() ALTER/CREATE USER WITH PASSWORD
        crdb-bcrypt yes (CRDB-BCRYPT$2a$...) recognized if enabled via cluster setting
        scram-sha-256 yes (SCRAM-SHA-256$4096:...) not implemented yet (issue #42519)
        md5 yes (md5...) obsolete, will likely not be implemented
  • Backported the server.user_login.store_client_pre_hashed_passwords.enabled cluster setting to v21.2. The backported default value in v21.2 is false. In v22.1 the default will be true. #73855

Operational changes

DB Console changes

  • Added new formatting functions to create summarized queries for SELECT, INSERT, and UPDATE statements. Also added new metadata fields, which will later be used to pass this information to the front-end Statements page. #73661
  • The jobs overview table in DB Console now shows when jobs have the status "reverting", and shows the badge "retrying" when running or reverting jobs are also retrying. Hovering over the status for a "retrying" job will show the "Next execution time" in UTC. Two new columns, "Last Execution Time (UTC)" and "Execution Count", were also added to the jobs overview table in DB Console, and the "Status" column was moved left to the second column in the table. The status query parameter in the /jobs endpoint now supports the values reverting and retrying. #73624
  • Added new statement summaries to the Statements page. This applies for SELECT, INSERT/UPSERT, and UPDATE statements, and will enable them to be more detailed and less ambiguous than our previous formats. #73661
  • Added new summarized formats for SELECT, INSERT/UPSERT, and UPDATE statements on the Sessions page and the Transactions page, to be consistent with the Statements page. Show "Mean rows written" as a metric for all statement types on the Statements page, instead of hiding this metric for SELECT statements. #73661
  • Made visual improvements to the DB Console. #73386
  • Updated text of filter drop-downs in the DB Console, replacing "usage" with "statement" for consistency. #74421

Bug fixes

  • Fixed a bug which caused corruption of partial indexes, which could cause incorrect query results. The bug was only present when two or more partial indexes in the same table had identical WHERE clauses. This bug has been present since v21.1.0. For more information, see Technical Advisory 74385. #74471
  • Fixed an internal error "empty Datums being compared to other" that could occur during planning for some SELECT queries over tables that included a DEFAULT partition value in a PARTITION BY LIST clause. This bug was present since v21.1.0. This bug does not exist in CockroachDB v20.2.x and earlier. #73664
  • Fixed a bug that could cause a CockroachDB node to deadlock upon startup in extremely rare cases. If encountered, a stack trace generated by SIGQUIT would show the function makeStartLine() near the top. This bug had existed since v21.1.0. #71407
  • Fixed a bug where CockroachDB could crash when reading a secondary index with a STORING clause in reverse direction (i.e. with ORDER BY col DESC). This bug was introduced in v21.2.0. #73699
  • Fixed a bug where the correct index count was not displayed in the Indexes column on the Databases page of the DB Console. #73747
  • Fixed a bug where a failed IMPORT INTO to a non-empty table would be unable to clean up the partially imported data when run in a serverless cluster because the operation to do so was incorrectly denied for tenants. #73541
  • Fixed a bug in database and schema restore cleanup that results in a dangling descriptor entry on job failure. #73411
  • Fixed a bug which allowed queries to reference internal columns created by the system for expression indexes. These columns, which had names prefixed with crdb_internal_idx_expr, can no longer be referenced in queries. This bug was present since version v21.2.0 when expression indexes were released. #74285
  • Fixed a bug with ungraceful shutdown of distributed queries in some rare cases. "Ungraceful" here means due to a statement_timeout (most likely) or because a node crashed. #73958
  • Fixed a bug where CockroachDB could return a spurious "context canceled" error for a query that actually succeeded in extremely rare cases. #73958
  • Fixed a bug where CockroachDB could encounter an internal error when executing queries with multiple window functions and one of those functions returned an INT2 or INT4 type. #74311
  • Fixed a bug where it was possible for cockroach debug zip and the log file viewer in the DB Console to observe incomplete log entries at the end of log files—especially the log file currently being written to by the CockroachDB process. This bug was introduced in a very early version of CockroachDB. #74153
  • Fixed a bug where Changefeeds would emit NULL values for virtual computed columns. Previously, the changefeeds would crash if these were set to NOT NULL. #74095
  • Internal columns created by the system to support expression indexes are now omitted from the output of SHOW COLUMNS statements and the information_schema.columns table. #73540
  • Fixed a bug where IMPORT TABLE ... PGDUMP DATA with a COPY FROM statement in the dump file that had fewer target columns than the inline table definition would result in a nil pointer exception. #74435
  • Fixed a bug where escape character processing was missing from constraint span generation, which resulted in incorrect results when doing escaped LIKE lookups. #74259
  • Fixed a bug affecting the redactability of logging tags in output log entries. This bug was introduced in the v21.2.0 release. #74155

Performance improvements

  • Bulk ingestion of small write batches (e.g. index backfill into a large number of ranges) is now throttled, to avoid buildup of read amplification and associated performance degradation. Concurrency is controlled by the new cluster setting kv.bulk_io_write.concurrent_addsstable_as_writes_requests. #74071



This release includes 57 merged PRs by 31 authors.

YesYes NoNo