Cloud Storage Authentication

The instructions in this section are specific to Amazon S3. To see instructions for other cloud storage providers, select a different option from the dropdown at the top right of this page.

You can use the following authentication options for Amazon S3 storage buckets:

Specified: You specify the AWS access key ID and secret access key in the URI when connecting.
Implicit: You store the needed AWS credentials as environment variables, and may omit them when connecting. As an alternative, you can use implicit authentication with the IAM role that CockroachDB Cloud automatically creates and manages for each CockroachDB Advanced cluster to avoid storing any credentials in your cluster.

To have users assume IAM roles to complete operations on an S3 bucket, you can also configure assume role authentication in addition to specified or implicit. If your CockroachDB cluster is deployed on Kubernetes, you can use workload identities with assume role authentication.

Amazon S3 specified

If the AUTH parameter is not provided, AWS connections default to specified and the access keys must be provided in the URI parameters.Use these parameters to specify your credentials:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

As an example:

BACKUP DATABASE &lt;database> INTO 's3://{bucket name}/{path in bucket}/?AWS_ACCESS_KEY_ID={access key ID}&AWS_SECRET_ACCESS_KEY={secret access key}';

See Amazon’s documentation Managing access keys for IAM users for more details on S3 credentials.

Assume role authentication allows you to use temporary short-lived credentials to authenticate to an Amazon S3 buckets. For more detail, see Amazon S3 assume role.

Amazon S3 implicit

To use implicit authentication on a CockroachDB Cloud cluster, it is necessary to authenticate using assume role authentication. See Amazon S3 assume role for more details.

Use implicit authentication with your own IAM role

If the AUTH parameter is implicit, the access keys can be omitted and the credentials will be loaded from the environment (i.e., the machines running the backup).

BACKUP DATABASE &lt;database> INTO 's3://{bucket name}/{path}?AUTH=implicit';

You can associate an EC2 instance with an IAM role to provide implicit access to S3 storage within the IAM role’s policy. In the following command, the instance example EC2 instance is associated with the example profile instance profile, giving the EC2 instance implicit access to any example profile S3 buckets.

aws ec2 associate-iam-instance-profile --iam-instance-profile Name={example profile} --region={us-east-2} --instance-id {instance example}

You can grant a user the EXTERNALIOIMPLICITACCESS to interact with external resources that require implicit access.

If the use of implicit credentials is disabled with the , you will receive an error when you access external cloud storage services with AUTH=implicit.

Amazon S3 assume role

CockroachDB supports assume role authentication on clusters running CockroachDB v22.2 and above. On earlier versions, an attempt to authenticate to cloud storage with ASSUME_ROLE will result in failed bulk operations.

Use implicit authentication with your own service account

To limit the control access to your Amazon S3 buckets, you can create IAM roles for users to assume. IAM roles do not have an association to a particular user. The role contains permissions that define the operations a user (or Principal) can complete. An IAM user can then assume a role to undertake a CockroachDB backup, restore, import, etc. As a result, the IAM user only has access to the assigned role, rather than having unlimited access to an S3 bucket.You can use the external_id option with ASSUME_ROLE to specify an external ID for third-party access to your Amazon S3 bucket. The external ID is a unique ID that the third party provides you along with their ARN. For guidance on external_id usage in CockroachDB, refer to the following example.

You must URL encode the entire value passed to ASSUME_ROLE.

Role assumption applies the principle of least privilege rather than directly providing privilege to a user. Creating IAM roles to manage access to AWS resources is Amazon’s recommended approach compared to giving access directly to IAM users.The following section demonstrates setting up assume role authentication between two users. For examples of assume role authentication for more than two roles, refer to Role chaining for additional detail.

Set up Amazon S3 assume role

For example, to configure a user to assume an IAM role that allows a bulk operation to an Amazon S3 bucket, take the following steps:

Create a role that contains a policy to interact with the S3 buckets depending on the operation your user needs to complete. See the section for details on the minimum permissions each CockroachDB bulk operation requires. You can create an IAM role in Amazon’s Management console, under the IAM and then Roles menu. Alternately, you can use the AWS CLI.

1. If you do not already have the user that needs to assume the role, create the user. Under IAM in the Amazon console, navigate to Users and Add users. You can then add the necessary permissions by clicking on the Permissions tab. Ensure that the IAM user has sts:AssumeRole permissions attached. The following policy will give the user assume role permissions:

{
"Version": "2012-10-17",
"Statement": {
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::{account ID}:role/{role name}"
    }
}

The Resource here is the Amazon Resource Name (ARN) of the role you created in step 1. You can copy this from the role’s Summary page.The sts:AssumeRole permission allows the user to obtain a temporary set of security credentials that gives them access to an S3 bucket to which they would not have access with their user-based permissions.

AWS user summary page showing the JSON policy in place

1. Return to your IAM role’s Summary page, and click on the Trust Relationships tab. Add a trust policy into the role, which will define the users that can assume the role.The following trust policy provides the user the privilege to assume the role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789123:user/{user}"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

When creating a trust policy consider the following:

In the trust policy you need to include the ARN of the user that you want to assume the role under Principal. You can also include the Condition attribute to further control access to the Amazon S3 bucket. For example, this could limit the operation to a specified date range, to users with multi-factor authentication enabled, or to specific IP addresses.
- The Condition attribute allows you to specify an external ID that the user has provided with their ARN. For example:
  "Condition": {"StringEquals": {"sts:ExternalId": "Unique ID Assigned by the user"}}
  For more detail on creating and implementing external IDs in your AWS account, refer to Amazon’s documentation How to use an external ID when granting access to your AWS resources to a third party.
If you set the Principal ARN to root, this will allow any IAM user in the account with the AssumeRole permission to access the Amazon S3 bucket as per the defined IAM role permissions.
When the IAM user takes on the role to perform a bulk operation, they are temporarily granted the permissions contained in the role. That is, not the permissions specified in their user profile.

Run the bulk operation. If using specified authentication, pass in the S3 bucket’s URL with the IAM user’s AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If using implicit authentication, specify AUTH=IMPLICIT instead. For assuming the role, pass the assumed role’s ARN, which you can copy from the IAM role’s summary page:

BACKUP DATABASE movr INTO 's3://{bucket name}?AWS_ACCESS_KEY_ID={user key}&AWS_SECRET_ACCESS_KEY={user secret key}&ASSUME_ROLE=arn%3Aaws%3Aiam%3A%3A{account ID}%3Arole%2F{role name}' AS OF SYSTEM TIME '-10s';

If your user also has an external ID, you can pass that with ASSUME_ROLE:

BACKUP DATABASE movr INTO 's3://{bucket name}?AWS_ACCESS_KEY_ID={user key}&AWS_SECRET_ACCESS_KEY={user secret key}&ASSUME_ROLE=arn%3Aaws%3Aiam%3A%3A{account ID}%3Arole%2F{role name}%3Bexternal_id%3D{Unique ID}' AS OF SYSTEM TIME '-10s';

CockroachDB also supports authentication for assuming roles when taking encrypted backups. To use with an encrypted backup, pass the ASSUME_ROLE parameter to the KMS URI as well as the bucket’s:

BACKUP INTO 's3://{bucket name}?AWS_ACCESS_KEY_ID={user key}&AWS_SECRET_ACCESS_KEY={user secret key}&ASSUME_ROLE={ARN}' WITH kms = 'aws:///{key}?AWS_ACCESS_KEY_ID={user key}&AWS_SECRET_ACCESS_KEY={user secret key}&REGION={region}&ASSUME_ROLE={ARN}';

For more information on AWS KMS URI formats, refer to .

Amazon S3 role chaining

Role chaining allows a user to assume a role through an intermediate role(s) instead of the user directly assuming a role. In this way, the role chain passes the request for access to the final role in the chain. Role chaining could be useful when a third-party organization needs access to your Amazon S3 bucket to complete a bulk operation. Or, your organization could grant roles based on limited-privilege levels.You can configure role chaining across multiple roles in the same way you configure assume role authentication for one role, except across all roles that will participate in the chain. For example, to configure role A to assume role B, ensure that the ARN of role A is present in role B’s trust policy with the sts:AssumeRole action.The role B’s trust policy must contain:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::{account-A-ID}:role/{role A name}"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then, to allow role B to assume a third role, role C, role C’s trust policy needs to include role B in the same way. For example, to chain three roles so that a user could assume role C, it is necessary to verify the following:

User →	Role A →	Role B →	Role C
Has permission to assume role A. See step 2.	Has a trust policy that permits the user to assume role A. See step 3.	Has a trust policy that permits role A to assume role B.	Has a trust policy that permits role B to assume role C.
Needs permission to assume role B.	Needs permission to assume role C.

When passing a chained role into BACKUP, it will follow this pattern:

BACKUP DATABASE movr INTO "s3://{bucket name}?AWS_ACCESS_KEY_ID={user's key}&AWS_SECRET_ACCESS_KEY={user's secret key}&ASSUME_ROLE={role A ARN}%2C{role B ARN}%2C{role C ARN}" AS OF SYSTEM TIME '-10s';

You can also specify a different external ID for each chained role. For example:

BACKUP DATABASE movr INTO "s3://{bucket name}?AWS_ACCESS_KEY_ID={user's key}&AWS_SECRET_ACCESS_KEY={user's secret key}&ASSUME_ROLE={role A ARN}%3Bexternal_id%3D{ID A}%2C{role B ARN}%3Bexternal_id%3D{ID B}%2C{role C ARN}%3Bexternal_id%3D{ID C}" AS OF SYSTEM TIME '-10s';

Each chained role is listed separated by a , character. You can copy the ARN of the role from its summary page in the AWS Management console.

Amazon S3 workload identity

With a CockroachDB cluster deployed on , you can allow your pods to authenticate as an IAM role that you have associated to a Kubernetes service account. You can then use assume role authentication to allow that IAM role to assume another role that has permissions to perform bulk operations to an S3 bucket.This means that a CockroachDB node will only be able to access credentials for the IAM role associated with the Kubernetes service account.You can use workload identities with assume role authentication to run the following operations:

To use assume role authentication, you will need at least two IAM roles:

An identity role: the IAM role associated with your Kubernetes service account.
An operation role: the IAM role to be assumed. This contains the permissions required to complete a CockroachDB operation.

For a walkthrough on how to set this up for CockroachDB self-hosted and CockroachDB Advanced clusters, see the following steps.

Step 1. Set up the identity role

The first step to setting up workload identity authentication is different for CockroachDB self-hosted and CockroachDB Advanced clusters. As you work through the step, ensure you use the relevant section for your cluster:

CockroachDB Advanced
CockroachDB self-hosted

Set up the identity role for CockroachDB Advanced clusters

Each CockroachDB Advanced cluster has a pre-configured IAM role for Amazon EKS service accounts that acts as the cluster’s identity. In addition, the clusters have a unique built-in, functionality IAM role, which you can configure as a trusted identity within the Trust Policy of your cloud IAM roles. This allows you to have the built-in IAM role assume another role (or roles as part of a chain). This section will refer to the built-in IAM role as the “identity role”.The prefixes for the built-in IAM identity roles are as follows:

IAM role name	Operation
`crl-dr-store-user`	Backup, Restore, Import, Export
`crl-cdc-sink-user`	Changefeeds
`crl-kms-user`	Encrypted backups and restores

Construct the ARN for your identity role. You will need this to add into the Trust Policy of an operation role or intermediary role. The IAM role’s ARN follows this pattern (in this example the crl-dr-store-user role name is used for a backup):

arn:aws:iam::{AWS account ID}:role/crl-dr-store-user-{cluster ID suffix}

You can find the AWS account ID and your cluster’s ID using the :

curl --request GET --url https://cockroachlabs.cloud/api/v1/clusters --header "Authorization: Bearer {secret key}"

Combine the last 12 digits of your cluster’s id and the full 12-digit account_id in the following fashion to form the ARN:

arn:aws:iam::{123456789123}:role/crl-dr-store-user-{123456789123}

See Step 2. Trust the identity role to add this ARN to an operation role’s Trust Policy.

Set up the identity role for CockroachDB Self-hosted clusters

First, create an IAM role for your Kubernetes service account to assume, and then configure your CockroachDB pods to use the service account. We will refer to this IAM role as an “identity role”. You can complete all of these steps with Amazon’s guide on IAM roles for service accounts.Once you have an identity role that your CockroachDB nodes can assume, you can configure the identity role to assume another IAM role that contains the permissions to perform a bulk operation.Copy the ARN of the identity role. In the Amazon management console, click on IAM, then Roles, and select the name of your identity role. From the Summary page, copy your ARN. You will need this when configuring the Trust Policy for the IAM role to be assumed.

Role summary page showing the ARN copied

See Step 2. Trust the identity role to add this ARN to an operation role’s Trust Policy.

Step 2. Trust the identity role

Create or open the operation role that your identity role will assume.

If you already have the role that contains permissions for the operation, ensure that you add the identity role ARN to the role’s Trust Relationships tab on the Summary page.

To create an operation role, click Create Role under the Roles menu. Select Custom trust policy and then add the ARN of your identity role (from Step 1) to the JSON by clicking Principal. This will open a dialog box. Select IAM Roles for Principal Type and paste the ARN. Click Add Principal and then Next.
On the Add Permissions page, search for the permission policies that the role will need to complete the bulk operation. Or, use the Create Policy button to define the required permissions. You can use the visual editor to select the service, actions, and resources. Or, use the JSON tab to specify the policy. For the JSON editor, see for an example and detail on the minimum permissions required for each operation to complete. Click Next.
Finally, give the role a name on the Name, review, and create page. The following screenshot shows the selected trust policy and permissions:

Step 3. Run the operation by assuming the role

Finally, you’ll use the ASSUME_ROLE parameter in your SQL statement to assume a role that contains the necessary permissions. In this step, ensure you read the section for your cluster type:

CockroachDB Advanced
CockroachDB self-hosted

Run the operation from a CockroachDB Advanced cluster

To run an operation, use implicit authentication so that your AWS cluster can authenticate directly. To the ASSUME_ROLE parameter, pass the pre-configured identity role from Step 1 followed by a comma, and finally the operation role(s) you need the identity to assume.For a backup to Amazon S3:

BACKUP DATABASE {database} INTO 's3://{bucket name}/{path}?AUTH=implicit&ASSUME_ROLE=arn%3Aaws%3Aiam%3A%3A{AWS account ID}%3Arole%2Fcrl-dr-store-user-{cluster ID suffix}%2Carn%3Aaws%3Aiam%3A%3A{account ID}%3Arole%2F{operation role name}' AS OF SYSTEM TIME '-10s';

In this SQL statement, the identity role assumes the operation role that has permission to write a backup to the S3 bucket.

Run the operation from a CockroachDB Self-hosted cluster

To run an operation, you can use implicit authentication for your identity role and pass the ASSUME_ROLE parameter for your operation role.For a backup to Amazon S3:

BACKUP DATABASE {database} INTO 's3://{bucket name}/{path}?AUTH=implicit&ASSUME_ROLE=arn%3Aaws%3Aiam%3A%3A{account ID}%3Arole%2F{operation role name}' AS OF SYSTEM TIME '-10s';

In this SQL statement, AUTH=implicit uses the identity role to authenticate to the S3 bucket. The identity role then assumes the operation role that has permission to write a backup to the S3 bucket.You can use the following authentication options for Google Cloud Storage buckets and :

Specified: You specify the Google Cloud credentials key in the URI when connecting.
Implicit: You store the needed Google Cloud credentials as environment variables, and may omit them when connecting. As an alternative, you can use implicit authentication with the service account that CockroachDB Cloud automatically creates and manages for each CockroachDB Advanced cluster to avoid storing any credentials in your cluster.

To have users assume IAM roles to complete operations on a Google Cloud Storage bucket, you can also configure assume role authentication in addition to specified or implicit. If your CockroachDB cluster is deployed on Kubernetes, you can use workload identities with assume role authentication.

Get Started

CockroachDB and AI

Feature Overview

Data Resilience

Connect to an Application

Self-Hosted Deployments

Schema Design

Reads and Writes

Stream Data

Cross-Cluster Replication

Multi-Region Capabilities

Optimize Performance

Integrate

Cloud Storage Authentication

Amazon S3 specified

Amazon S3 implicit

Use implicit authentication with your own IAM role

Amazon S3 assume role

Use implicit authentication with your own service account

Set up Amazon S3 assume role

Amazon S3 role chaining

Amazon S3 workload identity

Step 1. Set up the identity role

Set up the identity role for CockroachDB Advanced clusters

Set up the identity role for CockroachDB Self-hosted clusters

Step 2. Trust the identity role

Step 3. Run the operation by assuming the role

Run the operation from a CockroachDB Advanced cluster

Run the operation from a CockroachDB Self-hosted cluster

See also

​Amazon S3 specified

​Amazon S3 implicit

​Use implicit authentication with your own IAM role

​Amazon S3 assume role

​Use implicit authentication with your own service account

​Set up Amazon S3 assume role

​Amazon S3 role chaining

​Amazon S3 workload identity

​Step 1. Set up the identity role

​Set up the identity role for CockroachDB Advanced clusters

​Set up the identity role for CockroachDB Self-hosted clusters

​Step 2. Trust the identity role

​Step 3. Run the operation by assuming the role

​Run the operation from a CockroachDB Advanced cluster

​Run the operation from a CockroachDB Self-hosted cluster

​See also

Amazon S3 specified

Amazon S3 implicit

Use implicit authentication with your own IAM role

Amazon S3 assume role

Use implicit authentication with your own service account

Set up Amazon S3 assume role

Amazon S3 role chaining

Amazon S3 workload identity

Step 1. Set up the identity role

Set up the identity role for CockroachDB Advanced clusters

Set up the identity role for CockroachDB Self-hosted clusters

Step 2. Trust the identity role

Step 3. Run the operation by assuming the role

Run the operation from a CockroachDB Advanced cluster

Run the operation from a CockroachDB Self-hosted cluster

See also