Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1 +1 @@
You will encounter an error if you run multiple [backup collections]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) to the same storage URI. Each collection's URI must be unique.
You will encounter an error if you run multiple [backup collections]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) to the same storage URI. Backup collections can contain multiple full and incremental backups, but each collection's URI must be unique. If you are using backup schedules, each schedule must have a unique URI.
6 changes: 4 additions & 2 deletions src/current/v25.4/cluster-virtualization-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,17 @@ To upgrade to a new major version when cluster virtualization is enabled, you mu

For details, refer to [Work with virtual clusters]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#upgrade-a-cluster).

### Disaster recovery
### Backup and restore

Having a [disaster recovery]({% link {{ page.version.version }}/disaster-recovery-overview.md %}) plan is important to recover from unforseen incidents and minimize downtime and data loss.

When cluster virtualization is enabled, [backup]({% link {{ page.version.version }}/backup.md %}) and [restore]({% link {{ page.version.version }}/restore.md %}) commands are scoped to a virtual cluster. This means that:

- A backup taken from a virtual cluster contains all data for that virtual cluster, but does not contain modifications made via the system virtual cluster such as system-level cluster settings.
- If your deployment contains system-level customizations, you can take a separate backup of the system virtual cluster to capture them.
- A backup of a virtual cluster can be restored as a virtual cluster in any CockroachDB cluster with cluster virtualization enabled.

For more details about backing up and restoring a cluster with cluster virtualization enabled, refer to [Work with virtual clusters]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#disaster-recovery).
For more details about backing up and restoring a cluster with cluster virtualization enabled, refer to [Work with virtual clusters]({% link {{ page.version.version }}/work-with-virtual-clusters.md %}#backup-and-restore).

{{site.data.alerts.callout_success}}
For details about configuring and using PCR for disaster recovery, refer to [Physical Cluster Replication Overview]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}).
Expand Down
112 changes: 89 additions & 23 deletions src/current/v25.4/work-with-virtual-clusters.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,45 +114,111 @@ sql_txn_commit_count{tenant="demo"} 0

When connected to a virtual cluster from the DB Console, metrics which measure SQL and related activity show data scoped to the virtual cluster. All other metrics are collected system-wide and display the same data on all virtual clusters including the system virtual cluster.

## Disaster recovery
## Backup and restore

When cluster virtualization is enabled, [backup]({% link {{ page.version.version }}/backup.md %}) and [restore]({% link {{ page.version.version }}/restore.md %}) commands are scoped to the virtual cluster by default.
Cockroach Labs recommends that you regularly [back up]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#full-backups) your data. When using virtual clusters, perform backups on the _application virtual cluster (app VC)_. Only the app VC's data and settings are included in these backups, and data and settings for other virtual clusters or for the _system virtual cluster (system VC)_ are omitted. You can also back up your system VC to preserve metadata such as users and cluster settings, but this data is usually not critical.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why "data and settings"? could we just say "data"?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, a user may misinterpret the line "You can also back up your system VC to preserve metadata such as users and cluster settings" to mean that their app vc users are stored in the system tenant. this is not true.

I would rephrase "You can also back up your system VC to preserve metadata related to your system vc"


### Back up a virtual cluster
### Create a backup schedule for your app VC

To back up a virtual cluster:
Cockroach Labs recommends using [backup schedules]({% link {{ page.version.version }}/create-schedule-for-backup.md %}) to automate full and incremental backups of your data.

1. [Connect to the virtual cluster](#connect-to-a-virtual-cluster) you want to back up as a user with the `admin` role on the virtual cluster.
1. [Back up the cluster]({% link {{ page.version.version }}/backup.md %}). Only the virtual cluster's data and settings are included in the backup, and data and settings for other virtual clusters or for the system virtual cluster is omitted.
Use the following process to create a schedule for a cluster-level backup of your app VC. In this example, the schedule takes an incremental backup every day at midnight, and a full backup weekly. Consult [CREATE SCHEDULE FOR BACKUP]({% link {{ page.version.version }}/create-schedule-for-backup.md %}#parameters) for a full list of backup schedule options.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove 'cluster-level' and just say backup? and then add 'This happens to be a cluster-level backup' in the '2. Create a backup schedule:' line


For details about restoring a backup of a virtual cluster, refer to [Restore a virtual cluster](#restore-a-virtual-cluster).
1. [Connect](#connect-to-a-virtual-cluster) to the app VC as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the app VC. In this example the user has the `BACKUP` privilege.

### Back up the entire cluster
{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --url \
"postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \
--certs-dir "certs"
~~~

To back up the entire CockroachDB cluster, including all virtual clusters and the system virtual cluster:
1. [Create a backup schedule]({% link {{ page.version.version }}/create-schedule-for-backup.md %}#create-a-scheduled-backup-for-a-cluster):

1. [Connect to the system virtual cluster](#connect-to-the-system-virtual-cluster) as a user with the `admin` role on the system virtual cluster.
1. [Back up the cluster]({% link {{ page.version.version }}/backup.md %}), and include the `INCLUDE_ALL_SECONDARY_TENANTS` flag in the `BACKUP` command. All virtual clusters and the system virtual cluster are included in the backup.
{% include_cached copy-clipboard.html %}
~~~ sql
CREATE SCHEDULE schedule_label
FOR BACKUP INTO 's3://test/backups/schedule_test?AWS_ACCESS_KEY_ID=x&AWS_SECRET_ACCESS_KEY=x'
RECURRING '@daily';
~~~

### Restore a virtual cluster
~~~
schedule_id | name | status | first_run | schedule | backup_stmt
---------------------+----------------+------------------------------------------------+----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------
588796190000218113 | schedule_label | PAUSED: Waiting for initial backup to complete | NULL | @daily | BACKUP INTO LATEST IN 's3://test/schedule-test?AWS_ACCESS_KEY_ID=x&AWS_SECRET_ACCESS_KEY=x' WITH revision_history, detached
588796190012702721 | schedule_label | ACTIVE | 2020-09-10 16:52:17.280821+00:00 | @weekly | BACKUP INTO 's3://test/schedule-test?AWS_ACCESS_KEY_ID=x&AWS_SECRET_ACCESS_KEY=x' WITH revision_history, detached

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: revision history is in the result here.

(2 rows)
~~~

{% include {{ page.version.version }}/backups/backup-storage-collision.md %}

For information on scheduling backups at different levels or with other options, consult [CREATE SCHEDULE FOR BACKUP]({% link {{ page.version.version }}/create-schedule-for-backup.md %}).

### Take a one-off backup of your app VC

Use the following process to take a one-off full backup of your app VC. Even if you are using backup schedules, this can be useful if you want to create a separate copy.

1. [Connect](#connect-to-a-virtual-cluster) to the app VC as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the app VC. In this example the user has the `BACKUP` privilege.

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --url \
"postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \
--certs-dir "certs"
~~~

1. [Perform a full backup]({% link {{ page.version.version }}/backup.md %}#back-up-a-cluster):

{% include_cached copy-clipboard.html %}
~~~ sql
BACKUP INTO 'external://backup_s3/app' AS OF SYSTEM TIME '-10s';
~~~

You can restore a backup of a virtual cluster to:
You can also take one-off backups of a single [database]({% link {{ page.version.version }}/backup.md %}#back-up-a-database) or [table]({% link {{ page.version.version }}/backup.md %}#back-up-a-table-or-view) on your app VC. For more information on backup options, consult [BACKUP]({% link {{ page.version.version }}/backup.md %}).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove "single" and include "database(s)"


### Back up your system VC

You can also back up your system VC to preserve metadata such as users and cluster settings. Use the following process to back up your system VC.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/preserve metadata such as users and cluster settings/ preserve metadata stored in the system VC/r


1. [Connect](#connect-to-the-system-virtual-cluster) to the system VC as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the system VC. In this example the user has the `BACKUP` privilege.

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --url \
"postgresql://root@{primary node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \
--certs-dir "certs"
~~~

1. [Perform a full backup]({% link {{ page.version.version }}/backup.md %}#back-up-a-cluster):

{% include_cached copy-clipboard.html %}
~~~ sql
BACKUP INTO 'external://backup_s3/system' AS OF SYSTEM TIME '-10s';
~~~

{% include {{ page.version.version }}/backups/backup-storage-collision.md %}

### Restore a virtual cluster

- The original virtual cluster on the original CockroachDB cluster.
- A different virtual cluster on the original CockroachDB cluster.
- A different virtual cluster on a different CockroachDB cluster with cluster virtualization enabled.
If needed, you can restore a full backup to a new app VC with no user-created databases or tables. To restore your app VC from the latest full backup:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove "full". Also, the line "no user-created databases or tables" only applies to cluster level restores, so you may want to say this requirement is specific to the example below.


To restore only a virtual cluster:
1. [Connect to the destination app VC](#connect-to-a-virtual-cluster) as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the app VC. In this example the user has the `RESTORE` privilege.

1. [Connect to the destination virtual cluster](#connect-to-a-virtual-cluster) as a user with the `admin` role on the virtual cluster.
1. [Restore the cluster]({% link {{ page.version.version }}/restore.md %}). Only the virtual cluster's data and settings are restored.
{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --url \
"postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \
--certs-dir "certs"
~~~

### Restore the entire cluster
1. [Restore the cluster]({% link {{ page.version.version }}/restore.md %}):

To restore the entire CockroachDB cluster, including all virtual clusters and the system virtual cluster:
{% include_cached copy-clipboard.html %}
~~~ sql
RESTORE FROM LATEST IN 's3://bucket/path?AUTH=implicit';
~~~

1. [Connect to the destination system virtual cluster](#connect-to-the-system-virtual-cluster) as a user with the `admin` role on the system virtual cluster.
1. [Restore the cluster]({% link {{ page.version.version }}/restore.md %}) from a backup that included the the `INCLUDE_ALL_SECONDARY_VIRTUAL_CLUSTERS` flag. All virtual clusters and the system virtual cluster are restored.
You can also restore a [database-level]({% link {{ page.version.version }}/restore.md %}#restore-a-database) or [table-level]({% link {{ page.version.version }}/restore.md %}#restore-a-table) backup to the same app VC on which you created it. For more information on restore options, consult [RESTORE]({% link {{ page.version.version }}/restore.md %}).

## Configure cluster settings

Expand Down
Loading