-
Notifications
You must be signed in to change notification settings - Fork 477
Restructured DR backup section for clarity and thoroughness #20873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
935ee08
9ccf343
295db29
2b3fc7e
c26d7c0
3af9333
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| You will encounter an error if you run multiple [backup collections]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) to the same storage URI. Each collection's URI must be unique. | ||
| You will encounter an error if you run multiple [backup collections]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) to the same storage URI. Backup collections can contain multiple full and incremental backups, but each collection's URI must be unique. If you are using backup schedules, each schedule must have a unique URI. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -114,45 +114,111 @@ sql_txn_commit_count{tenant="demo"} 0 | |
|
|
||
| When connected to a virtual cluster from the DB Console, metrics which measure SQL and related activity show data scoped to the virtual cluster. All other metrics are collected system-wide and display the same data on all virtual clusters including the system virtual cluster. | ||
|
|
||
| ## Disaster recovery | ||
| ## Backup and restore | ||
|
|
||
| When cluster virtualization is enabled, [backup]({% link {{ page.version.version }}/backup.md %}) and [restore]({% link {{ page.version.version }}/restore.md %}) commands are scoped to the virtual cluster by default. | ||
| Cockroach Labs recommends that you regularly [back up]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#full-backups) your data. When using virtual clusters, perform backups on the _application virtual cluster (app VC)_. Only the app VC's data and settings are included in these backups, and data and settings for other virtual clusters or for the _system virtual cluster (system VC)_ are omitted. You can also back up your system VC to preserve metadata such as users and cluster settings, but this data is usually not critical. | ||
|
|
||
| ### Back up a virtual cluster | ||
peachdawnleach marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ### Create a backup schedule for your app VC | ||
|
|
||
| To back up a virtual cluster: | ||
| Cockroach Labs recommends using [backup schedules]({% link {{ page.version.version }}/create-schedule-for-backup.md %}) to automate full and incremental backups of your data. | ||
|
|
||
| 1. [Connect to the virtual cluster](#connect-to-a-virtual-cluster) you want to back up as a user with the `admin` role on the virtual cluster. | ||
| 1. [Back up the cluster]({% link {{ page.version.version }}/backup.md %}). Only the virtual cluster's data and settings are included in the backup, and data and settings for other virtual clusters or for the system virtual cluster is omitted. | ||
| Use the following process to create a schedule for a cluster-level backup of your app VC. In this example, the schedule takes an incremental backup every day at midnight, and a full backup weekly. Consult [CREATE SCHEDULE FOR BACKUP]({% link {{ page.version.version }}/create-schedule-for-backup.md %}#parameters) for a full list of backup schedule options. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can we remove 'cluster-level' and just say backup? and then add 'This happens to be a cluster-level backup' in the '2. Create a backup schedule:' line |
||
|
|
||
| For details about restoring a backup of a virtual cluster, refer to [Restore a virtual cluster](#restore-a-virtual-cluster). | ||
| 1. [Connect](#connect-to-a-virtual-cluster) to the app VC as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the app VC. In this example the user has the `BACKUP` privilege. | ||
|
|
||
| ### Back up the entire cluster | ||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ shell | ||
| cockroach sql --url \ | ||
| "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ | ||
| --certs-dir "certs" | ||
| ~~~ | ||
|
|
||
| To back up the entire CockroachDB cluster, including all virtual clusters and the system virtual cluster: | ||
| 1. [Create a backup schedule]({% link {{ page.version.version }}/create-schedule-for-backup.md %}#create-a-scheduled-backup-for-a-cluster): | ||
|
|
||
| 1. [Connect to the system virtual cluster](#connect-to-the-system-virtual-cluster) as a user with the `admin` role on the system virtual cluster. | ||
| 1. [Back up the cluster]({% link {{ page.version.version }}/backup.md %}), and include the `INCLUDE_ALL_SECONDARY_TENANTS` flag in the `BACKUP` command. All virtual clusters and the system virtual cluster are included in the backup. | ||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ sql | ||
| CREATE SCHEDULE schedule_label | ||
| FOR BACKUP INTO 's3://test/backups/schedule_test?AWS_ACCESS_KEY_ID=x&AWS_SECRET_ACCESS_KEY=x' | ||
| RECURRING '@daily'; | ||
| ~~~ | ||
|
|
||
| ### Restore a virtual cluster | ||
| ~~~ | ||
| schedule_id | name | status | first_run | schedule | backup_stmt | ||
| ---------------------+----------------+------------------------------------------------+----------------------------------+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------- | ||
| 588796190000218113 | schedule_label | PAUSED: Waiting for initial backup to complete | NULL | @daily | BACKUP INTO LATEST IN 's3://test/schedule-test?AWS_ACCESS_KEY_ID=x&AWS_SECRET_ACCESS_KEY=x' WITH revision_history, detached | ||
| 588796190012702721 | schedule_label | ACTIVE | 2020-09-10 16:52:17.280821+00:00 | @weekly | BACKUP INTO 's3://test/schedule-test?AWS_ACCESS_KEY_ID=x&AWS_SECRET_ACCESS_KEY=x' WITH revision_history, detached | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: revision history is in the result here. |
||
| (2 rows) | ||
| ~~~ | ||
|
|
||
| {% include {{ page.version.version }}/backups/backup-storage-collision.md %} | ||
msbutler marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| For information on scheduling backups at different levels or with other options, consult [CREATE SCHEDULE FOR BACKUP]({% link {{ page.version.version }}/create-schedule-for-backup.md %}). | ||
|
|
||
| ### Take a one-off backup of your app VC | ||
|
|
||
| Use the following process to take a one-off full backup of your app VC. Even if you are using backup schedules, this can be useful if you want to create a separate copy. | ||
|
|
||
| 1. [Connect](#connect-to-a-virtual-cluster) to the app VC as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the app VC. In this example the user has the `BACKUP` privilege. | ||
|
|
||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ shell | ||
| cockroach sql --url \ | ||
| "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ | ||
| --certs-dir "certs" | ||
| ~~~ | ||
|
|
||
| 1. [Perform a full backup]({% link {{ page.version.version }}/backup.md %}#back-up-a-cluster): | ||
|
|
||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ sql | ||
| BACKUP INTO 'external://backup_s3/app' AS OF SYSTEM TIME '-10s'; | ||
| ~~~ | ||
|
|
||
| You can restore a backup of a virtual cluster to: | ||
| You can also take one-off backups of a single [database]({% link {{ page.version.version }}/backup.md %}#back-up-a-database) or [table]({% link {{ page.version.version }}/backup.md %}#back-up-a-table-or-view) on your app VC. For more information on backup options, consult [BACKUP]({% link {{ page.version.version }}/backup.md %}). | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove "single" and include "database(s)" |
||
|
|
||
| ### Back up your system VC | ||
|
|
||
| You can also back up your system VC to preserve metadata such as users and cluster settings. Use the following process to back up your system VC. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. s/preserve metadata such as users and cluster settings/ preserve metadata stored in the system VC/r |
||
|
|
||
| 1. [Connect](#connect-to-the-system-virtual-cluster) to the system VC as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the system VC. In this example the user has the `BACKUP` privilege. | ||
|
|
||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ shell | ||
| cockroach sql --url \ | ||
| "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \ | ||
| --certs-dir "certs" | ||
| ~~~ | ||
|
|
||
| 1. [Perform a full backup]({% link {{ page.version.version }}/backup.md %}#back-up-a-cluster): | ||
|
|
||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ sql | ||
| BACKUP INTO 'external://backup_s3/system' AS OF SYSTEM TIME '-10s'; | ||
| ~~~ | ||
|
|
||
| {% include {{ page.version.version }}/backups/backup-storage-collision.md %} | ||
|
|
||
| ### Restore a virtual cluster | ||
|
|
||
| - The original virtual cluster on the original CockroachDB cluster. | ||
| - A different virtual cluster on the original CockroachDB cluster. | ||
| - A different virtual cluster on a different CockroachDB cluster with cluster virtualization enabled. | ||
| If needed, you can restore a full backup to a new app VC with no user-created databases or tables. To restore your app VC from the latest full backup: | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove "full". Also, the line "no user-created databases or tables" only applies to cluster level restores, so you may want to say this requirement is specific to the example below. |
||
|
|
||
| To restore only a virtual cluster: | ||
| 1. [Connect to the destination app VC](#connect-to-a-virtual-cluster) as a user with [supported privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) on the app VC. In this example the user has the `RESTORE` privilege. | ||
|
|
||
| 1. [Connect to the destination virtual cluster](#connect-to-a-virtual-cluster) as a user with the `admin` role on the virtual cluster. | ||
| 1. [Restore the cluster]({% link {{ page.version.version }}/restore.md %}). Only the virtual cluster's data and settings are restored. | ||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ shell | ||
| cockroach sql --url \ | ||
| "postgresql://root@{primary node IP or hostname}:26257?options=-ccluster={app_virtual_cluster_name}&sslmode=verify-full" \ | ||
| --certs-dir "certs" | ||
| ~~~ | ||
|
|
||
| ### Restore the entire cluster | ||
peachdawnleach marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| 1. [Restore the cluster]({% link {{ page.version.version }}/restore.md %}): | ||
|
|
||
| To restore the entire CockroachDB cluster, including all virtual clusters and the system virtual cluster: | ||
| {% include_cached copy-clipboard.html %} | ||
| ~~~ sql | ||
| RESTORE FROM LATEST IN 's3://bucket/path?AUTH=implicit'; | ||
| ~~~ | ||
|
|
||
| 1. [Connect to the destination system virtual cluster](#connect-to-the-system-virtual-cluster) as a user with the `admin` role on the system virtual cluster. | ||
| 1. [Restore the cluster]({% link {{ page.version.version }}/restore.md %}) from a backup that included the the `INCLUDE_ALL_SECONDARY_VIRTUAL_CLUSTERS` flag. All virtual clusters and the system virtual cluster are restored. | ||
| You can also restore a [database-level]({% link {{ page.version.version }}/restore.md %}#restore-a-database) or [table-level]({% link {{ page.version.version }}/restore.md %}#restore-a-table) backup to the same app VC on which you created it. For more information on restore options, consult [RESTORE]({% link {{ page.version.version }}/restore.md %}). | ||
|
|
||
msbutler marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## Configure cluster settings | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: why "data and settings"? could we just say "data"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, a user may misinterpret the line "You can also back up your system VC to preserve metadata such as users and cluster settings" to mean that their app vc users are stored in the system tenant. this is not true.
I would rephrase "You can also back up your system VC to preserve metadata related to your system vc"