Skip to content

tigris: document shadow write-through preservation on update #2470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tripledoublev wants to merge 1 commit into
base: vincent/fix-tigris-shadow-delete-docs
Choose a base branch
from
Draft

tigris: document shadow write-through preservation on update #2470

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tigris/index.html.markerb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@

* **`flyctl`** — for bucket lifecycle (create, list, destroy) and shadow buckets.
* **[Tigris CLI](https://www.tigrisdata.com/docs/cli/)** — for day-to-day bucket and object management, plus Tigris-specific features like snapshots, forks, and shadow buckets.
* **S3-compatible SDKs** — for application code in any language.

Check warning on line 94 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'app' instead of 'application'.

Raw Output:
{"message":"Use 'app' instead of 'application'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":94,"column":34},"end":{"line":94,"column":45}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

All three target the same bucket and accept the credentials that `fly storage create` set on your Fly app.

Expand All @@ -99,7 +99,7 @@

The canonical Tigris S3 endpoint is `https://t3.storage.dev`. We recommend using it for new code.

`fly storage create` currently sets the `AWS_ENDPOINT_URL_S3` secret on your Fly app to `https://fly.storage.tigris.dev`. Both endpoints reach the same Tigris service and the same buckets, so existing code keeps working. To switch a Fly app to the canonical endpoint:

Check warning on line 102 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Avoid time-based words like 'currently' in product documentation.

Raw Output:
{"message":"Avoid time-based words like 'currently' in product documentation.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":102,"column":22},"end":{"line":102,"column":31}}},"severity":"INFO","code":{"value":"Google.Timeless","url":"https://developers.google.com/style/timeless-documentation"}}

```cmd
fly secrets set AWS_ENDPOINT_URL_S3=https://t3.storage.dev
Expand All @@ -123,7 +123,7 @@
tigris login
```

When prompted, choose **As a user (OAuth2 flow)**, then select **Fly.io** on the login page that opens in your browser. This connects the CLI to buckets created through your Fly account. Then run commands like:

Check warning on line 126 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'OAuth 2.0' instead of 'OAuth2'.

Raw Output:
{"message":"Use 'OAuth 2.0' instead of 'OAuth2'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":126,"column":36},"end":{"line":126,"column":42}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

```cmd
tigris ls
Expand Down Expand Up @@ -151,7 +151,7 @@
tigris login
```

When prompted, choose **As a user (OAuth2 flow)**, then select **Fly.io** on the login page that opens in your browser. The CLI then manages every Tigris bucket on your Fly account — `tigris buckets list` should show the same buckets as `flyctl storage list`.

Check warning on line 154 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'OAuth 2.0' instead of 'OAuth2'.

Raw Output:
{"message":"Use 'OAuth 2.0' instead of 'OAuth2'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":154,"column":36},"end":{"line":154,"column":42}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

Snapshots can't be enabled on existing buckets, so create a new bucket with the `--enable-snapshots` flag, then migrate or write data into it:

Expand All @@ -165,9 +165,9 @@

## Migrate from another S3-compatible provider with zero downtime

Tigris supports **zero-downtime, lazy migration** from any S3-compatible storage provider — AWS S3, Cloudflare R2, Google Cloud Storage, MinIO, Backblaze B2, and others. Instead of copying everything up front, Tigris fetches objects from your old bucket as your application requests them and asynchronously stores copies in Tigris for future reads. Only data that's actually used moves over, so migration latency and egress costs stay low.

Check warning on line 168 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'app' instead of 'application'.

Raw Output:
{"message":"Use 'app' instead of 'application'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":168,"column":265},"end":{"line":168,"column":276}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

Check warning on line 168 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Is 'Backblaze' a typo?

Raw Output:
{"message":"Is 'Backblaze' a typo?","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":168,"column":147},"end":{"line":168,"column":156}}},"severity":"INFO","code":{"value":"Fly.Spelling"}}

The migration setup uses a **shadow bucket**: your existing source bucket, attached to a Tigris bucket through `flyctl`. Once attached, your application points at Tigris, and Tigris transparently fills in objects from the shadow bucket as needed. See the [Tigris migration guide](https://www.tigrisdata.com/docs/migration/) for provider-specific setup.

Check warning on line 170 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'app' instead of 'application'.

Raw Output:
{"message":"Use 'app' instead of 'application'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":170,"column":142},"end":{"line":170,"column":153}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

### How shadow bucket migration works

Expand All @@ -177,9 +177,9 @@
* **Deletes:** Removed from both Tigris and the shadow bucket.
* **Listing:** With write-through enabled, `ListObjects` returns the full shadow bucket contents. Without write-through, only objects already migrated into Tigris are listed.

### Write-through mode: zero-downtime cutover

Check warning on line 180 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Is 'cutover' a typo?

Raw Output:
{"message":"Is 'cutover' a typo?","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":180,"column":39},"end":{"line":180,"column":46}}},"severity":"INFO","code":{"value":"Fly.Spelling"}}

Write-through mode (the `--shadow-write-through` flag below) is what makes the migration zero-downtime and reversible. Because every new write also lands in your old bucket, your previous provider stays fully current while you run on Tigris in production. You can soak in this dual-write state for days, weeks, or months, validate behavior, and roll back to your old provider at any time without data loss. When you're ready, disable write-through and decommission the old bucket.

Check warning on line 182 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'turn off' or 'off' instead of 'disable'.

Raw Output:
{"message":"Use 'turn off' or 'off' instead of 'disable'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":182,"column":427},"end":{"line":182,"column":434}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

### Create a new bucket with a shadow bucket

Expand All @@ -199,7 +199,7 @@
flyctl storage update mybucket --clear-shadow
```

`--shadow-write-through` enables zero-downtime cutover by replicating every new write back to the source bucket.
When setting or changing a shadow configuration, supply all five required attributes again. On update, omit `--shadow-write-through` to preserve its current setting; pass `--shadow-write-through` to enable it or `--shadow-write-through=false` to disable it. Write-through enables zero-downtime cutover by replicating every new write back to the source bucket.

Check warning on line 202 in tigris/index.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
Use 'turn off' or 'off' instead of 'disable'.

Raw Output:
{"message":"Use 'turn off' or 'off' instead of 'disable'.","location":{"path":"tigris/index.html.markerb","range":{"start":{"line":202,"column":247},"end":{"line":202,"column":254}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

## Pricing and Billing

Expand Down
Loading