From 6f8d4cad226a9f1ee0f0a4307506369c941bf338 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 09:04:12 -0500 Subject: [PATCH 01/11] Add rollback (w db) steps to upgrade page Signed-off-by: Rachael Graham --- .../docs/kagent/operations/upgrade/page.mdx | 69 ++++++++++++++++++- 1 file changed, 68 insertions(+), 1 deletion(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index 06c8c3a2..3e56e1a2 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -27,7 +27,7 @@ Follow these steps to upgrade kagent to the latest version and keep your cluster - Any custom settings - PostgreSQL database -4. **v0.9.0 and later**: You must be running at least v0.8.0 before upgrading to v0.9.0. Check the [release notes](/docs/kagent/resources/release-notes#v09) for 0.9-specific upgrades related to database migrations and RBAC scope. +4. **v0.9.0 and later**: You must be running at least v0.8.0 before upgrading to v0.9.0. Check the [release notes](/docs/kagent/resources/release-notes#v09) for 0.9-specific upgrades related to database migrations and RBAC scope. ## Upgrade kagent @@ -81,3 +81,70 @@ After upgrading, verify that kagent is running. ```bash kubectl get pods -n kagent ``` + +## Roll back kagent + +If you need to roll back to a previous version after a successful upgrade, use the following steps. + +### Rollback compatibility window + +kagent guarantees `n-1` application compatibility with the database: an application at version `n-1` can start against a database schema applied by version `n`. This means that you can roll back the application without manually resetting the database first, as long as you stay within the supported window: + +- **Supported**: Rolling back one minor version (for example, `v0.10.x` → `v0.9.x`). +- **Requires DB reset first**: Rolling back more than one minor version. + +When the controller detects that the database schema is ahead of its own migration files, it starts in compatibility mode and skips migration application. The database schema is left unchanged. + +### Steps to roll back + +1. Save the version you want to roll back to in an environment variable. + ```bash + export ROLLBACK_VERSION= + ``` + +2. Roll back the kagent chart. + ```bash + helm upgrade kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \ + --namespace kagent \ + -f values.yaml \ + --version $ROLLBACK_VERSION + ``` + +3. Roll back the kagent-crds chart. + ```bash + helm upgrade kagent-crds oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \ + --namespace kagent \ + --version $ROLLBACK_VERSION + ``` + +4. Verify that the rollback succeeded. + ```bash + kubectl get pods -n kagent + ``` + +### Reset the database before rolling back further + +If you need to roll back more than one minor version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the target version's migration files. + +1. Check out the migration files for the target version. + ```bash + git clone https://github.com/kagent-dev/kagent.git + cd kagent && git checkout v$ROLLBACK_VERSION + ``` + +2. Run `golang-migrate` down to the target schema version for each track. You can check the `go/core/pkg/migrations/core/` directory in the target release to find the highest migration version number. + ```bash + # Core track + migrate -path go/core/pkg/migrations/core \ + -database "$DATABASE_URL" \ + goto + + # Vector track (if vector features are enabled) + migrate -path go/core/pkg/migrations/vector \ + -database "$DATABASE_URL" \ + goto + ``` + +3. After the database is at the correct schema version, roll back the kagent application using the steps above. + +> **Note**: Database migration version numbers do not correspond directly to kagent release versions. Always check the migration files in the target release to identify the correct target migration version. From 588997c32c6aa625abd5a0e78d59c5980434cd6e Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 09:52:50 -0500 Subject: [PATCH 02/11] Apply suggestions from code review Co-authored-by: Art Signed-off-by: Rachael Graham --- src/app/docs/kagent/operations/upgrade/page.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index 3e56e1a2..d5338ea6 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -97,7 +97,7 @@ When the controller detects that the database schema is ahead of its own migrati ### Steps to roll back -1. Save the version you want to roll back to in an environment variable. +1. Save the kagent version you want to roll back to in an environment variable. ```bash export ROLLBACK_VERSION= ``` @@ -124,9 +124,9 @@ When the controller detects that the database schema is ahead of its own migrati ### Reset the database before rolling back further -If you need to roll back more than one minor version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the target version's migration files. +If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the target version's migration files. Note that these migration files might have a different version number than the kagent version. -1. Check out the migration files for the target version. +1. Check out the migration files for the target kagent version. ```bash git clone https://github.com/kagent-dev/kagent.git cd kagent && git checkout v$ROLLBACK_VERSION @@ -145,6 +145,6 @@ If you need to roll back more than one minor version, you must first reset the d goto ``` -3. After the database is at the correct schema version, roll back the kagent application using the steps above. +3. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back). > **Note**: Database migration version numbers do not correspond directly to kagent release versions. Always check the migration files in the target release to identify the correct target migration version. From 8f1c6d021aae276f9b396832e9f7a3db22794f6b Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 10:27:08 -0500 Subject: [PATCH 03/11] Apply suggestions from code review Co-authored-by: Art Signed-off-by: Rachael Graham --- src/app/docs/kagent/operations/upgrade/page.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index d5338ea6..d35d524f 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -88,7 +88,7 @@ If you need to roll back to a previous version after a successful upgrade, use t ### Rollback compatibility window -kagent guarantees `n-1` application compatibility with the database: an application at version `n-1` can start against a database schema applied by version `n`. This means that you can roll back the application without manually resetting the database first, as long as you stay within the supported window: +As of v0.9, kagent guarantees `n-1` application compatibility with the database: an application at version `n-1` can start against a database schema applied by version `n`. This means that you can roll back the application without manually resetting the database first, as long as you stay within the supported window: - **Supported**: Rolling back one minor version (for example, `v0.10.x` → `v0.9.x`). - **Requires DB reset first**: Rolling back more than one minor version. From 9f8a0b2ac5a3a0f0b31f767fac83d8bdb3b24499 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 10:32:04 -0500 Subject: [PATCH 04/11] Update page.mdx Signed-off-by: Rachael Graham --- .../docs/kagent/operations/upgrade/page.mdx | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index d35d524f..247c6c06 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -124,27 +124,29 @@ When the controller detects that the database schema is ahead of its own migrati ### Reset the database before rolling back further -If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the target version's migration files. Note that these migration files might have a different version number than the kagent version. +If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the target version's migration files. -1. Check out the migration files for the target kagent version. +1. Install [`golang-migrate`](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate). + +2. Stop the kagent controller. ```bash - git clone https://github.com/kagent-dev/kagent.git - cd kagent && git checkout v$ROLLBACK_VERSION + kubectl -n kagent scale deploy/kagent-controller --replicas=0 ``` -2. Run `golang-migrate` down to the target schema version for each track. You can check the `go/core/pkg/migrations/core/` directory in the target release to find the highest migration version number. +3. Reset the core track to the target migration version. The `github://` source lets you reference the migration files directly from the kagent release tag without a local checkout. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of the target release. ```bash - # Core track - migrate -path go/core/pkg/migrations/core \ - -database "$DATABASE_URL" \ + migrate \ + -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$ROLLBACK_VERSION" \ + -database "$DATABASE_URL?x-migrations-table=schema_migrations" \ goto + ``` - # Vector track (if vector features are enabled) - migrate -path go/core/pkg/migrations/vector \ - -database "$DATABASE_URL" \ +4. If vector features are enabled, reset the vector track as well. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/vector/` directory of the target release. + ```bash + migrate \ + -source "github://kagent-dev/kagent/go/core/pkg/migrations/vector#v$ROLLBACK_VERSION" \ + -database "$DATABASE_URL?x-migrations-table=vector_schema_migrations" \ goto ``` -3. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back). - -> **Note**: Database migration version numbers do not correspond directly to kagent release versions. Always check the migration files in the target release to identify the correct target migration version. +5. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back). From 7ef5348b7539a4854b0c923f8c421ec4c9c36c69 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 10:38:05 -0500 Subject: [PATCH 05/11] Update page.mdx Signed-off-by: Rachael Graham --- src/app/docs/kagent/operations/upgrade/page.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index 247c6c06..efe30758 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -133,7 +133,7 @@ If you need to roll back more than one minor kagent version, you must first rese kubectl -n kagent scale deploy/kagent-controller --replicas=0 ``` -3. Reset the core track to the target migration version. The `github://` source lets you reference the migration files directly from the kagent release tag without a local checkout. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of the target release. +3. Reset the core track to the target migration version. The `github://` source lets you reference the migration files directly from the kagent release tag without a local checkout. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of the target release. For example, if the highest file is `000005_a2a_protocol_version.up.sql`, use `5`. ```bash migrate \ -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$ROLLBACK_VERSION" \ From e848d981a57029563eabf388884d5bfc6d8da90e Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 10:57:39 -0500 Subject: [PATCH 06/11] Update page.mdx Signed-off-by: Rachael Graham --- .../docs/kagent/operations/upgrade/page.mdx | 30 ++++++++++++------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index efe30758..9dc2f4b4 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -124,29 +124,39 @@ When the controller detects that the database schema is ahead of its own migrati ### Reset the database before rolling back further -If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the target version's migration files. +If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the migration files. -1. Install [`golang-migrate`](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate). +The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to. -2. Stop the kagent controller. +For example, if you are rolling back from `v0.10.2` (migrations up to `10`) to `v0.9.4` (migrations up to `8`), you use the `v0.10.2` source and `goto 8`. + +1. Save your current kagent version and the kagent version that you want to roll back to. + ```bash + export CURRENT_VERSION= + export ROLLBACK_VERSION= + ``` + +2. Install [`golang-migrate`](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate). + +3. Stop the kagent controller. ```bash kubectl -n kagent scale deploy/kagent-controller --replicas=0 ``` -3. Reset the core track to the target migration version. The `github://` source lets you reference the migration files directly from the kagent release tag without a local checkout. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of the target release. For example, if the highest file is `000005_a2a_protocol_version.up.sql`, use `5`. +4. Reset the core track. Use the `CURRENT_VERSION` source and `goto` the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of `ROLLBACK_VERSION`. The `github://` source references the migration files directly from the release tag without a local checkout. ```bash migrate \ - -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$ROLLBACK_VERSION" \ + -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$CURRENT_VERSION" \ -database "$DATABASE_URL?x-migrations-table=schema_migrations" \ - goto + goto ``` -4. If vector features are enabled, reset the vector track as well. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/vector/` directory of the target release. +5. If vector features are enabled, reset the vector track as well. ```bash migrate \ - -source "github://kagent-dev/kagent/go/core/pkg/migrations/vector#v$ROLLBACK_VERSION" \ + -source "github://kagent-dev/kagent/go/core/pkg/migrations/vector#v$CURRENT_VERSION" \ -database "$DATABASE_URL?x-migrations-table=vector_schema_migrations" \ - goto + goto ``` -5. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back). +6. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back). From 5de337d42f0ffea15f0f92ad072c746106a61279 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 11:06:35 -0500 Subject: [PATCH 07/11] Update page.mdx Signed-off-by: Rachael Graham --- src/app/docs/kagent/operations/upgrade/page.mdx | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index 9dc2f4b4..e2352daf 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -130,10 +130,9 @@ The source must be the current (newer) version that you are rolling back from, b For example, if you are rolling back from `v0.10.2` (migrations up to `10`) to `v0.9.4` (migrations up to `8`), you use the `v0.10.2` source and `goto 8`. -1. Save your current kagent version and the kagent version that you want to roll back to. +1. Save your current kagent version in an environment variable. ```bash export CURRENT_VERSION= - export ROLLBACK_VERSION= ``` 2. Install [`golang-migrate`](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate). @@ -143,19 +142,19 @@ For example, if you are rolling back from `v0.10.2` (migrations up to `10`) to ` kubectl -n kagent scale deploy/kagent-controller --replicas=0 ``` -4. Reset the core track. Use the `CURRENT_VERSION` source and `goto` the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of `ROLLBACK_VERSION`. The `github://` source references the migration files directly from the release tag without a local checkout. +4. Reset the core track. The `github://` source references the migration files directly from the release tag without a local checkout. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of your target (rollback) release, which you can browse at `https://github.com/kagent-dev/kagent/tree/v/go/core/pkg/migrations/core/`. For the database connection string, see [Database configuration](/docs/kagent/operations/operational-considerations#database-configuration). ```bash migrate \ -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$CURRENT_VERSION" \ - -database "$DATABASE_URL?x-migrations-table=schema_migrations" \ + -database "postgres://:@:5432/?sslmode=require&x-migrations-table=schema_migrations" \ goto ``` -5. If vector features are enabled, reset the vector track as well. +5. If vector features are enabled, reset the vector track as well. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/vector/` directory of your target release. ```bash migrate \ -source "github://kagent-dev/kagent/go/core/pkg/migrations/vector#v$CURRENT_VERSION" \ - -database "$DATABASE_URL?x-migrations-table=vector_schema_migrations" \ + -database "postgres://:@:5432/?sslmode=require&x-migrations-table=vector_schema_migrations" \ goto ``` From fc5dd025fec5b36a8eedf381b0da71c081d58eee Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 11:36:44 -0500 Subject: [PATCH 08/11] use real examples + env vars Signed-off-by: Rachael Graham --- .../docs/kagent/operations/upgrade/page.mdx | 39 +++++++++++++------ 1 file changed, 27 insertions(+), 12 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index e2352daf..4d9ba000 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -128,11 +128,12 @@ If you need to roll back more than one minor kagent version, you must first rese The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to. -For example, if you are rolling back from `v0.10.2` (migrations up to `10`) to `v0.9.4` (migrations up to `8`), you use the `v0.10.2` source and `goto 8`. +For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`. -1. Save your current kagent version in an environment variable. +1. Save your current kagent version and the kagent version you want to roll back to in environment variables. ```bash export CURRENT_VERSION= + export ROLLBACK_VERSION= ``` 2. Install [`golang-migrate`](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate). @@ -142,20 +143,34 @@ For example, if you are rolling back from `v0.10.2` (migrations up to `10`) to ` kubectl -n kagent scale deploy/kagent-controller --replicas=0 ``` -4. Reset the core track. The `github://` source references the migration files directly from the release tag without a local checkout. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/core/` directory of your target (rollback) release, which you can browse at `https://github.com/kagent-dev/kagent/tree/v/go/core/pkg/migrations/core/`. For the database connection string, see [Database configuration](/docs/kagent/operations/operational-considerations#database-configuration). +4. Open the core migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable. ```bash - migrate \ - -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$CURRENT_VERSION" \ - -database "postgres://:@:5432/?sslmode=require&x-migrations-table=schema_migrations" \ - goto + open "https://github.com/kagent-dev/kagent/tree/v${ROLLBACK_VERSION}/go/core/pkg/migrations/core/" + ``` + ```bash + export ROLLBACK_MIGRATION_VERSION= ``` -5. If vector features are enabled, reset the vector track as well. Replace `` with the highest migration sequence number in the `go/core/pkg/migrations/vector/` directory of your target release. +5. Reset the core track. The `github://` source references the migration files directly from the release tag without a local checkout. For the database connection string, see [Database configuration](/docs/kagent/operations/operational-considerations#database-configuration). ```bash migrate \ - -source "github://kagent-dev/kagent/go/core/pkg/migrations/vector#v$CURRENT_VERSION" \ - -database "postgres://:@:5432/?sslmode=require&x-migrations-table=vector_schema_migrations" \ - goto + -source "github://kagent-dev/kagent/go/core/pkg/migrations/core#v$CURRENT_VERSION" \ + -database "postgres://:@:5432/?sslmode=require&x-migrations-table=schema_migrations" \ + goto $ROLLBACK_MIGRATION_VERSION ``` -6. After the database is at the correct schema version, roll back the kagent application by following the steps in the [previous section](#steps-to-roll-back). +6. If vector features are enabled, reset the vector track as well. + 1. Open the vector migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable. + ```bash + open "https://github.com/kagent-dev/kagent/tree/v${ROLLBACK_VERSION}/go/core/pkg/migrations/vector/" + export ROLLBACK_VECTOR_MIGRATION_VERSION= + ``` + 2. Reset the vector track. + ```bash + migrate \ + -source "github://kagent-dev/kagent/go/core/pkg/migrations/vector#v$CURRENT_VERSION" \ + -database "postgres://:@:5432/?sslmode=require&x-migrations-table=vector_schema_migrations" \ + goto $ROLLBACK_VECTOR_MIGRATION_VERSION + ``` + +7. After the database is at the correct schema version, follow the steps to [roll back the kagent application](#steps-to-roll-back). From dd4c09906122436dd53d8d7266c81570d28f2a81 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 12:09:48 -0500 Subject: [PATCH 09/11] Update page.mdx Signed-off-by: Rachael Graham --- src/app/docs/kagent/operations/upgrade/page.mdx | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index 4d9ba000..89bfa36b 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -90,8 +90,8 @@ If you need to roll back to a previous version after a successful upgrade, use t As of v0.9, kagent guarantees `n-1` application compatibility with the database: an application at version `n-1` can start against a database schema applied by version `n`. This means that you can roll back the application without manually resetting the database first, as long as you stay within the supported window: -- **Supported**: Rolling back one minor version (for example, `v0.10.x` → `v0.9.x`). -- **Requires DB reset first**: Rolling back more than one minor version. +- **Supported**: Roll back one minor version (for example, `v0.10.x` → `v0.9.x`). +- **Requires DB reset, one minor at a time**: Roll back more than one minor version. Be sure to roll back only one minor version at a time, as skipping a minor version can result in data loss. When the controller detects that the database schema is ahead of its own migration files, it starts in compatibility mode and skips migration application. The database schema is left unchanged. @@ -130,6 +130,8 @@ The source must be the current (newer) version that you are rolling back from, b For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`. +> **Note**: If you need to roll back more than one minor kagent version, be sure to roll back one minor version at a time. Kagent guarantees backwards compatibility of only one minor version, so skipping minor versions can result in data loss. + 1. Save your current kagent version and the kagent version you want to roll back to in environment variables. ```bash export CURRENT_VERSION= From b6d572ce3cbb4693685c75718ac066d64958aac0 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 23 Jun 2026 12:23:22 -0500 Subject: [PATCH 10/11] Add snapshot as an option Signed-off-by: Rachael Graham --- .../docs/kagent/operations/upgrade/page.mdx | 31 ++++++++++++++++--- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index 89bfa36b..b0b75cef 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -25,7 +25,13 @@ Follow these steps to upgrade kagent to the latest version and keep your cluster 3. Back up your current configuration, including the following: - Agent definitions - Any custom settings - - PostgreSQL database + - PostgreSQL database: You can take a snapshot now so that you have a restore point if the upgrade fails. For the database connection string, see [Database configuration](/docs/kagent/operations/operational-considerations#database-configuration). + + ```bash + pg_dump "postgres://:@:5432/" \ + --format=custom \ + --file=kagent-pre-upgrade-snapshot.dump + ``` 4. **v0.9.0 and later**: You must be running at least v0.8.0 before upgrading to v0.9.0. Check the [release notes](/docs/kagent/resources/release-notes#v09) for 0.9-specific upgrades related to database migrations and RBAC scope. @@ -124,14 +130,31 @@ When the controller detects that the database schema is ahead of its own migrati ### Reset the database before rolling back further -If you need to roll back more than one minor kagent version, you must first reset the database schema to match the target version before rolling back the application. Use `golang-migrate` directly against the migration files. +If you need to roll back more than one minor kagent version, be sure to roll back one minor version at a time. Kagent guarantees backwards compatibility of only one minor version, so skipping minor versions can result in data loss. + +You have two options for resetting the database schema at each step. + +#### Option 1: Restore from snapshot + +If you took a snapshot before upgrading, you can restore it directly. This is simpler than running down migrations, but any data written after the snapshot was taken is lost. + +```bash +pg_restore \ + --clean \ + --dbname="postgres://:@:5432/" \ + kagent-pre-upgrade-snapshot.dump +``` + +After restoring, follow the steps to [roll back the kagent application](#steps-to-roll-back). + +#### Option 2: Run down migrations + +Use `golang-migrate` to run down migrations one minor version at a time. This preserves data written after the snapshot but requires more steps. The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to. For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`. -> **Note**: If you need to roll back more than one minor kagent version, be sure to roll back one minor version at a time. Kagent guarantees backwards compatibility of only one minor version, so skipping minor versions can result in data loss. - 1. Save your current kagent version and the kagent version you want to roll back to in environment variables. ```bash export CURRENT_VERSION= From ea4f71a98c0f3495cec7c15c6e9503e291a8bf9f Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Wed, 24 Jun 2026 09:26:34 -0500 Subject: [PATCH 11/11] Apply suggestions from code review Co-authored-by: Art Signed-off-by: Rachael Graham --- src/app/docs/kagent/operations/upgrade/page.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/app/docs/kagent/operations/upgrade/page.mdx b/src/app/docs/kagent/operations/upgrade/page.mdx index b0b75cef..87fbad37 100644 --- a/src/app/docs/kagent/operations/upgrade/page.mdx +++ b/src/app/docs/kagent/operations/upgrade/page.mdx @@ -153,7 +153,7 @@ Use `golang-migrate` to run down migrations one minor version at a time. This pr The source must be the current (newer) version that you are rolling back from, because it contains the down migrations needed to reverse the schema changes. The `goto` target is the highest migration sequence number present in the version that you are rolling back to. -For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4`. +For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` and `v0.9.3` has migrations up to `000004_feedback_single_pk.up.sql`. To roll back from `v0.9.9` to `v0.9.3`, you set `CURRENT_VERSION=0.9.9`, `ROLLBACK_VERSION=0.9.3`, and run `goto 4` because you want to go back to migration sequence 4 (v0.9.3's `000004`). 1. Save your current kagent version and the kagent version you want to roll back to in environment variables. ```bash @@ -168,7 +168,7 @@ For example, `v0.9.9` has migrations up to `000005_a2a_protocol_version.up.sql` kubectl -n kagent scale deploy/kagent-controller --replicas=0 ``` -4. Open the core migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable. +4. Open the core migration directory for your rollback version and save the sequence number of the highest-numbered file in an environment variable, such as `4` from the previous v0.9.3 `goto 4` example. ```bash open "https://github.com/kagent-dev/kagent/tree/v${ROLLBACK_VERSION}/go/core/pkg/migrations/core/" ```