[HOTE-1080] feat: Move goose migrator lambda to service repo

[iichr]

Description

https://nhsd-jira.digital.nhs.uk/browse/HOTE-1080

• Added lambdas/goose-migrator-lambda/ copying over the Go Lambda source (main.go, go.mod, go.sum), all 16 SQL migrations from the terraform repo, a build script, and a migration integration test script
• docker-compose.yml now sources migrations from lambdas/goose-migrator-lambda/migrations/ instead of the now-removed local database/migrations/ folder, so local development uses the same migrations as production
• Removed stale database/migrations/ folder

The new approach makes the seed script responsible only for what it should own: environment-specific overrides, patching service_url, oauth_token_path, order_path, and client_secret_name to local WireMock values via an UPDATE

Goose migrations (000002, 000009) become the canonical source of truth, they own supplier creation and UUIDs; the seed patches

• Updated seed data UUIDs in database/03-seed-hometest-data.sql to avoid primary key conflicts with production migration data
• Added a test-goose-migrations CI job to .github/workflows/stage-2-test.yaml that runs the migration test script against a real PostgreSQL container (was previously in terraform)
• Added go and goose CLI to .mise.toml tool definitions and registered test-migrations / test-migrations-keep tasks
• Updated .gitignore and .gitleaksignore (build artefacts and OAuth client ID false positives in migration data)

Context

Previously, the goose-migrator Lambda (Go source code, SQL migrations, along with build and test scripts) lived in the terraform repo, creating a split where application database migrations were owned by the infrastructure repo rather than the service repo.

This PR moves the Lambda into the service repo - close to the application code which depends on the migrations, making it the single source of truth for them all.

Terraform continues to own IAM, VPC config, and Lambda invocation - it calls the build script from the service repo at deploy time.

Type of changes

• Refactoring (non-breaking change)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would change existing functionality)
• Bug fix (non-breaking change which fixes an issue)

Checklist

• I am familiar with the contributing guidelines
• I have followed the code style of the project
• I have added tests to cover my changes
• I have updated the documentation accordingly
• This PR is a result of pair or mob programming

---

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

• I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

[Copilot]

Pull request overview

Moves the Goose database migrator Lambda into this service repo, alongside its migration set and developer/CI tooling to build and validate migrations.

Changes:

• Added a Go-based goose-migrator-lambda (handler + Go module files) and packaged migrations.
• Added local scripts to build the Lambda zip and to test migrations against a local Postgres container.
• Wired migration testing into CI and updated repo tooling/docs/ignores to support the new Lambda.

Reviewed changes

Copilot reviewed 24 out of 26 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
lambdas/goose-migrator-lambda/src/main.go	Implements the Go Lambda handler to run Goose migrations and manage schema/user setup.
lambdas/goose-migrator-lambda/src/go.mod	Adds a standalone Go module for the migrator Lambda.
lambdas/goose-migrator-lambda/src/go.sum	Locks Go dependency checksums for the migrator Lambda.
lambdas/goose-migrator-lambda/scripts/build.sh	Adds a build/packaging script that produces the Lambda zip and caches builds via hashing.
lambdas/goose-migrator-lambda/scripts/test-migrations.sh	Adds a Docker-based migration test script to validate migrations and privileges.
lambdas/goose-migrator-lambda/migrations/000001_create_initial_home_test_tables.sql	Adds initial schema migration for the migrator’s migration set.
lambdas/goose-migrator-lambda/migrations/000002_seed_home_test_data.sql	Adds a seed-data migration for suppliers and related config.
lambdas/goose-migrator-lambda/migrations/000003_add_order_status_columns.sql	Adds order status correlation ID column and index.
lambdas/goose-migrator-lambda/migrations/000004_static_data.sql	Seeds static reference data (status/result types).
lambdas/goose-migrator-lambda/migrations/000005_seed_test_type.sql	Seeds test_type reference data.
lambdas/goose-migrator-lambda/migrations/000006_supplier_data_update.sql	Updates supplier service URLs.
lambdas/goose-migrator-lambda/migrations/000007_supplier_data_update.sql	Updates supplier OAuth client ID data.
lambdas/goose-migrator-lambda/migrations/000008_schema_per_environment.sql	Adds pgcrypto extension setup for schema-per-environment approach.
lambdas/goose-migrator-lambda/migrations/000009_update_supplier_endpoints.sql	Adds/updates supplier endpoint path columns and values.
lambdas/goose-migrator-lambda/migrations/000010_update_order_status_codes.sql	Renames status codes with temporary FK constraint changes.
lambdas/goose-migrator-lambda/migrations/000011_add_consent_table.sql	Adds consent table and constraints.
lambdas/goose-migrator-lambda/migrations/000012_add_la_supplier_offering.sql	Seeds supplier offerings.
lambdas/goose-migrator-lambda/migrations/000013_remove_order_reference_from_order_status.sql	Removes order_reference from order_status.
lambdas/goose-migrator-lambda/migrations/000014_add_notification_audit_table.sql	Adds notification audit table + indexes.
lambdas/goose-migrator-lambda/migrations/000015_create_session_table.sql	Adds session table for auth/session storage.
lambdas/goose-migrator-lambda/migrations/000016_add_order_status_reminder_table.sql	Adds order status reminder tracking table/type/index.
lambdas/README.md	Documents the new Go migrator Lambda structure and how to build/test it.
.mise.toml	Adds mise tasks for migration testing and adds a Go tool version.
.gitleaksignore	Suppresses gitleaks false-positives for secret path names in new migrations.
.gitignore	Ignores Go migrator build artifacts (zip, bootstrap binary, cache).
.github/workflows/stage-2-test.yaml	Adds a CI job to run the new goose migration tests.

[github-actions]

Lambdas Coverage Report

Lines	Statements	Branches	Functions
	97.99% (2052/2094)	91.46% (579/633)	96.37% (346/359)

[github-actions]

UI Coverage Report

Lines	Statements	Branches	Functions
	96.32% (5927/6153)	88.4% (724/819)	88.44% (222/251)

[Copilot]

Pull request overview

Copilot reviewed 24 out of 27 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 32 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 32 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

lambdas/goose-migrator-lambda/migrations/000015_create_session_table.sql:6

• This migration was changed from CREATE TABLE IF NOT EXISTS to CREATE TABLE, which makes it non-idempotent and more brittle in environments where the table may already exist (e.g., partial restores or manual bootstrap); keep IF NOT EXISTS (or ensure the down migration is always run first in the same workflow).
  lambdas/goose-migrator-lambda/migrations/000015_create_session_table.sql:32
• The CREATE INDEX statement no longer uses IF NOT EXISTS, so rerunning the migration in a partially-initialised database will fail with “already exists”; use CREATE INDEX IF NOT EXISTS (Postgres 9.5+) to keep behavior consistent with other migrations in this repo.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 32 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 33 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 39 changed files in this pull request and generated 1 comment.

[iichr]

TL;DR:

• The new approach makes the seed script responsible only for what it should own: environment-specific overrides, patching service_url, oauth_token_path, order_path, and client_secret_name to local WireMock values via an UPDATE

• Goose migrations (000002, 000009) become the canonical source of truth, they own supplier creation and UUIDs; the seed patches

Previously the seed script duplicated supplier rows (Preventx as c1a2b3c4-..., SH:24 as d2b3c4d5-...) that only existed locally.
CI was testing against suppliers that don't exist in any deployed environment, and was masking config differences between environments. The same lambdas that talk to 11111111-...77777777-... on the pipeline were having to talk to completely different UUIDs locally.

API tests now reference the same supplier that exists in every environment.
In the future this means that if a goose migration changes a supplier UUID, the seed WHERE clause and test constants both need updating, which is one change across a single data model.

[billwirz1]

good stuff

[sonarqubecloud]

Quality Gate passed

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Pull request overview

Copilot reviewed 32 out of 40 changed files in this pull request and generated 3 comments.