diff --git a/.claude/agents/api-reviewer.md b/.claude/agents/api-reviewer.md deleted file mode 100644 index eb414ac8..00000000 --- a/.claude/agents/api-reviewer.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -name: api-reviewer -description: Reviews stacker REST API endpoints for design consistency, validation, and RBAC coverage. -tools: - - Read - - Grep - - Glob ---- - -You are a REST API design reviewer for the stacker platform. - -When API endpoints are added or modified: - -1. Check routing setup in src/startup.rs -2. Verify request validation using serde_valid -3. Check Casbin policy in access_control.conf covers the new endpoint -4. Verify response format matches existing API conventions -5. Check error responses use consistent error types -6. Verify pagination, filtering, and sorting follow existing patterns -7. Check rate limiting and authentication middleware applied - -Output a review: -- **Route**: method + path -- **Auth**: Casbin policy configured? Middleware applied? -- **Validation**: request body validated? Query params validated? -- **Response**: consistent format? Proper status codes? -- **Breaking Changes**: does this change existing API contracts? diff --git a/.claude/agents/code-reviewer.md b/.claude/agents/code-reviewer.md deleted file mode 100644 index 5dd73dbf..00000000 --- a/.claude/agents/code-reviewer.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -name: code-reviewer -description: Reviews stacker Rust code for safety, SQL injection, auth gaps, and API correctness. -tools: - - Read - - Grep - - Glob ---- - -You are a senior Rust code reviewer for the stacker platform API. - -Check for: -1. **SQL Safety** — all queries use sqlx macros (compile-time checked), no string interpolation in SQL -2. **Auth/RBAC** — new endpoints have Casbin policy entries, middleware applied correctly -3. **Memory Safety** — proper ownership, no unsafe blocks without justification -4. **Error Handling** — Result types propagated, no unwrap() in production paths -5. **Async Correctness** — no blocking calls in async context, proper tokio spawning -6. **Secret Safety** — Vault secrets not logged or leaked in responses -7. **SSH Security** — key material properly handled and cleaned up -8. **API Design** — proper HTTP methods, status codes, request validation with serde_valid -9. **Migration Safety** — new migrations have both up and down scripts -10. **Test Coverage** — new code paths have tests - -Output: severity-rated findings with file:line references. diff --git a/.claude/agents/migration-checker.md b/.claude/agents/migration-checker.md deleted file mode 100644 index b03a7d2c..00000000 --- a/.claude/agents/migration-checker.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -name: migration-checker -description: Validates sqlx PostgreSQL migrations for the stacker service. Checks for data safety and rollback correctness. -tools: - - Read - - Grep - - Glob ---- - -You are a PostgreSQL migration specialist reviewing sqlx migrations for a production Rust service. - -When a migration is created or modified: - -1. Read both .up.sql and .down.sql files in migrations/ -2. Check for destructive operations: DROP TABLE, DROP COLUMN, ALTER TYPE -3. Verify the .down.sql correctly reverses the .up.sql -4. Check for long-running locks (adding NOT NULL, creating indexes on large tables) -5. Verify new columns have sensible defaults or are nullable -6. Cross-reference with sqlx queries in src/ that reference affected tables -7. Check that `cargo sqlx prepare` has been run (sqlx-data.json updated) - -Output a safety report: -- **Risk Level**: LOW / MEDIUM / HIGH / CRITICAL -- **Destructive Operations**: list any data-loss risks -- **Lock Duration**: estimate for production table sizes -- **Rollback Safety**: does .down.sql correctly reverse changes? -- **Query Compatibility**: do existing sqlx queries still compile? diff --git a/.claude/agents/planner.md b/.claude/agents/planner.md deleted file mode 100644 index 3888ac95..00000000 --- a/.claude/agents/planner.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: planner -description: Plans changes for the stacker Rust service. Understands Actix-web, sqlx, Casbin RBAC, and the project/stack domain model. -tools: - - Read - - Grep - - Glob - - LS ---- - -You are a senior Rust engineer planning changes for the stacker platform API. - -This is the core service: Actix-web REST API with sqlx (PostgreSQL), Casbin RBAC, Redis caching, RabbitMQ messaging, and SSH remote management. - -1. Research the existing codebase — start with src/lib.rs and src/startup.rs for routing -2. Check existing patterns in project_app/, forms/, connectors/, middleware/ -3. Review sqlx migrations in migrations/ for schema understanding -4. Check Casbin policies in access_control.conf -5. Create a step-by-step implementation plan -6. Identify risks: SQL migration conflicts, auth policy gaps, breaking API changes - -RULES: -- NEVER write code. Only plan. -- ALWAYS check sqlx query patterns (compile-time checked) -- ALWAYS consider Casbin RBAC implications for new endpoints -- ALWAYS plan new migrations for schema changes -- Flag any changes to middleware/ (affects all routes) -- Consider backward compatibility for REST API changes -- Estimate complexity of each step (small / medium / large) diff --git a/.claude/agents/rbac-reviewer.md b/.claude/agents/rbac-reviewer.md deleted file mode 100644 index 2b724a84..00000000 --- a/.claude/agents/rbac-reviewer.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -name: rbac-reviewer -description: Reviews Casbin RBAC policies and authorization middleware in stacker. -tools: - - Read - - Grep - - Glob ---- - -You are an access control specialist reviewing Casbin RBAC in an Actix-web service. - -When authorization changes are made: - -1. Read access_control.conf for current policy definitions -2. Read src/middleware/authorization.rs for enforcement logic -3. Check that new endpoints have corresponding policy entries -4. Verify role hierarchy is correct -5. Check for privilege escalation paths -6. Verify policy changes don't remove access needed by existing features - -Output an RBAC review: -- **New Endpoints**: policy entries present for all new routes? -- **Role Coverage**: all roles have appropriate access levels? -- **Escalation Risk**: any path from lower to higher privilege? -- **Policy Consistency**: no conflicting or redundant rules? -- **Middleware Applied**: authorization middleware on all protected routes? diff --git a/.claude/agents/tester.md b/.claude/agents/tester.md deleted file mode 100644 index 4ebdaf00..00000000 --- a/.claude/agents/tester.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -name: tester -description: Writes and runs cargo tests for stacker. Uses wiremock for HTTP mocking and sqlx test fixtures. -tools: - - Read - - Write - - Bash - - Grep - - Glob ---- - -You are a QA engineer for a Rust/Actix-web API service. - -1. Read existing test patterns in src/project_app/tests.rs and other test modules -2. Write new tests following Rust testing conventions -3. Run the FULL test suite: `cargo test` -4. Report: what passed, what failed, root cause analysis - -RULES: -- TDD: Write failing test FIRST, then verify it fails, then implement fix -- ALWAYS run full suite: `cargo test` -- Use wiremock for mocking external HTTP services -- Use mockito for simple HTTP mocks -- Use sqlx test fixtures for database tests -- FOLLOW existing test patterns exactly -- Do NOT modify existing passing tests unless explicitly asked -- Test error paths: invalid input, auth failures, database errors -- Use `SQLX_OFFLINE=true cargo test` if no database available diff --git a/.claude/agents/vault-auditor.md b/.claude/agents/vault-auditor.md deleted file mode 100644 index 7e027756..00000000 --- a/.claude/agents/vault-auditor.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -name: vault-auditor -description: Audits Vault secrets integration in stacker. Checks src/project_app/vault.rs and all Vault access patterns. -tools: - - Read - - Grep - - Glob ---- - -You are a secrets management specialist auditing HashiCorp Vault integration in a Rust service. - -When Vault-related code is changed: - -1. Read src/project_app/vault.rs for core Vault logic -2. Grep for all Vault access patterns across the codebase -3. Check that secrets are never logged, serialized to responses, or stored in plain text -4. Verify Vault token renewal and error handling -5. Check that Vault paths follow naming conventions -6. Verify secrets are properly scoped (not over-privileged) - -Output an audit report: -- **Secret Exposure**: any paths where secrets could leak (logs, responses, errors) -- **Token Management**: proper renewal, expiration handling -- **Error Handling**: graceful degradation when Vault is unavailable -- **Access Scope**: secrets access follows least-privilege diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 8b97853b..4bc8344a 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -158,12 +158,20 @@ jobs: # `explain` too — a differently configured binary from the one the image # has always shipped. The second call is nearly free; only `console` and # the re-featured casbin dependency recompile. + # The tests and clippy above leave target/debug at a size where the + # release build below no longer fits on the runner's disk ("No space left + # on device" while compiling rustls). Nothing after this point reads the + # debug artifacts. + - name: Free debug artifacts before release build + run: rm -rf target/debug + - name: Build release binaries run: | cargo build --release \ --bin server \ --bin cleanup-notify \ - --bin backfill_field_policy + --bin backfill_field_policy \ + --bin bake cargo build --release --bin console --features explain - name: Set up Node.js @@ -202,6 +210,7 @@ jobs: cp target/release/console app/stacker/console cp target/release/cleanup-notify app/stacker/cleanup-notify cp target/release/backfill_field_policy app/stacker/backfill_field_policy + cp target/release/bake app/stacker/bake if [ -d web/dist ]; then cp -a web/dist/. app/stacker; fi cp Dockerfile app/Dockerfile cp access_control.conf.dist app/access_control.conf.dist @@ -248,8 +257,8 @@ jobs: tar -xzf app.tar.gz -C "${{ runner.temp }}/prebuilt" cd "${{ runner.temp }}/prebuilt" mv stacker/server stacker/console stacker/cleanup-notify \ - stacker/backfill_field_policy . - chmod +x server console cleanup-notify backfill_field_policy + stacker/backfill_field_policy stacker/bake . + chmod +x server console cleanup-notify backfill_field_policy bake rm -f "${GITHUB_WORKSPACE}/app.tar.gz" - diff --git a/.gitignore b/.gitignore index 9a6176b0..9cde9db7 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,6 @@ post-deploy-ran.txt bake/ docs/MARKETPLACE_FIELD_POLICY.md docs/IMMUTABLE_DEPLOY.md + +# Local-only files: never commit (see AGENTS.md) +local-only-settings/ diff --git a/.sqlx/query-0331ab318537e04c6ee6d191bca99145dd5400f45bdab15bfcb2406a78eb8a4e.json b/.sqlx/query-0331ab318537e04c6ee6d191bca99145dd5400f45bdab15bfcb2406a78eb8a4e.json new file mode 100644 index 00000000..79b1b10b --- /dev/null +++ b/.sqlx/query-0331ab318537e04c6ee6d191bca99145dd5400f45bdab15bfcb2406a78eb8a4e.json @@ -0,0 +1,130 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage\n SET status = 'active', started_at = now(), updated_at = now()\n WHERE deployment_hash = $1 AND status = 'pending'\n RETURNING *", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "0331ab318537e04c6ee6d191bca99145dd5400f45bdab15bfcb2406a78eb8a4e" +} diff --git a/.sqlx/query-0912eec85a56ad9503adc7a197c6c58310fbe7ed4388f3f235cbba88b47cc683.json b/.sqlx/query-0912eec85a56ad9503adc7a197c6c58310fbe7ed4388f3f235cbba88b47cc683.json new file mode 100644 index 00000000..d1c8c943 --- /dev/null +++ b/.sqlx/query-0912eec85a56ad9503adc7a197c6c58310fbe7ed4388f3f235cbba88b47cc683.json @@ -0,0 +1,131 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT * FROM managed_usage\n WHERE started_at IS NOT NULL\n AND (\n (status = 'active'\n AND (last_reported_on IS NULL OR last_reported_on < $1))\n OR (status IN ('stopped', 'suspended')\n AND (last_reported_on IS NULL\n OR last_reported_on < LEAST(\n (COALESCE(suspended_at, stopped_at) AT TIME ZONE 'UTC')::date,\n $1)))\n )\n ORDER BY last_reported_on ASC NULLS FIRST\n LIMIT $2", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Date", + "Int8" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "0912eec85a56ad9503adc7a197c6c58310fbe7ed4388f3f235cbba88b47cc683" +} diff --git a/.sqlx/query-1ee7eb9b87cfcc6ba3d2bbc6351277ac4a7f94d9f0f448b5549e30fc6cc66e19.json b/.sqlx/query-1ee7eb9b87cfcc6ba3d2bbc6351277ac4a7f94d9f0f448b5549e30fc6cc66e19.json index b4c6b1b8..68fdccad 100644 --- a/.sqlx/query-1ee7eb9b87cfcc6ba3d2bbc6351277ac4a7f94d9f0f448b5549e30fc6cc66e19.json +++ b/.sqlx/query-1ee7eb9b87cfcc6ba3d2bbc6351277ac4a7f94d9f0f448b5549e30fc6cc66e19.json @@ -157,6 +157,11 @@ "ordinal": 30, "name": "config_contract", "type_info": "Jsonb" + }, + { + "ordinal": 31, + "name": "runtime", + "type_info": "Jsonb" } ], "parameters": { @@ -196,6 +201,7 @@ true, true, true, + true, true ] }, diff --git a/.sqlx/query-2c7065ccf4a0a527087754db39a2077a054026cb2bc0c010aba218506e76110f.json b/.sqlx/query-2c7065ccf4a0a527087754db39a2077a054026cb2bc0c010aba218506e76110f.json index cc929cb1..53db6337 100644 --- a/.sqlx/query-2c7065ccf4a0a527087754db39a2077a054026cb2bc0c010aba218506e76110f.json +++ b/.sqlx/query-2c7065ccf4a0a527087754db39a2077a054026cb2bc0c010aba218506e76110f.json @@ -67,6 +67,11 @@ "ordinal": 12, "name": "deletion_warning_sent_at", "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "members_server_access", + "type_info": "Bool" } ], "parameters": { @@ -87,7 +92,8 @@ true, false, true, - true + true, + false ] }, "hash": "2c7065ccf4a0a527087754db39a2077a054026cb2bc0c010aba218506e76110f" diff --git a/.sqlx/query-3fd71974a7948b85a0fa72d2c583e29118c63af715e14d9b0a50ef672b8b4d97.json b/.sqlx/query-3fd71974a7948b85a0fa72d2c583e29118c63af715e14d9b0a50ef672b8b4d97.json index 629c9e0a..54b1aa7d 100644 --- a/.sqlx/query-3fd71974a7948b85a0fa72d2c583e29118c63af715e14d9b0a50ef672b8b4d97.json +++ b/.sqlx/query-3fd71974a7948b85a0fa72d2c583e29118c63af715e14d9b0a50ef672b8b4d97.json @@ -67,6 +67,11 @@ "ordinal": 12, "name": "deletion_warning_sent_at", "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "members_server_access", + "type_info": "Bool" } ], "parameters": { @@ -88,7 +93,8 @@ true, false, true, - true + true, + false ] }, "hash": "3fd71974a7948b85a0fa72d2c583e29118c63af715e14d9b0a50ef672b8b4d97" diff --git a/.sqlx/query-467365894a7f9a0888584e8879cac289299f4d03539b9c746324cd183e265553.json b/.sqlx/query-467365894a7f9a0888584e8879cac289299f4d03539b9c746324cd183e265553.json index e92dfd3b..4a0f4a48 100644 --- a/.sqlx/query-467365894a7f9a0888584e8879cac289299f4d03539b9c746324cd183e265553.json +++ b/.sqlx/query-467365894a7f9a0888584e8879cac289299f4d03539b9c746324cd183e265553.json @@ -157,6 +157,11 @@ "ordinal": 30, "name": "config_contract", "type_info": "Jsonb" + }, + { + "ordinal": 31, + "name": "runtime", + "type_info": "Jsonb" } ], "parameters": { @@ -195,6 +200,7 @@ true, true, true, + true, true ] }, diff --git a/.sqlx/query-53a76c5d7dbb79cb51cace5ffacc2cf689a650fb90bccfb80689ef3c5b73a2b0.json b/.sqlx/query-53a76c5d7dbb79cb51cace5ffacc2cf689a650fb90bccfb80689ef3c5b73a2b0.json index 28cc2e4b..9400d4f8 100644 --- a/.sqlx/query-53a76c5d7dbb79cb51cace5ffacc2cf689a650fb90bccfb80689ef3c5b73a2b0.json +++ b/.sqlx/query-53a76c5d7dbb79cb51cace5ffacc2cf689a650fb90bccfb80689ef3c5b73a2b0.json @@ -157,6 +157,11 @@ "ordinal": 30, "name": "config_contract", "type_info": "Jsonb" + }, + { + "ordinal": 31, + "name": "runtime", + "type_info": "Jsonb" } ], "parameters": { @@ -195,6 +200,7 @@ true, true, true, + true, true ] }, diff --git a/.sqlx/query-548b6569daa0654b52c79265b36b63e69b4bb2753f83921800ba74fc7939d93d.json b/.sqlx/query-548b6569daa0654b52c79265b36b63e69b4bb2753f83921800ba74fc7939d93d.json new file mode 100644 index 00000000..6c345585 --- /dev/null +++ b/.sqlx/query-548b6569daa0654b52c79265b36b63e69b4bb2753f83921800ba74fc7939d93d.json @@ -0,0 +1,131 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage\n SET status = 'closed', outcome = $2, stopped_at = now(), updated_at = now()\n WHERE deployment_hash = $1 AND status = 'pending'\n RETURNING *", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Text", + "Varchar" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "548b6569daa0654b52c79265b36b63e69b4bb2753f83921800ba74fc7939d93d" +} diff --git a/.sqlx/query-5d36c126c67a5b70ac168bc46fcff3ee63ae5548ce78f244099f9d61ca694312.json b/.sqlx/query-5d36c126c67a5b70ac168bc46fcff3ee63ae5548ce78f244099f9d61ca694312.json index 6465c2ce..dcf6d3e5 100644 --- a/.sqlx/query-5d36c126c67a5b70ac168bc46fcff3ee63ae5548ce78f244099f9d61ca694312.json +++ b/.sqlx/query-5d36c126c67a5b70ac168bc46fcff3ee63ae5548ce78f244099f9d61ca694312.json @@ -157,6 +157,11 @@ "ordinal": 30, "name": "config_contract", "type_info": "Jsonb" + }, + { + "ordinal": 31, + "name": "runtime", + "type_info": "Jsonb" } ], "parameters": { @@ -196,6 +201,7 @@ true, true, true, + true, true ] }, diff --git a/.sqlx/query-5fea60d7574cfd238a7cbae4d93423869bd7b79dd5b246d80f0b6f39ce4659dc.json b/.sqlx/query-5fea60d7574cfd238a7cbae4d93423869bd7b79dd5b246d80f0b6f39ce4659dc.json index 13c83c93..fa59919d 100644 --- a/.sqlx/query-5fea60d7574cfd238a7cbae4d93423869bd7b79dd5b246d80f0b6f39ce4659dc.json +++ b/.sqlx/query-5fea60d7574cfd238a7cbae4d93423869bd7b79dd5b246d80f0b6f39ce4659dc.json @@ -67,6 +67,11 @@ "ordinal": 12, "name": "deletion_warning_sent_at", "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "members_server_access", + "type_info": "Bool" } ], "parameters": { @@ -87,7 +92,8 @@ true, false, true, - true + true, + false ] }, "hash": "5fea60d7574cfd238a7cbae4d93423869bd7b79dd5b246d80f0b6f39ce4659dc" diff --git a/.sqlx/query-6580fb1fa48f2fc885a238625b6f59c0cc4c3fdc1a70811e61eaed7652d4b9aa.json b/.sqlx/query-6580fb1fa48f2fc885a238625b6f59c0cc4c3fdc1a70811e61eaed7652d4b9aa.json new file mode 100644 index 00000000..b70ede05 --- /dev/null +++ b/.sqlx/query-6580fb1fa48f2fc885a238625b6f59c0cc4c3fdc1a70811e61eaed7652d4b9aa.json @@ -0,0 +1,130 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage\n SET status = 'suspended', suspended_at = now(), updated_at = now()\n WHERE user_id = $1 AND status = 'active'\n RETURNING *", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "6580fb1fa48f2fc885a238625b6f59c0cc4c3fdc1a70811e61eaed7652d4b9aa" +} diff --git a/.sqlx/query-6da8bd589fe73a11fbdfabda9da6710d39e0f18404abba79e6e108a503a7be7a.json b/.sqlx/query-6da8bd589fe73a11fbdfabda9da6710d39e0f18404abba79e6e108a503a7be7a.json new file mode 100644 index 00000000..0def476d --- /dev/null +++ b/.sqlx/query-6da8bd589fe73a11fbdfabda9da6710d39e0f18404abba79e6e108a503a7be7a.json @@ -0,0 +1,135 @@ +{ + "db_name": "PostgreSQL", + "query": "INSERT INTO managed_usage\n (deployment_hash, user_id, stack, server_type, daily_rate_minor, monthly_cap_minor)\n VALUES ($1, $2, $3, $4, $5, $6)\n RETURNING *", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Varchar", + "Varchar", + "Varchar", + "Varchar", + "Int8", + "Int8" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "6da8bd589fe73a11fbdfabda9da6710d39e0f18404abba79e6e108a503a7be7a" +} diff --git a/.sqlx/query-7067df10149b1bb0810cb5b76ae6c36976c3a9cc996d5c145a7afa185a75580b.json b/.sqlx/query-7067df10149b1bb0810cb5b76ae6c36976c3a9cc996d5c145a7afa185a75580b.json new file mode 100644 index 00000000..6698933d --- /dev/null +++ b/.sqlx/query-7067df10149b1bb0810cb5b76ae6c36976c3a9cc996d5c145a7afa185a75580b.json @@ -0,0 +1,130 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT * FROM managed_usage\n WHERE status = 'suspended' AND suspended_at < $1\n ORDER BY suspended_at ASC", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Timestamptz" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "7067df10149b1bb0810cb5b76ae6c36976c3a9cc996d5c145a7afa185a75580b" +} diff --git a/.sqlx/query-7418ec7d0de4d21713237fdfc5651505d376a777f3dd50c4a1c73a3ee14dd4df.json b/.sqlx/query-7418ec7d0de4d21713237fdfc5651505d376a777f3dd50c4a1c73a3ee14dd4df.json new file mode 100644 index 00000000..1713e799 --- /dev/null +++ b/.sqlx/query-7418ec7d0de4d21713237fdfc5651505d376a777f3dd50c4a1c73a3ee14dd4df.json @@ -0,0 +1,130 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage\n SET status = 'closed', outcome = 'server_delete_due',\n stopped_at = now(), updated_at = now()\n WHERE deployment_hash = $1 AND status = 'suspended'\n RETURNING *", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "7418ec7d0de4d21713237fdfc5651505d376a777f3dd50c4a1c73a3ee14dd4df" +} diff --git a/.sqlx/query-7be70644fa8c16903d7608c86a2b1275361c8f05c3dd85eb58f755292d512a83.json b/.sqlx/query-7be70644fa8c16903d7608c86a2b1275361c8f05c3dd85eb58f755292d512a83.json new file mode 100644 index 00000000..3208279d --- /dev/null +++ b/.sqlx/query-7be70644fa8c16903d7608c86a2b1275361c8f05c3dd85eb58f755292d512a83.json @@ -0,0 +1,130 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage\n SET status = 'stopped', stopped_at = now(), updated_at = now()\n WHERE deployment_hash = $1 AND status IN ('active', 'suspended')\n RETURNING *", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "7be70644fa8c16903d7608c86a2b1275361c8f05c3dd85eb58f755292d512a83" +} diff --git a/.sqlx/query-837a91d5f0131bba50d2edce48c125367dfeda8042c0315a7754e42e649ab4a9.json b/.sqlx/query-837a91d5f0131bba50d2edce48c125367dfeda8042c0315a7754e42e649ab4a9.json new file mode 100644 index 00000000..1c6789bb --- /dev/null +++ b/.sqlx/query-837a91d5f0131bba50d2edce48c125367dfeda8042c0315a7754e42e649ab4a9.json @@ -0,0 +1,17 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage\n SET period_start = $2, period_reported_minor = $3, last_reported_on = $4,\n updated_at = now()\n WHERE id = $1 AND (last_reported_on IS NULL OR last_reported_on < $4)", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Int8", + "Date", + "Int8", + "Date" + ] + }, + "nullable": [] + }, + "hash": "837a91d5f0131bba50d2edce48c125367dfeda8042c0315a7754e42e649ab4a9" +} diff --git a/.sqlx/query-a61d069e5bd47694c3e022d3cba730f749bd6aaf7d8ea79fe2bf68c32852faa4.json b/.sqlx/query-a61d069e5bd47694c3e022d3cba730f749bd6aaf7d8ea79fe2bf68c32852faa4.json new file mode 100644 index 00000000..0c05ba27 --- /dev/null +++ b/.sqlx/query-a61d069e5bd47694c3e022d3cba730f749bd6aaf7d8ea79fe2bf68c32852faa4.json @@ -0,0 +1,130 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT * FROM managed_usage WHERE deployment_hash = $1", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "id", + "type_info": "Int8" + }, + { + "ordinal": 1, + "name": "deployment_hash", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "user_id", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "stack", + "type_info": "Varchar" + }, + { + "ordinal": 4, + "name": "server_type", + "type_info": "Varchar" + }, + { + "ordinal": 5, + "name": "daily_rate_minor", + "type_info": "Int8" + }, + { + "ordinal": 6, + "name": "monthly_cap_minor", + "type_info": "Int8" + }, + { + "ordinal": 7, + "name": "currency", + "type_info": "Bpchar" + }, + { + "ordinal": 8, + "name": "server_id", + "type_info": "Int8" + }, + { + "ordinal": 9, + "name": "status", + "type_info": "Varchar" + }, + { + "ordinal": 10, + "name": "outcome", + "type_info": "Varchar" + }, + { + "ordinal": 11, + "name": "started_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 12, + "name": "stopped_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 13, + "name": "suspended_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 14, + "name": "period_start", + "type_info": "Date" + }, + { + "ordinal": 15, + "name": "period_reported_minor", + "type_info": "Int8" + }, + { + "ordinal": 16, + "name": "last_reported_on", + "type_info": "Date" + }, + { + "ordinal": 17, + "name": "created_at", + "type_info": "Timestamptz" + }, + { + "ordinal": 18, + "name": "updated_at", + "type_info": "Timestamptz" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false, + false, + false, + false, + false, + false, + false, + true, + false, + true, + true, + true, + true, + true, + false, + true, + false, + false + ] + }, + "hash": "a61d069e5bd47694c3e022d3cba730f749bd6aaf7d8ea79fe2bf68c32852faa4" +} diff --git a/.sqlx/query-09211b75cd521772b4a9ca806efa60d355d2479811e2bb55d4f6b8163c7ad724.json b/.sqlx/query-b1eb696ce1c3159169bf12fa75110095300ae781f8a828f2af71d2eee4fd7588.json similarity index 89% rename from .sqlx/query-09211b75cd521772b4a9ca806efa60d355d2479811e2bb55d4f6b8163c7ad724.json rename to .sqlx/query-b1eb696ce1c3159169bf12fa75110095300ae781f8a828f2af71d2eee4fd7588.json index a7cf2d5e..461f25ed 100644 --- a/.sqlx/query-09211b75cd521772b4a9ca806efa60d355d2479811e2bb55d4f6b8163c7ad724.json +++ b/.sqlx/query-b1eb696ce1c3159169bf12fa75110095300ae781f8a828f2af71d2eee4fd7588.json @@ -1,6 +1,6 @@ { "db_name": "PostgreSQL", - "query": "\n INSERT INTO project_app (\n project_id, code, name, image, environment, ports, volumes,\n domain, ssl_enabled, resources, restart_policy, command,\n entrypoint, networks, depends_on, healthcheck, labels,\n config_files, template_source, enabled, deploy_order, parent_app_code,\n deployment_id, created_at, updated_at\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, NOW(), NOW())\n RETURNING *\n ", + "query": "\n INSERT INTO project_app (\n project_id, code, name, image, environment, ports, volumes,\n domain, ssl_enabled, resources, restart_policy, command,\n entrypoint, networks, depends_on, healthcheck, labels,\n config_files, template_source, enabled, deploy_order, parent_app_code,\n deployment_id, runtime, created_at, updated_at\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, NOW(), NOW())\n RETURNING *\n ", "describe": { "columns": [ { @@ -157,6 +157,11 @@ "ordinal": 30, "name": "config_contract", "type_info": "Jsonb" + }, + { + "ordinal": 31, + "name": "runtime", + "type_info": "Jsonb" } ], "parameters": { @@ -183,7 +188,8 @@ "Bool", "Int4", "Varchar", - "Int4" + "Int4", + "Jsonb" ] }, "nullable": [ @@ -217,8 +223,9 @@ true, true, true, + true, true ] }, - "hash": "09211b75cd521772b4a9ca806efa60d355d2479811e2bb55d4f6b8163c7ad724" + "hash": "b1eb696ce1c3159169bf12fa75110095300ae781f8a828f2af71d2eee4fd7588" } diff --git a/.sqlx/query-e4925b72d869ec1265180c5aaf98225ff6762b46efe67da2ec28448a6b3bf83c.json b/.sqlx/query-e4925b72d869ec1265180c5aaf98225ff6762b46efe67da2ec28448a6b3bf83c.json new file mode 100644 index 00000000..2024e029 --- /dev/null +++ b/.sqlx/query-e4925b72d869ec1265180c5aaf98225ff6762b46efe67da2ec28448a6b3bf83c.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE managed_usage SET server_id = $2, updated_at = now() WHERE deployment_hash = $1", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Text", + "Int8" + ] + }, + "nullable": [] + }, + "hash": "e4925b72d869ec1265180c5aaf98225ff6762b46efe67da2ec28448a6b3bf83c" +} diff --git a/.sqlx/query-fdb45a4fb83d33464cddc021f3cdfebd5dd137795ab393492b02ab517546a708.json b/.sqlx/query-fa0c97ff0bfdcf152814d34888024cc825de1909a2b592f6f1f53c290da5384b.json similarity index 92% rename from .sqlx/query-fdb45a4fb83d33464cddc021f3cdfebd5dd137795ab393492b02ab517546a708.json rename to .sqlx/query-fa0c97ff0bfdcf152814d34888024cc825de1909a2b592f6f1f53c290da5384b.json index 9f8aa0ed..dd8db6b8 100644 --- a/.sqlx/query-fdb45a4fb83d33464cddc021f3cdfebd5dd137795ab393492b02ab517546a708.json +++ b/.sqlx/query-fa0c97ff0bfdcf152814d34888024cc825de1909a2b592f6f1f53c290da5384b.json @@ -1,6 +1,6 @@ { "db_name": "PostgreSQL", - "query": "\n UPDATE project_app SET\n code = $2,\n name = $3,\n image = $4,\n environment = $5,\n ports = $6,\n volumes = $7,\n domain = $8,\n ssl_enabled = $9,\n resources = $10,\n restart_policy = $11,\n command = $12,\n entrypoint = $13,\n networks = $14,\n depends_on = $15,\n healthcheck = $16,\n labels = $17,\n config_files = $18,\n template_source = $19,\n enabled = $20,\n deploy_order = $21,\n parent_app_code = $22,\n deployment_id = $23,\n config_version = COALESCE(config_version, 0) + 1,\n updated_at = NOW()\n WHERE id = $1\n RETURNING *\n ", + "query": "\n UPDATE project_app SET\n code = $2,\n name = $3,\n image = $4,\n environment = $5,\n ports = $6,\n volumes = $7,\n domain = $8,\n ssl_enabled = $9,\n resources = $10,\n restart_policy = $11,\n command = $12,\n entrypoint = $13,\n networks = $14,\n depends_on = $15,\n healthcheck = $16,\n labels = $17,\n config_files = $18,\n template_source = $19,\n enabled = $20,\n deploy_order = $21,\n parent_app_code = $22,\n deployment_id = $23,\n runtime = $24,\n config_version = COALESCE(config_version, 0) + 1,\n updated_at = NOW()\n WHERE id = $1\n RETURNING *\n ", "describe": { "columns": [ { @@ -157,6 +157,11 @@ "ordinal": 30, "name": "config_contract", "type_info": "Jsonb" + }, + { + "ordinal": 31, + "name": "runtime", + "type_info": "Jsonb" } ], "parameters": { @@ -183,7 +188,8 @@ "Bool", "Int4", "Varchar", - "Int4" + "Int4", + "Jsonb" ] }, "nullable": [ @@ -217,8 +223,9 @@ true, true, true, + true, true ] }, - "hash": "fdb45a4fb83d33464cddc021f3cdfebd5dd137795ab393492b02ab517546a708" + "hash": "fa0c97ff0bfdcf152814d34888024cc825de1909a2b592f6f1f53c290da5384b" } diff --git a/BUGS.md b/BUGS.md index 1582587b..add11e7d 100644 --- a/BUGS.md +++ b/BUGS.md @@ -516,3 +516,41 @@ failing validation — that is the point (loud now, broken later otherwise). If a softer landing is wanted, emit a `W00x: unknown key 'x' ignored` warning from a pre-pass over the raw YAML first, then turn on the hard error after the catalog is clean. + +## 11. 🟡 NOT CONFIRMED — Quick Deploy 404s when `stacker.yml` `version` differs from the baked snapshot's (found 2026-10-17) + +**Severity:** Medium - the stack cannot be deployed through its Quick Deploy +link at all. **Files:** `src/routes/oneclick_deploy/clone.rs:122-145`, +`src/db/baked_snapshot.rs` (`resolve`, `resolve_latest`). + +### Observed (dev, 2026-10-17) + +`stacker-projects/linkding/stacker.yml` declares `version: "1.27.0"` (the +application's version). The only healthy baked snapshot for `linkding` / +`hetzner` is recorded as version `1.0.0`. The clone request carries the +`stacker.yml` version, so the clone answers: + +``` +404 {"error":"No baked snapshot found", + "details":"no healthy baked snapshot for stack 'linkding' (provider: hetzner) in the registry"} +``` + +Stacks whose `stacker.yml` has no `version` (`freshrss`, `mealie`, +`stirling-pdf`, `floci`) take the `resolve_latest` path and are not affected. + +### Root cause (by reading, not yet confirmed by a test) + +When the request has a `version`, `clone.rs:124` calls +`baked_snapshot::resolve(stack, version, provider)`, an exact match on +`baked_snapshots.version`. The `stacker.yml` `version` and the bake's version +are two different numbers with nothing keeping them in step, so any stack +whose declared version differs from the recorded bake version 404s. The +message also hides the cause: it does not say which version was asked for. + +### Open questions before a fix + +- Which number is the contract: the app version in `stacker.yml`, or a + separate bake/template version? Decide, then record bakes under it. +- Should a missing exact version fall back to the latest healthy snapshot + (with a warning), or stay a hard error with the requested version in the + message? diff --git a/Dockerfile b/Dockerfile index 50d6a87f..7dd0e04f 100644 --- a/Dockerfile +++ b/Dockerfile @@ -53,7 +53,8 @@ RUN apt-get update && apt-get install --no-install-recommends -y libssl-dev; \ cargo build --release --bin server; \ cargo build --release --bin console --features explain; \ cargo build --release --bin cleanup-notify; \ - cargo build --release --bin backfill_field_policy + cargo build --release --bin backfill_field_policy; \ + cargo build --release --bin bake #RUN ls -la /app/target/release/ >&2 @@ -78,6 +79,7 @@ COPY --from=builder /app/target/release/server /server COPY --from=builder /app/target/release/console /console COPY --from=builder /app/target/release/cleanup-notify /cleanup-notify COPY --from=builder /app/target/release/backfill_field_policy /backfill_field_policy +COPY --from=builder /app/target/release/bake /bake FROM ${BINARIES}-source AS binaries @@ -94,6 +96,7 @@ COPY --from=binaries /server . COPY --from=binaries /console . COPY --from=binaries /cleanup-notify . COPY --from=binaries /backfill_field_policy . +COPY --from=binaries /bake . COPY --from=config /app/.env . COPY --from=config /app/configuration.yaml . COPY --from=config /usr/local/cargo/bin/sqlx /usr/local/bin/sqlx diff --git a/docs/ONE_CLICK_DEPLOY.md b/docs/ONE_CLICK_DEPLOY.md index ee38626f..ca629ae4 100644 --- a/docs/ONE_CLICK_DEPLOY.md +++ b/docs/ONE_CLICK_DEPLOY.md @@ -86,6 +86,10 @@ stacker/ StackerConfig::from_str+validate_semantics registry lookup → cloud volumes, strip SSH host keys / machine-id / cloud-init state) — hence `bake --ssh-key`. - These three queries use runtime `sqlx::query_as`, not the compile-time macros, so they need no `.sqlx` entry. Run `cargo sqlx prepare` after changing any *macro* query. +- Bake is a manual CLI run today. Automating it - an admin endpoint + `POST /api/admin/bake { deployment_id, stack }` that needs no other input, a static + bake check of the contract, a smoke clone, and channel promotion via + `helpers/bake_registry.rs` - is planned in `config/docs/BAKE_PIPELINE_PLAN.md`. ### 3. `POST /api/v1/deploy/clone` (protected) - Request `{ stack, version, region, server_type, domain, admin_email, env{} }`. diff --git a/docs/STACKER_YML_REFERENCE.md b/docs/STACKER_YML_REFERENCE.md index e705a9ca..48b2e234 100644 --- a/docs/STACKER_YML_REFERENCE.md +++ b/docs/STACKER_YML_REFERENCE.md @@ -410,8 +410,16 @@ Additional containers deployed alongside your main application — databases, ca | `environment` | `map` | no | `{}` | Environment variables | | `volumes` | `string[]` | no | `[]` | Volume mounts (`"name:/path"` or `"./host:/container"`) | | `depends_on` | `string[]` | no | `[]` | Services this depends on (started first) | -| `command` | `string` | no | — | Override the container's default command / entrypoint | +| `command` | `string` | no | — | Override the container's default command | +| `entrypoint` | `string` | no | — | Override the image's entrypoint | | `healthcheck` | `object` | no | — | Docker health check for this service | +| `restart` | `string` | no | `unless-stopped` | `no`, `always`, `unless-stopped` or `on-failure` | +| `cap_add`, `cap_drop` | `string[]` | no | `[]` | Linux capabilities to add or drop | +| `devices` | `string[]` | no | `[]` | Host devices to map (`/dev/dri`) | +| `profiles` | `string[]` | no | `[]` | Start the service only when one of these compose profiles is active | +| `aliases` | `string[]` | no | `[]` | Other names the service answers to, on every network it joins | +| `deploy.replicas` | `integer` | no | `1` | `0` keeps the service defined but not started | +| `deploy.resources.reservations.devices` | `object[]` | no | `[]` | Devices to reserve, e.g. a GPU | #### `services[].command` @@ -500,6 +508,45 @@ services: > **Note:** Stacker detects port conflicts across services during validation. > If two services bind the same host port, you'll get a warning (`W001`). +#### One-shot jobs, profiles and GPUs + +`restart: "no"` keeps a job that runs once from being restarted after it +finishes. `profiles`, `aliases` and `deploy` let one stack carry a CPU and a GPU +variant of the same service: the CPU variant runs by default, the GPU variant +only under its profile, and both answer to the same name. + +```yaml +services: + - name: ollama-cpu + image: ollama/ollama:0.40.4 + aliases: [ollama] + deploy: + replicas: ${OLLAMA_CPU_REPLICAS} # 0 on a GPU machine + + - name: ollama-gpu + image: ollama/ollama:0.40.4 + profiles: [gpu-nvidia] + aliases: [ollama] + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: 1 + capabilities: [gpu] + + - name: model-pull + image: ollama/ollama:0.40.4 + restart: "no" + entrypoint: /bin/sh + command: -c "ollama pull nomic-embed-text" +``` + +On a GPU machine set `OLLAMA_CPU_REPLICAS=0` and `COMPOSE_PROFILES=gpu-nvidia`. +These settings reach every compose stacker writes from the service: the one the +CLI deploys, the author's own compose (`deploy.compose_file`), the one the +server renders from the project, and the one the status-panel agent applies. + --- ## `proxy` diff --git a/local-only-settings/.claude/settings.local.json b/local-only-settings/.claude/settings.local.json deleted file mode 100644 index 4a7cd6f2..00000000 --- a/local-only-settings/.claude/settings.local.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "permissions": { - "allow": [ - "Bash(SQLX_OFFLINE=true ./target/debug/stacker-cli secrets:*)", - "Bash(SQLX_OFFLINE=true cargo build:*)", - "Bash(grep:*)", - "Bash(wc:*)" - ] - } -} diff --git a/local-only-settings/.stacker/docker-compose.yml b/local-only-settings/.stacker/docker-compose.yml deleted file mode 100644 index 6761b81d..00000000 --- a/local-only-settings/.stacker/docker-compose.yml +++ /dev/null @@ -1,34 +0,0 @@ -services: - app: - build: - context: .. - dockerfile: .stacker/Dockerfile - ports: - - 3000:3000 - restart: unless-stopped - networks: - - app-network - web: - image: trydirect/stacker-website:latest - ports: - - 3456:3456 - volumes: - - .:/app - restart: unless-stopped - networks: - - app-network - nginx: - image: nginx:alpine - ports: - - 80:80 - - 443:443 - volumes: - - ./nginx/conf.d:/etc/nginx/conf.d:ro - depends_on: - - app - restart: unless-stopped - networks: - - app-network -networks: - app-network: - driver: bridge diff --git a/local-only-settings/Dockerfile b/local-only-settings/Dockerfile deleted file mode 100644 index 284bc9bb..00000000 --- a/local-only-settings/Dockerfile +++ /dev/null @@ -1,39 +0,0 @@ -## Use an official Node runtime as a parent image -#FROM node:16-alpine -# -## Set the working directory in the container -#WORKDIR /usr/src/app -# -## Copy package.json and package-lock.json -#COPY package*.json ./ -# -## Install dependencies -#RUN npm install --only=production -# -## Copy project files -#COPY . . -# -## Expose the application port -#EXPOSE 3456 -# -## Set environment variables if needed -#ENV NODE_ENV production -# -## Build the application -#RUN npm run build -# -## Command to run the application -#CMD ["npm", "run", "serve"] - -FROM node:16-alpine as builder -WORKDIR /app -COPY . . -RUN npm install --only=production -RUN npm run build - -# Stage 2: Final production image -FROM node:16-alpine -WORKDIR /app -COPY --from=builder /app/dist ./dist -EXPOSE 3456 -CMD ["npm", "run", "serve"] \ No newline at end of file diff --git a/local-only-settings/MULTI_SERVER_DEPLOYMENT.md b/local-only-settings/MULTI_SERVER_DEPLOYMENT.md deleted file mode 100644 index d1ee709d..00000000 --- a/local-only-settings/MULTI_SERVER_DEPLOYMENT.md +++ /dev/null @@ -1,50 +0,0 @@ -# Multi-Server Deployment (Current Workaround) - -## Overview - -Stacker currently uses a 1:1:1 model: one deployment → one agent → one server. True multi-server deployment of a single service is not yet supported. However, you can achieve a similar result today using multiple deployments. - -## Workaround: Multiple Deployments Per Project - -Create separate deployments under the same project, each targeting a different server: - -``` -Project: "my-app" -├── Deployment #1 (hash: abc...) → Server A → Agent A → openclaw -└── Deployment #2 (hash: def...) → Server B → Agent B → openclaw -``` - -### How to set up - -1. Create your project with openclaw in the service catalog -2. Deploy to Server A with one `stacker.yml` config pointing to the first server: - ```yaml - deploy: - target: server - server: - host: 203.0.113.10 - user: deploy - ssh_key: ~/.ssh/deploy_key - ``` -3. Create a second deployment for Server B by updating the target and deploying again: - ```yaml - deploy: - target: server - server: - host: 203.0.113.11 - user: deploy - ssh_key: ~/.ssh/deploy_key - ``` - -Each deployment gets its own: -- `deployment_hash` -- Status Panel agent instance -- Vault tokens and secrets -- Command queue -- Independent health monitoring - -### Limitations - -- Each deployment is **managed independently** — no unified status view across both servers -- Configuration changes must be applied to each deployment separately -- No built-in load balancing between the two instances; use an external reverse proxy or DNS-based routing diff --git a/local-only-settings/STACKER_CLI_PLAN.md b/local-only-settings/STACKER_CLI_PLAN.md deleted file mode 100644 index 55ec78f1..00000000 --- a/local-only-settings/STACKER_CLI_PLAN.md +++ /dev/null @@ -1,1973 +0,0 @@ -# Stacker CLI — Implementation Plan - -> **Approach**: Test-Driven Development — write automated tests first, then implement. -> **Principles**: Clean Code (Robert C. Martin), Rust idioms (Builder, From/Into, small functions, single responsibility). -> **Date**: 2026-02-22 -> **Status**: Draft - ---- - -## Table of Contents - -- [Overview](#overview) -- [Clean Code Principles Applied](#clean-code-principles-applied) -- [Architecture & Module Design](#architecture--module-design) -- [Type System Design](#type-system-design) -- [stacker.yml Configuration Schema](#stackeryml-configuration-schema) -- [CLI Command Interface](#cli-command-interface) -- [Test-First Implementation Plan](#test-first-implementation-plan) -- [Step-by-Step Implementation](#step-by-step-implementation) -- [Verification](#verification) -- [Decisions Log](#decisions-log) - ---- - -## Overview - -Extend the existing Stacker Rust console binary (`src/console/main.rs`) with new subcommands -enabling developers to deploy simple HTML+JS+CSS apps (and more) by adding a single `stacker.yml` -configuration file to their project. - -**Key capabilities:** -- Auto-detect project type, generate Dockerfile and docker-compose.yml when missing -- Deploy locally (docker compose), to cloud (via install container with Terraform/Ansible), or to existing servers (SSH) -- Auto-connect to existing nginx or Nginx Proxy Manager -- AI-assisted Dockerfile generation and troubleshooting (via LLM) -- Deploy Status Panel agent alongside app for monitoring -- Pull install container for Terraform/Ansible execution (no local install needed) -- Standalone for local deploys; optional TryDirect login for cloud/marketplace/AI - ---- - -## Clean Code Principles Applied - -### Naming: Intention-Revealing Names (Ch. 2) - -Every struct, function, and variable tells its purpose. No abbreviations, no generic names: - -``` -Bad: fn proc(c: &Config) -> Res -Good: fn generate_dockerfile(project: &ProjectDetection) -> Result - -Bad: struct Cfg { tp: String, p: String } -Good: struct AppSource { app_type: AppType, source_path: PathBuf } -``` - -### Functions: Small, Do One Thing (Ch. 3) - -Each function does exactly one thing at one level of abstraction. Command handlers -delegate to focused service functions: - -``` -deploy_command() → orchestration only -├── config_parser::load("stacker.yml") → parsing only -├── detector::detect(&source_path) → file scanning only -├── generator::dockerfile(...) → Dockerfile text only -├── generator::compose(...) → compose YAML only -└── runner::start_containers(...) → docker execution only -``` - -### Single Responsibility Principle (SRP) - -| Module | Single Responsibility | -|--------|-----------------------| -| `config_parser` | Parse and validate stacker.yml | -| `detector` | Identify project type from filesystem | -| `generator::dockerfile` | Produce Dockerfile content | -| `generator::compose` | Produce docker-compose.yml content | -| `proxy_manager` | Detect and configure reverse proxies | -| `credentials` | Store and retrieve auth tokens | -| `ai_client` | Communicate with LLM providers | -| `install_runner` | Run Terraform/Ansible via container | - -### Open/Closed Principle (OCP) - -New app types, cloud providers, and proxy types are added by extending enums -and implementing `From`/`Into` — not by modifying existing match arms: - -```rust -// Adding a new app type only requires: -// 1. Add variant to AppType enum -// 2. Implement From for DockerfileTemplate -// No existing code changes needed. -``` - -### Dependency Inversion Principle (DIP) - -Commands depend on traits, not concrete implementations. This enables testing -with mocks while keeping production code clean: - -```rust -// Trait (abstraction) -trait ContainerRuntime { - fn start(&self, compose_path: &Path) -> Result<(), RuntimeError>; - fn stop(&self, compose_path: &Path) -> Result<(), RuntimeError>; - fn logs(&self, service: &str, follow: bool) -> Result; -} - -// Production: DockerComposeRuntime -// Tests: MockContainerRuntime -``` - -### Error Handling: Use Exceptions (Types), Not Return Codes (Ch. 7) - -Following the existing codebase pattern — custom error enums with `Display`, -`From` conversions for error wrapping, and `anyhow` for ad-hoc contexts. -No string-based error passing. - -### DRY: Don't Repeat Yourself - -Shared template rendering, env var interpolation, and YAML generation are -extracted into reusable utilities — not duplicated across commands. - ---- - -## Architecture & Module Design - -``` -stacker/src/ -├── cli/ # NEW: Core CLI library modules -│ ├── mod.rs # Public API surface -│ ├── config_parser.rs # stacker.yml → StackerConfig -│ ├── detector.rs # Filesystem → ProjectDetection -│ ├── error.rs # CliError enum (single error hierarchy) -│ ├── generator/ -│ │ ├── mod.rs -│ │ ├── dockerfile.rs # AppType → Dockerfile content -│ │ ├── compose.rs # StackerConfig → docker-compose.yml -│ │ └── templates.rs # Embedded Dockerfile templates -│ ├── proxy_manager.rs # Proxy detection + configuration -│ ├── ai_client.rs # LLM provider abstraction -│ ├── credentials.rs # Token storage + refresh -│ └── install_runner.rs # Install container orchestration -│ -├── console/ -│ ├── main.rs # Extended Commands enum -│ └── commands/ -│ ├── cli/ # NEW: CLI command implementations -│ │ ├── mod.rs -│ │ ├── login.rs # CallableTrait impl -│ │ ├── init.rs -│ │ ├── deploy/ -│ │ │ ├── mod.rs # DeployCommand dispatcher -│ │ │ ├── local.rs # LocalDeployStrategy -│ │ │ ├── cloud.rs # CloudDeployStrategy -│ │ │ └── server.rs # ServerDeployStrategy -│ │ ├── logs.rs -│ │ ├── status.rs -│ │ ├── destroy.rs -│ │ ├── config.rs -│ │ ├── ai.rs -│ │ ├── proxy.rs -│ │ └── update.rs -│ └── ...existing commands... -``` - -### Dependency Graph (no cycles) - -``` -commands/cli/* - └── depends on → cli/* (library modules) - └── depends on → models, helpers, connectors (existing stacker infra) -``` - ---- - -## Type System Design - -### Core Types with Builder, From/Into - -All types follow existing codebase conventions: `Debug` always, `Clone + Serialize + Deserialize` -on DTOs, `Default` on structs with optional fields, `Validate` on user inputs. - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// cli/config_parser.rs — The central config type -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -/// Root configuration parsed from stacker.yml. -/// Every optional section defaults to sensible values via Default. -#[derive(Debug, Clone, Serialize, Deserialize, Default, Validate)] -pub struct StackerConfig { - #[validate(min_length = 1)] - #[validate(max_length = 128)] - pub name: String, - - #[serde(default)] - pub version: Option, - - #[serde(default)] - pub organization: Option, - - pub app: AppSource, - - #[serde(default)] - pub services: Vec, - - #[serde(default)] - pub proxy: ProxyConfig, - - #[serde(default)] - pub deploy: DeployConfig, - - #[serde(default)] - pub ai: AiConfig, - - #[serde(default)] - pub monitoring: MonitoringConfig, - - #[serde(default)] - pub hooks: HookConfig, - - #[serde(default)] - pub env_file: Option, - - #[serde(default)] - pub env: HashMap, -} - -impl StackerConfig { - /// Load from file path, resolving env vars and validating. - pub fn from_file(path: &Path) -> Result { ... } - - /// Validate cross-field constraints beyond serde_valid. - pub fn validate_semantics(&self) -> Result<(), Vec> { ... } -} -``` - -### Enums with Display, From, serde rename - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// AppType — Discoverable project types -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -pub enum AppType { - Static, - Node, - Python, - Rust, - Go, - Php, - Custom, -} - -impl std::fmt::Display for AppType { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - Self::Static => write!(f, "static"), - Self::Node => write!(f, "node"), - Self::Python => write!(f, "python"), - Self::Rust => write!(f, "rust"), - Self::Go => write!(f, "go"), - Self::Php => write!(f, "php"), - Self::Custom => write!(f, "custom"), - } - } -} - -impl Default for AppType { - fn default() -> Self { Self::Static } -} -``` - -### Builder Pattern — ConfigBuilder - -Fluent builder for constructing `StackerConfig` programmatically (used by `stacker init` -and tests). Follows the `JsonResponseBuilder` pattern already in the codebase: - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// ConfigBuilder — fluent stacker.yml construction -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -#[derive(Debug, Default)] -pub struct ConfigBuilder { - name: Option, - version: Option, - organization: Option, - app_type: Option, - app_path: Option, - services: Vec, - proxy: Option, - deploy_target: Option, - ai: Option, - monitoring: Option, -} - -impl ConfigBuilder { - pub fn new() -> Self { - Self::default() - } - - pub fn name>(mut self, name: S) -> Self { - self.name = Some(name.into()); - self - } - - pub fn app_type(mut self, app_type: AppType) -> Self { - self.app_type = Some(app_type); - self - } - - pub fn app_path>(mut self, path: P) -> Self { - self.app_path = Some(path.into()); - self - } - - pub fn add_service(mut self, service: ServiceDefinition) -> Self { - self.services.push(service); - self - } - - pub fn proxy(mut self, proxy: ProxyConfig) -> Self { - self.proxy = Some(proxy); - self - } - - pub fn deploy_target(mut self, target: DeployTarget) -> Self { - self.deploy_target = Some(target); - self - } - - pub fn ai(mut self, ai: AiConfig) -> Self { - self.ai = Some(ai); - self - } - - pub fn monitoring(mut self, monitoring: MonitoringConfig) -> Self { - self.monitoring = Some(monitoring); - self - } - - /// Consume the builder, validate, and produce StackerConfig. - /// Returns CliError::ConfigValidation if required fields are missing. - pub fn build(self) -> Result { - let name = self.name - .ok_or(CliError::ConfigValidation("name is required".into()))?; - - Ok(StackerConfig { - name, - version: self.version, - organization: self.organization, - app: AppSource { - app_type: self.app_type.unwrap_or_default(), - path: self.app_path.unwrap_or_else(|| PathBuf::from(".")), - ..Default::default() - }, - services: self.services, - proxy: self.proxy.unwrap_or_default(), - deploy: DeployConfig { - target: self.deploy_target.unwrap_or_default(), - ..Default::default() - }, - ai: self.ai.unwrap_or_default(), - monitoring: self.monitoring.unwrap_or_default(), - ..Default::default() - }) - } -} -``` - -### Builder Pattern — DockerfileBuilder - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// DockerfileBuilder — construct Dockerfile content -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -#[derive(Debug, Default)] -pub struct DockerfileBuilder { - base_image: Option, - workdir: Option, - copy_instructions: Vec<(String, String)>, - run_commands: Vec, - expose_ports: Vec, - entrypoint: Option>, - cmd: Option>, - build_args: Vec<(String, String)>, - stages: Vec, -} - -impl DockerfileBuilder { - pub fn new() -> Self { Self::default() } - - pub fn base_image>(mut self, image: S) -> Self { - self.base_image = Some(image.into()); - self - } - - pub fn workdir>(mut self, dir: S) -> Self { - self.workdir = Some(dir.into()); - self - } - - pub fn copy>(mut self, src: S, dst: S) -> Self { - self.copy_instructions.push((src.into(), dst.into())); - self - } - - pub fn run>(mut self, cmd: S) -> Self { - self.run_commands.push(cmd.into()); - self - } - - pub fn expose(mut self, port: u16) -> Self { - self.expose_ports.push(port); - self - } - - pub fn cmd(mut self, cmd: Vec) -> Self { - self.cmd = Some(cmd); - self - } - - pub fn build_arg>(mut self, key: S, default: S) -> Self { - self.build_args.push((key.into(), default.into())); - self - } - - /// Produce the Dockerfile content as a String. - pub fn build(self) -> Result { - let base = self.base_image - .ok_or(CliError::GeneratorError("base_image is required".into()))?; - let mut lines = Vec::new(); - lines.push(format!("FROM {base}")); - // ... assemble all instructions - Ok(lines.join("\n")) - } -} -``` - -### From/Into Conversions — Type-Safe Transformations - -Following the heavy `From`/`TryFrom` usage in the existing codebase (see -`DeploymentIdentifier`, form↔model conversions, view mappings): - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// From for DockerfileBuilder — auto-configure by type -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -impl From for DockerfileBuilder { - fn from(app_type: AppType) -> Self { - match app_type { - AppType::Static => DockerfileBuilder::new() - .base_image("nginx:alpine") - .copy(".", "/usr/share/nginx/html") - .expose(80), - - AppType::Node => DockerfileBuilder::new() - .base_image("node:lts-alpine") - .workdir("/app") - .copy("package*.json", ".") - .run("npm ci --only=production") - .copy(".", ".") - .expose(3000) - .cmd(vec!["node".into(), "server.js".into()]), - - AppType::Python => DockerfileBuilder::new() - .base_image("python:3.12-slim") - .workdir("/app") - .copy("requirements.txt", ".") - .run("pip install --no-cache-dir -r requirements.txt") - .copy(".", ".") - .expose(8000), - - AppType::Rust => DockerfileBuilder::new() - .base_image("rust:1-slim-bookworm") - .workdir("/app") - .copy("Cargo.toml", ".") - .copy("Cargo.lock", ".") - .copy("src", "src") - .run("cargo build --release") - .expose(8080), - - AppType::Go => DockerfileBuilder::new() - .base_image("golang:1.22-alpine") - .workdir("/app") - .copy("go.mod", ".") - .copy("go.sum", ".") - .run("go mod download") - .copy(".", ".") - .run("go build -o /app/main .") - .expose(8080), - - AppType::Php => DockerfileBuilder::new() - .base_image("php:8.3-apache") - .copy(".", "/var/www/html") - .expose(80), - - AppType::Custom => DockerfileBuilder::new(), - } - } -} - -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// From for AppType — detector → type -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -impl From<&ProjectDetection> for AppType { - fn from(detection: &ProjectDetection) -> Self { - detection.app_type - } -} - -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// From for ComposeDefinition — config → compose -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -impl TryFrom<&StackerConfig> for ComposeDefinition { - type Error = CliError; - - fn try_from(config: &StackerConfig) -> Result { - let mut services = Vec::new(); - - // App service - let app_service = ComposeService::from(&config.app); - services.push(app_service); - - // Additional services - for svc in &config.services { - services.push(ComposeService::from(svc)); - } - - // Proxy service (if configured) - if config.proxy.proxy_type != ProxyType::None { - let proxy_service = ComposeService::try_from(&config.proxy)?; - services.push(proxy_service); - } - - // Status panel (if monitoring enabled) - if config.monitoring.status_panel { - services.push(ComposeService::status_panel()); - } - - Ok(ComposeDefinition { services, ..Default::default() }) - } -} - -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// From<&ServiceDefinition> for ComposeService — user service → compose -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -impl From<&ServiceDefinition> for ComposeService { - fn from(svc: &ServiceDefinition) -> Self { - ComposeService { - name: svc.name.clone(), - image: svc.image.clone(), - ports: svc.ports.clone(), - environment: svc.environment.clone(), - volumes: svc.volumes.clone(), - ..Default::default() - } - } -} - -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// From for DeployPayload — config → install service format -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -impl TryFrom<&StackerConfig> for DeployPayload { - type Error = CliError; - - fn try_from(config: &StackerConfig) -> Result { - // Transforms stacker.yml config into the Payload format - // expected by InstallServiceClient::deploy() - // (see forms/project/payload.rs for the target structure) - ... - } -} -``` - -### Error Hierarchy — Single Unified Type - -Following the `ConnectorError` / `DeploymentValidationError` pattern in the codebase. -One `CliError` enum covers all CLI failure modes with structured variants: - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// cli/error.rs — Unified CLI error type -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -#[derive(Debug)] -pub enum CliError { - // Config errors - ConfigNotFound { path: PathBuf }, - ConfigParseFailed { source: serde_yaml::Error }, - ConfigValidation(String), - EnvVarNotFound { var_name: String }, - - // Detection errors - DetectionFailed { path: PathBuf, reason: String }, - - // Generator errors - GeneratorError(String), - DockerfileExists { path: PathBuf }, - - // Deployment errors - DeployFailed { target: DeployTarget, reason: String }, - LoginRequired { feature: String }, - CloudProviderMissing, - ServerHostMissing, - - // Runtime errors - ContainerRuntimeUnavailable, - CommandFailed { command: String, exit_code: i32 }, - - // Auth errors - AuthFailed(String), - TokenExpired, - - // AI errors - AiNotConfigured, - AiProviderError { provider: String, message: String }, - - // Proxy errors - ProxyConfigFailed(String), - - // IO errors - Io(std::io::Error), -} - -impl std::fmt::Display for CliError { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - Self::ConfigNotFound { path } => - write!(f, "Configuration file not found: {}", path.display()), - Self::ConfigParseFailed { source } => - write!(f, "Failed to parse stacker.yml: {source}"), - Self::ConfigValidation(msg) => - write!(f, "Configuration validation error: {msg}"), - Self::EnvVarNotFound { var_name } => - write!(f, "Environment variable not found: ${var_name}"), - Self::LoginRequired { feature } => - write!(f, "Login required for {feature}. Run: stacker login"), - Self::ContainerRuntimeUnavailable => - write!(f, "Docker is not running. Install Docker or start the Docker daemon."), - // ... every variant has a human-readable message - _ => write!(f, "{self:?}"), - } - } -} - -impl std::error::Error for CliError {} - -// From conversions for ergonomic error propagation with ? -impl From for CliError { - fn from(err: std::io::Error) -> Self { Self::Io(err) } -} - -impl From for CliError { - fn from(err: serde_yaml::Error) -> Self { Self::ConfigParseFailed { source: err } } -} -``` - -### Strategy Pattern — Deploy Targets - -Each deploy target is a strategy implementing a common trait. -New targets are added by implementing `DeployStrategy` — no modification of existing code (OCP): - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// Deploy strategy trait (DIP + Strategy pattern) -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -#[async_trait] -pub trait DeployStrategy { - /// Validate that all prerequisites are met for this target. - fn validate(&self, config: &StackerConfig) -> Result<(), CliError>; - - /// Execute the deployment. - async fn deploy(&self, context: &DeployContext) -> Result; - - /// Tear down a deployment created by this strategy. - async fn destroy(&self, context: &DeployContext) -> Result<(), CliError>; -} - -// Concrete strategies -pub struct LocalDeploy; // docker compose up/down -pub struct CloudDeploy; // install container → Terraform/Ansible -pub struct ServerDeploy; // SSH + docker compose or Ansible - -// Factory: DeployTarget enum → strategy (From pattern) -impl DeployTarget { - pub fn strategy(&self) -> Box { - match self { - DeployTarget::Local => Box::new(LocalDeploy), - DeployTarget::Cloud => Box::new(CloudDeploy), - DeployTarget::Server => Box::new(ServerDeploy), - } - } -} -``` - -### Trait-Based Abstractions for Testability (DIP) - -Following the `UserServiceConnector` / `MockUserServiceConnector` pattern: - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// Abstractions for testing — trait + mock pairs -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -// Container runtime -#[async_trait] -pub trait ContainerRuntime: Send + Sync { - async fn compose_up(&self, path: &Path, build: bool) -> Result<(), CliError>; - async fn compose_down(&self, path: &Path, volumes: bool) -> Result<(), CliError>; - async fn compose_logs(&self, path: &Path, service: Option<&str>, - follow: bool, tail: Option) -> Result<(), CliError>; - async fn list_containers(&self) -> Result, CliError>; - fn is_available(&self) -> bool; -} - -pub struct DockerComposeRuntime; // Production -pub struct MockContainerRuntime; // Tests - -// Filesystem (for detector/generator testing without tempdir) -pub trait FileSystem: Send + Sync { - fn exists(&self, path: &Path) -> bool; - fn read_to_string(&self, path: &Path) -> Result; - fn write(&self, path: &Path, content: &str) -> Result<(), CliError>; - fn list_dir(&self, path: &Path) -> Result, CliError>; -} - -pub struct RealFileSystem; // Production -pub struct MockFileSystem; // Tests — in-memory HashMap - -// AI provider -#[async_trait] -pub trait AiProvider: Send + Sync { - async fn complete(&self, prompt: &str, context: &str) -> Result; -} - -pub struct OpenAiProvider; -pub struct OllamaProvider; -pub struct MockAiProvider; // Tests — returns canned responses -``` - -### Validation — Structured Issues - -```rust -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// Validation results (like ValidateStackConfigTool pattern) -// ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct ValidationIssue { - pub severity: Severity, - pub code: String, - pub message: String, - pub field: Option, -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -pub enum Severity { - Error, - Warning, - Info, -} - -impl std::fmt::Display for Severity { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - Self::Error => write!(f, "error"), - Self::Warning => write!(f, "warning"), - Self::Info => write!(f, "info"), - } - } -} -``` - ---- - -## stacker.yml Configuration Schema - -```yaml -# ===== Project Identity ===== -name: my-landing-page # required -version: "1.0" # optional -organization: acme-corp # optional, set via `stacker login` - -# ===== Application Source ===== -app: - type: static # static | node | python | rust | go | php | custom - path: ./src # app source root (default: .) - dockerfile: ./Dockerfile # optional — auto-generated if missing - image: registry.io/myapp:v1 # optional — skip build if provided - build: - context: . - args: - NODE_ENV: production - ports: # optional — overrides default port from type - - "8080:3000" - volumes: # optional — volume mounts for the app - - "./uploads:/app/uploads" - environment: # optional — merged with top-level env (app wins) - NODE_ENV: production - -# ===== Services ===== -services: - - name: postgres - image: postgres:16 - ports: ["5432:5432"] - environment: - POSTGRES_DB: myapp - POSTGRES_PASSWORD: ${DB_PASSWORD} - volumes: - - pgdata:/var/lib/postgresql/data - - name: redis - image: redis:7-alpine - -# ===== Proxy / Ingress ===== -proxy: - type: nginx # nginx | nginx-proxy-manager | traefik | none - auto_detect: true - domains: - - domain: myapp.example.com - ssl: auto # auto | manual | off - upstream: app:3000 - config: ./nginx.conf - -# ===== Deployment Target ===== -deploy: - target: local # local | cloud | server - compose_file: ./docker-compose.yml - - cloud: - provider: hetzner # hetzner | digitalocean | aws | linode | vultr - region: fsn1 - size: cpx21 - ssh_key: ~/.ssh/id_ed25519 - - server: - host: 192.168.1.100 - user: deploy - ssh_key: ~/.ssh/id_ed25519 - port: 22 - -# ===== AI Assistant ===== -ai: - enabled: true - provider: openai # openai | anthropic | ollama | custom - model: gpt-4o - api_key: ${OPENAI_API_KEY} - endpoint: null - tasks: [dockerfile, compose, troubleshoot, security] - -# ===== Monitoring ===== -monitoring: - status_panel: true - healthcheck: - endpoint: /health - interval: 30s - metrics: - enabled: true - telegraf: false - -# ===== Lifecycle Hooks ===== -hooks: - pre_build: ./scripts/pre-build.sh - post_deploy: ./scripts/post-deploy.sh - on_failure: ./scripts/on-failure.sh - -# ===== Environment ===== -env_file: .env -env: - APP_PORT: "3000" - LOG_LEVEL: info -``` - -### Minimal config (static HTML app) - -```yaml -name: my-site -app: - type: static - path: ./public -``` - ---- - -## CLI Command Interface - -``` -stacker login [--org ] [--domain ] [--api-url ] -stacker init [--type static|node|python|rust|go|php] [--with-proxy] [--with-ai] -stacker deploy [--target local|cloud|server] [--file stacker.yml] [--dry-run] [--force-rebuild] -stacker logs [--service ] [--follow] [--tail ] [--since ] -stacker status [--json] [--watch] -stacker destroy [--volumes] [--confirm] -stacker config validate [--file stacker.yml] -stacker config show [--file stacker.yml] -stacker ai ask "" [--context deploy|compose|dockerfile] -stacker proxy add [--upstream ] [--ssl auto|manual|off] -stacker proxy detect -stacker update [--channel stable|nightly] -``` - ---- - -## Test-First Implementation Plan - -### Dev-Dependencies to Add - -```toml -# stacker/Cargo.toml [dev-dependencies] — additions -assert_cmd = "2.0" # CLI binary testing -predicates = "3.0" # Assertion helpers for assert_cmd -tempfile = "3" # Temp dirs for project scaffolding tests -mockito = "1" # HTTP mock server -``` - -### Test Phases and Files - -Tests are organized in dependency order — each phase builds on types from the previous. - ---- - -### Phase 0: Error Types + Core Enums (foundation) - -**File: `src/cli/error.rs` — `#[cfg(test)] mod tests`** - -``` -test_cli_error_display_config_not_found - — Create CliError::ConfigNotFound { path: "/tmp/stacker.yml".into() } - — Assert: display contains "Configuration file not found: /tmp/stacker.yml" - -test_cli_error_display_env_var_not_found - — Create CliError::EnvVarNotFound { var_name: "DB_PASSWORD".into() } - — Assert: display contains "DB_PASSWORD" - -test_cli_error_display_login_required - — Create CliError::LoginRequired { feature: "cloud deploy".into() } - — Assert: display contains "stacker login" - -test_cli_error_from_io_error - — Create std::io::Error → convert via From - — Assert: matches CliError::Io(_) - -test_cli_error_from_yaml_error - — Create serde_yaml parse error → convert via From - — Assert: matches CliError::ConfigParseFailed { .. } - -test_app_type_display - — For each AppType variant: assert display matches serde rename - — Static→"static", Node→"node", Python→"python" etc. - -test_app_type_serde_roundtrip - — Serialize AppType::Node → deserialize back - — Assert: equality preserved - -test_app_type_default_is_static - — Assert: AppType::default() == AppType::Static - -test_deploy_target_display - — DeployTarget::Cloud → "cloud" - — DeployTarget::Local → "local" - — DeployTarget::Server → "server" - -test_deploy_target_default_is_local - — Assert: DeployTarget::default() == DeployTarget::Local - -test_proxy_type_display - — ProxyType::Nginx → "nginx" - — ProxyType::NginxProxyManager → "nginx-proxy-manager" - — ProxyType::None → "none" - -test_severity_display - — Severity::Error → "error" - — Severity::Warning → "warning" - -test_validation_issue_serialize - — Create ValidationIssue with code/message/field - — Serialize to JSON → assert expected structure -``` - -**13 tests** — Establishes error hierarchy and enum behavior. - ---- - -### Phase 1: Config Parser + Builder - -**File: `src/cli/config_parser.rs` — `#[cfg(test)] mod tests`** - -``` -test_parse_minimal_config - — Parse: name + app.type + app.path only - — Assert: name="my-site", app.app_type=Static, app.path="./public" - — Assert: all optional sections are Default values - -test_parse_full_config - — Parse complete stacker.yml with every section populated - — Assert: every field parsed correctly, services.len() correct, proxy domains correct - -test_parse_env_var_interpolation - — Set env var TEST_DB_PASS=secret123 - — Parse config with password: ${TEST_DB_PASS} - — Assert: resolved to "secret123" - -test_parse_env_var_missing_returns_error - — Parse config with ${NONEXISTENT_VAR} - — Assert: CliError::EnvVarNotFound { var_name: "NONEXISTENT_VAR" } - -test_parse_env_file_loads_vars - — Write .env to tempdir with KEY=value - — Parse config with env_file: .env - — Assert: env map contains KEY=value - -test_parse_invalid_app_type_returns_error - — Parse: app.type: cobol - — Assert: serde error mentioning unknown variant - -test_parse_missing_name_returns_error - — Parse config without name field - — Assert: CliError::ConfigValidation mentioning "name" - -test_parse_services_array - — Parse config with 3 services (postgres, redis, minio) - — Assert: services.len()==3, each has correct name/image/ports - -test_parse_proxy_domains - — Parse proxy with 2 domains, different ssl settings - — Assert: domains[0].ssl == SslMode::Auto, domains[1].upstream correct - -test_parse_ai_section_with_ollama - — Parse ai with provider=ollama, custom endpoint - — Assert: ai.provider==AiProvider::Ollama, ai.endpoint present - -test_default_deploy_target_is_local - — Parse config without deploy section - — Assert: config.deploy.target == DeployTarget::Local - -test_default_proxy_type_is_none - — Parse config without proxy section - — Assert: config.proxy.proxy_type == ProxyType::None - -test_config_file_not_found - — StackerConfig::from_file(Path::new("/nonexistent/stacker.yml")) - — Assert: CliError::ConfigNotFound { path } - -test_config_invalid_yaml_syntax - — Parse "{{invalid: yaml: :::" - — Assert: CliError::ConfigParseFailed - -test_validate_semantics_cloud_without_provider - — Config with deploy.target=Cloud but cloud.provider=None - — Assert: validate_semantics() returns error with "provider" - -test_validate_semantics_server_without_host - — Config with deploy.target=Server but server.host=None - — Assert: error with "host" - -test_validate_semantics_port_conflict - — Config with two services both on port 8080 - — Assert: warning about port conflict - -test_validate_semantics_no_image_no_dockerfile_custom_type - — Config with app.type=Custom, no image, no dockerfile - — Assert: error "need image or dockerfile" - -test_validate_semantics_happy_path - — Valid full config - — Assert: no issues returned -``` - -**ConfigBuilder tests (same file):** - -``` -test_config_builder_minimal - — ConfigBuilder::new().name("test").build() - — Assert: Ok(config) with name="test", defaults everywhere - -test_config_builder_fluent_chain - — ConfigBuilder::new() - .name("my-app") - .app_type(AppType::Node) - .app_path("./src") - .add_service(postgres_service) - .proxy(proxy_config) - .build() - — Assert: all fields set correctly - -test_config_builder_missing_name_returns_error - — ConfigBuilder::new().app_type(AppType::Static).build() - — Assert: Err(CliError::ConfigValidation("name is required")) - -test_config_builder_default_app_type_is_static - — ConfigBuilder::new().name("x").build() - — Assert: config.app.app_type == AppType::Static - -test_config_builder_to_yaml_roundtrip - — Build config via builder → serialize to YAML → parse back - — Assert: both match - -test_config_builder_multiple_services - — Add 3 services via chained .add_service() - — Assert: config.services.len() == 3 -``` - -**25 tests** — Config parsing, env interpolation, validation, builder. - ---- - -### Phase 2: Project Detector - -**File: `src/cli/detector.rs` — `#[cfg(test)] mod tests`** - -Uses `MockFileSystem` trait mock to avoid tempdir I/O: - -``` -test_detect_static_html - — MockFileSystem with ["index.html", "style.css"] - — Assert: detection.app_type == AppType::Static - -test_detect_node_project - — MockFileSystem with ["package.json", "src/index.js"] - — Assert: detection.app_type == AppType::Node - -test_detect_python_requirements - — MockFileSystem with ["requirements.txt", "app.py"] - — Assert: detection.app_type == AppType::Python - -test_detect_python_pyproject - — MockFileSystem with ["pyproject.toml"] - — Assert: detection.app_type == AppType::Python - -test_detect_rust_project - — MockFileSystem with ["Cargo.toml", "src/main.rs"] - — Assert: detection.app_type == AppType::Rust - -test_detect_go_project - — MockFileSystem with ["go.mod", "main.go"] - — Assert: detection.app_type == AppType::Go - -test_detect_php_composer - — MockFileSystem with ["composer.json", "public/index.php"] - — Assert: detection.app_type == AppType::Php - -test_detect_empty_directory - — MockFileSystem with [] - — Assert: detection.app_type == AppType::Custom (unknown fallback) - -test_detect_priority_node_over_static - — MockFileSystem with ["package.json", "index.html"] - — Assert: detection.app_type == AppType::Node - -test_detect_existing_dockerfile_flag - — MockFileSystem with ["Dockerfile", "package.json"] - — Assert: detection.has_dockerfile == true - -test_detect_existing_compose_flag - — MockFileSystem with ["docker-compose.yml", "index.html"] - — Assert: detection.has_compose == true - -test_detect_env_file_flag - — MockFileSystem with [".env", "index.html"] - — Assert: detection.has_env_file == true - -test_detection_to_app_type_via_from - — Create ProjectDetection { app_type: AppType::Node, .. } - — let app_type: AppType = AppType::from(&detection); - — Assert: app_type == AppType::Node -``` - -**13 tests** — Project detection with mock filesystem. - ---- - -### Phase 3: Generators - -**File: `src/cli/generator/dockerfile.rs` — `#[cfg(test)] mod tests`** - -``` -test_dockerfile_builder_static - — DockerfileBuilder::from(AppType::Static).build() - — Assert: contains "FROM nginx:alpine", "COPY . /usr/share/nginx/html" - -test_dockerfile_builder_node - — DockerfileBuilder::from(AppType::Node).build() - — Assert: contains "FROM node:", "npm ci", "EXPOSE 3000" - -test_dockerfile_builder_python - — DockerfileBuilder::from(AppType::Python).build() - — Assert: contains "FROM python:3.12-slim", "pip install" - -test_dockerfile_builder_rust - — DockerfileBuilder::from(AppType::Rust).build() - — Assert: contains "FROM rust:", "cargo build --release" - -test_dockerfile_builder_go - — DockerfileBuilder::from(AppType::Go).build() - — Assert: contains "FROM golang:", "go build" - -test_dockerfile_builder_php - — DockerfileBuilder::from(AppType::Php).build() - — Assert: contains "FROM php:", "/var/www/html" - -test_dockerfile_builder_custom_base_image - — DockerfileBuilder::new().base_image("ubuntu:22.04").build() - — Assert: starts with "FROM ubuntu:22.04" - -test_dockerfile_builder_with_build_args - — DockerfileBuilder::from(AppType::Node) - .build_arg("NODE_ENV", "production") - .build() - — Assert: contains "ARG NODE_ENV" - -test_dockerfile_builder_with_custom_path - — DockerfileBuilder::new() - .base_image("nginx:alpine") - .copy("./dist", "/usr/share/nginx/html") - .build() - — Assert: contains "COPY ./dist" - -test_dockerfile_builder_missing_base_image_returns_error - — DockerfileBuilder::new().copy(".", ".").build() - — Assert: Err(CliError::GeneratorError) mentioning "base_image" - -test_dockerfile_builder_chaining_returns_self - — Verify each method returns Self for chaining: - DockerfileBuilder::new() - .base_image("x").workdir("/app").copy("a","b") - .run("cmd").expose(80).cmd(vec![]).build_arg("K","V") - .build() - — Assert: Ok — all methods chainable - -test_generate_does_not_overwrite_existing_dockerfile - — MockFileSystem with existing Dockerfile - — Call generate → assert CliError::DockerfileExists -``` - -**12 tests** — Dockerfile builder + From conversions. - -**File: `src/cli/generator/compose.rs` — `#[cfg(test)] mod tests`** - -``` -test_compose_from_minimal_config - — Config: app only, no services - — ComposeDefinition::try_from(&config) - — Assert: 1 service ("app"), valid YAML output - -test_compose_from_config_with_services - — Config with postgres + redis - — Assert: 3 services total, correct images/ports - -test_compose_from_config_with_nginx_proxy - — Config with proxy.type=Nginx, 1 domain - — Assert: nginx service present, depends_on app, port 80/443 - -test_compose_from_config_with_npm_proxy - — Config with proxy.type=NginxProxyManager - — Assert: NPM service with ports 80, 81, 443 - -test_compose_from_config_with_status_panel - — Config with monitoring.status_panel=true - — Assert: status-panel service present - -test_compose_service_from_service_definition - — ServiceDefinition { name: "pg", image: "postgres:16", ports: ["5432:5432"] } - — ComposeService::from(&svc) - — Assert: name, image, ports correct - -test_compose_output_is_valid_yaml - — Generate from full config → serde_yaml::from_str(&output) - — Assert: roundtrip succeeds - -test_compose_includes_named_volumes - — Config with service using named volumes - — Assert: top-level volumes section in output - -test_compose_includes_default_network - — Any config → compose output - — Assert: networks section with default bridge - -test_compose_env_vars_rendered - — Config with app env vars - — Assert: environment section in compose YAML - -test_compose_port_format - — Service with ports ["8080:80", "443:443"] - — Assert: ports rendered as "8080:80" strings in YAML -``` - -**11 tests** — Compose generation via TryFrom<&StackerConfig>. - ---- - -### Phase 4: Credentials - -**File: `src/cli/credentials.rs` — `#[cfg(test)] mod tests`** - -``` -test_credentials_save_and_load - — Save Credentials { token, org, domain, api_url } to tempdir - — Load from same path → assert fields match - -test_credentials_with_org_and_domain - — Save with org="acme-corp", domain="acme.com" - — Load → assert correct - -test_credentials_missing_file_returns_none - — Load from nonexistent path - — Assert: Ok(None) - -test_credentials_corrupted_file_returns_error - — Write "not json" to credentials path - — Load → assert CliError - -test_credentials_is_expired_true - — Credentials with expires_at in the past - — Assert: is_expired() == true - -test_credentials_is_expired_false - — Credentials with expires_at in the future - — Assert: is_expired() == false - -test_credentials_refresh_token - — Mock OAuth endpoint (mockito) returning new token - — Call refresh() → assert new token saved - -test_credentials_default_config_dir - — Assert: default_config_dir() returns ~/.config/stacker (Linux) - or ~/Library/Application Support/stacker (macOS) -``` - -**8 tests** — Token storage, expiry, refresh. - ---- - -### Phase 5: Proxy Manager - -**File: `src/cli/proxy_manager.rs` — `#[cfg(test)] mod tests`** - -``` -test_generate_nginx_server_block - — Input: DomainConfig { domain: "app.example.com", upstream: "app:3000", ssl: Auto } - — Assert: output contains server_name, proxy_pass, ssl directives - -test_generate_nginx_multiple_domains - — 2 DomainConfig entries → 2 server blocks - -test_detect_proxy_nginx_from_containers - — Mock container list with container named "nginx" on port 80 - — Assert: ProxyDetection { proxy_type: ProxyType::Nginx, ports: [80] } - -test_detect_proxy_npm_from_containers - — Mock container with ports 80, 81, 443 - — Assert: ProxyDetection { proxy_type: ProxyType::NginxProxyManager } - -test_detect_no_proxy - — Empty container list - — Assert: ProxyDetection { proxy_type: ProxyType::None } - -test_proxy_type_from_string - — "nginx" → ProxyType::Nginx - — "nginx-proxy-manager" → ProxyType::NginxProxyManager - — "traefik" → ProxyType::Traefik - — "none" → ProxyType::None -``` - -**6 tests** — Proxy detection and nginx config generation. - ---- - -### Phase 6: Install Container Runner - -**File: `src/cli/install_runner.rs` — `#[cfg(test)] mod tests`** - -``` -test_build_run_command_with_cloud_config - — Input: provider=hetzner, region=fsn1, compose_path, ssh_key_path - — Assert: docker run command has correct -v mounts and -e vars - -test_run_command_mounts_stacker_yml - — Assert: volume mount for stacker.yml at expected container path - -test_run_command_mounts_ssh_key - — ssh_key=~/.ssh/id_ed25519 - — Assert: -v mount for SSH key - -test_run_command_plan_mode - — dry_run=true - — Assert: command includes "plan" not "apply" - -test_run_command_apply_mode - — dry_run=false - — Assert: command includes "apply" - -test_install_container_image_tag - — Assert: default image is "trydirect/install-service:latest" -``` - -**6 tests** — Install container command construction. - ---- - -### Phase 7: AI Client - -**File: `src/cli/ai_client.rs` — `#[cfg(test)] mod tests`** - -``` -test_ai_provider_from_config_openai - — AiConfig { provider: "openai", model: "gpt-4o", api_key: "sk-..." } - — Build provider → assert correct type - -test_ai_provider_from_config_ollama - — AiConfig { provider: "ollama", endpoint: "http://localhost:11434" } - — Build provider → assert correct type - -test_mock_ai_complete - — MockAiProvider returning "Use FROM node:lts-alpine" - — Call complete("optimize dockerfile", context) - — Assert: response contains expected text - -test_ai_build_prompt_for_dockerfile - — Build prompt with project_type=Node, files=["package.json"] - — Assert: prompt mentions Node.js, asks for Dockerfile - -test_ai_build_prompt_for_troubleshoot - — Build prompt with error_log="connection refused" - — Assert: prompt includes error log, asks for diagnosis - -test_ai_not_configured_returns_error - — AiConfig { enabled: false } - — Call complete → CliError::AiNotConfigured -``` - -**6 tests** — AI provider abstraction and prompt building. - ---- - -### Phase 8: Integration Tests — CLI Commands - -**File: `tests/cli_login.rs`** - -``` -test_login_saves_credentials - — Mock OAuth server (wiremock) returning token - — Invoke LoginCommand with mock api_url - — Assert: credentials file created - -test_login_with_org_stores_org - — Login with org="acme" - — Assert: stored credentials.org == "acme" - -test_login_with_domain_stores_domain - — Login with domain="acme.com" - — Assert: stored credentials.domain == "acme.com" - -test_login_invalid_credentials_returns_error - — Mock OAuth returning 401 - — Assert: CliError::AuthFailed - -test_login_api_url_override - — Login with api_url="https://custom.api" - — Assert: stored api_url correct - -test_login_refresh_existing_token - — Pre-populate expired credentials - — Login → assert new token, file updated not duplicated -``` - -**6 tests** - -**File: `tests/cli_init.rs`** - -``` -test_init_static_project_creates_config - — tempdir with index.html - — Run InitCommand → assert stacker.yml exists with app.type=static - -test_init_node_project_detects_correctly - — tempdir with package.json - — Assert: generated config has app.type=node - -test_init_type_flag_overrides_detection - — tempdir with package.json, --type python - — Assert: app.type=python - -test_init_with_proxy_flag_adds_section - — --with-proxy → assert proxy section in generated YAML - -test_init_with_ai_flag_adds_section - — --with-ai → assert ai section with placeholder - -test_init_does_not_overwrite_existing - — tempdir with existing stacker.yml - — Assert: error, not overwritten - -test_init_output_parses_as_valid_config - — Init → parse generated file via StackerConfig::from_file() - — Assert: Ok - -test_init_empty_dir_defaults_to_static - — Empty tempdir → assert app.type=static -``` - -**8 tests** - -**File: `tests/cli_deploy.rs`** - -``` -test_deploy_local_dry_run_generates_files - — tempdir with index.html + minimal stacker.yml - — DeployCommand with --dry-run - — Assert: .stacker/Dockerfile and .stacker/docker-compose.yml created - -test_deploy_local_preserves_existing_dockerfile - — tempdir with custom Dockerfile, config referencing it - — Deploy --dry-run → assert Dockerfile unchanged - -test_deploy_local_uses_existing_compose - — tempdir with docker-compose.yml, config with compose_file set - — Deploy --dry-run → assert uses existing, no .stacker/ compose - -test_deploy_local_with_image_skips_build - — Config with app.image="nginx:latest" - — Deploy --dry-run → assert no Dockerfile generated - -test_deploy_cloud_requires_login - — Config: deploy.target=cloud, no credentials - — Assert: CliError::LoginRequired { feature: "cloud deploy" } - -test_deploy_cloud_requires_provider - — Config: deploy.target=cloud, cloud section empty - — Assert: validation error - -test_deploy_server_requires_host - — Config: deploy.target=server, no host - — Assert: validation error - -test_deploy_missing_config_file - — Deploy in empty dir → CliError::ConfigNotFound - -test_deploy_custom_file_flag - — --file custom.yml → assert loads custom file - -test_deploy_force_rebuild - — --force-rebuild → assert build steps included - -test_deploy_runs_pre_build_hook - — Config with hooks.pre_build → assert hook noted in --dry-run - -test_deploy_target_strategy_dispatch - — DeployTarget::Local.strategy() → assert is LocalDeploy - — DeployTarget::Cloud.strategy() → assert is CloudDeploy - — DeployTarget::Server.strategy() → assert is ServerDeploy - -test_deploy_payload_from_config - — Full config → DeployPayload::try_from(&config) - — Assert: payload has correct stack_code, compose, cloud settings -``` - -**13 tests** - -**File: `tests/cli_logs.rs`** - -``` -test_logs_constructs_compose_command - — LogsCommand with defaults - — Assert: command = "docker compose logs" - -test_logs_with_service_filter - — --service postgres - — Assert: command includes "postgres" - -test_logs_with_follow - — --follow → assert "-f" in command - -test_logs_with_tail - — --tail 100 → assert "--tail 100" - -test_logs_with_since - — --since 1h → assert "--since 1h" - -test_logs_no_deployment_returns_error - — Clean dir, no .stacker/ → error -``` - -**6 tests** - -**File: `tests/cli_status.rs`** - -``` -test_status_local_constructs_query - — StatusCommand → assert queries Docker API - -test_status_json_flag - — --json → assert output format is JSON - -test_status_no_deployment_returns_error - — Clean dir → error - -test_status_remote_queries_agent_api - — Config with cloud deployment + mock agent API - — Assert: queries /api/v1/agent/deployments/{hash} -``` - -**4 tests** - -**File: `tests/cli_destroy.rs`** - -``` -test_destroy_constructs_down_command - — DestroyCommand --confirm - — Assert: "docker compose down" - -test_destroy_with_volumes_flag - — --volumes → assert "--volumes" in command - -test_destroy_requires_confirmation - — No --confirm → error in non-interactive mode - -test_destroy_no_deployment_returns_error - — Clean dir → error - -test_destroy_cloud_triggers_terraform_destroy - — Config: deploy.target=cloud, mock install runner - — Assert: terraform destroy triggered -``` - -**5 tests** - -**File: `tests/cli_config.rs`** - -``` -test_config_validate_valid_returns_success - — Valid stacker.yml → assert success message - -test_config_validate_invalid_lists_errors - — Invalid config → assert errors displayed - -test_config_validate_custom_file - — --file custom.yml → assert validates custom file - -test_config_show_resolves_env_vars - — Set TEST_VAR=hello, config with ${TEST_VAR} - — Assert: output shows "hello" - -test_config_show_displays_defaults - — Minimal config → show → assert defaults visible -``` - -**5 tests** - -**File: `tests/cli_ai.rs`** - -``` -test_ai_ask_no_config_returns_error - — Config without ai section → CliError::AiNotConfigured - -test_ai_ask_openai_sends_correct_request - — Mock OpenAI endpoint - — Assert: POST with model, messages array - -test_ai_ask_ollama_sends_correct_request - — Mock Ollama endpoint - — Assert: POST to /api/generate - -test_ai_ask_with_context_includes_logs - — --context deploy → assert deployment context in prompt - -test_ai_dockerfile_generation - — Mock LLM returning Dockerfile - — Assert: valid result returned -``` - -**5 tests** - -**File: `tests/cli_proxy.rs`** - -``` -test_proxy_detect_finds_nginx - — Mock Docker API with nginx container - — Assert: "nginx detected" - -test_proxy_detect_finds_npm - — Mock with NPM container - — Assert: "Nginx Proxy Manager detected" - -test_proxy_detect_nothing - — Empty containers → "no proxy detected" - -test_proxy_add_domain_updates_config - — proxy add example.com --upstream app:3000 - — Assert: config updated - -test_proxy_add_invalid_domain_returns_error - — "not a domain" → validation error -``` - -**5 tests** - -**File: `tests/cli_update.rs`** - -``` -test_update_newer_version_available - — Mock release endpoint with v2.0.0 - — Assert: "new version available" - -test_update_already_latest - — Mock with same version - — Assert: "already up to date" - -test_update_channel_flag - — --channel nightly → assert checks nightly feed -``` - -**3 tests** - ---- - -### Test Fixtures - -**File: `tests/mock_data/stacker_minimal.yml`** -```yaml -name: test-app -app: - type: static - path: ./public -``` - -**File: `tests/mock_data/stacker_full.yml`** -```yaml -name: full-test-app -version: "2.0" -organization: test-org -app: - type: node - path: ./src - build: - context: . - args: - NODE_ENV: production -services: - - name: postgres - image: postgres:16 - ports: ["5432:5432"] - environment: - POSTGRES_DB: testdb - POSTGRES_PASSWORD: testpass - volumes: - - pgdata:/var/lib/postgresql/data - - name: redis - image: redis:7-alpine - ports: ["6379:6379"] -proxy: - type: nginx - domains: - - domain: test.example.com - ssl: auto - upstream: app:3000 -deploy: - target: local -ai: - enabled: true - provider: ollama - model: llama3 - endpoint: http://localhost:11434 - tasks: [dockerfile, troubleshoot] -monitoring: - status_panel: true - healthcheck: - endpoint: /health - interval: 30s -env: - APP_PORT: "3000" - LOG_LEVEL: debug -``` - -**File: `tests/mock_data/stacker_invalid.yml`** -```yaml -# Missing required name field -app: - type: cobol -``` - -**File: `tests/mock_data/stacker_cloud.yml`** -```yaml -name: cloud-test -app: - type: static - path: ./public -deploy: - target: cloud - cloud: - provider: hetzner - region: fsn1 - size: cpx21 - ssh_key: ~/.ssh/id_ed25519 -``` - ---- - -### Test Count Summary - -| Phase | Module | Tests | -|-------|--------|-------| -| 0 | error.rs + enums | 13 | -| 1 | config_parser + builder | 25 | -| 2 | detector | 13 | -| 3 | generator/dockerfile | 12 | -| 3 | generator/compose | 11 | -| 4 | credentials | 8 | -| 5 | proxy_manager | 6 | -| 6 | install_runner | 6 | -| 7 | ai_client | 6 | -| 8 | cli_login | 6 | -| 8 | cli_init | 8 | -| 8 | cli_deploy | 13 | -| 8 | cli_logs | 6 | -| 8 | cli_status | 4 | -| 8 | cli_destroy | 5 | -| 8 | cli_config | 5 | -| 8 | cli_ai | 5 | -| 8 | cli_proxy | 5 | -| 8 | cli_update | 3 | -| **Total** | | **160** | - ---- - -## Step-by-Step Implementation - -Each step: write tests → run (all fail) → implement → run (all pass) → refactor. - -### Step 1: Foundation — Error Types + Enums -1. Add dev-dependencies to `Cargo.toml` -2. Create `src/cli/mod.rs`, `src/cli/error.rs` -3. Define `CliError`, `AppType`, `DeployTarget`, `ProxyType`, `Severity`, `ValidationIssue` -4. Implement `Display`, `From`, `Default` for each -5. Write Phase 0 tests (13) → implement → green - -### Step 2: Config Parser + Builder -1. Create `src/cli/config_parser.rs` -2. Define `StackerConfig`, `AppSource`, `ServiceDefinition`, `ProxyConfig`, - `DeployConfig`, `AiConfig`, `MonitoringConfig`, `HookConfig` — all with - `Deserialize + Serialize + Default + Debug + Clone + Validate` -3. Define `ConfigBuilder` with fluent chaining -4. Implement `StackerConfig::from_file()` with env var interpolation -5. Implement `validate_semantics()` -6. Write Phase 1 tests (25) → implement → green - -### Step 3: Project Detector -1. Create `src/cli/detector.rs` -2. Define `FileSystem` trait + `MockFileSystem` + `RealFileSystem` -3. Define `ProjectDetection` struct -4. Implement `From<&ProjectDetection> for AppType` -5. Write Phase 2 tests (13) → implement → green - -### Step 4: Dockerfile Generator -1. Create `src/cli/generator/mod.rs`, `dockerfile.rs`, `templates.rs` -2. Define `DockerfileBuilder` with fluent builder pattern -3. Implement `From for DockerfileBuilder` -4. Write Phase 3 dockerfile tests (12) → implement → green - -### Step 5: Compose Generator -1. Create `src/cli/generator/compose.rs` -2. Define `ComposeDefinition`, `ComposeService` -3. Implement `TryFrom<&StackerConfig> for ComposeDefinition` -4. Implement `From<&ServiceDefinition> for ComposeService` -5. Write Phase 3 compose tests (11) → implement → green - -### Step 6: Credentials Manager -1. Create `src/cli/credentials.rs` -2. Define `Credentials` struct with `save()`, `load()`, `is_expired()`, `refresh()` -3. Write Phase 4 tests (8) → implement → green - -### Step 7: Proxy Manager -1. Create `src/cli/proxy_manager.rs` -2. Define `ProxyDetection`, nginx config generation -3. Implement `ContainerRuntime` trait dependency for detection -4. Write Phase 5 tests (6) → implement → green - -### Step 8: Install Runner -1. Create `src/cli/install_runner.rs` -2. Define `InstallContainerCommand` builder -3. Write Phase 6 tests (6) → implement → green - -### Step 9: AI Client -1. Create `src/cli/ai_client.rs` -2. Define `AiProvider` trait + `MockAiProvider` + `OpenAiProvider` + `OllamaProvider` -3. Implement prompt construction -4. Write Phase 7 tests (6) → implement → green - -### Step 10: CLI Command Stubs -1. Extend `Commands` enum in `src/console/main.rs` -2. Create `src/console/commands/cli/` with all command modules -3. Stub `CallableTrait` implementations (return `Ok(())`) -4. Verify `cargo build --features explain` - -### Step 11-21: Commands (one per step) -For each command (login, init, deploy, logs, status, destroy, config, ai, proxy, update): -1. Write integration tests from Phase 8 -2. Implement `CallableTrait` using library modules from Steps 1-9 -3. Green - -### Step 22: Binary Target + Distribution -1. Add `[[bin]] name = "stacker"` in `Cargo.toml` -2. Create `scripts/install.sh` for curl-based install -3. Update CI for multi-platform builds - ---- - -## Verification - -### Run all tests -```bash -cd stacker - -# All tests -cargo test --features explain - -# By phase -cargo test --features explain -- cli::error::tests # Phase 0 -cargo test --features explain -- cli::config_parser::tests # Phase 1 -cargo test --features explain -- cli::detector::tests # Phase 2 -cargo test --features explain -- cli::generator::dockerfile::tests # Phase 3 -cargo test --features explain -- cli::generator::compose::tests # Phase 3 -cargo test --features explain -- cli::credentials::tests # Phase 4 -cargo test --features explain -- cli::proxy_manager::tests # Phase 5 -cargo test --features explain -- cli::install_runner::tests # Phase 6 -cargo test --features explain -- cli::ai_client::tests # Phase 7 -cargo test --features explain --test cli_login # Phase 8 -cargo test --features explain --test cli_deploy # Phase 8 -# ... - -# Integration tests only -cargo test --features explain --test 'cli_*' -``` - -### Manual E2E smoke test -```bash -mkdir /tmp/test-site && echo "

Hello

" > /tmp/test-site/index.html -cd /tmp/test-site -stacker init -stacker config validate -stacker deploy --target local --dry-run -stacker deploy --target local -stacker status -stacker logs --follow -stacker destroy --confirm -``` - ---- - -## Decisions Log - -| Decision | Chosen | Over | Reason | -|----------|--------|------|--------| -| Builder pattern | `ConfigBuilder`, `DockerfileBuilder` with fluent chaining | Direct struct construction | Clean API for `stacker init`, testable, follows `JsonResponseBuilder` pattern in codebase | -| From/Into | `From for DockerfileBuilder`, `TryFrom<&StackerConfig> for ComposeDefinition`, etc. | Manual conversion functions | Idiomatic Rust, composable, follows 20+ existing From/Into impls in codebase | -| Error type | Single `CliError` enum with structured variants + `Display` + `From` | String-based errors | Clean Code Ch.7 — structured error handling. Follows `ConnectorError` pattern. No `thiserror` (codebase doesn't use it) | -| Trait abstraction | `ContainerRuntime`, `FileSystem`, `AiProvider` traits | Concrete types only | DIP — enables `MockContainerRuntime` etc. for testing. Follows `UserServiceConnector` mock pattern | -| Strategy pattern | `DeployStrategy` trait with `LocalDeploy`, `CloudDeploy`, `ServerDeploy` | Match arm in single function | OCP — new deploy targets don't modify existing code | -| Naming | Full intention-revealing names, no abbreviations | Short names | Clean Code Ch.2 — `generate_dockerfile` not `gen_df` | -| Function size | Single-purpose functions, one level of abstraction | Large orchestration functions | Clean Code Ch.3 — do one thing | -| Validation | `Validate` derive + `validate_semantics()` returning `Vec` | Boolean checks | Follows `serde_valid` pattern in codebase + `ValidateStackConfigTool` error/warning/info pattern | -| Test isolation | `MockFileSystem`, `MockContainerRuntime`, `MockAiProvider` | tempdir + real Docker | Fast, deterministic, no external dependencies. Integration tests use tempdir where needed | -| Config format | serde-derived structs with `#[serde(default)]` + `#[serde(rename_all)]` | Manual parsing | Follows every model/form in codebase. Declarative, auto-documented. | diff --git a/local-only-settings/stacker-deploy.yml b/local-only-settings/stacker-deploy.yml deleted file mode 100644 index 44581b9b..00000000 --- a/local-only-settings/stacker-deploy.yml +++ /dev/null @@ -1,23 +0,0 @@ -name: Deploy website with Stacker - -on: - push: - branches: [main] - workflow_dispatch: - -jobs: - deploy: - name: Deploy website - runs-on: ubuntu-latest - - steps: - - name: Checkout code - uses: actions/checkout@v4 - - - name: Install Stacker CLI - run: curl -fsSL https://get.try.direct/stacker | sh - - - name: Deploy stack - env: - STACKER_TOKEN: ${{ secrets.STACKER_TOKEN }} - run: stacker deploy --target server diff --git a/migrations/20261013090000_casbin_project_clouds_rule.down.sql b/migrations/20261013090000_casbin_project_clouds_rule.down.sql new file mode 100644 index 00000000..75948035 --- /dev/null +++ b/migrations/20261013090000_casbin_project_clouds_rule.down.sql @@ -0,0 +1 @@ +DELETE FROM public.casbin_rule WHERE ptype = 'p' AND v1 IN ('/project/:id/clouds', '/api/v1/project/:id/clouds'); diff --git a/migrations/20261013090000_casbin_project_clouds_rule.up.sql b/migrations/20261013090000_casbin_project_clouds_rule.up.sql new file mode 100644 index 00000000..cf4ef225 --- /dev/null +++ b/migrations/20261013090000_casbin_project_clouds_rule.up.sql @@ -0,0 +1,5 @@ +-- The project owner's saved clouds (no credentials), for the owner and members. +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_user', '/project/:id/clouds', 'GET', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_admin', '/project/:id/clouds', 'GET', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_user', '/api/v1/project/:id/clouds', 'GET', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_admin', '/api/v1/project/:id/clouds', 'GET', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; diff --git a/migrations/20261013120000_ai_chat_analysis_agreement.down.sql b/migrations/20261013120000_ai_chat_analysis_agreement.down.sql new file mode 100644 index 00000000..f4adf7c2 --- /dev/null +++ b/migrations/20261013120000_ai_chat_analysis_agreement.down.sql @@ -0,0 +1,5 @@ +DELETE FROM public.casbin_rule WHERE ptype = 'p' AND v1 = '/api/agreement/name/:name' AND v2 = 'GET'; +DELETE FROM public.casbin_rule +WHERE ptype = 'p' AND v0 = 'group_admin' AND v1 = '/api/admin/chat/recent' AND v2 = 'GET'; +DELETE FROM user_agreement WHERE agrt_id IN (SELECT id FROM agreement WHERE name = 'ai_chat_analysis'); +DELETE FROM agreement WHERE name = 'ai_chat_analysis'; diff --git a/migrations/20261013120000_ai_chat_analysis_agreement.up.sql b/migrations/20261013120000_ai_chat_analysis_agreement.up.sql new file mode 100644 index 00000000..1dee641b --- /dev/null +++ b/migrations/20261013120000_ai_chat_analysis_agreement.up.sql @@ -0,0 +1,28 @@ +-- Consent agreement for analysing AI chat conversations. Admin chat review +-- (GET /api/admin/chat/recent) only returns chats of users who accepted it. +INSERT INTO agreement (name, text, created_at, updated_at) +SELECT 'ai_chat_analysis', + 'Allow TryDirect to analyze my AI chat conversations to improve our AI assistant and services. Conversations are reviewed only by authorized TryDirect staff. You can withdraw this consent at any time by contacting support.', + NOW(), NOW() +WHERE NOT EXISTS (SELECT 1 FROM agreement WHERE name = 'ai_chat_analysis'); + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +SELECT 'p', 'group_admin', '/api/admin/chat/recent', 'GET', '', '', '' +WHERE NOT EXISTS ( + SELECT 1 FROM public.casbin_rule + WHERE ptype = 'p' AND v0 = 'group_admin' AND v1 = '/api/admin/chat/recent' AND v2 = 'GET' +); + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +SELECT 'p', 'group_user', '/api/agreement/name/:name', 'GET', '', '', '' +WHERE NOT EXISTS ( + SELECT 1 FROM public.casbin_rule + WHERE ptype = 'p' AND v0 = 'group_user' AND v1 = '/api/agreement/name/:name' AND v2 = 'GET' +); + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +SELECT 'p', 'group_admin', '/api/agreement/name/:name', 'GET', '', '', '' +WHERE NOT EXISTS ( + SELECT 1 FROM public.casbin_rule + WHERE ptype = 'p' AND v0 = 'group_admin' AND v1 = '/api/agreement/name/:name' AND v2 = 'GET' +); diff --git a/migrations/20261014100000_casbin_deployment_member_access.down.sql b/migrations/20261014100000_casbin_deployment_member_access.down.sql new file mode 100644 index 00000000..22e63ae3 --- /dev/null +++ b/migrations/20261014100000_casbin_deployment_member_access.down.sql @@ -0,0 +1,2 @@ +DELETE FROM public.casbin_rule +WHERE ptype = 'p' AND v1 = '/api/v1/deployments/:deployment_hash/member-access'; diff --git a/migrations/20261014100000_casbin_deployment_member_access.up.sql b/migrations/20261014100000_casbin_deployment_member_access.up.sql new file mode 100644 index 00000000..e358e191 --- /dev/null +++ b/migrations/20261014100000_casbin_deployment_member_access.up.sql @@ -0,0 +1,4 @@ +-- Service-to-service: the handler requires STACKER_SERVICE_TOKEN itself. +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'group_anonymous', '/api/v1/deployments/:deployment_hash/member-access', 'GET', '', '', '') +ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; diff --git a/migrations/20261014120000_project_members_server_access.down.sql b/migrations/20261014120000_project_members_server_access.down.sql new file mode 100644 index 00000000..7a0efcec --- /dev/null +++ b/migrations/20261014120000_project_members_server_access.down.sql @@ -0,0 +1,2 @@ +DELETE FROM public.casbin_rule WHERE ptype = 'p' AND v1 IN ('/project/:id/server-access', '/api/v1/project/:id/server-access'); +ALTER TABLE project DROP COLUMN IF EXISTS members_server_access; diff --git a/migrations/20261014120000_project_members_server_access.up.sql b/migrations/20261014120000_project_members_server_access.up.sql new file mode 100644 index 00000000..e80d955a --- /dev/null +++ b/migrations/20261014120000_project_members_server_access.up.sql @@ -0,0 +1,7 @@ +-- Team seats: the owner decides whether members get the servers' SSH key and +-- passwords (the deployment report). Off by default. +ALTER TABLE project ADD COLUMN IF NOT EXISTS members_server_access BOOLEAN NOT NULL DEFAULT FALSE; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_user', '/project/:id/server-access', 'PATCH', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_user', '/api/v1/project/:id/server-access', 'PATCH', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_admin', '/project/:id/server-access', 'PATCH', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_admin', '/api/v1/project/:id/server-access', 'PATCH', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; diff --git a/migrations/20261015100000_bake_job.down.sql b/migrations/20261015100000_bake_job.down.sql new file mode 100644 index 00000000..2d6d1e24 --- /dev/null +++ b/migrations/20261015100000_bake_job.down.sql @@ -0,0 +1,2 @@ +DELETE FROM public.casbin_rule WHERE ptype = 'p' AND v1 IN ('/api/admin/bake', '/api/admin/bake/:id'); +DROP TABLE IF EXISTS bake_job; diff --git a/migrations/20261015100000_bake_job.up.sql b/migrations/20261015100000_bake_job.up.sql new file mode 100644 index 00000000..8d0382b9 --- /dev/null +++ b/migrations/20261015100000_bake_job.up.sql @@ -0,0 +1,23 @@ +-- Bake started from the API: one row per POST /api/admin/bake. +CREATE TABLE IF NOT EXISTS bake_job ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + stack TEXT NOT NULL, + version TEXT NOT NULL, + deployment_id INTEGER NOT NULL, + server_id INTEGER NOT NULL, + server_ip TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'running' CHECK (status IN ('running', 'done', 'failed')), + image_id BIGINT, + snapshot_id INTEGER, + error TEXT, + created_by TEXT NOT NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS bake_job_created_at_idx ON bake_job (created_at DESC); + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_admin', '/api/admin/bake', 'POST', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_admin', '/api/admin/bake/:id', 'GET', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'admin_service', '/api/admin/bake', 'POST', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'admin_service', '/api/admin/bake/:id', 'GET', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; diff --git a/migrations/20261016100000_project_app_runtime.down.sql b/migrations/20261016100000_project_app_runtime.down.sql new file mode 100644 index 00000000..96bf01cb --- /dev/null +++ b/migrations/20261016100000_project_app_runtime.down.sql @@ -0,0 +1 @@ +ALTER TABLE project_app DROP COLUMN IF EXISTS runtime; diff --git a/migrations/20261016100000_project_app_runtime.up.sql b/migrations/20261016100000_project_app_runtime.up.sql new file mode 100644 index 00000000..bc057b72 --- /dev/null +++ b/migrations/20261016100000_project_app_runtime.up.sql @@ -0,0 +1,4 @@ +-- Compose settings of an app beyond its image and wiring: cap_add, cap_drop, +-- devices, profiles, aliases, deploy (replicas, device reservations). +-- See shared-fixtures/api-contracts/stack-service-runtime-fields.json. +ALTER TABLE project_app ADD COLUMN IF NOT EXISTS runtime JSONB; diff --git a/migrations/20261016120000_baked_snapshot_server_type.down.sql b/migrations/20261016120000_baked_snapshot_server_type.down.sql new file mode 100644 index 00000000..1a40f261 --- /dev/null +++ b/migrations/20261016120000_baked_snapshot_server_type.down.sql @@ -0,0 +1 @@ +ALTER TABLE baked_snapshots DROP COLUMN IF EXISTS server_type; diff --git a/migrations/20261016120000_baked_snapshot_server_type.up.sql b/migrations/20261016120000_baked_snapshot_server_type.up.sql new file mode 100644 index 00000000..b0326830 --- /dev/null +++ b/migrations/20261016120000_baked_snapshot_server_type.up.sql @@ -0,0 +1,4 @@ +-- Server type of the build box a snapshot was baked on. Quick Deploy clones the +-- snapshot onto that type and prices it at that type's rate. +-- See shared-fixtures/api-contracts/quick-deploy-server-type.json. +ALTER TABLE baked_snapshots ADD COLUMN IF NOT EXISTS server_type TEXT; diff --git a/migrations/20261016180000_managed_usage.down.sql b/migrations/20261016180000_managed_usage.down.sql new file mode 100644 index 00000000..9850a185 --- /dev/null +++ b/migrations/20261016180000_managed_usage.down.sql @@ -0,0 +1 @@ +DROP TABLE IF EXISTS managed_usage; diff --git a/migrations/20261016180000_managed_usage.up.sql b/migrations/20261016180000_managed_usage.up.sql new file mode 100644 index 00000000..09b57a9f --- /dev/null +++ b/migrations/20261016180000_managed_usage.up.sql @@ -0,0 +1,34 @@ +-- Daily usage of the Managed server behind a Quick Deploy: one row per +-- deployment. The sweeper reports each running day to the Stripe meter +-- through the User Service and records here how far it got. +-- Contract: config/shared-fixtures/api-contracts/managed-server-billing.json. +CREATE TABLE IF NOT EXISTS managed_usage ( + id BIGSERIAL PRIMARY KEY, + deployment_hash VARCHAR(120) NOT NULL UNIQUE, + user_id VARCHAR(64) NOT NULL, + stack VARCHAR(255) NOT NULL, + server_type VARCHAR(50) NOT NULL, + daily_rate_minor BIGINT NOT NULL CHECK (daily_rate_minor > 0), + monthly_cap_minor BIGINT NOT NULL CHECK (monthly_cap_minor > 0), + currency CHAR(3) NOT NULL DEFAULT 'USD', + -- Hetzner server id, set when the clone creates the VM; admins delete the + -- VM by hand from the alert (nothing deletes it automatically). + server_id BIGINT, + -- pending: deploy running; active: billed daily; stopped: server deleted; + -- suspended: invoice unpaid; closed: deploy failed, or unpaid and handed + -- to admins for deletion + status VARCHAR(16) NOT NULL DEFAULT 'pending' + CHECK (status IN ('pending', 'active', 'stopped', 'suspended', 'closed')), + outcome VARCHAR(40), + started_at TIMESTAMPTZ, + stopped_at TIMESTAMPTZ, + suspended_at TIMESTAMPTZ, + period_start DATE, + period_reported_minor BIGINT NOT NULL DEFAULT 0, + last_reported_on DATE, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT now() +); + +CREATE INDEX IF NOT EXISTS ix_managed_usage_status ON managed_usage (status, last_reported_on); +CREATE INDEX IF NOT EXISTS ix_managed_usage_user ON managed_usage (user_id, status); diff --git a/migrations/20261016181000_casbin_managed_usage_internal.down.sql b/migrations/20261016181000_casbin_managed_usage_internal.down.sql new file mode 100644 index 00000000..fcc6682f --- /dev/null +++ b/migrations/20261016181000_casbin_managed_usage_internal.down.sql @@ -0,0 +1,6 @@ +DELETE FROM public.casbin_rule WHERE ptype = 'p' AND v0 = 'group_anonymous' AND v1 IN ( + '/api/internal/managed-usage/:hash/start', + '/api/internal/managed-usage/:hash/fail', + '/api/internal/managed-usage/:hash/stop', + '/api/internal/managed-usage/suspend' +) AND v2 = 'POST'; diff --git a/migrations/20261016181000_casbin_managed_usage_internal.up.sql b/migrations/20261016181000_casbin_managed_usage_internal.up.sql new file mode 100644 index 00000000..2f535791 --- /dev/null +++ b/migrations/20261016181000_casbin_managed_usage_internal.up.sql @@ -0,0 +1,7 @@ +-- Internal Managed server usage endpoints, called by the User Service with +-- no user token. Casbin only makes them reachable; the handlers authorise +-- with require_internal_key (x-internal-key). +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_anonymous', '/api/internal/managed-usage/:hash/start', 'POST', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_anonymous', '/api/internal/managed-usage/:hash/fail', 'POST', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_anonymous', '/api/internal/managed-usage/suspend', 'POST', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'group_anonymous', '/api/internal/managed-usage/:hash/stop', 'POST', '', '', '') ON CONFLICT ON CONSTRAINT unique_key_sqlx_adapter DO NOTHING; diff --git a/migrations/20261016191200_casbin_admin_template_update.down.sql b/migrations/20261016191200_casbin_admin_template_update.down.sql new file mode 100644 index 00000000..8dc2be65 --- /dev/null +++ b/migrations/20261016191200_casbin_admin_template_update.down.sql @@ -0,0 +1,5 @@ +DELETE FROM public.casbin_rule +WHERE ptype = 'p' + AND v0 IN ('admin_service', 'group_admin', 'root') + AND v1 IN ('/api/admin/templates/:id', '/stacker/admin/templates/:id') + AND v2 = 'PUT'; diff --git a/migrations/20261016191200_casbin_admin_template_update.up.sql b/migrations/20261016191200_casbin_admin_template_update.up.sql new file mode 100644 index 00000000..b3c7b20d --- /dev/null +++ b/migrations/20261016191200_casbin_admin_template_update.up.sql @@ -0,0 +1,26 @@ +-- Allow admins to edit a template's content: PUT /admin/templates/:id +-- Mirrors the existing per-field admin rules (pricing, verifications) across +-- both gateway prefixes and all three admin roles. +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'admin_service', '/api/admin/templates/:id', 'PUT', '', '', '') +ON CONFLICT DO NOTHING; + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'group_admin', '/api/admin/templates/:id', 'PUT', '', '', '') +ON CONFLICT DO NOTHING; + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'root', '/api/admin/templates/:id', 'PUT', '', '', '') +ON CONFLICT DO NOTHING; + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'admin_service', '/stacker/admin/templates/:id', 'PUT', '', '', '') +ON CONFLICT DO NOTHING; + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'group_admin', '/stacker/admin/templates/:id', 'PUT', '', '', '') +ON CONFLICT DO NOTHING; + +INSERT INTO public.casbin_rule (ptype, v0, v1, v2, v3, v4, v5) +VALUES ('p', 'root', '/stacker/admin/templates/:id', 'PUT', '', '', '') +ON CONFLICT DO NOTHING; diff --git a/src/bin/bake.rs b/src/bin/bake.rs index b441c195..7b7d78fb 100644 --- a/src/bin/bake.rs +++ b/src/bin/bake.rs @@ -267,6 +267,7 @@ async fn main() -> Result<(), Box> { None, config_contract, required_env_keys, + None, ) .await .map_err(|e| e.to_string())?; diff --git a/src/cli/compose_service_sync.rs b/src/cli/compose_service_sync.rs index 17541f0d..3fc6321f 100644 --- a/src/cli/compose_service_sync.rs +++ b/src/cli/compose_service_sync.rs @@ -375,13 +375,56 @@ fn service_to_compose_value( serde_yaml::Value::String(platform.clone()), ); } - if !project_networks.is_empty() { - insert_string_sequence(&mut map, "networks", project_networks); + let runtime = &service.runtime; + if runtime.aliases.is_empty() { + if !project_networks.is_empty() { + insert_string_sequence(&mut map, "networks", project_networks); + } + } else { + // Aliases need the long form, on every network the service joins. + let default_network = ["default".to_string()]; + let networks = if project_networks.is_empty() { + &default_network[..] + } else { + project_networks + }; + let mut settings = serde_yaml::Mapping::new(); + insert_string_sequence(&mut settings, "aliases", &runtime.aliases); + let networks: serde_yaml::Mapping = networks + .iter() + .map(|n| { + ( + serde_yaml::Value::String(n.clone()), + serde_yaml::Value::Mapping(settings.clone()), + ) + }) + .collect(); + map.insert( + serde_yaml::Value::String("networks".to_string()), + serde_yaml::Value::Mapping(networks), + ); } map.insert( serde_yaml::Value::String("restart".to_string()), - serde_yaml::Value::String("unless-stopped".to_string()), + serde_yaml::Value::String( + runtime + .restart + .map_or("unless-stopped", |r| r.as_str()) + .to_string(), + ), ); + if let Some(entrypoint) = &runtime.entrypoint { + map.insert( + serde_yaml::Value::String("entrypoint".to_string()), + serde_yaml::Value::String(entrypoint.clone()), + ); + } + insert_string_sequence(&mut map, "profiles", &runtime.profiles); + if let Some(deploy) = &runtime.deploy { + if let Ok(value) = serde_yaml::to_value(deploy) { + map.insert(serde_yaml::Value::String("deploy".to_string()), value); + } + } serde_yaml::Value::Mapping(map) } @@ -732,6 +775,7 @@ mod tests { config: None, }, services: vec![ServiceDefinition { + runtime: Default::default(), name: service_name.to_string(), image: "myapp:latest".to_string(), ports: vec!["3000:3000".to_string()], @@ -787,6 +831,7 @@ mod tests { // proxy points to "api" but we are syncing "smtp" let mut config = npm_stacker_config(dir.path(), "api"); config.services = vec![ServiceDefinition { + runtime: Default::default(), name: "smtp".to_string(), image: "trydirect/smtp".to_string(), ports: vec![], @@ -852,6 +897,7 @@ volumes: ..Default::default() }, services: vec![ServiceDefinition { + runtime: Default::default(), name: "smtp".to_string(), image: "trydirect/smtp".to_string(), ports: vec!["1025:25".to_string()], diff --git a/src/cli/compose_targets.rs b/src/cli/compose_targets.rs index 6891490a..f19a1a87 100644 --- a/src/cli/compose_targets.rs +++ b/src/cli/compose_targets.rs @@ -175,6 +175,7 @@ fn collect_compose_services( }; services.push(ServiceDefinition { + runtime: Default::default(), name: service_name.to_string(), image, ports: mapping_sequence(definition, "ports") diff --git a/src/cli/config_parser.rs b/src/cli/config_parser.rs index 2a3964da..92252191 100644 --- a/src/cli/config_parser.rs +++ b/src/cli/config_parser.rs @@ -244,7 +244,7 @@ pub struct AppSource { pub cap_add: Vec, #[serde(default)] pub cap_drop: Vec, - #[serde(default)] + #[serde(default, deserialize_with = "deserialize_flag")] pub privileged: bool, #[serde(default)] pub platform: Option, @@ -293,14 +293,17 @@ fn default_health_retries_compose() -> u32 { /// /// This is distinct from `MonitoringConfig::healthcheck`, which is an /// app-level HTTP endpoint polling configuration. -#[derive(Debug, Clone, Serialize, Deserialize, Default)] +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Default)] pub struct ComposeHealthcheck { pub test: String, #[serde(default = "default_health_interval")] pub interval: String, #[serde(default = "default_health_timeout_compose")] pub timeout: String, - #[serde(default = "default_health_retries_compose")] + #[serde( + default = "default_health_retries_compose", + deserialize_with = "deserialize_number" + )] pub retries: u32, } @@ -334,12 +337,271 @@ pub struct ServiceDefinition { pub cap_add: Vec, #[serde(default)] pub cap_drop: Vec, - #[serde(default)] + #[serde(default, deserialize_with = "deserialize_flag")] pub privileged: bool, #[serde(default)] pub platform: Option, #[serde(default)] pub devices: Vec, + + /// restart, entrypoint, profiles, aliases, deploy. + #[serde(flatten)] + pub runtime: ServiceRuntime, +} + +/// How a service runs, beyond its image and wiring. Each maps to the compose +/// key of the same name (`aliases` to `networks..aliases`); left +/// unset, the compose keeps its defaults. +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct ServiceRuntime { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub restart: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub entrypoint: Option, + /// The service starts only when one of these compose profiles is active. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub profiles: Vec, + /// Other names the service answers to on every network it joins. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub aliases: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub deploy: Option, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum RestartPolicy { + No, + Always, + UnlessStopped, + OnFailure, +} + +impl RestartPolicy { + pub fn as_str(&self) -> &'static str { + match self { + RestartPolicy::No => "no", + RestartPolicy::Always => "always", + RestartPolicy::UnlessStopped => "unless-stopped", + RestartPolicy::OnFailure => "on-failure", + } + } +} + +/// The subset of compose `deploy:` a single host honours. +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ServiceDeploy { + /// `0` keeps the service defined but not started. Also read from a + /// `${VAR}` reference: filled, it arrives as a string; read without + /// filling variables, the reference is kept as written. + #[serde( + default, + deserialize_with = "deserialize_optional_count", + skip_serializing_if = "Option::is_none" + )] + pub replicas: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub resources: Option, +} + +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ServiceResources { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub reservations: Option, +} + +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ServiceReservations { + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub devices: Vec, +} + +/// A device the container reserves, e.g. an NVIDIA GPU: +/// `{driver: nvidia, count: 1, capabilities: [gpu]}`. +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct DeviceRequest { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub driver: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub count: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub device_ids: Vec, + pub capabilities: Vec, +} + +/// A count: a whole number, or a `${VAR}` reference the deploy fills in. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Count { + Value(u32), + Reference(String), +} + +impl Count { + /// The number, when it is known before the deploy fills variables. + pub fn value(&self) -> Option { + match self { + Count::Value(n) => Some(*n), + Count::Reference(_) => None, + } + } +} + +impl Serialize for Count { + fn serialize(&self, serializer: S) -> Result { + match self { + Count::Value(n) => serializer.serialize_u32(*n), + Count::Reference(r) => serializer.serialize_str(r), + } + } +} + +/// `count: 2`, `count: all` or `count: ${VAR}`. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DeviceCount { + Count(Count), + All, +} + +impl Serialize for DeviceCount { + fn serialize(&self, serializer: S) -> Result { + match self { + DeviceCount::Count(c) => c.serialize(serializer), + DeviceCount::All => serializer.serialize_str("all"), + } + } +} + +impl<'de> Deserialize<'de> for DeviceCount { + fn deserialize>(deserializer: D) -> Result { + match serde_yaml::Value::deserialize(deserializer)? { + serde_yaml::Value::String(s) if s == "all" => Ok(DeviceCount::All), + other => count_from_value(&other) + .map(DeviceCount::Count) + .ok_or_else(|| serde::de::Error::custom("count must be a number or `all`")), + } + } +} + +fn count_from_value(value: &serde_yaml::Value) -> Option { + match value { + serde_yaml::Value::Number(n) => n + .as_u64() + .and_then(|n| u32::try_from(n).ok()) + .map(Count::Value), + serde_yaml::Value::String(s) if is_variable_reference(s.trim()) => { + Some(Count::Reference(s.trim().to_string())) + } + serde_yaml::Value::String(s) => s.trim().parse().ok().map(Count::Value), + _ => None, + } +} + +/// A boolean, also read from the string a filled `${VAR}` leaves behind. +fn deserialize_flag<'de, D: Deserializer<'de>>(deserializer: D) -> Result { + match serde_yaml::Value::deserialize(deserializer)? { + serde_yaml::Value::Bool(b) => Ok(b), + serde_yaml::Value::String(s) if s.trim() == "true" => Ok(true), + serde_yaml::Value::String(s) if s.trim() == "false" => Ok(false), + other => Err(serde::de::Error::custom(format!( + "expected true or false, got {}", + scalar_text(&other) + ))), + } +} + +/// A whole number, also read from the string a filled `${VAR}` leaves behind. +fn number_from_value>(value: &serde_yaml::Value) -> Option { + let n = match value { + serde_yaml::Value::Number(n) => n.as_u64()?, + serde_yaml::Value::String(s) => s.trim().parse().ok()?, + _ => return None, + }; + T::try_from(n).ok() +} + +fn deserialize_number<'de, D, T>(deserializer: D) -> Result +where + D: Deserializer<'de>, + T: TryFrom, +{ + let value = serde_yaml::Value::deserialize(deserializer)?; + number_from_value(&value).ok_or_else(|| { + serde::de::Error::custom(format!( + "expected a whole number, got {}", + scalar_text(&value) + )) + }) +} + +fn deserialize_optional_number<'de, D, T>(deserializer: D) -> Result, D::Error> +where + D: Deserializer<'de>, + T: TryFrom, +{ + match Option::::deserialize(deserializer)? { + None | Some(serde_yaml::Value::Null) => Ok(None), + Some(value) => number_from_value(&value).map(Some).ok_or_else(|| { + serde::de::Error::custom(format!( + "expected a whole number, got {}", + scalar_text(&value) + )) + }), + } +} + +fn scalar_text(value: &serde_yaml::Value) -> String { + match value { + serde_yaml::Value::String(s) => format!("`{s}`"), + other => serde_yaml::to_string(other) + .map(|s| format!("`{}`", s.trim())) + .unwrap_or_else(|_| "a non-scalar value".to_string()), + } +} + +/// `${NAME}`, `${NAME:-default}` (and the other compose modifiers) or `$NAME`. +fn is_variable_reference(value: &str) -> bool { + let name = match value.strip_prefix("${") { + Some(rest) => match rest.strip_suffix('}') { + Some(inner) => { + let end = inner + .find(|c: char| !(c == '_' || c.is_ascii_alphanumeric())) + .unwrap_or(inner.len()); + let modifier = &inner[end..]; + if !(modifier.is_empty() + || [":-", "-", ":?", "?", ":+", "+"] + .iter() + .any(|m| modifier.starts_with(m))) + { + return false; + } + &inner[..end] + } + None => return false, + }, + None => match value.strip_prefix('$') { + Some(name) => name, + None => return false, + }, + }; + let mut chars = name.chars(); + matches!(chars.next(), Some(c) if c == '_' || c.is_ascii_alphabetic()) + && chars.all(|c| c == '_' || c.is_ascii_alphanumeric()) +} + +fn deserialize_optional_count<'de, D>(deserializer: D) -> Result, D::Error> +where + D: Deserializer<'de>, +{ + match Option::::deserialize(deserializer)? { + None | Some(serde_yaml::Value::Null) => Ok(None), + Some(value) => count_from_value(&value) + .map(Some) + .ok_or_else(|| serde::de::Error::custom("replicas must be a whole number, 0 or more")), + } } fn deserialize_services<'de, D>(deserializer: D) -> Result, D::Error> @@ -399,7 +661,7 @@ pub struct ProxyConfig { #[serde(rename = "type", default)] pub proxy_type: ProxyType, - #[serde(default = "default_auto_detect")] + #[serde(default = "default_auto_detect", deserialize_with = "deserialize_flag")] pub auto_detect: bool, #[serde(default)] @@ -450,14 +712,30 @@ pub struct PipeSpec { #[serde(default = "default_pipe_trigger")] pub trigger: String, /// Poll interval (seconds) when trigger = poll. - #[serde(default, skip_serializing_if = "Option::is_none")] + #[serde( + default, + skip_serializing_if = "Option::is_none", + deserialize_with = "deserialize_optional_number" + )] pub poll_interval: Option, /// Max delivery retries (→ pipe config). - #[serde(default, skip_serializing_if = "Option::is_none")] + #[serde( + default, + skip_serializing_if = "Option::is_none", + deserialize_with = "deserialize_optional_number" + )] pub retry: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] + #[serde( + default, + skip_serializing_if = "Option::is_none", + deserialize_with = "deserialize_optional_number" + )] pub retry_backoff_ms: Option, - #[serde(default, skip_serializing_if = "Option::is_none")] + #[serde( + default, + skip_serializing_if = "Option::is_none", + deserialize_with = "deserialize_optional_number" + )] pub retry_backoff_max_ms: Option, /// Run another pipe (by name) on failure / success. #[serde(default, skip_serializing_if = "Option::is_none")] @@ -746,7 +1024,7 @@ pub struct ServerConfig { #[serde(default)] pub ssh_key: Option, - #[serde(default = "default_ssh_port")] + #[serde(default = "default_ssh_port", deserialize_with = "deserialize_number")] pub port: u16, } @@ -780,7 +1058,7 @@ fn default_ai_timeout() -> u64 { /// AI/LLM assistant configuration. #[derive(Debug, Clone, Serialize, Deserialize, Default)] pub struct AiConfig { - #[serde(default)] + #[serde(default, deserialize_with = "deserialize_flag")] pub enabled: bool, #[serde(default)] @@ -797,7 +1075,10 @@ pub struct AiConfig { /// Request timeout in seconds. Default: 300 (5 minutes). /// Can be overridden via `STACKER_AI_TIMEOUT` env var. - #[serde(default = "default_ai_timeout")] + #[serde( + default = "default_ai_timeout", + deserialize_with = "deserialize_number" + )] pub timeout: u64, #[serde(default)] @@ -807,7 +1088,7 @@ pub struct AiConfig { /// Monitoring and health check configuration. #[derive(Debug, Clone, Serialize, Deserialize, Default)] pub struct MonitoringConfig { - #[serde(default)] + #[serde(default, deserialize_with = "deserialize_flag")] pub status_panel: bool, #[serde(default)] @@ -827,12 +1108,15 @@ pub struct MonitoringConfig { #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct AlertConfig { /// Poll interval in seconds for the watch loop (default 60). - #[serde(default = "default_alert_interval")] + #[serde( + default = "default_alert_interval", + deserialize_with = "deserialize_number" + )] pub interval: u64, /// Where to deliver the alert (required). pub target: AlertTarget, /// Also notify when containers recover to healthy (default true). - #[serde(default = "default_true")] + #[serde(default = "default_true", deserialize_with = "deserialize_flag")] pub on_recovery: bool, } @@ -896,10 +1180,10 @@ fn default_health_interval() -> String { /// Metrics collection settings. #[derive(Debug, Clone, Serialize, Deserialize, Default)] pub struct MetricsConfig { - #[serde(default)] + #[serde(default, deserialize_with = "deserialize_flag")] pub enabled: bool, - #[serde(default)] + #[serde(default, deserialize_with = "deserialize_flag")] pub telegraf: bool, } @@ -916,7 +1200,11 @@ pub struct HookConfig { pub on_failure: Option, /// Send a terminal/desktop notification on deploy completion. - #[serde(default, skip_serializing_if = "is_false")] + #[serde( + default, + skip_serializing_if = "is_false", + deserialize_with = "deserialize_flag" + )] pub notify: bool, } @@ -2126,7 +2414,16 @@ fn validate_deploy_semantics( // dropped during cloud firewall provisioning. Bare numbers and // "port/proto" specs are both accepted. for port in &cloud.public_ports { - if let Err(err) = crate::forms::firewall::normalize_public_port(port) { + // A `${VAR}` port is only known once the deploy fills it; the + // protocol after it is still checked. + let checked = match port.split_once('/') { + Some((number, protocol)) if is_variable_reference(number.trim()) => { + format!("1/{protocol}") + } + None if is_variable_reference(port.trim()) => "1".to_string(), + _ => port.clone(), + }; + if let Err(err) = crate::forms::firewall::normalize_public_port(&checked) { issues.push(ValidationIssue { severity: Severity::Error, code: "E005".to_string(), @@ -2208,6 +2505,9 @@ fn load_env_file_vars_from_yaml(path: &Path, raw_content: &str) -> HashMap `"80"`, `"80:80/tcp"` -> `"80"`. /// Validate one numeric port, rejecting anything outside Docker's 1-65535. fn validate_port_number(value: &str) -> Result<(), String> { + if value == VARIABLE_MASK { + return Ok(()); + } let parsed: u64 = value .parse() .map_err(|_| format!("'{value}' is not a port number"))?; @@ -2220,6 +2520,25 @@ fn validate_port_number(value: &str) -> Result<(), String> { Ok(()) } +/// Stands in for a `${VAR}` while a port spec is checked. +const VARIABLE_MASK: &str = "__variable__"; + +/// Replace each `${...}` in `spec` with [`VARIABLE_MASK`]. +fn mask_variable_references(spec: &str) -> String { + let mut out = String::with_capacity(spec.len()); + let mut rest = spec; + while let Some(start) = rest.find("${") { + let Some(len) = rest[start..].find('}') else { + break; + }; + out.push_str(&rest[..start]); + out.push_str(VARIABLE_MASK); + rest = &rest[start + len + 1..]; + } + out.push_str(rest); + out +} + /// Validate one side of a mapping, which may be a range (`8000-8010`). fn validate_port_segment(segment: &str) -> Result<(), String> { match segment.split_once('-') { @@ -2247,6 +2566,10 @@ fn validate_port_mapping(spec: &str) -> Result<(), String> { if spec.is_empty() { return Err("port mapping is empty".to_string()); } + // A `${VAR}` part is only known once the deploy fills it, and its + // `:-default` would split like a mapping. Check the rest without it. + let masked = mask_variable_references(spec); + let spec = masked.as_str(); let (mapping, protocol) = match spec.rsplit_once('/') { Some((mapping, proto)) => (mapping, Some(proto)), @@ -4180,6 +4503,7 @@ app: #[test] fn e007_names_the_offending_service() { let svc = ServiceDefinition { + runtime: Default::default(), name: "db".to_string(), image: "postgres:16".to_string(), ports: vec!["5432:5432".to_string(), "99999:5432".to_string()], @@ -4691,6 +5015,7 @@ deploy: .app_type(AppType::Node) .app_path("./src") .add_service(ServiceDefinition { + runtime: Default::default(), name: "postgres".to_string(), image: "postgres:16".to_string(), ports: vec!["5432:5432".to_string()], @@ -4770,6 +5095,7 @@ deploy: let config = ConfigBuilder::new() .name("multi") .add_service(ServiceDefinition { + runtime: Default::default(), name: "pg".to_string(), image: "postgres:16".to_string(), ports: vec![], @@ -4785,6 +5111,7 @@ deploy: devices: Vec::new(), }) .add_service(ServiceDefinition { + runtime: Default::default(), name: "redis".to_string(), image: "redis:7".to_string(), ports: vec![], @@ -4800,6 +5127,7 @@ deploy: devices: Vec::new(), }) .add_service(ServiceDefinition { + runtime: Default::default(), name: "minio".to_string(), image: "minio/minio".to_string(), ports: vec![], diff --git a/src/cli/generator/compose.rs b/src/cli/generator/compose.rs index 3bf8e6fd..0a5fe3cb 100644 --- a/src/cli/generator/compose.rs +++ b/src/cli/generator/compose.rs @@ -5,7 +5,8 @@ use std::fmt; use std::path::Path; use crate::cli::config_parser::{ - AppType, ComposeHealthcheck, DomainConfig, ProxyType, ServiceDefinition, SslMode, StackerConfig, + AppType, ComposeHealthcheck, DomainConfig, ProxyType, ServiceDefinition, ServiceDeploy, + SslMode, StackerConfig, }; use crate::cli::error::CliError; @@ -41,6 +42,11 @@ pub struct ComposeService { pub shm_size: Option, /// Container `user:` (e.g. `"0:0"`). pub user: Option, + pub entrypoint: Option, + pub profiles: Vec, + /// Added as `aliases:` on every network in `networks`. + pub aliases: Vec, + pub deploy: Option, } impl Default for ComposeService { @@ -67,6 +73,10 @@ impl Default for ComposeService { devices: Vec::new(), shm_size: None, user: None, + entrypoint: None, + profiles: Vec::new(), + aliases: Vec::new(), + deploy: None, } } } @@ -88,8 +98,15 @@ impl From<&ServiceDefinition> for ComposeService { privileged: svc.privileged, platform: svc.platform.clone(), devices: svc.devices.clone(), + entrypoint: svc.runtime.entrypoint.clone(), + profiles: svc.runtime.profiles.clone(), + aliases: svc.runtime.aliases.clone(), + deploy: svc.runtime.deploy.clone(), ..Default::default() }; + if let Some(restart) = svc.runtime.restart { + compose_service.restart = restart.as_str().to_string(); + } crate::helpers::stacker_labels::insert_runtime_labels( &mut compose_service.labels, None::, @@ -712,6 +729,7 @@ impl ComposeDefinition { ("cap_add", &svc.cap_add), ("cap_drop", &svc.cap_drop), ("devices", &svc.devices), + ("profiles", &svc.profiles), ] { if !values.is_empty() { out.push_str(&format!(" {key}:\n")); @@ -741,6 +759,10 @@ impl ComposeDefinition { out.push_str(&format!(" command: {}\n", yaml_quote(cmd))); } + if let Some(ref entrypoint) = svc.entrypoint { + out.push_str(&format!(" entrypoint: {}\n", yaml_quote(entrypoint))); + } + if !svc.volumes.is_empty() { out.push_str(" volumes:\n"); for v in &svc.volumes { @@ -766,12 +788,32 @@ impl ComposeDefinition { out.push_str(&format!(" retries: {}\n", hc.retries)); } - out.push_str(&format!(" restart: {}\n", svc.restart)); + // `no` unquoted is a boolean to YAML 1.1 readers. + if svc.restart == "no" { + out.push_str(" restart: \"no\"\n"); + } else { + out.push_str(&format!(" restart: {}\n", svc.restart)); + } + + if let Some(ref deploy) = svc.deploy { + out.push_str(" deploy:\n"); + let body = serde_yaml::to_string(deploy).unwrap_or_default(); + for line in body.lines().filter(|l| !l.is_empty() && *l != "---") { + out.push_str(&format!(" {}\n", line)); + } + } if !svc.networks.is_empty() { out.push_str(" networks:\n"); for n in &svc.networks { - out.push_str(&format!(" - {}\n", n)); + if svc.aliases.is_empty() { + out.push_str(&format!(" - {}\n", n)); + } else { + out.push_str(&format!(" {}:\n aliases:\n", n)); + for alias in &svc.aliases { + out.push_str(&format!(" - {}\n", yaml_quote(alias))); + } + } } } @@ -1503,6 +1545,7 @@ services: config.app_present = false; // simulate no declared app: section config.app.image = Some("myorg/app:1.0".to_string()); config.services = vec![ServiceDefinition { + runtime: Default::default(), name: "db".to_string(), image: "postgres:16".to_string(), ports: vec![], @@ -1551,6 +1594,7 @@ services: #[test] fn test_compose_includes_additional_services() { let svc = ServiceDefinition { + runtime: Default::default(), name: "postgres".into(), image: "postgres:16".into(), ports: vec!["5432:5432".into()], @@ -1883,6 +1927,7 @@ services: #[test] fn test_compose_render_contains_volumes_section() { let svc = ServiceDefinition { + runtime: Default::default(), name: "redis".into(), image: "redis:7".into(), ports: Vec::new(), @@ -1989,6 +2034,7 @@ services: #[test] fn test_service_definition_to_compose_service() { let svc_def = ServiceDefinition { + runtime: Default::default(), name: "mysql".into(), image: "mysql:8".into(), ports: vec!["3306:3306".into()], @@ -2020,6 +2066,7 @@ services: #[test] fn service_definition_adds_project_scope_labels() { let svc_def = ServiceDefinition { + runtime: Default::default(), name: "smtp".into(), image: "trydirect/smtp:latest".into(), ports: Vec::new(), @@ -2282,6 +2329,7 @@ services: #[test] fn npm_proxy_injects_default_network_into_proxied_service() { let svc = ServiceDefinition { + runtime: Default::default(), name: "api".into(), image: "myapp:latest".into(), ports: vec!["8080:8080".into()], @@ -2336,6 +2384,7 @@ services: #[test] fn npm_proxy_does_not_inject_into_unproxied_service() { let smtp = ServiceDefinition { + runtime: Default::default(), name: "smtp".into(), image: "trydirect/smtp".into(), ports: vec![], @@ -2423,6 +2472,7 @@ services: // proxied service must also join `default_network` for the managed // proxy to reach it. (Previously only NginxProxyManager did this.) let svc = ServiceDefinition { + runtime: Default::default(), name: "web".into(), image: "nginx:latest".into(), ports: vec![], @@ -2494,7 +2544,6 @@ services: ); } - #[test] /// A container that insists on its own variable name is fed from one the /// author defined: `MYSQL_PASSWORD: "${DB_PASSWORD}"`. Emitting /// `${MYSQL_PASSWORD}` there names something no `.env` defines — compose @@ -2604,6 +2653,7 @@ services: assert!(!aliases.contains_key("MYSQL_DATABASE")); } + #[test] fn parameterize_replaces_secret_values_with_env_refs() { let compose = "\ services: diff --git a/src/cli/service_catalog.rs b/src/cli/service_catalog.rs index 20253137..cf7357cf 100644 --- a/src/cli/service_catalog.rs +++ b/src/cli/service_catalog.rs @@ -139,6 +139,7 @@ impl ServiceCatalog { if let Some(services) = stack_def.get("services") { if let Some(first_svc) = services.as_array().and_then(|arr| arr.first()) { let service = ServiceDefinition { + runtime: Default::default(), name: first_svc["name"].as_str().unwrap_or(slug).to_string(), image: first_svc["image"].as_str().unwrap_or("").to_string(), ports: first_svc["ports"] @@ -282,6 +283,7 @@ fn build_hardcoded_catalog() -> Vec { category: "database".into(), description: "Reliable open-source relational database".into(), service: ServiceDefinition { + runtime: Default::default(), name: "postgres".into(), image: "postgres:16-alpine".into(), ports: vec!["5432:5432".into()], @@ -308,6 +310,7 @@ fn build_hardcoded_catalog() -> Vec { category: "database".into(), description: "Popular open-source relational database".into(), service: ServiceDefinition { + runtime: Default::default(), name: "mysql".into(), image: "mysql:8.0".into(), ports: vec!["3306:3306".into()], @@ -338,6 +341,7 @@ fn build_hardcoded_catalog() -> Vec { category: "database".into(), description: "Document-oriented NoSQL database".into(), service: ServiceDefinition { + runtime: Default::default(), name: "mongodb".into(), image: "mongo:7".into(), ports: vec!["27017:27017".into()], @@ -369,6 +373,7 @@ fn build_hardcoded_catalog() -> Vec { category: "cache".into(), description: "In-memory data store for caching and message broker".into(), service: ServiceDefinition { + runtime: Default::default(), name: "redis".into(), image: "redis:7-alpine".into(), ports: vec!["6379:6379".into()], @@ -391,6 +396,7 @@ fn build_hardcoded_catalog() -> Vec { category: "cache".into(), description: "High-performance distributed memory caching system".into(), service: ServiceDefinition { + runtime: Default::default(), name: "memcached".into(), image: "memcached:1.6-alpine".into(), ports: vec!["11211:11211".into()], @@ -414,6 +420,7 @@ fn build_hardcoded_catalog() -> Vec { category: "queue".into(), description: "Advanced message broker with management UI".into(), service: ServiceDefinition { + runtime: Default::default(), name: "rabbitmq".into(), image: "rabbitmq:3-management-alpine".into(), ports: vec!["5672:5672".into(), "15672:15672".into()], @@ -443,6 +450,7 @@ fn build_hardcoded_catalog() -> Vec { category: "proxy".into(), description: "Cloud-native reverse proxy with automatic SSL".into(), service: ServiceDefinition { + runtime: Default::default(), name: "traefik".into(), image: "traefik:v3.0".into(), ports: vec!["80:80".into(), "443:443".into()], @@ -468,6 +476,7 @@ fn build_hardcoded_catalog() -> Vec { category: "proxy".into(), description: "High-performance web server and reverse proxy".into(), service: ServiceDefinition { + runtime: Default::default(), name: "nginx".into(), image: "nginx:1.25-alpine".into(), ports: vec!["80:80".into(), "443:443".into()], @@ -490,6 +499,7 @@ fn build_hardcoded_catalog() -> Vec { category: "proxy".into(), description: "Web UI for managing Nginx reverse proxy with SSL".into(), service: ServiceDefinition { + runtime: Default::default(), name: "nginx_proxy_manager".into(), image: "jc21/nginx-proxy-manager:latest".into(), ports: vec!["80:80".into(), "443:443".into(), "81:81".into()], @@ -516,6 +526,7 @@ fn build_hardcoded_catalog() -> Vec { category: "web".into(), description: "Popular CMS and blogging platform".into(), service: ServiceDefinition { + runtime: Default::default(), name: "wordpress".into(), image: "wordpress:latest".into(), ports: vec!["8080:80".into()], @@ -547,6 +558,7 @@ fn build_hardcoded_catalog() -> Vec { category: "search".into(), description: "Distributed search and analytics engine".into(), service: ServiceDefinition { + runtime: Default::default(), name: "elasticsearch".into(), image: "elasticsearch:8.12.0".into(), ports: vec!["9200:9200".into()], @@ -573,6 +585,7 @@ fn build_hardcoded_catalog() -> Vec { category: "search".into(), description: "Visualization dashboard for Elasticsearch".into(), service: ServiceDefinition { + runtime: Default::default(), name: "kibana".into(), image: "kibana:8.12.0".into(), ports: vec!["5601:5601".into()], @@ -599,6 +612,7 @@ fn build_hardcoded_catalog() -> Vec { category: "database".into(), description: "Vector similarity search engine for AI applications".into(), service: ServiceDefinition { + runtime: Default::default(), name: "qdrant".into(), image: "qdrant/qdrant:latest".into(), ports: vec!["6333:6333".into(), "6334:6334".into()], @@ -622,6 +636,7 @@ fn build_hardcoded_catalog() -> Vec { category: "monitoring".into(), description: "Agent for collecting and reporting metrics".into(), service: ServiceDefinition { + runtime: Default::default(), name: "telegraf".into(), image: "telegraf:1.30-alpine".into(), ports: vec![], @@ -645,6 +660,7 @@ fn build_hardcoded_catalog() -> Vec { category: "devtool".into(), description: "Web-based MySQL database management (development)".into(), service: ServiceDefinition { + runtime: Default::default(), name: "phpmyadmin".into(), image: "phpmyadmin:latest".into(), ports: vec!["8081:80".into()], @@ -671,6 +687,7 @@ fn build_hardcoded_catalog() -> Vec { description: "Attachable SMTP companion app for local delivery and relay testing" .into(), service: ServiceDefinition { + runtime: Default::default(), name: "smtp".into(), image: "trydirect/smtp".into(), ports: vec!["1025:25".into()], @@ -699,6 +716,7 @@ fn build_hardcoded_catalog() -> Vec { category: "devtool".into(), description: "Email testing tool — catches all outgoing mail (development)".into(), service: ServiceDefinition { + runtime: Default::default(), name: "mailhog".into(), image: "mailhog/mailhog:latest".into(), ports: vec!["1025:1025".into(), "8025:8025".into()], @@ -722,6 +740,7 @@ fn build_hardcoded_catalog() -> Vec { category: "storage".into(), description: "S3-compatible object storage".into(), service: ServiceDefinition { + runtime: Default::default(), name: "minio".into(), image: "minio/minio:latest".into(), ports: vec!["9000:9000".into(), "9001:9001".into()], @@ -751,6 +770,7 @@ fn build_hardcoded_catalog() -> Vec { category: "devtool".into(), description: "Docker container management web UI".into(), service: ServiceDefinition { + runtime: Default::default(), name: "portainer".into(), image: "portainer/portainer-ce:latest".into(), ports: vec!["9443:9443".into()], @@ -777,6 +797,7 @@ fn build_hardcoded_catalog() -> Vec { category: "ai".into(), description: "Personal AI assistant with multi-channel gateway".into(), service: ServiceDefinition { + runtime: Default::default(), name: "openclaw".into(), image: "ghcr.io/openclaw/openclaw:latest".into(), ports: vec!["18789:18789".into()], diff --git a/src/cli/service_import.rs b/src/cli/service_import.rs index d6188a05..d05ac2d5 100644 --- a/src/cli/service_import.rs +++ b/src/cli/service_import.rs @@ -154,6 +154,7 @@ pub fn import_plan_from_compose_str( has_mail_server_shape |= looks_like_mail_server(source_name, &image, &ports); services.push(ServiceDefinition { + runtime: Default::default(), name: destination_name.clone(), image: image.clone(), ports: ports.clone(), diff --git a/src/cli/stacker_client.rs b/src/cli/stacker_client.rs index 238343ba..e3f4cfad 100644 --- a/src/cli/stacker_client.rs +++ b/src/cli/stacker_client.rs @@ -3826,6 +3826,26 @@ fn parse_port_mapping(port_str: &str) -> (String, String) { } } +/// The `shared_ports` entry for a port mapping from stacker.yml. +/// +/// A host IP must reach the server: `127.0.0.1:5432:5432` is host-local, and +/// the server reads the IP from `container_port` to keep the port out of the +/// firewall and to publish it on 127.0.0.1 only. Sending just `5432` opened a +/// database bound to loopback to the internet. +fn port_mapping_json(port_str: &str) -> serde_json::Value { + let (host, container) = parse_port_mapping(port_str); + let mapping = port_str.split('/').next().unwrap_or(port_str); + let container_port = if mapping.matches(':').count() >= 2 { + mapping.to_string() + } else { + container + }; + serde_json::json!({ + "host_port": host, + "container_port": container_port, + }) +} + /// Parse a volume mapping string like "./dist:/usr/share/nginx/html" or "data:/var/lib/db" /// into (host_path, container_path, read_only) tuple. /// Handles optional `:ro` / `:rw` suffix (e.g. "/var/run/docker.sock:/var/run/docker.sock:ro"). @@ -3847,17 +3867,8 @@ fn service_to_app_json(svc: &ServiceDefinition, network_ids: &[String]) -> serde let (dockerhub_user, dockerhub_name, dockerhub_tag) = parse_docker_image(&svc.image); let id = generate_app_id(); - let shared_ports: Vec = svc - .ports - .iter() - .map(|p| { - let (host, container) = parse_port_mapping(p); - serde_json::json!({ - "host_port": host, - "container_port": container, - }) - }) - .collect(); + let shared_ports: Vec = + svc.ports.iter().map(|p| port_mapping_json(p)).collect(); let volumes: Vec = svc .volumes @@ -3889,7 +3900,7 @@ fn service_to_app_json(svc: &ServiceDefinition, network_ids: &[String]) -> serde "code": svc.name.to_lowercase(), "type": "web", "dockerhub_name": dockerhub_name, - "restart": "always", + "restart": svc.runtime.restart.map_or("always", |r| r.as_str()), "custom": true, "shared_ports": shared_ports, "volumes": volumes, @@ -3898,6 +3909,26 @@ fn service_to_app_json(svc: &ServiceDefinition, network_ids: &[String]) -> serde }); let obj = app.as_object_mut().unwrap(); + // Everything else the service declares reaches the server's compose too + // (shared-fixtures/api-contracts/stack-service-runtime-fields.json). + for (key, values) in [ + ("depends_on", &svc.depends_on), + ("cap_add", &svc.cap_add), + ("cap_drop", &svc.cap_drop), + ("devices", &svc.devices), + ("profiles", &svc.runtime.profiles), + ("aliases", &svc.runtime.aliases), + ] { + if !values.is_empty() { + obj.insert(key.to_string(), serde_json::json!(values)); + } + } + if let Some(ref entrypoint) = svc.runtime.entrypoint { + obj.insert("entrypoint".to_string(), serde_json::json!(entrypoint)); + } + if let Some(ref deploy) = svc.runtime.deploy { + obj.insert("deploy".to_string(), serde_json::json!(deploy)); + } if let Some(user) = dockerhub_user { obj.insert("dockerhub_user".to_string(), serde_json::json!(user)); } @@ -3977,13 +4008,7 @@ fn app_source_to_app_json( .app .ports .iter() - .map(|p| { - let (host, container) = parse_port_mapping(p); - serde_json::json!({ - "host_port": host, - "container_port": container, - }) - }) + .map(|p| port_mapping_json(p)) .collect() }; @@ -5373,6 +5398,7 @@ mod tests { #[test] fn test_build_project_body_skips_declared_npm_service_when_proxy_is_managed() { let npm_service = ServiceDefinition { + runtime: Default::default(), name: "nginx_proxy_manager".to_string(), image: "jc21/nginx-proxy-manager:latest".to_string(), ports: vec![ @@ -5392,6 +5418,7 @@ mod tests { devices: Vec::new(), }; let redis_service = ServiceDefinition { + runtime: Default::default(), name: "redis".to_string(), image: "redis:7-alpine".to_string(), ports: vec![], @@ -5431,6 +5458,7 @@ mod tests { #[test] fn test_build_project_body_skips_platform_managed_services_even_without_proxy() { let npm_service = ServiceDefinition { + runtime: Default::default(), name: "nginx_proxy_manager".to_string(), image: "jc21/nginx-proxy-manager:latest".to_string(), ports: vec![ @@ -5450,6 +5478,7 @@ mod tests { devices: Vec::new(), }; let statuspanel_service = ServiceDefinition { + runtime: Default::default(), name: "statuspanel".to_string(), image: "trydirect/status:latest".to_string(), ports: vec!["5000:5000".to_string()], @@ -5465,6 +5494,7 @@ mod tests { devices: Vec::new(), }; let smtp_service = ServiceDefinition { + runtime: Default::default(), name: "smtp".to_string(), image: "trydirect/smtp:latest".to_string(), ports: vec!["127.0.0.1:1025:25".to_string()], @@ -5525,6 +5555,7 @@ mod tests { #[test] fn test_scn_001_stacker_yml_service_serializes_as_service_target() { let upload_service = ServiceDefinition { + runtime: Default::default(), name: "upload".to_string(), image: "ghcr.io/example/upload:1.0".to_string(), ports: vec!["8081:8080".to_string()], diff --git a/src/connectors/install_service/client.rs b/src/connectors/install_service/client.rs index 695ce92d..d0919cae 100644 --- a/src/connectors/install_service/client.rs +++ b/src/connectors/install_service/client.rs @@ -107,6 +107,7 @@ impl InstallServiceConnector for InstallServiceClient { server_public_key: Option, server_private_key: Option, proxy_domains: Option, + initiator_email: Option, ) -> Result { // Build payload for the install service let mut payload = crate::forms::project::Payload::try_from(project) @@ -140,6 +141,7 @@ impl InstallServiceConnector for InstallServiceClient { payload.common_domain = stack_var_string(&form_stack.vars, "commonDomain"); } payload.user_token = Some(user_id); + payload.initiator_email = initiator_email.filter(|email| *email != user_email); payload.user_email = Some(user_email); payload.docker_compose = Some(compress(fc.as_str())); payload.registry = registry; diff --git a/src/connectors/install_service/mock.rs b/src/connectors/install_service/mock.rs index 12fd793d..fe1295c3 100644 --- a/src/connectors/install_service/mock.rs +++ b/src/connectors/install_service/mock.rs @@ -27,6 +27,7 @@ impl InstallServiceConnector for MockInstallServiceConnector { _server_public_key: Option, _server_private_key: Option, _proxy_domains: Option, + _initiator_email: Option, ) -> Result { Ok(project_id) } diff --git a/src/connectors/install_service/mod.rs b/src/connectors/install_service/mod.rs index a9a70d9b..1a9eb38f 100644 --- a/src/connectors/install_service/mod.rs +++ b/src/connectors/install_service/mod.rs @@ -53,6 +53,7 @@ pub trait InstallServiceConnector: Send + Sync { server_public_key: Option, server_private_key: Option, proxy_domains: Option, + initiator_email: Option, ) -> Result; async fn configure_cloud_firewall( diff --git a/src/connectors/user_service/client.rs b/src/connectors/user_service/client.rs index fe70ea8a..7780ed62 100644 --- a/src/connectors/user_service/client.rs +++ b/src/connectors/user_service/client.rs @@ -7,8 +7,8 @@ use uuid::Uuid; use super::connector::UserServiceConnector; use super::types::{ - AuthorizationHandle, BillingCapability, CategoryInfo, PlanDefinition, ProductInfo, - StackResponse, UserPlanInfo, UserProfile, + AuthorizationHandle, BillingCapability, CategoryInfo, ManagedBillingEvent, ManagedUsageReport, + PlanDefinition, ProductInfo, StackResponse, UserPlanInfo, UserProfile, }; use super::utils::is_plan_higher_tier; @@ -781,6 +781,53 @@ impl UserServiceConnector for UserServiceClient { serde_json::from_str::(&text) .map_err(|e| ConnectorError::InvalidResponse(e.to_string())) } + + async fn report_managed_usage( + &self, + service_token: &str, + usage: &ManagedUsageReport, + ) -> Result<(), ConnectorError> { + self.post_managed_billing("usage", service_token, usage) + .await + } + + async fn record_managed_billing_event( + &self, + service_token: &str, + event: &ManagedBillingEvent, + ) -> Result<(), ConnectorError> { + self.post_managed_billing("event", service_token, event) + .await + } +} + +impl UserServiceClient { + /// POST to `/api/1.0/marketplace/billing/managed/` with the service + /// token; non-2xx responses map like the other billing calls. + async fn post_managed_billing( + &self, + path: &str, + service_token: &str, + body: &T, + ) -> Result<(), ConnectorError> { + let url = format!( + "{}/api/1.0/marketplace/billing/managed/{}", + self.base_url, path + ); + let resp = self + .http_client + .post(&url) + .header("Authorization", format!("Bearer {}", service_token)) + .json(body) + .send() + .await?; + let status = resp.status(); + if status.is_success() { + return Ok(()); + } + let text = resp.text().await.unwrap_or_default(); + Err(map_billing_error_status(status.as_u16(), &text)) + } } /// Map a non-2xx billing response to the appropriate ConnectorError. diff --git a/src/connectors/user_service/connector.rs b/src/connectors/user_service/connector.rs index 64ee8a38..e4c68d90 100644 --- a/src/connectors/user_service/connector.rs +++ b/src/connectors/user_service/connector.rs @@ -1,8 +1,8 @@ use uuid::Uuid; use super::types::{ - AuthorizationHandle, BillingCapability, CategoryInfo, PlanDefinition, ProductInfo, - StackResponse, UserPlanInfo, UserProfile, + AuthorizationHandle, BillingCapability, CategoryInfo, ManagedBillingEvent, ManagedUsageReport, + PlanDefinition, ProductInfo, StackResponse, UserPlanInfo, UserProfile, }; use crate::connectors::errors::ConnectorError; @@ -150,4 +150,20 @@ pub trait UserServiceConnector: Send + Sync { amount_minor: i64, deployment_hash: &str, ) -> Result; + + /// Send one day of a Managed server to the meter (the User Service resolves + /// the billing email and calls payment). `PaymentRequired` when the user + /// has no payment method. + async fn report_managed_usage( + &self, + service_token: &str, + usage: &ManagedUsageReport, + ) -> Result<(), ConnectorError>; + + /// Add one row to the User Service's `managed_billing_events`. + async fn record_managed_billing_event( + &self, + service_token: &str, + event: &ManagedBillingEvent, + ) -> Result<(), ConnectorError>; } diff --git a/src/connectors/user_service/mock.rs b/src/connectors/user_service/mock.rs index 268df061..79e5c115 100644 --- a/src/connectors/user_service/mock.rs +++ b/src/connectors/user_service/mock.rs @@ -3,8 +3,9 @@ use uuid::Uuid; use crate::connectors::errors::ConnectorError; use super::{ - AuthorizationHandle, BillingCapability, CategoryInfo, PlanDefinition, ProductInfo, - StackResponse, UserPlanInfo, UserProduct, UserProfile, UserServiceConnector, + AuthorizationHandle, BillingCapability, CategoryInfo, ManagedBillingEvent, ManagedUsageReport, + PlanDefinition, ProductInfo, StackResponse, UserPlanInfo, UserProduct, UserProfile, + UserServiceConnector, }; /// Mock User Service for testing - always succeeds @@ -314,4 +315,20 @@ impl UserServiceConnector for MockUserServiceConnector { status: "captured".to_string(), }) } + + async fn report_managed_usage( + &self, + _service_token: &str, + _usage: &ManagedUsageReport, + ) -> Result<(), ConnectorError> { + Ok(()) + } + + async fn record_managed_billing_event( + &self, + _service_token: &str, + _event: &ManagedBillingEvent, + ) -> Result<(), ConnectorError> { + Ok(()) + } } diff --git a/src/connectors/user_service/mod.rs b/src/connectors/user_service/mod.rs index 02e7cc54..781d4752 100644 --- a/src/connectors/user_service/mod.rs +++ b/src/connectors/user_service/mod.rs @@ -26,8 +26,8 @@ pub use marketplace_webhook::{ }; pub use mock::MockUserServiceConnector; pub use types::{ - AuthorizationHandle, BillingCapability, CategoryInfo, PlanDefinition, ProductInfo, - StackResponse, UserPlanInfo, UserProduct, UserProfile, + AuthorizationHandle, BillingCapability, CategoryInfo, ManagedBillingEvent, ManagedUsageReport, + PlanDefinition, ProductInfo, StackResponse, UserPlanInfo, UserProduct, UserProfile, }; #[cfg(test)] diff --git a/src/connectors/user_service/tests.rs b/src/connectors/user_service/tests.rs index 8757ace3..72f7e46e 100644 --- a/src/connectors/user_service/tests.rs +++ b/src/connectors/user_service/tests.rs @@ -553,3 +553,98 @@ fn test_category_info_deserialization() { assert_eq!(category.title, "AI Agents"); assert_eq!(category.priority, Some(5)); } + +fn managed_usage() -> super::ManagedUsageReport { + super::ManagedUsageReport { + user_id: "user-1".to_string(), + deployment_hash: "deployment_abc".to_string(), + value_minor: 86, + identifier: "deployment_abc:2026-10-16".to_string(), + timestamp: Some(1792108800), + } +} + +/// Contract: shared-fixtures/api-contracts/managed-server-billing.json +#[tokio::test] +async fn report_managed_usage_posts_the_day_with_the_service_token() { + let mut server = Server::new_async().await; + let mock = server + .mock("POST", "/api/1.0/marketplace/billing/managed/usage") + .match_header("authorization", "Bearer service-token") + .match_body(Matcher::Json(json!({ + "user_id": "user-1", + "deployment_hash": "deployment_abc", + "value_minor": 86, + "identifier": "deployment_abc:2026-10-16", + "timestamp": 1792108800 + }))) + .with_status(200) + .with_body(json!({"identifier": "deployment_abc:2026-10-16"}).to_string()) + .create_async() + .await; + + let client = UserServiceClient::new_public(&server.url()); + client + .report_managed_usage("service-token", &managed_usage()) + .await + .expect("usage reported"); + mock.assert_async().await; +} + +#[tokio::test] +async fn report_managed_usage_without_a_card_is_payment_required() { + let mut server = Server::new_async().await; + let _mock = server + .mock("POST", "/api/1.0/marketplace/billing/managed/usage") + .with_status(402) + .with_body(json!({"reason": "no_payment_method"}).to_string()) + .create_async() + .await; + + let client = UserServiceClient::new_public(&server.url()); + let err = client + .report_managed_usage("service-token", &managed_usage()) + .await + .expect_err("402 is an error"); + assert!( + matches!( + err, + crate::connectors::errors::ConnectorError::PaymentRequired(_) + ), + "{err:?}" + ); +} + +#[tokio::test] +async fn record_managed_billing_event_posts_one_event() { + let mut server = Server::new_async().await; + let mock = server + .mock("POST", "/api/1.0/marketplace/billing/managed/event") + .match_header("authorization", "Bearer service-token") + .match_body(Matcher::Json(json!({ + "user_id": "user-1", + "deployment_hash": "deployment_abc", + "event": "server_deleted" + }))) + .with_status(200) + .with_body(json!({"id": "6f1c6a3e-0000-0000-0000-000000000000"}).to_string()) + .create_async() + .await; + + let client = UserServiceClient::new_public(&server.url()); + client + .record_managed_billing_event( + "service-token", + &super::ManagedBillingEvent { + user_id: "user-1".to_string(), + deployment_hash: "deployment_abc".to_string(), + event: "server_deleted".to_string(), + amount_minor: None, + currency: None, + metadata: None, + }, + ) + .await + .expect("event recorded"); + mock.assert_async().await; +} diff --git a/src/connectors/user_service/types.rs b/src/connectors/user_service/types.rs index b757dbbf..cb97abad 100644 --- a/src/connectors/user_service/types.rs +++ b/src/connectors/user_service/types.rs @@ -112,3 +112,32 @@ pub struct BillingCapability { #[serde(default)] pub reason: Option, } + +/// One day of a Managed server for `/marketplace/billing/managed/usage` +/// (contract: shared-fixtures/api-contracts/managed-server-billing.json). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct ManagedUsageReport { + pub user_id: String, + pub deployment_hash: String, + pub value_minor: i64, + /// `:`, so a resend counts once. + pub identifier: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub timestamp: Option, +} + +/// One row of `managed_billing_events` for `/marketplace/billing/managed/event`. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct ManagedBillingEvent { + pub user_id: String, + pub deployment_hash: String, + /// usage_started, deploy_failed, server_deleted, invoice_paid, + /// invoice_failed, server_suspended, server_deleted_unpaid + pub event: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub amount_minor: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub currency: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub metadata: Option, +} diff --git a/src/console/commands/cli/init.rs b/src/console/commands/cli/init.rs index 3120714f..def07c74 100644 --- a/src/console/commands/cli/init.rs +++ b/src/console/commands/cli/init.rs @@ -1118,6 +1118,7 @@ fn compose_service_to_definition( }; Some(ServiceDefinition { + runtime: Default::default(), name: service.name.clone(), image, ports: service @@ -1609,6 +1610,7 @@ fn convert_compose_to_stacker(ai_output: &str, repo_name: &str) -> Option ServiceDefinition { ServiceDefinition { + runtime: Default::default(), name: name.to_string(), image: "optimum/syncopia-upload:latest".to_string(), ports: vec!["8000:8000".to_string()], diff --git a/src/db/agreement.rs b/src/db/agreement.rs index 2bc582c6..f64f4d90 100644 --- a/src/db/agreement.rs +++ b/src/db/agreement.rs @@ -218,3 +218,27 @@ pub async fn delete(pool: &PgPool, id: i32) -> Result { "Failed to delete agreement".to_string() }) } + +pub async fn fetch_by_name(pool: &PgPool, name: &str) -> Result, String> { + use sqlx::Row; + sqlx::query("SELECT id, name, text, created_at, updated_at FROM agreement WHERE name = $1 ORDER BY id LIMIT 1") + .bind(name) + .fetch_optional(pool) + .await + .and_then(|row| { + row.map(|r| { + Ok(models::Agreement { + id: r.try_get("id")?, + name: r.try_get("name")?, + text: r.try_get("text")?, + created_at: r.try_get("created_at")?, + updated_at: r.try_get("updated_at")?, + }) + }) + .transpose() + }) + .map_err(|err| { + tracing::error!("Failed to fetch agreement by name, error: {:?}", err); + "Could not fetch data".to_string() + }) +} diff --git a/src/db/bake_job.rs b/src/db/bake_job.rs new file mode 100644 index 00000000..04ce2e2d --- /dev/null +++ b/src/db/bake_job.rs @@ -0,0 +1,80 @@ +use crate::models::BakeJob; +use sqlx::PgPool; + +pub struct NewBakeJob<'a> { + pub stack: &'a str, + pub version: &'a str, + pub deployment_id: i32, + pub server_id: i32, + pub server_ip: &'a str, + pub created_by: &'a str, +} + +pub async fn insert(pool: &PgPool, job: NewBakeJob<'_>) -> Result { + sqlx::query_as::<_, BakeJob>( + r#" + INSERT INTO bake_job (stack, version, deployment_id, server_id, server_ip, created_by) + VALUES ($1, $2, $3, $4, $5, $6) + RETURNING * + "#, + ) + .bind(job.stack) + .bind(job.version) + .bind(job.deployment_id) + .bind(job.server_id) + .bind(job.server_ip) + .bind(job.created_by) + .fetch_one(pool) + .await + .map_err(|e| format!("Failed to record bake job: {e}")) +} + +pub async fn fetch(pool: &PgPool, id: uuid::Uuid) -> Result, String> { + sqlx::query_as::<_, BakeJob>("SELECT * FROM bake_job WHERE id = $1") + .bind(id) + .fetch_optional(pool) + .await + .map_err(|e| format!("Failed to read bake job: {e}")) +} + +/// A bake already running for this build box. One at a time: two would +/// sanitize and snapshot the same box concurrently. +pub async fn running_for_server(pool: &PgPool, server_id: i32) -> Result, String> { + sqlx::query_as::<_, BakeJob>( + "SELECT * FROM bake_job WHERE server_id = $1 AND status = 'running' LIMIT 1", + ) + .bind(server_id) + .fetch_optional(pool) + .await + .map_err(|e| format!("Failed to read bake jobs: {e}")) +} + +pub async fn mark_done( + pool: &PgPool, + id: uuid::Uuid, + image_id: i64, + snapshot_id: i32, +) -> Result<(), String> { + sqlx::query( + "UPDATE bake_job SET status = 'done', image_id = $2, snapshot_id = $3, updated_at = NOW() WHERE id = $1", + ) + .bind(id) + .bind(image_id) + .bind(snapshot_id) + .execute(pool) + .await + .map(|_| ()) + .map_err(|e| format!("Failed to finish bake job: {e}")) +} + +pub async fn mark_failed(pool: &PgPool, id: uuid::Uuid, error: &str) -> Result<(), String> { + sqlx::query( + "UPDATE bake_job SET status = 'failed', error = $2, updated_at = NOW() WHERE id = $1", + ) + .bind(id) + .bind(error) + .execute(pool) + .await + .map(|_| ()) + .map_err(|e| format!("Failed to mark bake job failed: {e}")) +} diff --git a/src/db/baked_snapshot.rs b/src/db/baked_snapshot.rs index 160839f2..ab65abfd 100644 --- a/src/db/baked_snapshot.rs +++ b/src/db/baked_snapshot.rs @@ -65,11 +65,12 @@ pub async fn record( digests: Option, config_contract: Option, required_env_keys: Option, + server_type: Option<&str>, ) -> Result { sqlx::query_as::<_, BakedSnapshot>( r#" - INSERT INTO baked_snapshots (stack, version, provider, image_id, healthy, digests, config_contract, required_env_keys) - VALUES ($1, $2, $3, $4, $5, $6, $7, $8) + INSERT INTO baked_snapshots (stack, version, provider, image_id, healthy, digests, config_contract, required_env_keys, server_type) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) RETURNING * "#, ) @@ -81,6 +82,7 @@ pub async fn record( .bind(digests) .bind(config_contract) .bind(required_env_keys) + .bind(server_type) .fetch_one(pool) .await .map_err(|e| format!("Failed to record baked snapshot: {e}")) diff --git a/src/db/chat_admin.rs b/src/db/chat_admin.rs new file mode 100644 index 00000000..f1843cbc --- /dev/null +++ b/src/db/chat_admin.rs @@ -0,0 +1,62 @@ +use serde::Serialize; +use sqlx::{PgPool, Row}; + +/// Agreement name that gates admin access to chat content. +pub const AI_CHAT_AGREEMENT: &str = "ai_chat_analysis"; + +#[derive(Debug, Serialize)] +pub struct ConsentedChat { + pub source: &'static str, + pub id: String, + pub user_id: String, + pub title: Option, + pub updated_at: chrono::DateTime, + /// Encrypted blob for `chat_sessions`; empty for `chat_conversations`. + #[serde(skip)] + pub messages_encrypted: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub plain_messages: Option, +} + +/// Most recently updated chats of users who accepted the AI chat analysis +/// agreement, from both `chat_sessions` and `chat_conversations`. +pub async fn recent_consented(pool: &PgPool, limit: i64) -> Result, sqlx::Error> { + let rows = sqlx::query( + r#" + SELECT * FROM ( + SELECT 'session' AS source, s.id::text AS id, s.user_id, s.title, + s.updated_at, s.messages_encrypted, NULL::jsonb AS plain_messages + FROM chat_sessions s + UNION ALL + SELECT 'conversation', c.id::text, c.user_id, NULL, c.updated_at, '', c.messages + FROM chat_conversations c + ) chats + WHERE EXISTS ( + SELECT 1 FROM user_agreement ua + JOIN agreement a ON a.id = ua.agrt_id + WHERE a.name = $1 AND ua.user_id = chats.user_id + ) + ORDER BY updated_at DESC + LIMIT $2 + "#, + ) + .bind(AI_CHAT_AGREEMENT) + .bind(limit) + .fetch_all(pool) + .await?; + + rows.into_iter() + .map(|r| { + let source: String = r.try_get("source")?; + Ok(ConsentedChat { + source: if source == "session" { "session" } else { "conversation" }, + id: r.try_get("id")?, + user_id: r.try_get("user_id")?, + title: r.try_get("title")?, + updated_at: r.try_get("updated_at")?, + messages_encrypted: r.try_get("messages_encrypted")?, + plain_messages: r.try_get("plain_messages")?, + }) + }) + .collect() +} diff --git a/src/db/managed_usage.rs b/src/db/managed_usage.rs new file mode 100644 index 00000000..e092039c --- /dev/null +++ b/src/db/managed_usage.rs @@ -0,0 +1,262 @@ +//! Persistence of the Managed server usage behind a Quick Deploy. +//! +//! One row per deployment (`managed_usage`): the rate and cap it is billed +//! at, its lifecycle, and how far the daily report to the Stripe meter got. +//! The arithmetic lives in `services::managed_usage`. + +use chrono::{DateTime, NaiveDate, Utc}; +use serde::Serialize; +use sqlx::PgPool; + +#[derive(Debug, Clone, PartialEq, Serialize)] +pub struct ManagedUsage { + pub id: i64, + pub deployment_hash: String, + pub user_id: String, + pub stack: String, + pub server_type: String, + pub daily_rate_minor: i64, + pub monthly_cap_minor: i64, + pub currency: String, + pub server_id: Option, + pub status: String, + pub outcome: Option, + pub started_at: Option>, + pub stopped_at: Option>, + pub suspended_at: Option>, + pub period_start: Option, + pub period_reported_minor: i64, + pub last_reported_on: Option, + pub created_at: DateTime, + pub updated_at: DateTime, +} + +impl ManagedUsage { + /// The last day to report: today for a running server, the day it was + /// suspended or deleted otherwise (never after today). + pub fn report_until(&self, today: NaiveDate) -> NaiveDate { + match self.status.as_str() { + "active" => today, + _ => self + .suspended_at + .or(self.stopped_at) + .map(|at| at.date_naive().min(today)) + .unwrap_or(today), + } + } +} + +#[derive(Debug, Clone)] +pub struct NewManagedUsage { + pub deployment_hash: String, + pub user_id: String, + pub stack: String, + pub server_type: String, + pub daily_rate_minor: i64, + pub monthly_cap_minor: i64, +} + +pub async fn insert_pending(pool: &PgPool, new: &NewManagedUsage) -> Result { + sqlx::query_as!( + ManagedUsage, + r#"INSERT INTO managed_usage + (deployment_hash, user_id, stack, server_type, daily_rate_minor, monthly_cap_minor) + VALUES ($1, $2, $3, $4, $5, $6) + RETURNING *"#, + new.deployment_hash, + new.user_id, + new.stack, + new.server_type, + new.daily_rate_minor, + new.monthly_cap_minor, + ) + .fetch_one(pool) + .await + .map_err(|e| format!("managed_usage insert: {e}")) +} + +pub async fn fetch_by_hash( + pool: &PgPool, + deployment_hash: &str, +) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + "SELECT * FROM managed_usage WHERE deployment_hash = $1", + deployment_hash, + ) + .fetch_optional(pool) + .await + .map_err(|e| format!("managed_usage fetch: {e}")) +} + +/// The deploy succeeded: pending -> active. `None` when not pending. +pub async fn start(pool: &PgPool, deployment_hash: &str) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"UPDATE managed_usage + SET status = 'active', started_at = now(), updated_at = now() + WHERE deployment_hash = $1 AND status = 'pending' + RETURNING *"#, + deployment_hash, + ) + .fetch_optional(pool) + .await + .map_err(|e| format!("managed_usage start: {e}")) +} + +/// The deploy failed: pending -> closed with `outcome`. `None` when not pending. +pub async fn close( + pool: &PgPool, + deployment_hash: &str, + outcome: &str, +) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"UPDATE managed_usage + SET status = 'closed', outcome = $2, stopped_at = now(), updated_at = now() + WHERE deployment_hash = $1 AND status = 'pending' + RETURNING *"#, + deployment_hash, + outcome, + ) + .fetch_optional(pool) + .await + .map_err(|e| format!("managed_usage close: {e}")) +} + +/// The server was deleted: active or suspended -> stopped. +pub async fn stop(pool: &PgPool, deployment_hash: &str) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"UPDATE managed_usage + SET status = 'stopped', stopped_at = now(), updated_at = now() + WHERE deployment_hash = $1 AND status IN ('active', 'suspended') + RETURNING *"#, + deployment_hash, + ) + .fetch_optional(pool) + .await + .map_err(|e| format!("managed_usage stop: {e}")) +} + +/// An invoice is unpaid: every active server of the user -> suspended. +pub async fn suspend_for_user(pool: &PgPool, user_id: &str) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"UPDATE managed_usage + SET status = 'suspended', suspended_at = now(), updated_at = now() + WHERE user_id = $1 AND status = 'active' + RETURNING *"#, + user_id, + ) + .fetch_all(pool) + .await + .map_err(|e| format!("managed_usage suspend: {e}")) +} + +/// Rows with a day left to report up to `today`: running servers, and +/// stopped or suspended ones up to the day they stopped. +pub async fn list_due( + pool: &PgPool, + today: NaiveDate, + limit: i64, +) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"SELECT * FROM managed_usage + WHERE started_at IS NOT NULL + AND ( + (status = 'active' + AND (last_reported_on IS NULL OR last_reported_on < $1)) + OR (status IN ('stopped', 'suspended') + AND (last_reported_on IS NULL + OR last_reported_on < LEAST( + (COALESCE(suspended_at, stopped_at) AT TIME ZONE 'UTC')::date, + $1))) + ) + ORDER BY last_reported_on ASC NULLS FIRST + LIMIT $2"#, + today, + limit, + ) + .fetch_all(pool) + .await + .map_err(|e| format!("managed_usage list_due: {e}")) +} + +/// Store one reported day. `false` when that day was already recorded, so a +/// retry cannot count it twice. +pub async fn record_day( + pool: &PgPool, + id: i64, + period_start: NaiveDate, + period_reported_minor: i64, + day: NaiveDate, +) -> Result { + let result = sqlx::query!( + r#"UPDATE managed_usage + SET period_start = $2, period_reported_minor = $3, last_reported_on = $4, + updated_at = now() + WHERE id = $1 AND (last_reported_on IS NULL OR last_reported_on < $4)"#, + id, + period_start, + period_reported_minor, + day, + ) + .execute(pool) + .await + .map_err(|e| format!("managed_usage record_day: {e}"))?; + Ok(result.rows_affected() == 1) +} + +pub async fn list_suspended_before( + pool: &PgPool, + cutoff: DateTime, +) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"SELECT * FROM managed_usage + WHERE status = 'suspended' AND suspended_at < $1 + ORDER BY suspended_at ASC"#, + cutoff, + ) + .fetch_all(pool) + .await + .map_err(|e| format!("managed_usage list_suspended_before: {e}")) +} + +/// Keep the Hetzner server id: admins delete the VM by hand from the alert. +pub async fn set_server_id( + pool: &PgPool, + deployment_hash: &str, + server_id: i64, +) -> Result<(), String> { + sqlx::query!( + "UPDATE managed_usage SET server_id = $2, updated_at = now() WHERE deployment_hash = $1", + deployment_hash, + server_id, + ) + .execute(pool) + .await + .map(|_| ()) + .map_err(|e| format!("managed_usage set_server_id: {e}")) +} + +/// Suspended for 3 days: closed and handed to admins to delete the VM by hand. +pub async fn mark_delete_due( + pool: &PgPool, + deployment_hash: &str, +) -> Result, String> { + sqlx::query_as!( + ManagedUsage, + r#"UPDATE managed_usage + SET status = 'closed', outcome = 'server_delete_due', + stopped_at = now(), updated_at = now() + WHERE deployment_hash = $1 AND status = 'suspended' + RETURNING *"#, + deployment_hash, + ) + .fetch_optional(pool) + .await + .map_err(|e| format!("managed_usage mark_delete_due: {e}")) +} diff --git a/src/db/marketplace.rs b/src/db/marketplace.rs index adf0a6cb..38944cc0 100644 --- a/src/db/marketplace.rs +++ b/src/db/marketplace.rs @@ -6,7 +6,7 @@ use crate::models::{ }; use chrono::{Duration, Utc}; use serde_json::{Map, Value}; -use sqlx::{PgPool, Row}; +use sqlx::{PgConnection, PgPool, Row}; use tracing::Instrument; pub const SLUG_UNIQUE_CONSTRAINT: &str = "stack_template_slug_key"; @@ -946,6 +946,90 @@ pub async fn update_metadata( Ok(res.rows_affected() > 0) } +/// Update a template's metadata as an administrator, whatever its status. +/// +/// `update_metadata` refuses anything outside draft/rejected/needs_changes and +/// `update_metadata_for_resubmit` refuses draft/submitted/under_review, because +/// both model an *author* editing their own listing through the review +/// workflow. An admin correcting a published listing is not in that workflow, +/// so the status gate does not apply - same reasoning as `admin_update_pricing` +/// just below. Status itself still moves only through approve/reject/unapprove. +#[allow(clippy::too_many_arguments)] +pub async fn admin_update_metadata( + pool: &PgPool, + template_id: &uuid::Uuid, + name: Option<&str>, + short_description: Option<&str>, + long_description: Option<&str>, + category_code: Option<&str>, + tags: Option, + tech_stack: Option, + infrastructure_requirements: Option, + price: Option, + billing_cycle: Option<&str>, + required_plan_name: Option<&str>, + currency: Option<&str>, + public_ports: Option, + vendor_url: Option<&str>, +) -> Result { + let query_span = + tracing::info_span!("marketplace_admin_update_metadata", template_id = %template_id); + + if let Some(category_code) = category_code { + sqlx::query(r#"INSERT INTO stack_category (name) VALUES ($1) ON CONFLICT DO NOTHING"#) + .bind(category_code) + .execute(pool) + .instrument(query_span.clone()) + .await + .map_err(|e| { + tracing::error!("admin_update_metadata category upsert error: {:?}", e); + "Internal Server Error".to_string() + })?; + } + + let res = sqlx::query( + r#"UPDATE stack_template SET + name = COALESCE($2, name), + short_description = COALESCE($3, short_description), + long_description = COALESCE($4, long_description), + category_id = COALESCE((SELECT id FROM stack_category WHERE name = $5), category_id), + tags = COALESCE($6, tags), + tech_stack = COALESCE($7, tech_stack), + infrastructure_requirements = COALESCE($8, infrastructure_requirements), + price = COALESCE($9, price), + billing_cycle = COALESCE($10, billing_cycle), + required_plan_name = COALESCE($11, required_plan_name), + currency = COALESCE($12, currency), + public_ports = COALESCE($13, public_ports), + vendor_url = COALESCE($14, vendor_url), + updated_at = now() + WHERE id = $1::uuid"#, + ) + .bind(template_id) + .bind(name) + .bind(short_description) + .bind(long_description) + .bind(category_code) + .bind(tags) + .bind(tech_stack) + .bind(infrastructure_requirements) + .bind(price) + .bind(billing_cycle) + .bind(required_plan_name) + .bind(currency) + .bind(public_ports) + .bind(vendor_url) + .execute(pool) + .instrument(query_span) + .await + .map_err(|e| { + tracing::error!("admin_update_metadata error: {:?}", e); + "Internal Server Error".to_string() + })?; + + Ok(res.rows_affected() > 0) +} + pub async fn update_metadata_for_resubmit( pool: &PgPool, template_id: &uuid::Uuid, @@ -2367,27 +2451,87 @@ pub async fn update_verifications( Ok(res.rows_affected() > 0) } -/// Increment view_count for a marketplace template +/// Writes one view/deploy event to both event tables: `marketplace_template_event` +/// feeds creator analytics, `marketplace_event` is the raw event log. +async fn insert_event_rows( + conn: &mut PgConnection, + template_id: &uuid::Uuid, + event_type: &str, + viewer_user_id: Option<&str>, + deployer_user_id: Option<&str>, + cloud_provider: Option<&str>, + metadata: &serde_json::Value, +) -> Result<(), sqlx::Error> { + sqlx::query( + r#"INSERT INTO marketplace_template_event + (template_id, event_type, user_id, viewer_user_id, deployer_user_id, cloud_provider, metadata) + VALUES ($1, $2, $3, $4, $5, $6, $7)"#, + ) + .bind(*template_id) + .bind(event_type) + .bind(viewer_user_id.or(deployer_user_id)) + .bind(viewer_user_id) + .bind(deployer_user_id) + .bind(cloud_provider) + .bind(metadata) + .execute(&mut *conn) + .await?; + + sqlx::query( + r#"INSERT INTO marketplace_event + (template_id, event_type, viewer_user_id, deployer_user_id, cloud_provider, metadata) + VALUES ($1, $2, $3, $4, $5, $6)"#, + ) + .bind(*template_id) + .bind(event_type) + .bind(viewer_user_id) + .bind(deployer_user_id) + .bind(cloud_provider) + .bind(metadata) + .execute(&mut *conn) + .await?; + Ok(()) +} + +/// Increment view_count for a marketplace template and record a view event pub async fn increment_view_count(pool: &PgPool, template_id: &uuid::Uuid) -> Result { let query_span = tracing::info_span!("marketplace_increment_view_count", template_id = %template_id); + let internal = |e: sqlx::Error| { + tracing::error!("increment_view_count error: {:?}", e); + "Internal Server Error".to_string() + }; + let mut tx = pool.begin().await.map_err(internal)?; let res = sqlx::query( r#"UPDATE stack_template SET view_count = COALESCE(view_count, 0) + 1 WHERE id = $1"#, ) .bind(*template_id) - .execute(pool) + .execute(&mut *tx) .instrument(query_span) .await - .map_err(|e| { - tracing::error!("increment_view_count error: {:?}", e); - "Internal Server Error".to_string() - })?; + .map_err(internal)?; - Ok(res.rows_affected() > 0) + let found = res.rows_affected() > 0; + if found { + insert_event_rows( + &mut tx, + template_id, + "view", + None, + None, + None, + &serde_json::json!({}), + ) + .await + .map_err(internal)?; + } + tx.commit().await.map_err(internal)?; + Ok(found) } -/// Increment deploy_count for a marketplace template +/// Increment deploy_count for a marketplace template. +/// No event is recorded: this counter is unauthenticated; verified deploy events come from `record_deploy_complete_once`. pub async fn increment_deploy_count( pool: &PgPool, template_id: &uuid::Uuid, @@ -2493,6 +2637,21 @@ pub async fn record_deploy_complete_once( "Internal Server Error".to_string() })?; + insert_event_rows( + &mut tx, + template_id, + "deploy", + None, + None, + None, + &serde_json::json!({ "source": "deploy-complete" }), + ) + .await + .map_err(|e| { + tracing::error!("record_deploy_complete_once event error: {:?}", e); + "Internal Server Error".to_string() + })?; + tx.commit().await.map_err(|e| { tracing::error!("record_deploy_complete_once commit error: {:?}", e); "Internal Server Error".to_string() @@ -2502,37 +2661,74 @@ pub async fn record_deploy_complete_once( } // ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -// TDD Stub Functions for Metrics/Analytics -// These are intentionally unimplemented - tests will FAIL until implemented +// Metrics/Analytics // ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -/// Insert a view event into marketplace_event table -/// -/// TDD stub - unimplemented. Requires: -/// - marketplace_event table migration -/// - INSERT query with template_id, event_type='view', viewer_user_id, occurred_at, metadata +/// Insert a view event (both event tables) in one transaction pub async fn insert_view_event( - _pool: &PgPool, - _template_id: uuid::Uuid, - _viewer_user_id: &str, - _metadata: serde_json::Value, + pool: &PgPool, + template_id: uuid::Uuid, + viewer_user_id: &str, + metadata: serde_json::Value, ) -> Result<(), String> { - Err("insert_view_event not implemented - requires marketplace_event table".to_string()) + insert_event_tx( + pool, + &template_id, + "view", + Some(viewer_user_id), + None, + None, + &metadata, + ) + .await } -/// Insert a deploy event into marketplace_event table -/// -/// TDD stub - unimplemented. Requires: -/// - marketplace_event table migration with cloud_provider column -/// - INSERT query with template_id, event_type='deploy', deployer_user_id, cloud_provider, occurred_at, metadata +/// Insert a deploy event (both event tables) in one transaction pub async fn insert_deploy_event( - _pool: &PgPool, - _template_id: uuid::Uuid, - _deployer_user_id: &str, - _cloud_provider: &str, - _metadata: serde_json::Value, + pool: &PgPool, + template_id: uuid::Uuid, + deployer_user_id: &str, + cloud_provider: &str, + metadata: serde_json::Value, +) -> Result<(), String> { + insert_event_tx( + pool, + &template_id, + "deploy", + None, + Some(deployer_user_id), + Some(cloud_provider), + &metadata, + ) + .await +} + +async fn insert_event_tx( + pool: &PgPool, + template_id: &uuid::Uuid, + event_type: &str, + viewer: Option<&str>, + deployer: Option<&str>, + cloud: Option<&str>, + metadata: &serde_json::Value, ) -> Result<(), String> { - Err("insert_deploy_event not implemented - requires marketplace_event table".to_string()) + let internal = |e: sqlx::Error| { + tracing::error!("insert {} event error: {:?}", event_type, e); + "Internal Server Error".to_string() + }; + let mut tx = pool.begin().await.map_err(internal)?; + insert_event_rows( + &mut tx, + template_id, + event_type, + viewer, + deployer, + cloud, + metadata, + ) + .await + .map_err(internal)?; + tx.commit().await.map_err(internal) } /// Get vendor analytics for all templates owned by creator_user_id diff --git a/src/db/mod.rs b/src/db/mod.rs index 66d6488a..85d65a78 100644 --- a/src/db/mod.rs +++ b/src/db/mod.rs @@ -1,8 +1,10 @@ pub mod agent; pub mod agent_audit_log; pub(crate) mod agreement; +pub mod bake_job; pub mod baked_snapshot; pub mod chat; +pub mod chat_admin; pub mod chat_session; pub mod client; pub(crate) mod cloud; @@ -11,6 +13,7 @@ pub mod dag; pub(crate) mod deployment; pub mod deployment_container; pub mod marketplace; +pub mod managed_usage; pub mod marketplace_billing; pub mod pipe; pub mod product; diff --git a/src/db/project.rs b/src/db/project.rs index 6292ea64..45eda422 100644 --- a/src/db/project.rs +++ b/src/db/project.rs @@ -247,6 +247,37 @@ pub async fn set_protected( }) } +/// Whether the owner lets project members get the servers' SSH key and +/// passwords (the deployment report). +pub async fn members_server_access(pool: &PgPool, id: i32) -> Result { + sqlx::query_scalar::<_, bool>("SELECT members_server_access FROM project WHERE id = $1") + .bind(id) + .fetch_optional(pool) + .await + .map(|flag| flag.unwrap_or(false)) + .map_err(|err| { + tracing::error!("Failed to read members_server_access: {:?}", err); + "Failed to read project".to_string() + }) +} + +pub async fn set_members_server_access( + pool: &PgPool, + id: i32, + enabled: bool, +) -> Result { + sqlx::query("UPDATE project SET members_server_access = $2 WHERE id = $1") + .bind(id) + .bind(enabled) + .execute(pool) + .await + .map(|r| r.rows_affected() > 0) + .map_err(|err| { + tracing::error!("Failed to set members_server_access: {:?}", err); + "Failed to update project".to_string() + }) +} + /// Summary of resources that block project deletion. #[derive(Debug, Clone, serde::Serialize)] pub struct DeletionBlockers { diff --git a/src/db/project_app.rs b/src/db/project_app.rs index b612db2d..c9d73dc7 100644 --- a/src/db/project_app.rs +++ b/src/db/project_app.rs @@ -174,9 +174,9 @@ pub async fn insert(pool: &PgPool, app: &models::ProjectApp) -> Result Result Result Result, pub entrypoint: Option, + /// Compose settings a stacker.yml service declares + /// (shared-fixtures/api-contracts/stack-service-runtime-fields.json). + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub depends_on: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub healthcheck: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub cap_add: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub cap_drop: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub devices: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub profiles: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub aliases: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub deploy: Option, pub volumes: Option>, #[serde(flatten)] pub environment: forms::project::Environment, @@ -114,7 +132,20 @@ impl App { let networks = dctypes::Networks::try_from(&self.network).unwrap_or_default(); let networks = replace_id_with_name(networks, all_networks); - service.networks = dctypes::Networks::Simple(networks); + service.networks = if self.aliases.is_empty() { + dctypes::Networks::Simple(networks) + } else { + let settings = dctypes::AdvancedNetworkSettings { + aliases: self.aliases.clone(), + ..Default::default() + }; + dctypes::Networks::Advanced(dctypes::AdvancedNetworks( + networks + .into_iter() + .map(|name| (name, dctypes::MapOrEmpty::Map(settings.clone()))) + .collect(), + )) + }; let ports: Vec = match &self.shared_ports { Some(ports) => { @@ -169,11 +200,80 @@ impl App { } service.volumes = volumes; service.environment = dctypes::Environment::KvPair(envs); + if !self.depends_on.is_empty() { + service.depends_on = dctypes::DependsOnOptions::Simple(self.depends_on.clone()); + } + service.healthcheck = self.healthcheck.as_ref().map(compose_healthcheck); + service.cap_add = self.cap_add.clone(); + service.cap_drop = self.cap_drop.clone(); + service.devices = self.devices.clone(); + service.profiles = self.profiles.clone(); + service.deploy = self.deploy.as_ref().map(compose_deploy); Ok(service) } } +fn compose_healthcheck(hc: &crate::cli::config_parser::ComposeHealthcheck) -> dctypes::Healthcheck { + let test = match crate::cli::compose_service_sync::healthcheck_test_value(&hc.test) { + serde_yaml::Value::Sequence(parts) => dctypes::HealthcheckTest::Multiple( + parts + .iter() + .filter_map(|p| p.as_str().map(str::to_string)) + .collect(), + ), + _ => dctypes::HealthcheckTest::Single(hc.test.clone()), + }; + dctypes::Healthcheck { + test: Some(test), + interval: Some(hc.interval.clone()), + timeout: Some(hc.timeout.clone()), + retries: i64::from(hc.retries), + ..Default::default() + } +} + +fn compose_deploy(deploy: &crate::cli::config_parser::ServiceDeploy) -> dctypes::Deploy { + use crate::cli::config_parser::DeviceCount; + let devices = deploy + .resources + .as_ref() + .and_then(|r| r.reservations.as_ref()) + .map(|r| { + r.devices + .iter() + .map(|d| dctypes::Device { + driver: d.driver.clone(), + // A `${VAR}` count cannot be typed here and is left out. + count: d.count.as_ref().and_then(|c| match c { + DeviceCount::All => Some(dctypes::DeviceCount::All), + DeviceCount::Count(n) => { + n.value().map(|n| dctypes::DeviceCount::Count(u64::from(n))) + } + }), + device_ids: (!d.device_ids.is_empty()).then(|| d.device_ids.clone()), + capabilities: Some(d.capabilities.clone()), + ..Default::default() + }) + .collect::>() + }); + dctypes::Deploy { + replicas: deploy + .replicas + .as_ref() + .and_then(|r| r.value()) + .map(i64::from), + resources: devices.map(|devices| dctypes::Resources { + reservations: Some(dctypes::Limits { + devices: Some(devices), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + } +} + impl AsRef for App { fn as_ref(&self) -> &forms::project::DockerImage { &self.docker_image diff --git a/src/forms/project/deploy.rs b/src/forms/project/deploy.rs index c0d8db4e..8d403701 100644 --- a/src/forms/project/deploy.rs +++ b/src/forms/project/deploy.rs @@ -88,6 +88,11 @@ pub struct Deploy { /// not use this. #[serde(default, skip_serializing_if = "Option::is_none")] pub(crate) proxy_domains: Option, + + /// Set by the server when a project member deploys as the owner, never + /// read from the request. + #[serde(skip)] + pub(crate) initiator_email: Option, } impl std::fmt::Debug for Deploy { diff --git a/src/forms/project/payload.rs b/src/forms/project/payload.rs index d948491c..16e91ed8 100644 --- a/src/forms/project/payload.rs +++ b/src/forms/project/payload.rs @@ -13,6 +13,11 @@ pub struct Payload { pub(crate) deployment_hash: Option, pub(crate) user_token: Option, pub(crate) user_email: Option, + /// A project member who started this deployment as the owner; the install + /// service reports to them as well. Set by the server only: project + /// metadata comes from the client, so it is never read from there. + #[serde(default, skip_deserializing, skip_serializing_if = "Option::is_none")] + pub(crate) initiator_email: Option, #[serde( default, rename = "commonDomain", @@ -223,4 +228,30 @@ mod tests { "proxy_domains must be omitted when unset" ); } + + #[test] + fn initiator_email_is_sent_but_never_read_from_project_metadata() { + // A member deploys as the owner; the install service also reports to + // the member. Project metadata comes from the client, so an address + // there must not become a report recipient. + let project = models::Project::new( + "owner-1".to_string(), + "team-stack".to_string(), + json!({ + "stack": {"stack_code": "team-stack"}, + "custom": {"web": [], "custom_stack_code": "team-stack"}, + "initiator_email": "someone@example.com" + }), + json!({}), + ); + let mut payload = Payload::try_from(&project).expect("payload"); + assert_eq!(payload.initiator_email, None); + + let empty = serde_json::to_value(&payload).expect("serialize"); + assert!(empty.get("initiator_email").is_none()); + + payload.initiator_email = Some("member@example.com".to_string()); + let serialized = serde_json::to_value(&payload).expect("serialize"); + assert_eq!(serialized["initiator_email"], json!("member@example.com")); + } } diff --git a/src/forms/user.rs b/src/forms/user.rs index 4de809d0..82f5ec7d 100644 --- a/src/forms/user.rs +++ b/src/forms/user.rs @@ -61,6 +61,9 @@ pub struct User { #[serde(rename = "suspension_hints")] pub suspension_hints: Option, pub role: String, + /// Every role of the user; `role` carries one of them in no fixed order. + #[serde(default)] + pub roles: Vec, } #[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)] @@ -129,6 +132,15 @@ pub struct SuspensionHints { pub reason: String, } +/// The role Casbin checks. `root` is the User Service role that grants admin +/// access, so a user who has it is an admin whichever role `role` carries. +fn access_role(role: String, roles: &[String]) -> String { + if roles.iter().any(|r| r == "root") { + return "root".to_string(); + } + role +} + impl TryInto for UserForm { type Error = String; fn try_into(self) -> Result { @@ -138,10 +150,44 @@ impl TryInto for UserForm { last_name: self.user.last_name.unwrap_or("Noname".to_string()), email: self.user.email, email_confirmed: self.user.email_confirmed, - role: self.user.role, + role: access_role(self.user.role, &self.user.roles), mfa_verified: self.user.mfa_verified.unwrap_or(false) || self.user.two_factor_verified.unwrap_or(false), access_token: None, }) } } + +#[cfg(test)] +mod tests { + use super::*; + + fn me(roles: serde_json::Value) -> UserModel { + let mut user = serde_json::json!({ + "_id": "u1", "email": "u@example.com", "email_confirmed": true, + "deployments_left": 1, "role": "creator", "currency": null, + "password_change_required": false, "billing_first_name": null, "billing_last_name": null + }); + if !roles.is_null() { + user["roles"] = roles; + } + let form: UserForm = serde_json::from_value(serde_json::json!({ "user": user })).unwrap(); + form.try_into().unwrap() + } + + #[test] + fn root_among_the_roles_is_the_access_role() { + // `role` holds one role in no fixed order: creator here, root as well. + assert_eq!(me(serde_json::json!(["creator", "root"])).role, "root"); + } + + #[test] + fn without_root_the_single_role_stays() { + assert_eq!(me(serde_json::json!(["creator", "user"])).role, "creator"); + assert_eq!( + me(serde_json::Value::Null).role, + "creator", + "an older /me without `roles`" + ); + } +} diff --git a/src/health/checks.rs b/src/health/checks.rs index 872f95b3..619c04e6 100644 --- a/src/health/checks.rs +++ b/src/health/checks.rs @@ -321,7 +321,7 @@ impl HealthChecker { let vault_address = &self.settings.vault.address; let health_url = format!("{}/v1/sys/health", vault_address); - match reqwest::Client::builder() + match crate::helpers::vault::http_client_builder(&self.settings.vault) .timeout(Duration::from_secs(5)) .build() { diff --git a/src/helpers/bake_job.rs b/src/helpers/bake_job.rs new file mode 100644 index 00000000..3e73d10d --- /dev/null +++ b/src/helpers/bake_job.rs @@ -0,0 +1,250 @@ +//! Bake started from the API (`POST /api/admin/bake`) instead of the `bake` +//! binary. +//! +//! The binary needs the operator to type the box's IP, its SSH key, the +//! version and the health URL. Here they come from what stacker already +//! holds: the deployment's server row (IP, SSH user and port), the server's +//! SSH key in Vault, and the template's latest approved version. These pure +//! helpers check what the box reports before anything on it is changed. + +/// The stack's project directory on the build box, from +/// `docker compose ls --all --format json`. +/// +/// The deploy puts each stack in its own directory under the deploy root; +/// platform services (status panel, proxy manager) have their own. Anything +/// but exactly one project directory is refused: sanitizing the wrong one +/// would publish the author's secrets. +pub fn project_dir_from_compose_ls(stdout: &str, platform_dirs: &[&str]) -> Result { + let projects: Vec = serde_json::from_str(stdout.trim()) + .map_err(|e| format!("could not read `docker compose ls` output: {e}"))?; + + let mut dirs: Vec = projects + .iter() + .filter_map(|p| p.get("ConfigFiles").and_then(|v| v.as_str())) + .filter_map(|files| files.split(',').next()) + .filter_map(|file| file.trim().rsplit_once('/').map(|(dir, _)| dir.to_string())) + .filter(|dir| !platform_dirs.contains(&dir.as_str())) + .collect(); + dirs.sort(); + dirs.dedup(); + + match dirs.as_slice() { + [dir] if is_safe_dir(dir) => Ok(dir.clone()), + [dir] => Err(format!("refusing the stack directory `{dir}`: not a plain absolute path")), + [] => Err("no stack is deployed on the build box (only platform services)".to_string()), + many => Err(format!( + "more than one stack is deployed on the build box: {}. Bake needs a box with one stack.", + many.join(", ") + )), + } +} + +/// An absolute path of letters, digits and `/ _ . -` only: it goes into a +/// shell command on the box. +fn is_safe_dir(dir: &str) -> bool { + dir.starts_with('/') + && !dir.contains("..") + && dir + .chars() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, '/' | '_' | '.' | '-')) +} + +/// Every service of the stack is up, from `docker compose ps --all --format +/// json` (one JSON object per line in compose v2.21+, a JSON array before). +/// +/// Running without a healthcheck or healthy counts as up; a one-shot job that +/// exited 0 (migrations, init) is done, not down. +pub fn check_compose_services(stdout: &str) -> Result<(), String> { + let text = stdout.trim(); + let services: Vec = if text.starts_with('[') { + serde_json::from_str(text) + .map_err(|e| format!("could not read `docker compose ps`: {e}"))? + } else { + text.lines() + .filter(|line| !line.trim().is_empty()) + .map(serde_json::from_str) + .collect::>() + .map_err(|e| format!("could not read `docker compose ps`: {e}"))? + }; + if services.is_empty() { + return Err("the stack has no services on the build box".to_string()); + } + + let mut down = Vec::new(); + for service in &services { + let field = |name: &str| service.get(name).and_then(|v| v.as_str()).unwrap_or(""); + let name = field("Service"); + let state = field("State"); + let health = field("Health"); + let exit_code = service + .get("ExitCode") + .and_then(|v| v.as_i64()) + .unwrap_or(-1); + let up = match state { + "running" => health.is_empty() || health == "healthy", + "exited" => exit_code == 0, + _ => false, + }; + if !up { + let detail = if health.is_empty() { + state.to_string() + } else { + format!("{state}, {health}") + }; + down.push(format!("{name} ({detail})")); + } + } + if down.is_empty() { + Ok(()) + } else { + Err(format!("services not up: {}", down.join(", "))) + } +} + +/// The health URL must point at the build box itself: the bake runs on the +/// stacker backend, so any other host would make it a request the caller +/// chose from inside the platform's network. +pub fn check_health_url(url: &str, server_ip: &str) -> Result<(), String> { + let parsed = reqwest::Url::parse(url).map_err(|e| format!("health_url is not a URL: {e}"))?; + if !matches!(parsed.scheme(), "http" | "https") { + return Err("health_url must be http or https".to_string()); + } + if !parsed.username().is_empty() || parsed.password().is_some() { + return Err("health_url must not carry credentials".to_string()); + } + match parsed.host_str() { + Some(host) if host == server_ip => Ok(()), + _ => Err(format!( + "health_url must point at the build box ({server_ip})" + )), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const PLATFORM: &[&str] = &[ + "/home/trydirect/statuspanel", + "/home/trydirect/nginx_proxy_manager", + ]; + + #[test] + fn the_stack_directory_is_found_next_to_platform_services() { + let ls = r#"[ + {"Name":"statuspanel","Status":"running(1)","ConfigFiles":"/home/trydirect/statuspanel/docker-compose.yml"}, + {"Name":"ai-workflows-v2-12","Status":"running(4)","ConfigFiles":"/home/trydirect/ai-workflows-v2-12/docker-compose.yml"} + ]"#; + assert_eq!( + project_dir_from_compose_ls(ls, PLATFORM).unwrap(), + "/home/trydirect/ai-workflows-v2-12" + ); + } + + #[test] + fn several_config_files_of_one_project_point_at_its_directory() { + let ls = r#"[{"Name":"app","Status":"running(2)","ConfigFiles":"/home/trydirect/app/docker-compose.yml,/home/trydirect/app/docker-compose.override.yml"}]"#; + assert_eq!( + project_dir_from_compose_ls(ls, PLATFORM).unwrap(), + "/home/trydirect/app" + ); + } + + #[test] + fn no_stack_or_two_stacks_are_refused() { + let only_platform = r#"[{"Name":"statuspanel","Status":"running(1)","ConfigFiles":"/home/trydirect/statuspanel/docker-compose.yml"}]"#; + assert!(project_dir_from_compose_ls(only_platform, PLATFORM).is_err()); + + let two = r#"[ + {"Name":"a","Status":"running(1)","ConfigFiles":"/home/trydirect/a/docker-compose.yml"}, + {"Name":"b","Status":"running(1)","ConfigFiles":"/home/trydirect/b/docker-compose.yml"} + ]"#; + let err = project_dir_from_compose_ls(two, PLATFORM).unwrap_err(); + assert!( + err.contains("/home/trydirect/a") && err.contains("/home/trydirect/b"), + "{err}" + ); + + assert!(project_dir_from_compose_ls("not json", PLATFORM).is_err()); + } + + #[test] + fn a_directory_unsafe_for_a_shell_command_is_refused() { + // The directory is put into `cd && ...` on the box. + for dir in [ + "/home/trydirect/a b", + "/home/trydirect/a;rm -rf /", + "/home/trydirect/$(id)", + "relative/dir", + ] { + let ls = format!( + r#"[{{"Name":"x","Status":"running(1)","ConfigFiles":"{dir}/docker-compose.yml"}}]"# + ); + assert!(project_dir_from_compose_ls(&ls, PLATFORM).is_err(), "{dir}"); + } + } + + #[test] + fn services_running_healthy_or_finished_are_up() { + let ps = "\ +{\"Service\":\"n8n\",\"State\":\"running\",\"Health\":\"healthy\",\"ExitCode\":0} +{\"Service\":\"redis\",\"State\":\"running\",\"Health\":\"\",\"ExitCode\":0} +{\"Service\":\"migrate\",\"State\":\"exited\",\"Health\":\"\",\"ExitCode\":0}"; + assert_eq!(check_compose_services(ps), Ok(())); + + let array = r#"[{"Service":"n8n","State":"running","Health":"healthy","ExitCode":0}]"#; + assert_eq!(check_compose_services(array), Ok(())); + } + + #[test] + fn a_starting_unhealthy_crashed_or_missing_service_fails_the_gate() { + for (ps, culprit) in [ + ( + r#"{"Service":"db","State":"running","Health":"starting","ExitCode":0}"#, + "db", + ), + ( + r#"{"Service":"db","State":"running","Health":"unhealthy","ExitCode":0}"#, + "db", + ), + ( + r#"{"Service":"api","State":"exited","Health":"","ExitCode":1}"#, + "api", + ), + ( + r#"{"Service":"api","State":"restarting","Health":"","ExitCode":1}"#, + "api", + ), + ] { + let err = check_compose_services(ps).unwrap_err(); + assert!(err.contains(culprit), "{err}"); + } + assert!( + check_compose_services("").is_err(), + "no services is not healthy" + ); + assert!(check_compose_services("garbage").is_err()); + } + + #[test] + fn the_health_url_must_point_at_the_build_box() { + assert_eq!( + check_health_url("http://203.0.113.7/health", "203.0.113.7"), + Ok(()) + ); + assert_eq!( + check_health_url("https://203.0.113.7:8443/api/ping", "203.0.113.7"), + Ok(()) + ); + for url in [ + "http://198.51.100.1/health", + "http://203.0.113.7.evil.example/health", + "http://user@198.51.100.1/", + "file:///etc/passwd", + "http://localhost/health", + "not a url", + ] { + assert!(check_health_url(url, "203.0.113.7").is_err(), "{url}"); + } + } +} diff --git a/src/helpers/mod.rs b/src/helpers/mod.rs index 95919c6d..3a6876aa 100644 --- a/src/helpers/mod.rs +++ b/src/helpers/mod.rs @@ -36,6 +36,7 @@ pub use dockerhub::*; pub use cloud::*; pub mod audit_cache; pub mod bake; +pub mod bake_job; pub mod bake_finalize; pub mod bake_registry; pub mod cloud_init; diff --git a/src/helpers/redact.rs b/src/helpers/redact.rs index 5ba1f349..be12910d 100644 --- a/src/helpers/redact.rs +++ b/src/helpers/redact.rs @@ -159,6 +159,31 @@ const CONFIG_CONTRACT_KEY: &str = "config_contract"; /// Replace, in place, the values of env entries whose key is in `keys`. /// Handles the same shapes as [`redact_sensitive_json_values`] plus `KEY=value` /// strings in environment arrays. +/// Whether `value` only names another variable (`${NAME}` or `$NAME`). +/// +/// Such a value is not a secret: it is how one generated secret feeds several +/// keys, so stripping keeps it. `${NAME:-default}` does not count, the default +/// can be the secret itself. +fn is_env_reference(value: &str) -> bool { + let name = match value.strip_prefix("${") { + Some(rest) => match rest.strip_suffix('}') { + Some(name) => name, + None => return false, + }, + None => match value.strip_prefix('$') { + Some(name) => name, + None => return false, + }, + }; + let mut chars = name.chars(); + matches!(chars.next(), Some(c) if c == '_' || c.is_ascii_alphabetic()) + && chars.all(|c| c == '_' || c.is_ascii_alphanumeric()) +} + +fn json_is_env_reference(value: &serde_json::Value) -> bool { + value.as_str().map_or(false, is_env_reference) +} + pub fn strip_json_values_for_keys( value: &mut serde_json::Value, keys: &BTreeSet, @@ -170,7 +195,7 @@ pub fn strip_json_values_for_keys( if let Some(name) = map.get("key").and_then(|v| v.as_str()) { if keys.contains(name) { if let Some(val) = map.get_mut("value") { - if !val.is_null() { + if !val.is_null() && !json_is_env_reference(val) { *val = serde_json::Value::String(replacement.to_string()); } } @@ -187,7 +212,9 @@ pub fn strip_json_values_for_keys( continue; } if keys.contains(key) && !val.is_null() { - *val = serde_json::Value::String(replacement.to_string()); + if !json_is_env_reference(val) { + *val = serde_json::Value::String(replacement.to_string()); + } } else { strip_json_values_for_keys(val, keys, replacement); } @@ -198,7 +225,9 @@ pub fn strip_json_values_for_keys( if let serde_json::Value::String(s) = item { if let Some(eq) = s.find('=') { if keys.contains(&s[..eq]) { - *s = format!("{}={}", &s[..eq], replacement); + if !is_env_reference(&s[eq + 1..]) { + *s = format!("{}={}", &s[..eq], replacement); + } continue; } } @@ -225,7 +254,9 @@ fn strip_yaml_values_for_keys( continue; } if keys.contains(k) && !val.is_null() { - *val = serde_yaml::Value::String(replacement.to_string()); + if !val.as_str().map_or(false, is_env_reference) { + *val = serde_yaml::Value::String(replacement.to_string()); + } continue; } } @@ -236,7 +267,7 @@ fn strip_yaml_values_for_keys( for item in seq.iter_mut() { if let serde_yaml::Value::String(s) = item { if let Some(eq) = s.find('=') { - if keys.contains(&s[..eq]) { + if keys.contains(&s[..eq]) && !is_env_reference(&s[eq + 1..]) { *s = format!("{}={}", &s[..eq], replacement); } } @@ -501,4 +532,70 @@ config_contract: "policy survives:\n{out}" ); } + + // A value that only names another variable is not a secret. It is how one + // generated secret feeds two keys (plane: the app's AWS_SECRET_ACCESS_KEY + // is "${MINIO_ROOT_PASSWORD}"); blanking it broke that link and the two + // services got different values. A default inside the reference can hold + // the secret itself, so `${VAR:-value}` is still blanked. + + #[test] + fn strip_json_keeps_references_to_other_variables() { + let mut v = json!({ + "services": { + "api": { + "environment": { + "SECRET_KEY": "${MINIO_ROOT_PASSWORD}", + "POSTGRES_PASSWORD": "${DB_PASSWORD:-author-secret}" + } + }, + "worker": { "environment": ["SECRET_KEY=$MINIO_ROOT_PASSWORD"] } + }, + "vars": [{ "key": "SECRET_KEY", "value": "${MINIO_ROOT_PASSWORD}" }] + }); + + strip_json_values_for_keys(&mut v, &generated_keys(), ""); + + let env = &v["services"]["api"]["environment"]; + assert_eq!(env["SECRET_KEY"], "${MINIO_ROOT_PASSWORD}"); + assert_eq!(env["POSTGRES_PASSWORD"], "", "a default may be the secret"); + assert_eq!( + v["services"]["worker"]["environment"][0], + "SECRET_KEY=$MINIO_ROOT_PASSWORD" + ); + assert_eq!(v["vars"][0]["value"], "${MINIO_ROOT_PASSWORD}"); + } + + #[test] + fn strip_yaml_keeps_references_to_other_variables() { + let yaml = "\ +services: + api: + environment: + SECRET_KEY: ${MINIO_ROOT_PASSWORD} + POSTGRES_PASSWORD: ${DB_PASSWORD:-author-secret} + worker: + environment: + - SECRET_KEY=${MINIO_ROOT_PASSWORD} +"; + let out = strip_yaml_string_for_keys(yaml, &generated_keys(), ""); + let parsed: serde_yaml::Value = serde_yaml::from_str(&out).unwrap(); + + assert_eq!( + parsed["services"]["api"]["environment"]["SECRET_KEY"].as_str(), + Some("${MINIO_ROOT_PASSWORD}"), + "{out}" + ); + assert_eq!( + parsed["services"]["api"]["environment"]["POSTGRES_PASSWORD"].as_str(), + Some(""), + "{out}" + ); + assert_eq!( + parsed["services"]["worker"]["environment"][0].as_str(), + Some("SECRET_KEY=${MINIO_ROOT_PASSWORD}"), + "{out}" + ); + assert!(!out.contains("author-secret"), "{out}"); + } } diff --git a/src/helpers/vault.rs b/src/helpers/vault.rs index 24d0e41c..6f904bf5 100644 --- a/src/helpers/vault.rs +++ b/src/helpers/vault.rs @@ -25,48 +25,58 @@ impl std::fmt::Debug for VaultClient { } } -impl VaultClient { - pub fn new(settings: &VaultSettings) -> Self { - let mut client_builder = Client::builder(); - if let (Some(cert), Some(key)) = (&settings.client_cert, &settings.client_key) { - match Identity::from_pkcs8_pem(cert.as_bytes(), key.as_bytes()) { - Ok(identity) => { - client_builder = client_builder.identity(identity); - } - Err(e) => { - tracing::warn!("Failed to load mTLS identity for Vault client: {}", e); - } +/// The HTTP client for Vault: mTLS identity and the private CA from the +/// settings. Every Vault call must use it, the health check included - a bare +/// client fails TLS verification against a private-CA Vault while Vault itself +/// is fine. +pub fn http_client_builder(settings: &VaultSettings) -> reqwest::ClientBuilder { + let mut client_builder = Client::builder(); + if let (Some(cert), Some(key)) = (&settings.client_cert, &settings.client_key) { + match Identity::from_pkcs8_pem(cert.as_bytes(), key.as_bytes()) { + Ok(identity) => { + client_builder = client_builder.identity(identity); + } + Err(e) => { + tracing::warn!("Failed to load mTLS identity for Vault client: {}", e); } } - match &settings.ca_cert { - Some(ca_pem) => match Certificate::from_pem(ca_pem.as_bytes()) { - Ok(ca) => { - client_builder = client_builder.add_root_certificate(ca); - } - Err(e) => { - tracing::error!( - "Failed to load CA certificate for Vault client, falling back to the \ - system trust store — every Vault call will fail if Vault uses a \ - private CA: {}", - e - ); - } - }, - // Silence here is what made this expensive to find: a private-CA - // Vault over https with no CA configured fails every call with - // "unable to get local issuer certificate" and nothing ever says - // the CA was missing. Set VAULT_CACERT to the CA's PEM *contents*. - None if settings.address.starts_with("https://") => { + } + match &settings.ca_cert { + Some(ca_pem) => match Certificate::from_pem(ca_pem.as_bytes()) { + Ok(ca) => { + client_builder = client_builder.add_root_certificate(ca); + } + Err(e) => { tracing::error!( - "Vault address {} is https but no CA certificate is configured \ - (VAULT_CACERT unset or empty) — relying on the system trust store. \ - If Vault uses a private CA, every Vault call will fail TLS verification.", - settings.address + "Failed to load CA certificate for Vault client, falling back to the \ + system trust store — every Vault call will fail if Vault uses a \ + private CA: {}", + e ); } - None => {} + }, + // Silence here is what made this expensive to find: a private-CA + // Vault over https with no CA configured fails every call with + // "unable to get local issuer certificate" and nothing ever says + // the CA was missing. Set VAULT_CACERT to the CA's PEM *contents*. + None if settings.address.starts_with("https://") => { + tracing::error!( + "Vault address {} is https but no CA certificate is configured \ + (VAULT_CACERT unset or empty) — relying on the system trust store. \ + If Vault uses a private CA, every Vault call will fail TLS verification.", + settings.address + ); } - let client = client_builder.build().unwrap_or_else(|_| Client::new()); + None => {} + } + client_builder +} + +impl VaultClient { + pub fn new(settings: &VaultSettings) -> Self { + let client = http_client_builder(settings) + .build() + .unwrap_or_else(|_| Client::new()); Self { client, diff --git a/src/models/bake_job.rs b/src/models/bake_job.rs new file mode 100644 index 00000000..ea745c05 --- /dev/null +++ b/src/models/bake_job.rs @@ -0,0 +1,22 @@ +use chrono::{DateTime, Utc}; +use serde::{Deserialize, Serialize}; + +/// A bake started through `POST /api/admin/bake`. `running` until the job ends +/// as `done` (with the snapshot) or `failed` (with the reason). The build box +/// (`server_id`, `server_ip`) is never removed by the job. +#[derive(Debug, Clone, Serialize, Deserialize, sqlx::FromRow)] +pub struct BakeJob { + pub id: uuid::Uuid, + pub stack: String, + pub version: String, + pub deployment_id: i32, + pub server_id: i32, + pub server_ip: String, + pub status: String, + pub image_id: Option, + pub snapshot_id: Option, + pub error: Option, + pub created_by: String, + pub created_at: DateTime, + pub updated_at: DateTime, +} diff --git a/src/models/baked_snapshot.rs b/src/models/baked_snapshot.rs index 18f2a56a..e1c17fc5 100644 --- a/src/models/baked_snapshot.rs +++ b/src/models/baked_snapshot.rs @@ -28,6 +28,11 @@ pub struct BakedSnapshot { /// check rather than becoming undeployable. #[sqlx(default)] pub required_env_keys: Option, + /// Server type of the build box (e.g. `cpx42`). Quick Deploy provisions + /// this type: the image carries that type's disk. `None` for snapshots + /// baked before this column existed — the default type applies. + #[sqlx(default)] + pub server_type: Option, pub created_at: DateTime, } diff --git a/src/models/mod.rs b/src/models/mod.rs index b99d3855..cac5b82c 100644 --- a/src/models/mod.rs +++ b/src/models/mod.rs @@ -2,6 +2,7 @@ mod agent; pub mod agent_audit_log; pub mod agent_protocol; mod agreement; +pub mod bake_job; pub mod baked_snapshot; pub mod cdc; mod chat; @@ -31,6 +32,7 @@ pub mod user; pub use agent::*; pub use agent_audit_log::AgentAuditLog; pub use agreement::*; +pub use bake_job::*; pub use baked_snapshot::*; pub use chat::*; pub use client::*; diff --git a/src/models/project.rs b/src/models/project.rs index 349dd45c..477a3f66 100644 --- a/src/models/project.rs +++ b/src/models/project.rs @@ -163,6 +163,10 @@ pub struct Project { pub is_protected: bool, // prevent deletion when true pub deletion_scheduled_at: Option>, // auto-deletion date for stale projects pub deletion_warning_sent_at: Option>, // when the last warning was sent + /// Owner's choice: members get server access and deploy reports. + /// Read and set through db::project; kept out of API bodies. + #[serde(skip)] + pub members_server_access: bool, } #[derive(Debug, Clone, PartialEq, Serialize, Deserialize, sqlx::FromRow)] @@ -189,6 +193,7 @@ impl Project { is_protected: false, deletion_scheduled_at: None, deletion_warning_sent_at: None, + members_server_access: false, } } @@ -236,6 +241,7 @@ impl Default for Project { is_protected: false, deletion_scheduled_at: None, deletion_warning_sent_at: None, + members_server_access: false, } } } diff --git a/src/models/project_app.rs b/src/models/project_app.rs index 4c37f51d..2d60c066 100644 --- a/src/models/project_app.rs +++ b/src/models/project_app.rs @@ -108,6 +108,12 @@ pub struct ProjectApp { #[serde(skip_serializing_if = "Option::is_none")] #[sqlx(default)] pub config_contract: Option, + /// Compose settings beyond image and wiring, as declared in stacker.yml: + /// `{cap_add, cap_drop, devices, profiles, aliases, deploy}`. See + /// shared-fixtures/api-contracts/stack-service-runtime-fields.json. + #[serde(default, skip_serializing_if = "Option::is_none")] + #[sqlx(default)] + pub runtime: Option, } impl ProjectApp { @@ -146,6 +152,7 @@ impl ProjectApp { parent_app_code: None, deployment_id: None, config_contract: None, + runtime: None, } } @@ -218,6 +225,7 @@ impl Default for ProjectApp { parent_app_code: None, deployment_id: None, config_contract: None, + runtime: None, } } } diff --git a/src/project_app/mapping.rs b/src/project_app/mapping.rs index 99010659..55ea8ab9 100644 --- a/src/project_app/mapping.rs +++ b/src/project_app/mapping.rs @@ -370,5 +370,6 @@ pub(crate) fn merge_project_app(existing: ProjectApp, incoming: ProjectApp) -> P .config_contract .clone() .or_else(|| existing.config_contract.clone()), + runtime: incoming.runtime.or(existing.runtime), } } diff --git a/src/project_app/mod.rs b/src/project_app/mod.rs index 49d31a0d..38a4f5f1 100644 --- a/src/project_app/mod.rs +++ b/src/project_app/mod.rs @@ -50,7 +50,7 @@ const PLATFORM_DISPLAY_CODES: &[&str] = &[ /// Directories platform-managed services are deployed into, by the convention /// in `stacker/docs/APP_DEPLOYMENT.md`: project services live under the /// project directory, platform ones get their own. -const PLATFORM_INSTALL_DIRS: &[&str] = &[ +pub(crate) const PLATFORM_INSTALL_DIRS: &[&str] = &[ "/home/trydirect/statuspanel", "/home/trydirect/nginx_proxy_manager", ]; diff --git a/src/project_app/sync.rs b/src/project_app/sync.rs index f148393f..30431b9c 100644 --- a/src/project_app/sync.rs +++ b/src/project_app/sync.rs @@ -54,6 +54,43 @@ fn app_networks_json( } } +/// The healthcheck in the shape the config renderer reads: `test` as a list. +fn healthcheck_json(hc: &crate::cli::config_parser::ComposeHealthcheck) -> Value { + let test = match crate::cli::compose_service_sync::healthcheck_test_value(&hc.test) { + serde_yaml::Value::Sequence(parts) => parts + .iter() + .filter_map(|p| p.as_str().map(str::to_string)) + .collect::>(), + _ => vec!["CMD-SHELL".to_string(), hc.test.clone()], + }; + json!({ + "test": test, + "interval": hc.interval, + "timeout": hc.timeout, + "retries": hc.retries, + }) +} + +/// `project_app.runtime`: the settings with no column of their own. +fn app_runtime_json(app: &ProjectFormApp) -> Option { + let mut runtime = serde_json::Map::new(); + for (key, values) in [ + ("cap_add", &app.cap_add), + ("cap_drop", &app.cap_drop), + ("devices", &app.devices), + ("profiles", &app.profiles), + ("aliases", &app.aliases), + ] { + if !values.is_empty() { + runtime.insert(key.to_string(), json!(values)); + } + } + if let Some(deploy) = &app.deploy { + runtime.insert("deploy".to_string(), json!(deploy)); + } + (!runtime.is_empty()).then_some(Value::Object(runtime)) +} + fn build_project_app( project_id: i32, deploy_order: i32, @@ -84,6 +121,9 @@ fn build_project_app( project_app.command = non_empty_string(app.command.as_deref()); project_app.entrypoint = non_empty_string(app.entrypoint.as_deref()); project_app.networks = app_networks_json(app, all_networks); + project_app.depends_on = (!app.depends_on.is_empty()).then(|| json!(app.depends_on)); + project_app.healthcheck = app.healthcheck.as_ref().map(healthcheck_json); + project_app.runtime = app_runtime_json(app); project_app.enabled = Some(true); project_app.deploy_order = Some(deploy_order); project_app.config_contract = app.config_contract.clone(); diff --git a/src/routes/admin_bake.rs b/src/routes/admin_bake.rs new file mode 100644 index 00000000..014963ed --- /dev/null +++ b/src/routes/admin_bake.rs @@ -0,0 +1,374 @@ +//! `POST /api/admin/bake` and `GET /api/admin/bake/{id}`: bake a deployed +//! build box into a Quick Deploy snapshot without the operator typing its IP, +//! SSH key or version. +//! +//! Everything that can be checked without touching the box is checked before +//! the job starts, and answered at once: the deployment and its server, the +//! approved template with a usable contract, a health URL on the box, and the +//! box being one Hetzner knows. Only then does the job connect, check that +//! every service is up, sanitize the box, snapshot it and record the snapshot. +//! The box is never removed: the result lists it for the operator. + +use std::collections::BTreeSet; +use std::sync::Arc; + +use actix_web::{get, post, web, HttpResponse, Responder, Result}; +use serde::Deserialize; +use sqlx::PgPool; + +use crate::connectors::config::HetznerConfig; +use crate::connectors::hetzner::{ + HetznerCloudClient, HetznerCloudConnector, HetznerSnapshotTarget, +}; +use crate::db; +use crate::helpers::bake_job::{ + check_compose_services, check_health_url, project_dir_from_compose_ls, +}; +use crate::helpers::{JsonResponse, VaultClient}; +use crate::models; + +#[derive(Debug, Deserialize)] +pub struct BakeRequest { + /// The build box's deployment (from the CLI deployment lock). + pub deployment_id: i32, + /// Approved marketplace template slug; its latest version is baked. + pub stack: String, + /// The stack's health endpoint on the build box. + pub health_url: String, + /// The build box's Hetzner server id, as the `bake` binary takes it. + /// Needed when the project holds several server records - every + /// `deploy --force-new` adds one, and Hetzner reuses IPs. + #[serde(default)] + pub server_id: Option, +} + +/// What the job needs, all resolved and checked before it starts. +struct Prepared { + server: models::Server, + server_ip: String, + version: String, + contract: serde_json::Value, + parsed_contract: crate::cli::config_parser::ConfigContract, + protected_keys: BTreeSet, + token: String, + connector: Arc, + target: HetznerSnapshotTarget, +} + +async fn prepare(pool: &PgPool, form: &BakeRequest) -> Result { + let deployment = db::deployment::fetch(pool, form.deployment_id) + .await + .map_err(JsonResponse::internal_server_error)? + .ok_or_else(|| JsonResponse::not_found("deployment not found"))?; + + let servers: Vec = db::server::fetch_by_project(pool, deployment.project_id) + .await + .map_err(JsonResponse::internal_server_error)? + .into_iter() + .filter(|s| { + s.srv_ip + .as_deref() + .map_or(false, |ip| !ip.trim().is_empty()) + }) + .collect(); + if servers.is_empty() { + return Err(JsonResponse::<()>::build() + .conflict("the deployment's project has no server with an IP")); + } + if servers.len() > 1 && form.server_id.is_none() { + return Err(JsonResponse::<()>::build().conflict( + "the deployment's project has more than one server; pass server_id, the build \ + box's Hetzner server id", + )); + } + + // The latest approved version: the contract describes it, so version and + // contract cannot disagree. + let (template, version) = db::marketplace::get_by_slug_with_latest(pool, &form.stack) + .await + .map_err(|_| JsonResponse::not_found("no approved template with this slug"))?; + let version = version + .map(|v| v.version) + .ok_or_else(|| JsonResponse::<()>::build().conflict("the template has no version"))?; + let contract = db::marketplace::get_config_contract(pool, template.id) + .await + .map_err(JsonResponse::internal_server_error)?; + if contract.is_null() { + return Err(JsonResponse::<()>::build() + .conflict("the template declares no config_contract; nothing would be sanitized")); + } + let protected_keys = crate::helpers::bake_finalize::protected_keys_from_contract(&contract); + let parsed_contract = serde_json::from_value(contract.clone()).map_err(|e| { + JsonResponse::<()>::build().conflict(format!( + "the stored config_contract could not be parsed: {e}" + )) + })?; + crate::helpers::bake_finalize::check_contract_usable(&protected_keys, false) + .map_err(|e| JsonResponse::<()>::build().conflict(e.to_string()))?; + + let htz = HetznerConfig::from_env(); + let token = htz.token.filter(|t| !t.trim().is_empty()).ok_or_else(|| { + JsonResponse::internal_server_error("HETZNER_TOKEN is not set on the stacker backend") + })?; + let client = HetznerCloudClient::new(htz.base_url) + .map_err(|e| JsonResponse::internal_server_error(e.to_string()))?; + let connector: Arc = Arc::new(client); + + // Sanitizing removes SSH access, so the box must be the one Hetzner has at + // the record's IP - and, when server_id is given, that very server - + // before anything on it changes. Newest record first: an older record at + // the same IP belongs to a removed box. + let mut newest_first = servers.clone(); + newest_first.sort_by_key(|s| std::cmp::Reverse(s.id)); + let mut tried = BTreeSet::new(); + let mut chosen = None; + for candidate in &newest_first { + let ip = candidate.srv_ip.clone().unwrap_or_default(); + if !tried.insert(ip.clone()) { + continue; + } + let target = HetznerSnapshotTarget { + provider_server_id: None, + server_name: None, + public_ip: Some(ip.clone()), + }; + match connector.resolve_snapshot_target(&token, &target).await { + Ok(id) if form.server_id.map_or(true, |wanted| wanted == id) => { + chosen = Some((candidate.clone(), id, target)); + break; + } + Ok(_) => continue, + Err(e) if servers.len() == 1 => { + return Err(JsonResponse::<()>::build().conflict(format!( + "Hetzner does not know a server at {ip} in the platform's project: {e}" + ))) + } + Err(_) => continue, + } + } + let (server, provider_server_id, target) = chosen.ok_or_else(|| { + JsonResponse::<()>::build().conflict(format!( + "Hetzner server {} is not at the IP of any server of this deployment's project", + form.server_id.unwrap_or_default() + )) + })?; + let server_ip = server.srv_ip.clone().unwrap_or_default(); + check_health_url(&form.health_url, &server_ip).map_err(JsonResponse::bad_request)?; + let target = HetznerSnapshotTarget { + provider_server_id: Some(provider_server_id), + ..target + }; + + Ok(Prepared { + server, + server_ip, + version, + contract, + parsed_contract, + protected_keys, + token, + connector, + target, + }) +} + +#[tracing::instrument(name = "Start a bake (admin)", skip_all)] +#[post("")] +pub async fn start_handler( + admin: web::ReqData>, // role enforced by Casbin + form: web::Json, + pg_pool: web::Data, + vault_client: web::Data, +) -> Result { + let form = form.into_inner(); + let prepared = prepare(pg_pool.get_ref(), &form).await?; + + if db::bake_job::running_for_server(pg_pool.get_ref(), prepared.server.id) + .await + .map_err(JsonResponse::internal_server_error)? + .is_some() + { + return Err( + JsonResponse::<()>::build().conflict("a bake of this build box is already running") + ); + } + + let job = db::bake_job::insert( + pg_pool.get_ref(), + db::bake_job::NewBakeJob { + stack: &form.stack, + version: &prepared.version, + deployment_id: form.deployment_id, + server_id: prepared.server.id, + server_ip: &prepared.server_ip, + created_by: &admin.id, + }, + ) + .await + .map_err(JsonResponse::internal_server_error)?; + + let pool = pg_pool.get_ref().clone(); + let vault = vault_client.clone(); // web::Data: an Arc, moved into the job + let job_id = job.id; + let stack = form.stack.clone(); + let health_url = form.health_url.clone(); + tokio::spawn(async move { + let outcome = run_job(&pool, &vault, &stack, &health_url, prepared).await; + let saved = match outcome { + Ok((image_id, snapshot_id)) => { + tracing::info!(%job_id, image_id, "bake finished"); + db::bake_job::mark_done(&pool, job_id, image_id, snapshot_id).await + } + Err(err) => { + tracing::warn!(%job_id, error = %err, "bake failed"); + db::bake_job::mark_failed(&pool, job_id, &err).await + } + }; + if let Err(err) = saved { + tracing::error!(%job_id, error = %err, "could not save the bake result"); + } + }); + + Ok(HttpResponse::Accepted().json(serde_json::json!({ "message": "accepted", "item": job }))) +} + +/// Connect, check, sanitize, snapshot, record. Returns (image_id, snapshot row id). +async fn run_job( + pool: &PgPool, + vault: &VaultClient, + stack: &str, + health_url: &str, + p: Prepared, +) -> Result<(i64, i32), String> { + use crate::helpers::ssh_client::{disconnect_ssh, exec_remote, open_ssh, HostKeyPolicy}; + + let response = reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(30)) + .build() + .map_err(|e| e.to_string())? + .get(health_url) + .send() + .await + .map_err(|e| format!("health check: {health_url} unreachable: {e}"))?; + if !response.status().is_success() { + return Err(format!( + "health check: {health_url} returned {}", + response.status() + )); + } + + let private_key_pem = vault + .fetch_ssh_key(&p.server.user_id, p.server.id) + .await + .map_err(|e| format!("SSH key of the build box: {e}"))?; + let user = p + .server + .ssh_user + .clone() + .unwrap_or_else(|| "root".to_string()); + let port = p + .server + .ssh_port + .and_then(|v| u16::try_from(v).ok()) + .unwrap_or(22); + + let session = open_ssh( + &p.server_ip, + port, + &user, + &private_key_pem, + std::time::Duration::from_secs(30), + &HostKeyPolicy::Pin(p.server.host_key_fingerprint.clone()), + ) + .await + .map_err(|e| format!("ssh connect: {e}"))?; + let inspect = async { + let (ls, stderr, code) = exec_remote(&session, "docker compose ls --all --format json", 60) + .await + .map_err(|e| e.to_string())?; + if code != 0 { + return Err(format!("docker compose ls exited {code}: {stderr}")); + } + let dir = project_dir_from_compose_ls(&ls, crate::project_app::PLATFORM_INSTALL_DIRS)?; + let (ps, stderr, code) = exec_remote( + &session, + &format!("cd {dir} && docker compose ps --all --format json"), + 60, + ) + .await + .map_err(|e| e.to_string())?; + if code != 0 { + return Err(format!("docker compose ps exited {code}: {stderr}")); + } + check_compose_services(&ps)?; + Ok::(dir) + } + .await; + disconnect_ssh(session).await; + let project_dir = inspect?; + + let outcome = crate::helpers::bake_finalize::finalize_build_box( + &crate::helpers::bake_finalize::FinalizeContext { + host: p.server_ip.clone(), + port, + user, + private_key_pem, + project_dir, + stack: stack.to_string(), + contract: p.parsed_contract, + protected_keys: p.protected_keys, + }, + ) + .await + .map_err(|e| e.to_string())?; + + let record = crate::helpers::bake::run_bake( + p.connector.as_ref(), + &p.token, + p.target, + stack, + &p.version, + true, + "", + ) + .await + .map_err(|e| e.to_string())?; + + let required_env_keys = serde_json::Value::Array( + outcome + .required_env_keys + .into_iter() + .map(serde_json::Value::String) + .collect(), + ); + let row = db::baked_snapshot::record( + pool, + &record.stack, + &record.version, + &record.provider, + record.image_id, + record.healthy, + None, + Some(p.contract), + Some(required_env_keys), + p.server.server.as_deref(), + ) + .await?; + Ok((record.image_id, row.id)) +} + +#[tracing::instrument(name = "Bake status (admin)", skip_all)] +#[get("/{id}")] +pub async fn status_handler( + _admin: web::ReqData>, // role enforced by Casbin + path: web::Path<(String,)>, + pg_pool: web::Data, +) -> Result { + let id = uuid::Uuid::parse_str(&path.into_inner().0) + .map_err(|_| JsonResponse::bad_request("invalid bake id"))?; + let job = db::bake_job::fetch(pg_pool.get_ref(), id) + .await + .map_err(JsonResponse::internal_server_error)? + .ok_or_else(|| JsonResponse::not_found("bake not found"))?; + Ok(JsonResponse::build().set_item(job).ok("OK")) +} diff --git a/src/routes/agreement/get.rs b/src/routes/agreement/get.rs index 1df27fe5..5571e0aa 100644 --- a/src/routes/agreement/get.rs +++ b/src/routes/agreement/get.rs @@ -40,3 +40,19 @@ pub async fn accept_handler( None => Err(JsonResponse::not_found("not found")), }) } + +#[tracing::instrument(name = "Get agreement by name.", skip_all)] +#[get("/name/{name}")] +pub async fn get_by_name_handler( + _user: web::ReqData>, + path: web::Path<(String,)>, + pg_pool: web::Data, +) -> Result { + db::agreement::fetch_by_name(pg_pool.get_ref(), &path.0) + .await + .map_err(|err| JsonResponse::internal_server_error(err)) + .and_then(|item| match item { + Some(item) => Ok(JsonResponse::build().set_item(Some(item)).ok("OK")), + None => Err(JsonResponse::not_found("not found")), + }) +} diff --git a/src/routes/chat/admin.rs b/src/routes/chat/admin.rs new file mode 100644 index 00000000..e5179f25 --- /dev/null +++ b/src/routes/chat/admin.rs @@ -0,0 +1,97 @@ +use crate::db; +use crate::helpers::{self, JsonResponse}; +use actix_web::{get, web, Responder, Result}; +use serde::Deserialize; +use serde_json::{json, Value}; +use sqlx::PgPool; + +const MAX_LIMIT: i64 = 50; +const MESSAGES_PER_CHAT: usize = 5; +const MAX_CONTENT_CHARS: usize = 500; + +#[derive(Debug, Deserialize)] +pub struct Query { + pub limit: Option, +} + +fn truncate_content(message: &Value) -> Value { + let mut message = message.clone(); + if let Some(content) = message.get("content").and_then(Value::as_str) { + if content.chars().count() > MAX_CONTENT_CHARS { + let cut: String = content.chars().take(MAX_CONTENT_CHARS).collect(); + message["content"] = Value::String(format!("{cut}…")); + } + } + message +} + +fn last_messages(messages: &Value) -> (usize, Vec) { + match messages.as_array() { + Some(all) => ( + all.len(), + all.iter() + .skip(all.len().saturating_sub(MESSAGES_PER_CHAT)) + .map(truncate_content) + .collect(), + ), + None => (0, vec![]), + } +} + +/// GET /api/admin/chat/recent[?limit=N] +/// Admin-only. Returns the last messages of the most recently active chats, +/// restricted to users who accepted the `ai_chat_analysis` agreement. +#[tracing::instrument(name = "Admin recent consented chats.", skip_all)] +#[get("/recent")] +pub async fn recent( + query: web::Query, + pg_pool: web::Data, +) -> Result { + let limit = query.limit.unwrap_or(20).clamp(1, MAX_LIMIT); + let chats = db::chat_admin::recent_consented(pg_pool.get_ref(), limit) + .await + .map_err(|err| JsonResponse::internal_server_error(err.to_string()))?; + + let list: Vec = chats + .into_iter() + .map(|chat| { + let messages = match chat.plain_messages { + Some(plain) => plain, + None => helpers::chat::decrypt_messages(&chat.messages_encrypted) + .unwrap_or_else(|err| { + tracing::warn!("admin chat decrypt failed for {}: {}", chat.id, err); + Value::Array(vec![]) + }), + }; + let (message_count, last) = last_messages(&messages); + json!({ + "source": chat.source, + "id": chat.id, + "user_id": chat.user_id, + "title": chat.title, + "updated_at": chat.updated_at, + "message_count": message_count, + "messages": last, + }) + }) + .collect(); + + Ok(JsonResponse::build().set_list(list).ok("OK")) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn keeps_only_last_messages_and_truncates() { + let long = "x".repeat(MAX_CONTENT_CHARS + 10); + let msgs: Vec = (0..8) + .map(|i| json!({"role": "user", "content": if i == 7 { long.clone() } else { i.to_string() }})) + .collect(); + let (count, last) = last_messages(&Value::Array(msgs)); + assert_eq!(count, 8); + assert_eq!(last.len(), MESSAGES_PER_CHAT); + assert!(last[4]["content"].as_str().unwrap().ends_with('…')); + } +} diff --git a/src/routes/chat/mod.rs b/src/routes/chat/mod.rs index 98f7fe92..b562f756 100644 --- a/src/routes/chat/mod.rs +++ b/src/routes/chat/mod.rs @@ -1,3 +1,4 @@ +pub mod admin; pub mod delete; pub mod get; pub mod sessions; diff --git a/src/routes/deployment/member_access.rs b/src/routes/deployment/member_access.rs new file mode 100644 index 00000000..d0c02807 --- /dev/null +++ b/src/routes/deployment/member_access.rs @@ -0,0 +1,62 @@ +//! Which role a user holds on a deployment, for the User Service. +//! +//! Deployments of a shared project are recorded under the project owner, so +//! the User Service installation endpoints only recognise the owner. They ask +//! here before letting a project member operate the server. Roles live only in +//! Stacker (`project_member`), so this keeps them in one place. + +use crate::db; +use crate::helpers::JsonResponse; +use crate::routes::marketplace::public::require_stacker_service_auth; +use crate::routes::project::access::{access_of, Access}; +use actix_web::{get, web, HttpRequest, Responder, Result}; +use serde::{Deserialize, Serialize}; +use sqlx::PgPool; + +#[derive(Debug, Deserialize)] +pub struct MemberAccessQuery { + pub user_id: String, +} + +#[derive(Debug, Serialize)] +struct MemberAccess { + /// "owner", "member", "viewer", or null when the user has no access or + /// the deployment is unknown. + role: Option<&'static str>, +} + +#[tracing::instrument(name = "Deployment member access", skip_all)] +#[get("/{deployment_hash}/member-access")] +pub async fn member_access_handler( + req: HttpRequest, + path: web::Path<(String,)>, + query: web::Query, + pg_pool: web::Data, +) -> Result { + require_stacker_service_auth(&req)?; + + let deployment = db::deployment::fetch_by_deployment_hash(pg_pool.get_ref(), &path.0) + .await + .map_err(JsonResponse::internal_server_error)?; + let project = match deployment { + Some(deployment) => db::project::fetch(pg_pool.get_ref(), deployment.project_id) + .await + .map_err(JsonResponse::internal_server_error)?, + None => None, + }; + let access = match project { + Some(project) => access_of(pg_pool.get_ref(), &project, &query.user_id) + .await + .map_err(JsonResponse::internal_server_error)?, + None => None, + }; + let role = access.map(|access| match access { + Access::Owner => "owner", + Access::Member => "member", + Access::Viewer => "viewer", + }); + + Ok(JsonResponse::build() + .set_item(MemberAccess { role }) + .ok("OK")) +} diff --git a/src/routes/deployment/mod.rs b/src/routes/deployment/mod.rs index 95a6b675..7308abf3 100644 --- a/src/routes/deployment/mod.rs +++ b/src/routes/deployment/mod.rs @@ -1,6 +1,7 @@ pub mod capabilities; pub mod events; pub mod force_complete; +pub mod member_access; pub mod plan; pub mod state; pub mod status; @@ -8,6 +9,7 @@ pub mod status; pub use capabilities::*; pub use events::*; pub use force_complete::*; +pub use member_access::*; pub use plan::*; pub use state::*; pub use status::*; diff --git a/src/routes/managed_usage.rs b/src/routes/managed_usage.rs new file mode 100644 index 00000000..f00aecb5 --- /dev/null +++ b/src/routes/managed_usage.rs @@ -0,0 +1,186 @@ +//! Internal endpoints the User Service calls for the Managed server of a +//! Quick Deploy (contract: shared-fixtures/api-contracts/managed-server-billing.json). +//! +//! Casbin admits them for `group_anonymous` (the caller carries no user +//! token); `require_internal_key` is the authorisation. + +use actix_web::{post, web, HttpRequest, HttpResponse, Result}; +use serde::Deserialize; +use serde_json::json; +use sqlx::PgPool; + +use crate::db::managed_usage; +use crate::helpers::internal_key::require_internal_key; + +/// 404 when the deployment has no usage record, 409 when it is in another state. +async fn not_changed(pool: &PgPool, deployment_hash: &str, action: &str) -> HttpResponse { + match managed_usage::fetch_by_hash(pool, deployment_hash).await { + Ok(None) => HttpResponse::NotFound().json(json!({ + "error": "no usage record for the deployment", + "deployment_hash": deployment_hash, + })), + Ok(Some(row)) => HttpResponse::Conflict().json(json!({ + "error": format!("cannot {action} usage in status {}", row.status), + "deployment_hash": deployment_hash, + "status": row.status, + })), + Err(err) => HttpResponse::InternalServerError().json(json!({ "error": err })), + } +} + +/// `POST /api/internal/managed-usage/{deployment_hash}/start`: the deploy succeeded. +#[post("/{deployment_hash}/start")] +pub async fn start( + req: HttpRequest, + path: web::Path, + pool: web::Data, +) -> Result { + require_internal_key(&req)?; + let deployment_hash = path.into_inner(); + Ok(match managed_usage::start(&pool, &deployment_hash).await { + Ok(Some(row)) => HttpResponse::Ok().json(json!({ + "deployment_hash": row.deployment_hash, + "status": row.status, + })), + Ok(None) => not_changed(&pool, &deployment_hash, "start").await, + Err(err) => HttpResponse::InternalServerError().json(json!({ "error": err })), + }) +} + +#[derive(Debug, Default, Deserialize)] +pub struct FailBody { + #[serde(default)] + pub error: Option, +} + +/// `POST /api/internal/managed-usage/{deployment_hash}/fail`: the deploy failed. +#[post("/{deployment_hash}/fail")] +pub async fn fail( + req: HttpRequest, + path: web::Path, + body: Option>, + pool: web::Data, +) -> Result { + require_internal_key(&req)?; + let deployment_hash = path.into_inner(); + if let Some(error) = body.as_ref().and_then(|b| b.error.as_deref()) { + tracing::info!(deployment_hash = %deployment_hash, error = %error, "quick deploy failed"); + } + Ok( + match managed_usage::close(&pool, &deployment_hash, "deploy_failed").await { + Ok(Some(row)) => HttpResponse::Ok().json(json!({ + "deployment_hash": row.deployment_hash, + "status": row.status, + "outcome": row.outcome, + })), + Ok(None) => not_changed(&pool, &deployment_hash, "close").await, + Err(err) => HttpResponse::InternalServerError().json(json!({ "error": err })), + }, + ) +} + +/// `POST /api/internal/managed-usage/{deployment_hash}/stop`: the buyer deleted +/// the deployment. Usage stops now; the response carries the Hetzner server id +/// so the User Service can ask admins to delete the VM. +#[post("/{deployment_hash}/stop")] +pub async fn stop( + req: HttpRequest, + path: web::Path, + pool: web::Data, +) -> Result { + require_internal_key(&req)?; + let deployment_hash = path.into_inner(); + Ok(match managed_usage::stop(&pool, &deployment_hash).await { + Ok(Some(row)) => HttpResponse::Ok().json(json!({ + "deployment_hash": row.deployment_hash, + "status": row.status, + "server_id": row.server_id, + })), + Ok(None) => not_changed(&pool, &deployment_hash, "stop").await, + Err(err) => HttpResponse::InternalServerError().json(json!({ "error": err })), + }) +} + +#[derive(Debug, Deserialize)] +pub struct SuspendBody { + #[serde(default)] + pub user_id: Option, +} + +/// `POST /api/internal/managed-usage/suspend`: the user's invoice is unpaid +/// after Stripe's retries; every active server of the user stops being billed. +#[post("/suspend")] +pub async fn suspend( + req: HttpRequest, + body: web::Json, + pool: web::Data, +) -> Result { + require_internal_key(&req)?; + let Some(user_id) = body.user_id.as_deref().filter(|id| !id.trim().is_empty()) else { + return Ok(HttpResponse::BadRequest().json(json!({ "error": "user_id is required" }))); + }; + let rows = match managed_usage::suspend_for_user(&pool, user_id).await { + Ok(rows) => rows, + Err(err) => { + return Ok(HttpResponse::InternalServerError().json(json!({ "error": err }))); + } + }; + for row in &rows { + stop_containers(&pool, &row.deployment_hash, &row.user_id).await; + } + Ok(HttpResponse::Ok().json(json!({ + "suspended": rows.iter().map(|r| r.deployment_hash.clone()).collect::>(), + }))) +} + +/// Queue a `stop` for every app of the deployment: the containers stop, the +/// data stays on the server. Failures are logged; the server is already no +/// longer billed and the admins get it after 3 days. +async fn stop_containers(pool: &PgPool, deployment_hash: &str, user_id: &str) { + use crate::models::{Command, CommandPriority}; + + let apps = match crate::db::deployment::fetch_by_deployment_hash(pool, deployment_hash).await { + Ok(Some(deployment)) => { + crate::db::project_app::fetch_by_project(pool, deployment.project_id) + .await + .unwrap_or_default() + } + Ok(None) => Vec::new(), + Err(err) => { + tracing::warn!(deployment_hash = %deployment_hash, error = %err, "suspend: deployment lookup failed"); + Vec::new() + } + }; + for app in apps { + let command = Command::new( + uuid::Uuid::new_v4().to_string(), + deployment_hash.to_string(), + "stop".to_string(), + user_id.to_string(), + ) + .with_priority(CommandPriority::High) + .with_parameters(json!({ + "name": "stacker.stop", + "params": { + "deployment_hash": deployment_hash, + "app_code": app.code, + "timeout": 30 + } + })); + let queued = match crate::db::command::insert(pool, &command).await { + Ok(command) => { + crate::db::command::add_to_queue( + pool, + &command.command_id, + deployment_hash, + &CommandPriority::High, + ) + .await + } + Err(err) => Err(err), + }; + if let Err(err) = queued { + tracing::warn!(deployment_hash = %deployment_hash, app_code = %app.code, error = %err, "suspend: stop command failed"); + } + } +} diff --git a/src/routes/marketplace/admin.rs b/src/routes/marketplace/admin.rs index ea2e4dc6..17975fda 100644 --- a/src/routes/marketplace/admin.rs +++ b/src/routes/marketplace/admin.rs @@ -4,7 +4,7 @@ use crate::db; use crate::helpers::security_validator; use crate::helpers::JsonResponse; use crate::models; -use actix_web::{get, patch, post, web, Responder, Result}; +use actix_web::{get, patch, post, put, web, Responder, Result}; use sqlx::PgPool; use std::sync::Arc; use tracing::Instrument; @@ -587,6 +587,44 @@ pub struct AdminPricingRequest { pub monthly_cap: Option, } +/// Edit any template's content and version payload as an administrator. +/// +/// Deliberately the same write path as the author's own +/// `PUT /api/v1/templates/{id}` (`creator::apply_template_update`), so an admin +/// edit cannot slip past the config-contract validation or the stripping of +/// author-declared generated secrets. The only difference is +/// `UpdateAuthority::Admin`, which lifts the review-workflow status gate: the +/// point of this route is correcting a listing that is already approved. +/// +/// Ownership is not checked, by design - Casbin restricts the route to admin +/// roles, and an admin editing someone else's listing is the feature. The edit +/// is logged with the acting admin so the change is attributable. +#[tracing::instrument(name = "Admin update template", skip_all)] +#[put("/{id}")] +pub async fn update_template_handler( + admin: web::ReqData>, + path: web::Path<(String,)>, + pg_pool: web::Data, + body: web::Json, +) -> Result>> { + let id = uuid::Uuid::parse_str(&path.into_inner().0) + .map_err(|_| actix_web::error::ErrorBadRequest("Invalid UUID"))?; + + tracing::info!( + admin_id = %admin.id, + template_id = %id, + "admin edits marketplace template" + ); + + crate::routes::marketplace::creator::apply_template_update( + pg_pool.get_ref(), + id, + body.into_inner(), + crate::routes::marketplace::creator::UpdateAuthority::Admin, + ) + .await +} + #[tracing::instrument(name = "Admin update template pricing", skip_all)] #[patch("/{id}/pricing")] pub async fn pricing_handler( diff --git a/src/routes/marketplace/creator.rs b/src/routes/marketplace/creator.rs index 2aeb2cce..b485b742 100644 --- a/src/routes/marketplace/creator.rs +++ b/src/routes/marketplace/creator.rs @@ -846,7 +846,38 @@ pub async fn update_handler( return Err(template_not_visible()); } - let req = body.into_inner(); + apply_template_update( + pg_pool.get_ref(), + id, + body.into_inner(), + UpdateAuthority::Author, + ) + .await +} + +/// Apply an `UpdateTemplateRequest` to a template the caller is already +/// authorised for. +/// +/// The creator route authorises by ownership, the admin route by role. The +/// write itself lives here so the two cannot drift: an admin edit runs the +/// same contract validation, the same generated-secret stripping and the same +/// version upsert as the author's own edit. +/// Who is editing, which decides whether the review-workflow status gate +/// applies to the metadata write. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum UpdateAuthority { + /// The template's author: editable only in draft/rejected/needs_changes. + Author, + /// An administrator correcting a listing: any status. + Admin, +} + +pub(crate) async fn apply_template_update( + pg_pool: &PgPool, + id: uuid::Uuid, + req: UpdateTemplateRequest, + authority: UpdateAuthority, +) -> Result>> { validate_config_contract(req.config_contract.as_ref())?; let infrastructure_requirements = req.infrastructure_requirements.clone(); @@ -856,24 +887,48 @@ pub async fn update_handler( _ => req.price, }; - let updated = db::marketplace::update_metadata( - pg_pool.get_ref(), - &id, - req.name.as_deref(), - req.short_description.as_deref(), - req.long_description.as_deref(), - req.category_code.as_deref(), - req.tags, - req.tech_stack, - infrastructure_requirements, - price, - req.plan_type.as_deref(), - req.required_plan_name.as_deref(), - req.currency.as_deref(), - req.public_ports.clone(), - req.vendor_url.as_deref(), - ) - .await + let updated = match authority { + UpdateAuthority::Author => { + db::marketplace::update_metadata( + pg_pool, + &id, + req.name.as_deref(), + req.short_description.as_deref(), + req.long_description.as_deref(), + req.category_code.as_deref(), + req.tags, + req.tech_stack, + infrastructure_requirements, + price, + req.plan_type.as_deref(), + req.required_plan_name.as_deref(), + req.currency.as_deref(), + req.public_ports.clone(), + req.vendor_url.as_deref(), + ) + .await + } + UpdateAuthority::Admin => { + db::marketplace::admin_update_metadata( + pg_pool, + &id, + req.name.as_deref(), + req.short_description.as_deref(), + req.long_description.as_deref(), + req.category_code.as_deref(), + req.tags, + req.tech_stack, + infrastructure_requirements, + price, + req.plan_type.as_deref(), + req.required_plan_name.as_deref(), + req.currency.as_deref(), + req.public_ports.clone(), + req.vendor_url.as_deref(), + ) + .await + } + } .map_err(|err| JsonResponse::::build().bad_request(err))?; if req.stack_definition.is_some() @@ -885,7 +940,7 @@ pub async fn update_handler( || req.post_deploy_hooks.is_some() || req.update_mode_capabilities.is_some() { - let latest_version = db::marketplace::get_latest_version_by_template(pg_pool.get_ref(), id) + let latest_version = db::marketplace::get_latest_version_by_template(pg_pool, id) .await .map_err(|err| JsonResponse::::build().bad_request(err))?; let current_version = latest_version.unwrap_or_default(); @@ -930,7 +985,7 @@ pub async fn update_handler( // against the effective contract (this request's, else the stored one). let effective_contract = match req.config_contract.clone() { Some(cc) => cc, - None => db::marketplace::get_config_contract(pg_pool.get_ref(), id) + None => db::marketplace::get_config_contract(pg_pool, id) .await .unwrap_or(serde_json::Value::Null), }; @@ -938,7 +993,7 @@ pub async fn update_handler( strip_generated_field_values(&stack_definition, definition_format, &effective_contract); db::marketplace::upsert_latest_version( - pg_pool.get_ref(), + pg_pool, &id, if version.is_empty() { "1.0.0" @@ -959,13 +1014,13 @@ pub async fn update_handler( } if let Some(source_project_id) = req.source_project_id { - db::marketplace::set_source_project_id(pg_pool.get_ref(), &id, source_project_id) + db::marketplace::set_source_project_id(pg_pool, &id, source_project_id) .await .map_err(|err| JsonResponse::::build().internal_server_error(err))?; } if let Some(config_contract) = req.config_contract.clone() { - db::marketplace::set_config_contract(pg_pool.get_ref(), &id, config_contract) + db::marketplace::set_config_contract(pg_pool, &id, config_contract) .await .map_err(|err| JsonResponse::::build().internal_server_error(err))?; } diff --git a/src/routes/marketplace/public.rs b/src/routes/marketplace/public.rs index bd08b369..ff5f0bd9 100644 --- a/src/routes/marketplace/public.rs +++ b/src/routes/marketplace/public.rs @@ -215,7 +215,7 @@ struct DeployCompleteResponse { deploy_count_incremented: bool, } -fn require_stacker_service_auth(req: &HttpRequest) -> Result<()> { +pub(crate) fn require_stacker_service_auth(req: &HttpRequest) -> Result<()> { let expected_token = std::env::var("STACKER_SERVICE_TOKEN").unwrap_or_default(); if expected_token.trim().is_empty() { return Err(JsonResponse::::build() diff --git a/src/routes/mod.rs b/src/routes/mod.rs index 89dee705..3fbb55e0 100644 --- a/src/routes/mod.rs +++ b/src/routes/mod.rs @@ -1,3 +1,4 @@ +pub(crate) mod admin_bake; pub(crate) mod admin_notifications; pub(crate) mod agent; pub mod audit; @@ -8,6 +9,7 @@ pub(crate) mod dockerhub; pub(crate) mod handoff; pub mod health_checks; pub(crate) mod legacy_installations; +pub(crate) mod managed_usage; pub(crate) mod oneclick_deploy; pub(crate) mod rating; pub(crate) mod test; diff --git a/src/routes/oneclick_deploy/clone.rs b/src/routes/oneclick_deploy/clone.rs index b842c9a6..2541135f 100644 --- a/src/routes/oneclick_deploy/clone.rs +++ b/src/routes/oneclick_deploy/clone.rs @@ -82,8 +82,8 @@ pub struct CloneResponse { /// SSH private key (PEM) for the deploy key injected into the cloned server. /// The user service must pass this to the install service for Ansible access. pub ssh_private_key: String, - /// Present only for deployment_daily templates. The user service stores - /// this so it can void on failure or pass to deploy-complete for capture. + /// No longer set: the Managed server is metered daily (`managed_usage`) + /// instead of held on the card. Kept so existing readers still parse. #[serde(skip_serializing_if = "Option::is_none")] pub authorization_id: Option, } @@ -375,6 +375,70 @@ pub async fn clone_server( GateLookup::Block(response) => return response, } + // ── Managed server billing: rate and card, before any record ───────── + // Every Quick Deploy runs on a TryDirect-managed server billed daily to + // the buyer (shared-fixtures/api-contracts/managed-server-billing.json). + let template = match crate::db::marketplace::get_approved_by_slug(&pg_pool, &form.stack).await { + Ok(Some(template)) => template, + Ok(None) => { + tracing::warn!( + template_slug = %form.stack, + "stack not registered in stack_template; refusing to deploy" + ); + return HttpResponse::NotFound().json(json!({ + "error": "Unknown stack", + "details": format!( + "stack '{}' is not registered in the marketplace catalog", + form.stack + ), + })); + } + Err(err) => { + // Fail closed: without the template the rate cannot be known. + tracing::error!(error = %err, "template lookup for billing failed"); + return HttpResponse::ServiceUnavailable().json(json!({ + "error": "billing lookup failed", + "details": err, + })); + } + }; + let server_type_rate = + match crate::db::server_type_daily_rate::fetch(&pg_pool, &form.server_type).await { + Ok(rate) => rate.map(|cfg| (cfg.daily_rate, cfg.monthly_cap)), + Err(err) => { + tracing::error!(error = %err, "server type rate lookup failed"); + return HttpResponse::ServiceUnavailable().json(json!({ + "error": "billing lookup failed", + "details": err, + })); + } + }; + let capability = match user_service + .can_charge(user.access_token.as_deref().unwrap_or("")) + .await + { + Ok(capability) => capability, + Err(err) => { + tracing::error!(error = ?err, "can_charge failed"); + return HttpResponse::ServiceUnavailable().json(json!({ + "error": "payment check failed", + "details": err.to_string(), + })); + } + }; + let billed = match managed_billing_check( + crate::services::managed_usage::resolve_daily_rate( + template.daily_rate, + template.monthly_cap, + server_type_rate, + ), + &capability, + &form.server_type, + ) { + Ok(billed) => billed, + Err(response) => return response, + }; + let mut project_model = crate::models::Project::new( user.id.clone(), project_name, @@ -445,145 +509,6 @@ pub async fn clone_server( "clone deployment records created" ); - // ── Deployment-daily billing: authorize before server creation ──────── - let mut authorization_id: Option = None; - match crate::db::marketplace::get_approved_by_slug(&pg_pool, &form.stack).await { - Ok(Some(template)) => { - tracing::info!( - template_slug = %form.stack, - billing_cycle = ?template.billing_cycle, - daily_rate = ?template.daily_rate, - "template found for billing check" - ); - if template.billing_cycle.as_deref() == Some("deployment_daily") { - // Resolve daily_rate: template override or server-type default - let daily_rate = if let Some(rate) = template.daily_rate { - rate - } else if let Ok(Some(cfg)) = - crate::db::server_type_daily_rate::fetch(&pg_pool, &form.server_type).await - { - cfg.daily_rate - } else { - 0.87 // fallback default - }; - let monthly_cap = template.monthly_cap.unwrap_or_else(|| daily_rate * 30.0); - - // Convert to minor units (cents) - let amount_minor = (daily_rate * 100.0).round() as i64; - let currency = template - .currency - .clone() - .unwrap_or_else(|| "USD".to_string()); - let idem_key = format!("oneclick-{}", deployment_hash); - - // Get user's access token for authorization - let user_token = user.access_token.as_deref().unwrap_or(""); - if !user_token.is_empty() { - match user_service - .authorize_install_charge( - user_token, - &template.id, - amount_minor, - ¤cy, - &idem_key, - ) - .await - { - Ok(handle) => { - let expires_at = handle - .expires_at - .as_deref() - .and_then(|s| chrono::DateTime::parse_from_rfc3339(s).ok()) - .map(|dt| dt.with_timezone(&chrono::Utc)); - - match crate::db::marketplace_billing::insert_authorization( - &pg_pool, - crate::db::marketplace_billing::NewAuthorization { - user_id: user.id.clone(), - template_id: template.id, - idempotency_key: idem_key, - authorization_id: handle.authorization_id.clone(), - amount_minor: handle.amount_minor, - currency: handle.currency.clone(), - expires_at, - billing_cycle: Some("deployment_daily".to_string()), - daily_rate: Some(daily_rate), - monthly_cap: Some(monthly_cap), - }, - ) - .await - { - Ok(auth_row) => { - crate::db::marketplace_billing::attach_deployment_hash( - &pg_pool, - auth_row.id, - &deployment_hash, - ) - .await - .ok(); - authorization_id = Some(handle.authorization_id); - tracing::info!( - deployment_hash = %deployment_hash, - daily_rate = daily_rate, - monthly_cap = monthly_cap, - "deployment_daily authorization created" - ); - } - Err(err) => { - tracing::error!("Failed to store authorization: {}", err); - let _ = user_service - .void_install_charge( - user_token, - &handle.authorization_id, - "db_write_failed", - ) - .await; - return HttpResponse::InternalServerError().json(json!({ - "error": "authorization storage failed", - "details": err, - })); - } - } - } - Err(err) => { - tracing::error!("authorize_install_charge failed: {:?}", err); - return HttpResponse::PaymentRequired().json(json!({ - "error": "Payment authorization failed", - "details": format!("{:?}", err), - })); - } - } - } else { - tracing::warn!( - "deployment_daily template but user has no access_token, skipping authorize" - ); - } - } else { - tracing::info!( - template_slug = %form.stack, - billing_cycle = ?template.billing_cycle, - "template is not deployment_daily, skipping billing" - ); - } - } - Ok(None) => { - tracing::warn!( - template_slug = %form.stack, - "stack not registered in stack_template; refusing to deploy" - ); - return HttpResponse::NotFound().json(json!({ - "error": "Unknown stack", - "details": format!( - "stack '{}' is not registered in the marketplace catalog", - form.stack - ), - })); - } - Err(err) => { - tracing::warn!(error = %err, "failed to look up template for billing"); - } - } - // Generate a per-deploy SSH keypair so Ansible can reach the server post-boot. let (public_key, private_key) = match VaultClient::generate_ssh_keypair() { Ok(pair) => pair, @@ -643,10 +568,36 @@ pub async fn clone_server( user_data: Some(user_data), }; + // Usage is recorded now and starts only when the deploy succeeds. + if let Err(err) = crate::db::managed_usage::insert_pending( + &pg_pool, + &crate::db::managed_usage::NewManagedUsage { + deployment_hash: deployment_hash.clone(), + user_id: user.id.clone(), + stack: form.stack.clone(), + server_type: form.server_type.clone(), + daily_rate_minor: billed.rate_minor, + monthly_cap_minor: billed.cap_minor, + }, + ) + .await + { + tracing::error!(error = %err, "failed to record managed usage"); + return HttpResponse::InternalServerError().json(json!({ + "error": "usage record failed", + "details": err, + })); + } + let provisioned = match client.create_server_from_image(token, request).await { Ok(server) => server, Err(err) => { tracing::error!(error = %err, "clone from snapshot failed"); + if let Err(close_err) = + crate::db::managed_usage::close(&pg_pool, &deployment_hash, "deploy_failed").await + { + tracing::warn!(error = %close_err, "failed to close managed usage"); + } return HttpResponse::BadGateway().json(serde_json::json!({ "error": "Hetzner clone failed", "details": err.to_string(), @@ -654,6 +605,14 @@ pub async fn clone_server( } }; + // Keep the VM id for the admins: nothing deletes a Quick Deploy VM + // automatically; they delete it by hand from the alert. + if let Err(err) = + crate::db::managed_usage::set_server_id(&pg_pool, &deployment_hash, provisioned.id).await + { + tracing::warn!(error = %err, "failed to record the server id on managed usage"); + } + // Register the server in Stacker's inventory so secrets, agent, and // monitoring features can resolve it. Non-fatal: the Hetzner server is // already created; a missing record only breaks Stacker-managed features. @@ -715,10 +674,36 @@ pub async fn clone_server( project_id: project.id, deployment_hash, ssh_private_key: private_key, - authorization_id, + authorization_id: None, }) } +/// Whether a Quick Deploy may start, and at what rate its Managed server is +/// billed. The server type must have a rate (no fallback rate), then the +/// buyer must have a verified card: the server is billed to every buyer, +/// whatever the plan (quick-deploy-managed-server-billing.feature). +fn managed_billing_check( + rate: Option, + capability: &crate::connectors::user_service::BillingCapability, + server_type: &str, +) -> Result { + let Some(rate) = rate else { + return Err(HttpResponse::UnprocessableEntity().json(json!({ + "error": "no_server_rate", + "details": format!("server type '{server_type}' has no daily rate"), + "server_type": server_type, + }))); + }; + if !capability.can_charge { + return Err(HttpResponse::PaymentRequired().json(json!({ + "error": "Payment method required", + "details": "A verified card is required for the Managed server", + "reason": capability.reason.as_deref().unwrap_or("no_payment_method"), + }))); + } + Ok(rate) +} + /// What the clone access gate does with its catalog lookup, before any /// user-service call. enum GateLookup { @@ -991,6 +976,52 @@ mod tests { use actix_web::web; use actix_web::HttpMessage; + fn capability(can_charge: bool) -> crate::connectors::user_service::BillingCapability { + crate::connectors::user_service::BillingCapability { + can_charge, + reason: (!can_charge).then(|| "no_payment_method".to_string()), + } + } + + const CPX32_RATE: crate::services::managed_usage::DailyRate = + crate::services::managed_usage::DailyRate { + rate_minor: 86, + cap_minor: 2600, + }; + + #[actix_web::test] + async fn a_server_type_without_a_rate_is_refused_before_the_card() { + let resp = managed_billing_check(None, &capability(false), "ccx63").unwrap_err(); + assert_eq!( + resp.status(), + actix_web::http::StatusCode::UNPROCESSABLE_ENTITY + ); + let body: serde_json::Value = + serde_json::from_slice(&actix_web::body::to_bytes(resp.into_body()).await.unwrap()) + .unwrap(); + assert_eq!(body["error"], "no_server_rate"); + assert_eq!(body["server_type"], "ccx63"); + } + + #[actix_web::test] + async fn a_buyer_without_a_card_is_asked_for_one() { + let resp = + managed_billing_check(Some(CPX32_RATE), &capability(false), "cpx32").unwrap_err(); + assert_eq!(resp.status(), actix_web::http::StatusCode::PAYMENT_REQUIRED); + let body: serde_json::Value = + serde_json::from_slice(&actix_web::body::to_bytes(resp.into_body()).await.unwrap()) + .unwrap(); + assert_eq!(body["reason"], "no_payment_method"); + } + + #[actix_web::test] + async fn a_buyer_with_a_card_is_billed_at_the_resolved_rate() { + assert_eq!( + managed_billing_check(Some(CPX32_RATE), &capability(true), "cpx32").unwrap(), + CPX32_RATE + ); + } + /// The User Service stores this as `installations.stack_id`; without it /// `_sync_apps_from_stacker` returns on its first line and the /// Applications panel is empty for every one-click deployment. diff --git a/src/routes/oneclick_deploy/mod.rs b/src/routes/oneclick_deploy/mod.rs index 0b1b7159..b56541f9 100644 --- a/src/routes/oneclick_deploy/mod.rs +++ b/src/routes/oneclick_deploy/mod.rs @@ -12,6 +12,7 @@ use actix_web::web::ServiceConfig; use actix_web::{post, web, HttpResponse, Responder}; use serde::Serialize; +use sqlx::PgPool; use std::collections::BTreeMap; use crate::cli::config_parser::StackerConfig; @@ -82,6 +83,65 @@ struct ValidateOk { version: Option, composition: Composition, config_contract: crate::cli::config_parser::ConfigContract, + quick_deploy: QuickDeploy, +} + +/// What a Quick Deploy of this stack provisions and what each type costs. +/// See shared-fixtures/api-contracts/quick-deploy-server-type.json. +#[derive(Debug, Serialize)] +struct QuickDeploy { + /// Server type the snapshot the clone uses was baked on; `None` + /// when unknown, and the caller's default type applies. + server_type: Option, + rates: BTreeMap, +} + +#[derive(Debug, Serialize)] +struct Rate { + daily_rate: f64, + monthly_cap: f64, +} + +async fn quick_deploy_for( + pool: Option<&PgPool>, + stack: &str, + version: Option<&str>, +) -> QuickDeploy { + let Some(pool) = pool else { + return QuickDeploy { + server_type: None, + rates: BTreeMap::new(), + }; + }; + // The snapshot the clone will use: the exact version when given, else the latest. + let snapshot = match version { + Some(version) => crate::db::baked_snapshot::resolve(pool, stack, version, "hetzner").await, + None => crate::db::baked_snapshot::resolve_latest(pool, stack, "hetzner").await, + }; + let server_type = snapshot + .unwrap_or_else(|e| { + tracing::warn!("quick deploy server type: {e}"); + None + }) + .and_then(|snapshot| snapshot.server_type); + let rates = crate::db::server_type_daily_rate::list(pool) + .await + .unwrap_or_else(|e| { + tracing::warn!("quick deploy rates: {e}"); + Vec::new() + }) + .into_iter() + .map(|r| { + ( + r.server_type, + Rate { + daily_rate: r.daily_rate, + monthly_cap: r.monthly_cap, + }, + ) + }) + .collect(); + QuickDeploy { server_type, rates } } #[derive(Debug, Serialize)] @@ -127,7 +187,7 @@ fn to_issue_json(issues: &[&crate::cli::error::ValidationIssue]) -> Vec impl Responder { +async fn validate(body: String, pool: Option>) -> impl Responder { // Parse. A malformed YAML is a config error, not a server error. // Use from_str_raw so config_contract install inputs (`${VAR}`) that are // filled at deploy time do not fail validation when undefined here. @@ -172,6 +232,12 @@ async fn validate(body: String) -> impl Responder { version: config.version.clone(), composition: compose_from(&config), config_contract: config.config_contract.clone(), + quick_deploy: quick_deploy_for( + pool.as_ref().map(|p| p.get_ref()), + &config.name, + config.version.as_deref(), + ) + .await, }) } @@ -221,6 +287,32 @@ deploy: "name: [unclosed" } + // local-ai-packaged starts its CPU Ollama unless OLLAMA_CPU_REPLICAS=0. + // Validate reads stacker.yml without filling ${VAR}, so the reference + // must be accepted as written. + #[actix_web::test] + async fn replicas_from_an_env_reference_is_valid() { + let yaml = r#"name: local-ai +app: + type: custom + image: nginx:alpine +services: + - name: ollama-cpu + image: ollama/ollama:0.40.4 + deploy: + replicas: ${OLLAMA_CPU_REPLICAS} +"#; + let app = actix_web::test::init_service(actix_web::App::new().service(validate)).await; + let req = test::TestRequest::post() + .uri("/validate") + .set_payload(yaml) + .to_request(); + let resp = test::call_service(&app, req).await; + let status = resp.status(); + let body: serde_json::Value = test::read_body_json(resp).await; + assert_eq!(status, actix_web::http::StatusCode::OK, "{body}"); + } + #[actix_web::test] async fn valid_config_returns_200_with_composition() { let app = actix_web::test::init_service(actix_web::App::new().service(validate)).await; diff --git a/src/routes/project/access.rs b/src/routes/project/access.rs index fda29557..7bcb1354 100644 --- a/src/routes/project/access.rs +++ b/src/routes/project/access.rs @@ -180,6 +180,32 @@ pub async fn acting_user( }) } +/// The member's address when a member deploys as the owner, so the deployment +/// reports reach them as well as the owner. +pub fn initiator_email(user: &models::User, access: Access) -> Option { + (access != Access::Owner).then(|| user.email.clone()) +} + +/// The member to send the deployment report to as well as the owner, when the +/// owner turned on server access for members. The report carries the server's +/// SSH key and passwords, so a lookup failure means no. +pub async fn report_initiator( + pool: &PgPool, + project: &models::Project, + user: &models::User, + access: Access, +) -> Option { + initiator_email(user, access)?; + match db::project::members_server_access(pool, project.id).await { + Ok(true) => initiator_email(user, access), + Ok(false) => None, + Err(err) => { + tracing::warn!("Server access lookup failed, the report goes to the owner only: {err}"); + None + } + } +} + /// The owner's address from the User Service internal user lookup. async fn owner_email(user_service_url: &str, owner_id: &str) -> Result { let key = std::env::var("INTERNAL_SERVICES_ACCESS_KEY").unwrap_or_default(); @@ -217,6 +243,25 @@ async fn owner_email(user_service_url: &str, owner_id: &str) -> Result, + pub updated_at: DateTime, +} + +impl From for ProjectCloud { + fn from(cloud: models::Cloud) -> Self { + ProjectCloud { + id: cloud.id, + name: cloud.name, + provider: cloud.provider, + created_at: cloud.created_at, + updated_at: cloud.updated_at, + } + } +} + +#[tracing::instrument(name = "List the project owner's clouds", skip_all)] +#[get("/{id}/clouds")] +pub async fn list( + user: web::ReqData>, + path: web::Path<(i32,)>, + pg_pool: web::Data, +) -> Result { + let (project, _) = access::require(pg_pool.get_ref(), path.0, &user.id, Need::Manage).await?; + let clouds = db::cloud::fetch_by_user(pg_pool.get_ref(), &project.user_id) + .await + .map_err(JsonResponse::internal_server_error)? + .into_iter() + .map(ProjectCloud::from) + .collect::>(); + Ok(JsonResponse::build().set_list(clouds).ok("OK")) +} diff --git a/src/routes/project/deploy.rs b/src/routes/project/deploy.rs index 0665694b..cef426e8 100644 --- a/src/routes/project/deploy.rs +++ b/src/routes/project/deploy.rs @@ -1536,6 +1536,7 @@ async fn execute_deployment( new_public_key, new_private_key, form.proxy_domains.clone(), + form.initiator_email.clone(), ) .await .map_err(|err| JsonResponse::::build().internal_server_error(err))?; @@ -1856,6 +1857,9 @@ pub async fn item( "not found", ) .await?; + let mut form = form.into_inner(); + form.initiator_email = + access::report_initiator(pg_pool.get_ref(), &project, &user, access).await; let user = Arc::new( access::acting_user(&project, user.as_ref(), access, &sets.user_service_url).await?, ); @@ -1863,7 +1867,7 @@ pub async fn item( let (project_id, deployment_id) = deploy_project( user.as_ref(), project, - form.into_inner(), + form, pg_pool.get_ref(), mq_manager.get_ref(), sets.get_ref(), @@ -1917,6 +1921,8 @@ pub async fn saved_item( "Project not found", ) .await?; + form.initiator_email = + access::report_initiator(pg_pool.get_ref(), &project, &user, access).await; let user = Arc::new( access::acting_user(&project, user.as_ref(), access, &sets.user_service_url).await?, ); @@ -2155,6 +2161,8 @@ pub async fn rollback( "Project not found", ) .await?; + let initiator_email = + access::report_initiator(pg_pool.get_ref(), &project, &user, access).await; let user = Arc::new( access::acting_user(&project, user.as_ref(), access, &sets.user_service_url).await?, ); @@ -2227,7 +2235,8 @@ pub async fn rollback( let (metadata, template_stack_code) = build_rollback_project_payload(target_version.stack_definition.clone()) .map_err(|msg| JsonResponse::::build().bad_request(msg))?; - let deploy_form = build_rollback_deploy_form(template_stack_code); + let mut deploy_form = build_rollback_deploy_form(template_stack_code); + deploy_form.initiator_email = initiator_email; validate_reused_cloud_server(&cloud, &server) .await @@ -3060,3 +3069,92 @@ mod tests { assert!(embedded_marketplace_compose(&metadata).is_none()); } } + +/// Contract across the CLI and the server for host-local ports. +/// +/// The loopback handling broke more than once because each half was tested +/// alone: the CLI parser was pinned to drop the host IP (`127.0.0.1:1025:25` -> +/// `1025`), while the server expects the IP to arrive with the mapping and +/// skips such ports. Together they opened databases bound to 127.0.0.1 to the +/// internet: the firewall rule allowed 5432/6379 from anywhere and the compose +/// file published them on every interface. This test runs a stacker.yml +/// through the CLI's payload builder and then through what the server does +/// with it. +#[cfg(test)] +mod loopback_ports_contract { + use super::derive_public_ports_from_metadata; + use crate::cli::config_parser::StackerConfig; + use crate::cli::stacker_client::build_project_body; + + const STACKER_YML: &str = r#" +name: loopback-contract +app: + type: custom + image: example/web:1.0 + ports: + - "8000:8000" + - "127.0.0.1:8025:8025" +services: + - name: postgres + image: postgres:16 + ports: + - "127.0.0.1:5432:5432" + - name: redis + image: redis:7 + ports: + - "127.0.0.1:6379:6379/tcp" + - name: proxy + image: nginx:alpine + ports: + - "443:443" +"#; + + fn shared_ports(body: &serde_json::Value) -> Vec { + ["web", "service", "feature"] + .iter() + .filter_map(|group| body["custom"][group].as_array()) + .flatten() + .filter_map(|app| app["shared_ports"].as_array()) + .flatten() + .cloned() + .collect() + } + + #[test] + fn ports_bound_to_loopback_in_stacker_yml_are_not_opened_to_the_internet() { + let config = StackerConfig::from_str(STACKER_YML).expect("stacker.yml parses"); + let body = build_project_body(&config); + + let mut public = derive_public_ports_from_metadata(&body); + public.sort(); + assert_eq!( + public, + vec!["443/tcp".to_string(), "8000/tcp".to_string()], + "only the ports not bound to 127.0.0.1 may reach the firewall" + ); + } + + #[test] + fn ports_bound_to_loopback_in_stacker_yml_stay_on_loopback_in_compose() { + let config = StackerConfig::from_str(STACKER_YML).expect("stacker.yml parses"); + let body = build_project_body(&config); + + let mut loopback_published = Vec::new(); + for raw in shared_ports(&body) { + let port: crate::forms::project::Port = + serde_json::from_value(raw.clone()).expect("shared port"); + let dc: docker_compose_types::Port = (&port).try_into().expect("compose port"); + if let Some(docker_compose_types::PublishedPort::Single(p)) = dc.published { + if dc.host_ip.as_deref() == Some("127.0.0.1") { + loopback_published.push(p); + } + } + } + loopback_published.sort(); + assert_eq!( + loopback_published, + vec![5432, 6379, 8025], + "compose must publish these on 127.0.0.1 only" + ); + } +} diff --git a/src/routes/project/discover.rs b/src/routes/project/discover.rs index 6727ff52..2ea17256 100644 --- a/src/routes/project/discover.rs +++ b/src/routes/project/discover.rs @@ -424,6 +424,7 @@ pub async fn import_containers( parent_app_code: None, deployment_id: None, config_contract: None, + runtime: None, }; match db::project_app::insert(pg_pool.get_ref(), &app).await { diff --git a/src/routes/project/invitation.rs b/src/routes/project/invitation.rs index 83d7656a..a40ee40d 100644 --- a/src/routes/project/invitation.rs +++ b/src/routes/project/invitation.rs @@ -311,6 +311,7 @@ struct Seats { seats_used: i64, seats_free: i64, can_invite: bool, + members_server_access: bool, } #[tracing::instrument(name = "Project seats", skip_all)] @@ -336,6 +337,9 @@ pub async fn seats( .await .map_err(|err| JsonResponse::internal_server_error(err.to_string()))?; + let members_server_access = db::project::members_server_access(pg_pool.get_ref(), project.id) + .await + .map_err(JsonResponse::internal_server_error)?; let free = (PROJECT_SEATS - used).max(0); let on_team = matches!(plan_code(settings.get_ref(), &user).await, Ok(Some(code)) if can_share_projects(&code)); Ok(JsonResponse::build() @@ -344,6 +348,7 @@ pub async fn seats( seats_used: used, seats_free: free, can_invite: on_team && free > 0, + members_server_access, }) .ok("OK")) } diff --git a/src/routes/project/member.rs b/src/routes/project/member.rs index 01ed8835..4ce082fa 100644 --- a/src/routes/project/member.rs +++ b/src/routes/project/member.rs @@ -4,7 +4,7 @@ use crate::helpers::JsonResponse; use crate::models; use crate::routes::project::access::{self, Need}; use crate::routes::project::invitation::{require_team_plan, reserve_seat, INVITABLE_ROLES}; -use actix_web::{delete, get, post, web, HttpResponse, Responder, Result}; +use actix_web::{delete, get, patch, post, web, HttpResponse, Responder, Result}; use serde::Deserialize; use sqlx::PgPool; use std::sync::Arc; @@ -87,6 +87,28 @@ pub async fn add( Ok(JsonResponse::build().set_item(member).ok("OK")) } +#[derive(Debug, Deserialize)] +pub struct ServerAccessForm { + pub enabled: bool, +} + +/// Owner only: whether members get the servers' SSH key and passwords in the +/// deployment report. +#[tracing::instrument(name = "Set members server access", skip_all)] +#[patch("/{id}/server-access")] +pub async fn set_server_access( + user: web::ReqData>, + path: web::Path<(i32,)>, + web::Json(form): web::Json, + pg_pool: web::Data, +) -> Result { + let project = fetch_owned_project(pg_pool.get_ref(), path.0, &user.id).await?; + db::project::set_members_server_access(pg_pool.get_ref(), project.id, form.enabled) + .await + .map_err(|err| JsonResponse::::build().internal_server_error(err))?; + Ok(JsonResponse::::build().ok("OK")) +} + #[tracing::instrument(name = "List project members", skip_all)] #[get("/{id}/members")] pub async fn list( diff --git a/src/routes/project/mod.rs b/src/routes/project/mod.rs index 585947db..7ff19b95 100644 --- a/src/routes/project/mod.rs +++ b/src/routes/project/mod.rs @@ -1,6 +1,7 @@ pub mod access; pub mod add; pub mod app; +pub mod clouds; pub(crate) mod compose; pub(crate) mod delete; pub mod deploy; diff --git a/src/routes/server/get.rs b/src/routes/server/get.rs index ac8ad8b0..157633ca 100644 --- a/src/routes/server/get.rs +++ b/src/routes/server/get.rs @@ -1,6 +1,7 @@ use crate::db; use crate::helpers::JsonResponse; use crate::models; +use crate::routes::project::access::{self, Need}; use actix_web::{get, web, Responder, Result}; use sqlx::PgPool; use std::sync::Arc; @@ -55,17 +56,22 @@ pub async fn list_by_project( ) -> Result { let project_id = path.0; - // Verify user owns the project - let _project = db::project::fetch(pg_pool.get_ref(), project_id) + // The owner and members deploy to these servers; a viewer does not. + let project = db::project::fetch(pg_pool.get_ref(), project_id) .await .map_err(|_err| JsonResponse::::build().internal_server_error("")) .and_then(|p| match p { - Some(proj) if proj.user_id != user.id => { - Err(JsonResponse::::build().not_found("Project not found")) - } Some(proj) => Ok(proj), None => Err(JsonResponse::::build().not_found("Project not found")), })?; + access::check( + pg_pool.get_ref(), + &project, + &user.id, + Need::Manage, + "Project not found", + ) + .await?; db::server::fetch_by_project(pg_pool.get_ref(), project_id) .await diff --git a/src/services/config_renderer.rs b/src/services/config_renderer.rs index fffa7007..829a6951 100644 --- a/src/services/config_renderer.rs +++ b/src/services/config_renderer.rs @@ -434,6 +434,68 @@ pub struct AppRenderContext { pub healthcheck: Option, /// Container runtime override (e.g., "kata" for hardware isolation) pub runtime: Option, + /// command, entrypoint and the settings in `project_app.runtime`. + #[serde(flatten)] + pub run: AppRunSettings, +} + +/// What the app runs and how, beyond image and wiring. See +/// shared-fixtures/api-contracts/stack-service-runtime-fields.json. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +pub struct AppRunSettings { + pub command: Option, + pub entrypoint: Option, + #[serde(default)] + pub cap_add: Vec, + #[serde(default)] + pub cap_drop: Vec, + #[serde(default)] + pub devices: Vec, + #[serde(default)] + pub profiles: Vec, + #[serde(default)] + pub aliases: Vec, + pub replicas: Option, + /// `deploy.resources.reservations.devices` entries, as stored. + #[serde(default)] + pub gpu_devices: Vec, +} + +impl AppRunSettings { + fn from_app(app: &ProjectApp) -> Self { + let runtime = app.runtime.as_ref(); + let strings = |key: &str| -> Vec { + runtime + .and_then(|r| r.get(key)) + .and_then(Value::as_array) + .map(|items| { + items + .iter() + .filter_map(|v| v.as_str().map(str::to_string)) + .collect() + }) + .unwrap_or_default() + }; + let deploy = runtime.and_then(|r| r.get("deploy")); + Self { + command: app.command.clone().filter(|c| !c.is_empty()), + entrypoint: app.entrypoint.clone().filter(|e| !e.is_empty()), + cap_add: strings("cap_add"), + cap_drop: strings("cap_drop"), + devices: strings("devices"), + profiles: strings("profiles"), + aliases: strings("aliases"), + replicas: deploy + .and_then(|d| d.get("replicas")) + .and_then(Value::as_u64) + .and_then(|n| u32::try_from(n).ok()), + gpu_devices: deploy + .and_then(|d| d.pointer("/resources/reservations/devices")) + .and_then(Value::as_array) + .cloned() + .unwrap_or_default(), + } + } } #[derive(Debug, Clone, Serialize, Deserialize)] @@ -642,6 +704,7 @@ impl ConfigRenderer { labels, healthcheck, runtime: None, + run: AppRunSettings::from_app(app), }) } @@ -1182,12 +1245,24 @@ services: runtime: {{ app.runtime }} {% endif %} {% if app.command %} - command: {{ app.command }} + command: {{ app.command | yaml_quote }} {% endif %} {% if app.entrypoint %} - entrypoint: {{ app.entrypoint }} + entrypoint: {{ app.entrypoint | yaml_quote }} +{% endif %} + restart: {{ app.restart_policy | yaml_quote }} +{% if app.profiles | length > 0 %} + profiles: {{ app.profiles | json_encode() }} +{% endif %} +{% if app.cap_add | length > 0 %} + cap_add: {{ app.cap_add | json_encode() }} +{% endif %} +{% if app.cap_drop | length > 0 %} + cap_drop: {{ app.cap_drop | json_encode() }} +{% endif %} +{% if app.devices | length > 0 %} + devices: {{ app.devices | json_encode() }} {% endif %} - restart: {{ app.restart_policy }} {% if app.environment | length > 0 %} environment: {% for key, value in app.environment %}{% set entry = key ~ "=" ~ value %} @@ -1207,13 +1282,20 @@ services: {% endfor %} {% endif %} -{% if app.networks | length > 0 %} networks: +{% if app.networks | length > 0 %} {% for network in app.networks %} +{% if app.aliases | length > 0 %} + {{ network }}: + aliases: {{ app.aliases | json_encode() }} +{% else %} - {{ network }} +{% endif %} {% endfor %} +{% elif app.aliases | length > 0 %} + {{ default_network }}: + aliases: {{ app.aliases | json_encode() }} {% else %} - networks: - {{ default_network }} {% endif %} {% if app.depends_on | length > 0 %} @@ -1244,9 +1326,16 @@ services: start_period: {{ app.healthcheck.start_period }} {% endif %} {% endif %} -{% if app.resources.memory_limit or app.resources.cpu_limit %} +{% set has_limits = app.resources.memory_limit or app.resources.cpu_limit %} +{% set has_gpu = app.gpu_devices | length > 0 %} +{% if has_limits or has_gpu or app.replicas is number %} deploy: +{% if app.replicas is number %} + replicas: {{ app.replicas }} +{% endif %} +{% if has_limits or has_gpu %} resources: +{% if has_limits %} limits: {% if app.resources.memory_limit %} memory: {{ app.resources.memory_limit }} @@ -1254,14 +1343,20 @@ services: {% if app.resources.cpu_limit %} cpus: '{{ app.resources.cpu_limit }}' {% endif %} -{% if app.resources.memory_reservation or app.resources.cpu_reservation %} +{% endif %} +{% set limit_reservations = has_limits and (app.resources.memory_reservation or app.resources.cpu_reservation) %} +{% if limit_reservations or has_gpu %} reservations: -{% if app.resources.memory_reservation %} +{% if limit_reservations and app.resources.memory_reservation %} memory: {{ app.resources.memory_reservation }} {% endif %} -{% if app.resources.cpu_reservation %} +{% if limit_reservations and app.resources.cpu_reservation %} cpus: '{{ app.resources.cpu_reservation }}' {% endif %} +{% if has_gpu %} + devices: {{ app.gpu_devices | json_encode() }} +{% endif %} +{% endif %} {% endif %} {% endif %} @@ -1842,9 +1937,78 @@ mod tests { assert_eq!(ctx.image, "nginx:latest"); } + /// Every key of `expected` is in `actual` with the same value; + /// `networks_aliases`: every network the service joins carries them. + fn assert_compose_subset(case: &str, actual: &serde_json::Value, expected: &serde_json::Value) { + for (key, want) in expected.as_object().unwrap() { + if key == "networks_aliases" { + let networks = actual["networks"].as_object().unwrap_or_else(|| { + panic!( + "{case}: networks with aliases expected, got {}", + actual["networks"] + ) + }); + assert!(!networks.is_empty(), "{case}: no network"); + for (name, net) in networks { + assert_eq!(&net["aliases"], want, "{case}: aliases on {name}"); + } + } else if want.is_object() { + assert_compose_subset(case, &actual[key], want); + } else { + assert_eq!(&actual[key], want, "{case}: `{key}`\nservice: {actual}"); + } + } + } + + /// shared-fixtures/api-contracts/stack-service-runtime-fields.json, path + /// agent_render: stacker.yml -> registered project app -> project_app row + /// -> the compose the status-panel agent applies. + #[test] + fn agent_render_carries_every_declared_setting() { + let contract: serde_json::Value = serde_json::from_str(include_str!( + "../../tests/contracts/stack-service-runtime-fields.contract.json" + )) + .unwrap(); + let renderer = ConfigRenderer::new().unwrap(); + let project = Project { + name: "runtime-fields".to_string(), + ..Project::default() + }; + for case in contract["cases"].as_array().unwrap() { + let name = case["name"].as_str().unwrap(); + let service = &case["stacker_yml"]; + let yml = json!({ + "name": "runtime-fields", + "app": { "type": "custom", "image": "nginx:1.27" }, + "services": [service], + }); + let config = crate::cli::config_parser::StackerConfig::from_str( + &serde_yaml::to_string(&yml).unwrap(), + ) + .unwrap(); + let body = crate::cli::stacker_client::build_project_body(&config); + let form: crate::forms::project::ProjectForm = + serde_json::from_value(json!({ "custom": body["custom"] })).unwrap(); + let apps = crate::project_app::sync::project_level_apps_from_form(1, &form); + let code = service["name"].as_str().unwrap(); + let app = apps + .iter() + .find(|a| a.code == code) + .unwrap_or_else(|| panic!("{name}: no project_app row")); + let ctx = renderer + .project_app_to_context(app, HashMap::new()) + .unwrap(); + let compose = renderer.render_compose(&[ctx], &project).unwrap(); + let doc: serde_json::Value = serde_yaml::from_str(&compose) + .unwrap_or_else(|e| panic!("{name}: rendered compose is YAML: {e}\n{compose}")); + assert_compose_subset(name, &doc["services"][code], &case["compose"]); + } + } + #[test] fn render_compose_includes_kata_runtime() { let ctx = AppRenderContext { + run: Default::default(), code: "web".to_string(), name: "web".to_string(), image: "nginx:latest".to_string(), @@ -1874,6 +2038,7 @@ mod tests { #[test] fn render_compose_runtime_none_serializes_null() { let ctx = AppRenderContext { + run: Default::default(), code: "web".to_string(), name: "web".to_string(), image: "nginx:latest".to_string(), @@ -1916,6 +2081,7 @@ mod tests { ..Project::default() }; let app_ctx = AppRenderContext { + run: Default::default(), code: "app".to_string(), name: "app".to_string(), image: "miniflux/miniflux:latest".to_string(), @@ -1980,6 +2146,7 @@ mod tests { ..Project::default() }; let app_ctx = AppRenderContext { + run: Default::default(), code: "miniflux".to_string(), name: "miniflux".to_string(), image: "miniflux/miniflux:latest".to_string(), @@ -2021,6 +2188,7 @@ mod tests { ..Project::default() }; let postgres_ctx = AppRenderContext { + run: Default::default(), code: "postgres".to_string(), name: "postgres".to_string(), image: "postgres:16-alpine".to_string(), @@ -2115,6 +2283,7 @@ mod tests { } let ctx = AppRenderContext { + run: Default::default(), code: "web".to_string(), name: "web".to_string(), image: "nginx:latest".to_string(), @@ -2167,6 +2336,7 @@ mod tests { ..Project::default() }; let ctx = AppRenderContext { + run: Default::default(), code: "web".to_string(), name: "web".to_string(), image: "nginx:latest".to_string(), @@ -2200,6 +2370,7 @@ mod tests { ..Project::default() }; let ctx = AppRenderContext { + run: Default::default(), code: "web".to_string(), name: "web".to_string(), image: "nginx:latest".to_string(), @@ -2239,6 +2410,7 @@ mod tests { "web", ); let ctx = AppRenderContext { + run: Default::default(), code: "web".to_string(), name: "web".to_string(), image: "nginx:latest".to_string(), diff --git a/src/services/managed_usage.rs b/src/services/managed_usage.rs new file mode 100644 index 00000000..75348366 --- /dev/null +++ b/src/services/managed_usage.rs @@ -0,0 +1,279 @@ +//! Daily usage of the Managed server behind a Quick Deploy. +//! +//! Each day a server runs is one Stripe meter event worth that day's rate in +//! cents; the month's events become one invoice, never more than the monthly +//! cap. Contract: config/shared-fixtures/api-contracts/managed-server-billing.json. +//! +//! Pure functions only: the sweeper reads a usage row, asks [`next_report`] +//! for the next day to send, sends it, and stores the returned state. + +use chrono::{Datelike, NaiveDate}; + +/// The rate and cap a deployment is billed at, in cents. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct DailyRate { + pub rate_minor: i64, + pub cap_minor: i64, +} + +/// The daily rate is the higher of the template's and the server type's; the +/// cap goes with the rate that wins (a template without its own cap is capped +/// at 30 days of its rate). A server type without a rate is refused: `None`. +pub fn resolve_daily_rate( + template_rate: Option, + template_cap: Option, + server_type_rate: Option<(f64, f64)>, +) -> Option { + let (type_rate, type_cap) = server_type_rate?; + let type_rate_minor = to_minor(type_rate); + match template_rate.map(to_minor) { + Some(template_rate_minor) if template_rate_minor > type_rate_minor => Some(DailyRate { + rate_minor: template_rate_minor, + cap_minor: template_cap + .map(to_minor) + .unwrap_or(template_rate_minor * 30), + }), + _ => Some(DailyRate { + rate_minor: type_rate_minor, + cap_minor: to_minor(type_cap), + }), + } +} + +fn to_minor(amount: f64) -> i64 { + (amount * 100.0).round() as i64 +} + +fn month_start(day: NaiveDate) -> NaiveDate { + day.with_day(1).expect("day 1 exists in every month") +} + +/// What is known about a deployment's reported usage. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct UsageState { + pub rate_minor: i64, + pub cap_minor: i64, + /// The day the deploy succeeded and usage started. + pub started_on: NaiveDate, + /// First day of the month `period_reported_minor` counts. + pub period_start: Option, + pub period_reported_minor: i64, + pub last_reported_on: Option, +} + +/// One day to report. `value_minor` is 0 once the month's cap is reached: +/// the day is marked as done, nothing is sent. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct DayReport { + pub day: NaiveDate, + pub value_minor: i64, + pub period_start: NaiveDate, + pub period_reported_minor: i64, +} + +/// The earliest day not yet reported, up to and including `until` (today for a +/// running server, the deletion day for a stopped one). Missed days are caught +/// up one by one; the monthly counter restarts on the 1st. +pub fn next_report(state: &UsageState, until: NaiveDate) -> Option { + let day = match state.last_reported_on { + Some(last) => last.succ_opt()?, + None => state.started_on, + }; + if day < state.started_on || day > until { + return None; + } + let period_start = month_start(day); + let reported = if state.period_start == Some(period_start) { + state.period_reported_minor + } else { + 0 + }; + let value_minor = state.rate_minor.min(state.cap_minor - reported).max(0); + Some(DayReport { + day, + value_minor, + period_start, + period_reported_minor: reported + value_minor, + }) +} + +/// The meter event identifier: one per deployment and day, so a resend of the +/// same day is not counted twice. +pub fn identifier(deployment_hash: &str, day: NaiveDate) -> String { + format!("{}:{}", deployment_hash, day.format("%Y-%m-%d")) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn d(y: i32, m: u32, day: u32) -> NaiveDate { + NaiveDate::from_ymd_opt(y, m, day).unwrap() + } + + const CPX32: (f64, f64) = (0.86, 26.00); + const CPX42: (f64, f64) = (1.67, 50.00); + + #[test] + fn the_rate_is_the_higher_of_the_template_and_the_server_type() { + assert_eq!( + resolve_daily_rate(None, None, Some(CPX32)), + Some(DailyRate { + rate_minor: 86, + cap_minor: 2600 + }) + ); + assert_eq!( + resolve_daily_rate(Some(0.50), Some(15.00), Some(CPX42)), + Some(DailyRate { + rate_minor: 167, + cap_minor: 5000 + }) + ); + assert_eq!( + resolve_daily_rate(Some(2.00), Some(55.00), Some(CPX32)), + Some(DailyRate { + rate_minor: 200, + cap_minor: 5500 + }) + ); + } + + #[test] + fn a_winning_template_rate_without_a_cap_is_capped_at_30_days() { + assert_eq!( + resolve_daily_rate(Some(2.00), None, Some(CPX32)), + Some(DailyRate { + rate_minor: 200, + cap_minor: 6000 + }) + ); + } + + #[test] + fn a_server_type_without_a_rate_is_refused() { + assert_eq!(resolve_daily_rate(Some(2.00), Some(60.0), None), None); + assert_eq!(resolve_daily_rate(None, None, None), None); + } + + fn running(started_on: NaiveDate) -> UsageState { + UsageState { + rate_minor: 86, + cap_minor: 2600, + started_on, + period_start: None, + period_reported_minor: 0, + last_reported_on: None, + } + } + + fn apply(state: &mut UsageState, report: &DayReport) { + state.period_start = Some(report.period_start); + state.period_reported_minor = report.period_reported_minor; + state.last_reported_on = Some(report.day); + } + + /// Drive the sweeper day by day; returns the value sent for each day. + fn run(state: &mut UsageState, until: NaiveDate) -> Vec<(NaiveDate, i64)> { + let mut sent = Vec::new(); + while let Some(report) = next_report(state, until) { + apply(state, &report); + sent.push((report.day, report.value_minor)); + } + sent + } + + #[test] + fn the_first_day_is_the_day_usage_started() { + let state = running(d(2026, 10, 16)); + let report = next_report(&state, d(2026, 10, 16)).unwrap(); + assert_eq!(report.day, d(2026, 10, 16)); + assert_eq!(report.value_minor, 86); + assert_eq!(report.period_start, d(2026, 10, 1)); + assert_eq!(report.period_reported_minor, 86); + } + + #[test] + fn a_day_already_reported_is_not_reported_again() { + let mut state = running(d(2026, 10, 16)); + run(&mut state, d(2026, 10, 16)); + assert_eq!(next_report(&state, d(2026, 10, 16)), None); + } + + #[test] + fn a_31_day_month_stops_at_the_cap() { + let mut state = running(d(2026, 10, 1)); + let sent = run(&mut state, d(2026, 10, 31)); + assert_eq!(sent.len(), 31); + assert!(sent[..30].iter().all(|(_, v)| *v == 86)); + assert_eq!(sent[30], (d(2026, 10, 31), 20)); + assert_eq!(sent.iter().map(|(_, v)| v).sum::(), 2600); + } + + #[test] + fn days_after_the_cap_are_marked_with_nothing_sent() { + let mut state = UsageState { + rate_minor: 167, + cap_minor: 500, + ..running(d(2026, 10, 1)) + }; + let sent = run(&mut state, d(2026, 10, 5)); + assert_eq!( + sent.iter().map(|(_, v)| *v).collect::>(), + vec![167, 167, 166, 0, 0] + ); + } + + #[test] + fn the_cap_starts_again_on_the_first() { + let mut state = running(d(2026, 10, 1)); + run(&mut state, d(2026, 10, 31)); + let report = next_report(&state, d(2026, 11, 1)).unwrap(); + assert_eq!(report.day, d(2026, 11, 1)); + assert_eq!(report.value_minor, 86); + assert_eq!(report.period_start, d(2026, 11, 1)); + assert_eq!(report.period_reported_minor, 86); + } + + #[test] + fn missed_days_are_caught_up_in_order() { + let mut state = running(d(2026, 10, 16)); + run(&mut state, d(2026, 10, 16)); + let sent = run(&mut state, d(2026, 10, 19)); + assert_eq!( + sent, + vec![ + (d(2026, 10, 17), 86), + (d(2026, 10, 18), 86), + (d(2026, 10, 19), 86) + ] + ); + } + + #[test] + fn a_server_deleted_on_the_10th_is_billed_up_to_the_10th() { + let mut state = UsageState { + rate_minor: 167, + cap_minor: 5000, + ..running(d(2026, 10, 1)) + }; + let sent = run(&mut state, d(2026, 10, 10)); + assert_eq!(sent.len(), 10); + assert_eq!(sent.iter().map(|(_, v)| v).sum::(), 1670); + assert_eq!(next_report(&state, d(2026, 10, 10)), None); + } + + #[test] + fn nothing_is_reported_before_usage_started() { + let state = running(d(2026, 10, 16)); + assert_eq!(next_report(&state, d(2026, 10, 15)), None); + } + + #[test] + fn the_identifier_names_the_deployment_and_the_day() { + assert_eq!( + identifier("deployment_abc", d(2026, 10, 6)), + "deployment_abc:2026-10-06" + ); + } +} diff --git a/src/services/managed_usage_sweeper.rs b/src/services/managed_usage_sweeper.rs new file mode 100644 index 00000000..8dd9f48b --- /dev/null +++ b/src/services/managed_usage_sweeper.rs @@ -0,0 +1,200 @@ +//! Daily report of the Managed server usage behind a Quick Deploy. +//! +//! Each tick sends every day not yet reported, up to today for a running +//! server and up to the stop day for a deleted or suspended one, as one +//! meter event per day through the User Service. A server suspended for 3 +//! days is handed to the admins (`server_delete_due`): nothing deletes a VM +//! automatically. Contract: shared-fixtures/api-contracts/managed-server-billing.json. + +use std::sync::Arc; +use std::time::Duration as StdDuration; + +use async_trait::async_trait; +use chrono::{DateTime, Duration, NaiveDate, Utc}; +use sqlx::PgPool; + +use crate::connectors::user_service::{ + ManagedBillingEvent, ManagedUsageReport, UserServiceConnector, +}; +use crate::db::managed_usage; +use crate::services::managed_usage::{identifier, next_report, UsageState}; + +const TICK: StdDuration = StdDuration::from_secs(600); +const BATCH_LIMIT: i64 = 500; +/// Stripe accepts a meter event up to 35 days old; older days are skipped. +const MAX_DAY_AGE: i64 = 34; +/// A suspended server is handed to the admins after this long. +pub const SUSPENSION_HANDOFF_DAYS: i64 = 3; + +/// Where usage and billing events go. The User Service in production. +#[async_trait] +pub trait UsageSink: Send + Sync { + async fn report(&self, usage: &ManagedUsageReport) -> Result<(), String>; + async fn event(&self, event: &ManagedBillingEvent) -> Result<(), String>; +} + +pub struct UserServiceSink { + pub user_service: Arc, + pub service_token: String, +} + +#[async_trait] +impl UsageSink for UserServiceSink { + async fn report(&self, usage: &ManagedUsageReport) -> Result<(), String> { + self.user_service + .report_managed_usage(&self.service_token, usage) + .await + .map_err(|e| e.to_string()) + } + + async fn event(&self, event: &ManagedBillingEvent) -> Result<(), String> { + self.user_service + .record_managed_billing_event(&self.service_token, event) + .await + .map_err(|e| e.to_string()) + } +} + +#[derive(Debug, Default, PartialEq, Eq)] +pub struct TickSummary { + pub days_reported: usize, + pub handed_to_admins: usize, +} + +pub fn spawn(pg_pool: PgPool, user_service: Arc) { + let sink = UserServiceSink { + user_service, + service_token: std::env::var("STACKER_SERVICE_TOKEN").unwrap_or_default(), + }; + tokio::spawn(async move { + tracing::info!("managed_usage_sweeper started (tick={:?})", TICK); + loop { + let now = Utc::now(); + if let Err(err) = tick_once(&pg_pool, &sink, now.date_naive(), now).await { + tracing::warn!("managed_usage_sweeper tick error: {}", err); + } + tokio::time::sleep(TICK).await; + } + }); +} + +pub async fn tick_once( + pool: &PgPool, + sink: &dyn UsageSink, + today: NaiveDate, + now: DateTime, +) -> Result { + let mut summary = TickSummary::default(); + + for row in managed_usage::list_due(pool, today, BATCH_LIMIT).await? { + let Some(started_at) = row.started_at else { + continue; + }; + let mut state = UsageState { + rate_minor: row.daily_rate_minor, + cap_minor: row.monthly_cap_minor, + started_on: started_at.date_naive(), + period_start: row.period_start, + period_reported_minor: row.period_reported_minor, + last_reported_on: row.last_reported_on, + }; + let until = row.report_until(today); + let oldest = today - Duration::days(MAX_DAY_AGE); + while let Some(report) = next_report(&state, until) { + if report.day < oldest { + tracing::error!( + deployment_hash = %row.deployment_hash, + day = %report.day, + "managed usage day too old for the Stripe meter; skipped" + ); + } else if report.value_minor > 0 { + let usage = ManagedUsageReport { + user_id: row.user_id.clone(), + deployment_hash: row.deployment_hash.clone(), + value_minor: report.value_minor, + identifier: identifier(&row.deployment_hash, report.day), + timestamp: Some(timestamp_for(report.day, today, now, started_at)), + }; + if let Err(err) = sink.report(&usage).await { + // Not marked: the next tick sends this day again. + tracing::warn!( + deployment_hash = %row.deployment_hash, + day = %report.day, + error = %err, + "managed usage report failed" + ); + break; + } + summary.days_reported += 1; + } + if !managed_usage::record_day( + pool, + row.id, + report.period_start, + report.period_reported_minor, + report.day, + ) + .await? + { + break; + } + state.period_start = Some(report.period_start); + state.period_reported_minor = report.period_reported_minor; + state.last_reported_on = Some(report.day); + } + } + + let cutoff = now - Duration::days(SUSPENSION_HANDOFF_DAYS); + for row in managed_usage::list_suspended_before(pool, cutoff).await? { + // The alert first: if it does not get through, the row stays + // suspended and the next tick tries again. + let event = ManagedBillingEvent { + user_id: row.user_id.clone(), + deployment_hash: row.deployment_hash.clone(), + event: "server_delete_due".to_string(), + amount_minor: None, + currency: None, + metadata: Some(serde_json::json!({ + "server_id": row.server_id, + "stack": row.stack, + "server_type": row.server_type, + "suspended_at": row.suspended_at, + })), + }; + if let Err(err) = sink.event(&event).await { + tracing::warn!( + deployment_hash = %row.deployment_hash, + error = %err, + "server_delete_due event failed" + ); + continue; + } + if managed_usage::mark_delete_due(pool, &row.deployment_hash) + .await? + .is_some() + { + summary.handed_to_admins += 1; + } + } + + Ok(summary) +} + +/// Today's day is stamped now; a past day late in that day (but after the +/// usage started), so Stripe puts it in the right invoice period. +fn timestamp_for( + day: NaiveDate, + today: NaiveDate, + now: DateTime, + started_at: DateTime, +) -> i64 { + if day >= today { + return now.timestamp(); + } + let late = day + .and_hms_opt(23, 0, 0) + .expect("23:00 exists") + .and_utc() + .max(started_at + Duration::seconds(1)); + late.min(now).timestamp() +} diff --git a/src/services/marketplace_access.rs b/src/services/marketplace_access.rs index dda3b3c9..10f84d79 100644 --- a/src/services/marketplace_access.rs +++ b/src/services/marketplace_access.rs @@ -112,7 +112,8 @@ pub enum AccessMode { /// post-deploy settlement, where a card alone would deploy for free. RequireOwnership, /// A saved payment method is enough — the caller settles the charge after - /// a successful deploy (the one-click path's completion hook). + /// a successful deploy (the one-click path's completion hook). The Pro + /// feature plan is not required on this path. AllowCardOnFile, } @@ -141,12 +142,17 @@ pub async fn validate_marketplace_template_access_with_mode( .as_deref() .ok_or(MarketplaceAccessError::MissingUserToken)?; - let has_feature_plan = user_service - .user_has_plan(&user.id, MARKETPLACE_INSTALL_MIN_PLAN, Some(user_token)) - .await - .map_err(validation_failed)?; - if !has_feature_plan { - return Err(MarketplaceAccessError::InsufficientFeaturePlan); + // Quick Deploy (AllowCardOnFile) is plan-independent: every buyer adds a + // card and pays for the Managed server. CLI and marketplace installs keep + // the Pro feature plan. + if mode == AccessMode::RequireOwnership { + let has_feature_plan = user_service + .user_has_plan(&user.id, MARKETPLACE_INSTALL_MIN_PLAN, Some(user_token)) + .await + .map_err(validation_failed)?; + if !has_feature_plan { + return Err(MarketplaceAccessError::InsufficientFeaturePlan); + } } if let Some(required_plan) = template @@ -532,6 +538,22 @@ mod tests { status: "captured".to_string(), }) } + + async fn report_managed_usage( + &self, + _service_token: &str, + _usage: &crate::connectors::user_service::ManagedUsageReport, + ) -> Result<(), ConnectorError> { + Ok(()) + } + + async fn record_managed_billing_event( + &self, + _service_token: &str, + _event: &crate::connectors::user_service::ManagedBillingEvent, + ) -> Result<(), ConnectorError> { + Ok(()) + } } fn test_user() -> models::User { @@ -1082,4 +1104,100 @@ mod tests { "ownership alone must settle the check — no card probe" ); } + + // ── Quick Deploy is plan-independent (quick-deploy-managed-server-billing.feature) ── + // Every buyer adds a card and pays for the Managed server, so the Pro + // feature plan does not gate the one-click path. CLI and marketplace + // installs (RequireOwnership) keep it. + + #[tokio::test] + async fn deferred_mode_lets_a_free_plan_user_deploy_a_free_template() { + let svc = Arc::new(TestUserService::new(&[("professional", false)], &[])); + let user_service: Arc = svc.clone(); + let template = models::StackTemplate { + slug: "local-ai-packaged".to_string(), + ..Default::default() + }; + + let result = validate_marketplace_template_access_with_mode( + &user_service, + &test_user(), + &template, + AccessMode::AllowCardOnFile, + ) + .await; + + assert!(result.is_ok(), "{result:?}"); + } + + #[tokio::test] + async fn deferred_mode_asks_a_free_plan_user_for_a_card_on_a_priced_template() { + let svc = Arc::new( + TestUserService::new(&[("professional", false)], &[]) + .with_can_charge(false, Some("no_payment_method")), + ); + let user_service: Arc = svc.clone(); + let template = models::StackTemplate { + slug: "paid-template".to_string(), + product_id: Some(100), + price: Some(15.0), + ..Default::default() + }; + + let result = validate_marketplace_template_access_with_mode( + &user_service, + &test_user(), + &template, + AccessMode::AllowCardOnFile, + ) + .await; + + assert!( + matches!(result, Err(MarketplaceAccessError::NoPaymentMethod { .. })), + "{result:?}" + ); + } + + #[tokio::test] + async fn deferred_mode_still_requires_the_templates_own_plan() { + let svc = Arc::new(TestUserService::new( + &[("professional", false), ("enterprise", false)], + &[], + )); + let user_service: Arc = svc.clone(); + + let result = validate_marketplace_template_access_with_mode( + &user_service, + &test_user(), + &test_template(), + AccessMode::AllowCardOnFile, + ) + .await; + + assert!( + matches!( + result, + Err(MarketplaceAccessError::InsufficientTemplatePlan { .. }) + ), + "{result:?}" + ); + } + + #[tokio::test] + async fn ownership_mode_still_requires_the_pro_feature_plan() { + let svc = Arc::new(TestUserService::new(&[("professional", false)], &[])); + let user_service: Arc = svc.clone(); + let template = models::StackTemplate { + slug: "local-ai-packaged".to_string(), + ..Default::default() + }; + + let result = + validate_marketplace_template_access(&user_service, &test_user(), &template).await; + + assert!(matches!( + result, + Err(MarketplaceAccessError::InsufficientFeaturePlan) + )); + } } diff --git a/src/services/mod.rs b/src/services/mod.rs index 0b41e9ec..74bb79c7 100644 --- a/src/services/mod.rs +++ b/src/services/mod.rs @@ -16,6 +16,8 @@ pub mod handoff; pub mod install_authorization_sweeper; pub mod log_cache; pub mod marketplace_access; +pub mod managed_usage; +pub mod managed_usage_sweeper; pub mod marketplace_assets; pub mod payout_provider; pub mod project; diff --git a/src/startup.rs b/src/startup.rs index 25077947..f7442c52 100644 --- a/src/startup.rs +++ b/src/startup.rs @@ -32,6 +32,7 @@ fn project_scope(path: &str) -> actix_web::Scope { .service(crate::routes::project::invitation::list) .service(crate::routes::project::invitation::revoke) .service(crate::routes::project::invitation::seats) + .service(crate::routes::project::clouds::list) .service(crate::routes::project::compose::add) .service(crate::routes::project::get::list) .service(crate::routes::project::get::shared_list) @@ -41,6 +42,7 @@ fn project_scope(path: &str) -> actix_web::Scope { .service(crate::routes::project::sync::item) .service(crate::routes::project::delete::item) .service(crate::routes::project::protection::toggle) + .service(crate::routes::project::member::set_server_access) .service(crate::routes::project::app::list_apps) .service(crate::routes::project::app::create_app) .service(crate::routes::project::app::get_app) @@ -126,6 +128,13 @@ pub async fn run( user_service_connector.get_ref().clone(), settings.per_install_billing_enabled, ); + // Reports each running day of a Quick Deploy Managed server to the + // Stripe meter through the User Service, and hands servers unpaid for 3 + // days of suspension to the admins. Without it the servers are not billed. + crate::services::managed_usage_sweeper::spawn( + api_pool.get_ref().clone(), + user_service_connector.get_ref().clone(), + ); // Deletes container rows retired more than a month ago. Housekeeping only: // it touches nothing the dashboard shows, and skipping it costs a slowly // growing table rather than correctness. @@ -289,9 +298,17 @@ pub async fn run( ) .service( web::scope("/api") + .service( + web::scope("/internal/managed-usage") + .service(crate::routes::managed_usage::suspend) + .service(crate::routes::managed_usage::start) + .service(crate::routes::managed_usage::fail) + .service(crate::routes::managed_usage::stop), + ) .service( web::scope("/agreement") .service(crate::routes::agreement::user_add_handler) + .service(crate::routes::agreement::get_by_name_handler) .service(crate::routes::agreement::get_handler) .service(crate::routes::agreement::accept_handler), ) @@ -387,6 +404,7 @@ pub async fn run( .service( web::scope("/v1/deployments") .service(routes::deployment::capabilities_handler) + .service(routes::deployment::member_access_handler) .service(routes::deployment::events_handler) .service(routes::deployment::list_handler) .service(routes::deployment::plan_handler) @@ -454,6 +472,11 @@ pub async fn run( ) .service( web::scope("/admin") + .service( + web::scope("/bake") + .service(crate::routes::admin_bake::start_handler) + .service(crate::routes::admin_bake::status_handler), + ) .service( web::scope("/templates") .service( @@ -469,6 +492,7 @@ pub async fn run( .service(crate::routes::marketplace::admin::security_scan_handler) .service(crate::routes::marketplace::admin::detect_secrets_handler) .service(crate::routes::marketplace::admin::pricing_handler) + .service(crate::routes::marketplace::admin::update_template_handler) .service(crate::routes::marketplace::admin::update_verifications_handler) .service(crate::routes::marketplace::admin::update_vendor_profile_handler), ) @@ -488,6 +512,10 @@ pub async fn run( crate::routes::admin_notifications::unread_count_handler, ) .service(crate::routes::admin_notifications::update_handler), + ) + .service( + web::scope("/chat") + .service(crate::routes::chat::admin::recent), ), ) .service( diff --git a/tests/admin_bake.rs b/tests/admin_bake.rs new file mode 100644 index 00000000..917e51ad --- /dev/null +++ b/tests/admin_bake.rs @@ -0,0 +1,266 @@ +//! `POST /api/admin/bake`: everything that can be checked without touching the +//! build box is refused at once, with no job recorded and nothing done to the +//! box. Hetzner is a wiremock; `HETZNER_TOKEN` / `HETZNER_API_BASE_URL` are +//! process-wide, so every case runs inside one test. + +mod common; + +use chrono::{Duration, Utc}; +use serde_json::{json, Value}; +use wiremock::matchers::{method, path}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +const BOX_IP: &str = "203.0.113.7"; +const STACK: &str = "bake-endpoint-stack"; + +fn admin_jwt() -> String { + use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine}; + let header = URL_SAFE_NO_PAD.encode(json!({"alg": "HS256", "typ": "JWT"}).to_string()); + let payload = URL_SAFE_NO_PAD.encode( + json!({ + "role": "admin_service", + "email": "ops@test.com", + "exp": (Utc::now() + Duration::minutes(30)).timestamp(), + }) + .to_string(), + ); + let signature = common::sign_test_jwt(&header, &payload); + format!("{header}.{payload}.{signature}") +} + +fn contract() -> Value { + json!({ "services": { "app": { "fields": { + "APP_SECRET": { "mutability": "generated", "type": "hex", "length": 32 } + } } } }) +} + +async fn seed_template(pool: &sqlx::PgPool, slug: &str, status: &str, contract: Option) { + let id: uuid::Uuid = sqlx::query_scalar( + r#"INSERT INTO stack_template (creator_user_id, creator_name, name, slug, status, approved_at) + VALUES ('vendor', 'Vendor', $1, $1, $2, CASE WHEN $2 = 'approved' THEN NOW() END) + RETURNING id"#, + ) + .bind(slug) + .bind(status) + .fetch_one(pool) + .await + .expect("seed template"); + sqlx::query( + r#"INSERT INTO stack_template_version (template_id, version, stack_definition, definition_format, is_latest, config_contract) + VALUES ($1, '1.2.0', '{}'::jsonb, 'json', true, $2)"#, + ) + .bind(id) + .bind(contract) + .execute(pool) + .await + .expect("seed version"); +} + +/// A deployment whose project has one server at `ip`. +async fn seed_build_box(pool: &sqlx::PgPool, ip: &str) -> i32 { + let project = common::create_test_project(pool, common::USER_A_ID).await; + let hash = format!("bake-{}", uuid::Uuid::new_v4()); + let deployment = common::create_test_deployment(pool, common::USER_A_ID, project, &hash).await; + let server = common::create_test_server(pool, common::USER_A_ID, project, "active", None).await; + sqlx::query("UPDATE server SET srv_ip = $2 WHERE id = $1") + .bind(server) + .bind(ip) + .execute(pool) + .await + .unwrap(); + deployment +} + +/// Another server record in the deployment's project, at `ip`; returns its id. +async fn add_server(pool: &sqlx::PgPool, deployment: i32, ip: &str) -> i32 { + let project: i32 = sqlx::query_scalar("SELECT project_id FROM deployment WHERE id = $1") + .bind(deployment) + .fetch_one(pool) + .await + .unwrap(); + let server = common::create_test_server(pool, common::USER_A_ID, project, "active", None).await; + sqlx::query("UPDATE server SET srv_ip = $2 WHERE id = $1") + .bind(server) + .bind(ip) + .execute(pool) + .await + .unwrap(); + server +} + +async fn job_count(pool: &sqlx::PgPool) -> i64 { + sqlx::query_scalar("SELECT COUNT(*) FROM bake_job") + .fetch_one(pool) + .await + .unwrap() +} + +async fn post(app: &common::TestApp, body: Value) -> (u16, Value) { + let resp = reqwest::Client::new() + .post(format!("{}/api/admin/bake", app.address)) + .bearer_auth(admin_jwt()) + .json(&body) + .send() + .await + .expect("request"); + let status = resp.status().as_u16(); + (status, resp.json().await.unwrap_or(Value::Null)) +} + +#[tokio::test] +async fn a_bake_is_refused_before_the_box_is_touched_and_accepted_when_everything_checks_out() { + let Some(app) = common::spawn_app().await else { + return; + }; + let hetzner = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/servers")) + .respond_with( + ResponseTemplate::new(200).set_body_json(json!({ "servers": [ + { "id": 4242, "name": "build-box", "public_net": { "ipv4": { "ip": BOX_IP } } } + ] })), + ) + .mount(&hetzner) + .await; + std::env::set_var("HETZNER_API_BASE_URL", hetzner.uri()); + std::env::set_var("HETZNER_TOKEN", "test-hetzner-token"); + + seed_template(&app.db_pool, STACK, "approved", Some(contract())).await; + seed_template( + &app.db_pool, + "bake-submitted-stack", + "submitted", + Some(contract()), + ) + .await; + seed_template(&app.db_pool, "bake-no-contract-stack", "approved", None).await; + let deployment = seed_build_box(&app.db_pool, BOX_IP).await; + let unknown_box = seed_build_box(&app.db_pool, "198.51.100.9").await; + let health = format!("http://{BOX_IP}/health"); + + for (body, expected, why) in [ + ( + json!({"deployment_id": 999999, "stack": STACK, "health_url": health}), + 404, + "unknown deployment", + ), + ( + json!({"deployment_id": deployment, "stack": "bake-submitted-stack", "health_url": health}), + 404, + "template not approved", + ), + ( + json!({"deployment_id": deployment, "stack": "bake-no-contract-stack", "health_url": health}), + 409, + "no contract to sanitize against", + ), + ( + json!({"deployment_id": deployment, "stack": STACK, "health_url": "http://198.51.100.1/health"}), + 400, + "health URL off the box", + ), + ( + json!({"deployment_id": unknown_box, "stack": STACK, "health_url": "http://198.51.100.9/health"}), + 409, + "box Hetzner does not know", + ), + ] { + let (status, resp) = post(&app, body).await; + assert_eq!(status, expected, "{why}: {resp}"); + assert_eq!( + job_count(&app.db_pool).await, + 0, + "{why}: no job may be recorded" + ); + } + + let (status, resp) = post( + &app, + json!({"deployment_id": deployment, "stack": STACK, "health_url": health}), + ) + .await; + assert_eq!(status, 202, "{resp}"); + let job = &resp["item"]; + assert_eq!(job["stack"], STACK); + assert_eq!( + job["version"], "1.2.0", + "the latest approved version is baked" + ); + assert_eq!(job["server_ip"], BOX_IP); + + let (status, resp) = post( + &app, + json!({"deployment_id": deployment, "stack": STACK, "health_url": health}), + ) + .await; + // Either still running (409) or already failed (no SSH in tests): never two at once. + if status == 202 { + assert_eq!(job_count(&app.db_pool).await, 2); + } else { + assert_eq!(status, 409, "{resp}"); + } + + let id = job["id"].as_str().unwrap(); + let status = reqwest::Client::new() + .get(format!("{}/api/admin/bake/{id}", app.address)) + .bearer_auth(admin_jwt()) + .send() + .await + .unwrap(); + assert_eq!(status.status().as_u16(), 200); + let body: Value = status.json().await.unwrap(); + assert_eq!(body["item"]["id"], id); + + // A project redeployed with --force-new keeps a server record per attempt, + // and Hetzner hands the new box an old box's IP. `server_id` - the Hetzner + // id, as the `bake` binary takes it - names the build box among them. + let redeployed = seed_build_box(&app.db_pool, "198.51.100.40").await; + add_server(&app.db_pool, redeployed, BOX_IP).await; + let build_box = add_server(&app.db_pool, redeployed, BOX_IP).await; + let jobs_before = job_count(&app.db_pool).await; + for (server_id, why) in [ + (Value::Null, "several servers and no server_id"), + ( + json!(9999), + "a server_id that is none of the project's boxes", + ), + ] { + let (status, resp) = post( + &app, + json!({"deployment_id": redeployed, "stack": STACK, "health_url": health, "server_id": server_id}), + ) + .await; + assert_eq!(status, 409, "{why}: {resp}"); + assert_eq!(job_count(&app.db_pool).await, jobs_before, "{why}: no job"); + } + let (status, resp) = post( + &app, + json!({"deployment_id": redeployed, "stack": STACK, "health_url": health, "server_id": 4242}), + ) + .await; + assert_eq!(status, 202, "{resp}"); + assert_eq!(resp["item"]["server_ip"], BOX_IP); + assert_eq!( + resp["item"]["server_id"], build_box, + "the newest record at the box's IP: older ones belong to removed boxes" + ); +} + +#[tokio::test] +async fn a_user_who_is_not_an_admin_cannot_bake() { + let Some(app) = common::spawn_app().await else { + return; + }; + let resp = reqwest::Client::new() + .post(format!("{}/api/admin/bake", app.address)) + .bearer_auth(common::USER_A_TOKEN) + .json(&json!({"deployment_id": 1, "stack": STACK, "health_url": "http://203.0.113.7/"})) + .send() + .await + .unwrap(); + assert!( + matches!(resp.status().as_u16(), 401 | 403), + "got {}", + resp.status() + ); +} diff --git a/tests/admin_chat_recent.rs b/tests/admin_chat_recent.rs new file mode 100644 index 00000000..801363b8 --- /dev/null +++ b/tests/admin_chat_recent.rs @@ -0,0 +1,92 @@ +//! Admin chat review must only expose chats of users who accepted the +//! `ai_chat_analysis` agreement, and must be closed to non-admins. +mod common; + +use common::{USER_A_ID, USER_A_TOKEN, USER_B_ID}; + +async fn insert_conversation(pool: &sqlx::PgPool, user_id: &str, marker: &str) { + sqlx::query( + "INSERT INTO chat_conversations (user_id, messages) VALUES ($1, $2) \ + ON CONFLICT DO NOTHING", + ) + .bind(user_id) + .bind(serde_json::json!([{"role": "user", "content": marker}])) + .execute(pool) + .await + .unwrap(); +} + +#[tokio::test] +async fn only_consenting_users_chats_are_returned() { + let Some(app) = common::spawn_app_two_users().await else { + return; + }; + let pool = &app.db_pool; + insert_conversation(pool, USER_A_ID, "consented-marker").await; + insert_conversation(pool, USER_B_ID, "private-marker").await; + + let agreement: i32 = + sqlx::query_scalar("SELECT id FROM agreement WHERE name = 'ai_chat_analysis'") + .fetch_one(pool) + .await + .expect("agreement seeded by migration"); + sqlx::query( + "INSERT INTO user_agreement (agrt_id, user_id, created_at, updated_at) \ + VALUES ($1, $2, NOW(), NOW())", + ) + .bind(agreement) + .bind(USER_A_ID) + .execute(pool) + .await + .unwrap(); + + let chats = stacker::db::chat_admin::recent_consented(pool, 50) + .await + .unwrap(); + let dump = serde_json::to_string(&chats).unwrap(); + assert!(chats.iter().any(|c| c.user_id == USER_A_ID)); + assert!(chats.iter().all(|c| c.user_id != USER_B_ID), "{dump}"); +} + +#[tokio::test] +async fn non_admin_cannot_read_recent_chats() { + let Some(app) = common::spawn_app_two_users().await else { + return; + }; + let resp = reqwest::Client::new() + .get(format!("{}/api/admin/chat/recent", app.address)) + .header("Authorization", format!("Bearer {}", USER_A_TOKEN)) + .send() + .await + .unwrap(); + assert_eq!(resp.status().as_u16(), 403); +} + +#[tokio::test] +async fn consent_agreement_and_casbin_rules_are_seeded() { + let Some(app) = common::spawn_app_two_users().await else { + return; + }; + let text: String = + sqlx::query_scalar("SELECT text FROM agreement WHERE name = 'ai_chat_analysis'") + .fetch_one(&app.db_pool) + .await + .expect("seeded agreement"); + assert!(text.contains("analyze my AI chat conversations")); + + for (role, path) in [ + ("group_admin", "/api/admin/chat/recent"), + ("group_user", "/api/agreement/name/:name"), + ("group_admin", "/api/agreement/name/:name"), + ] { + let n: i64 = sqlx::query_scalar( + "SELECT COUNT(*) FROM casbin_rule WHERE ptype='p' AND v0=$1 AND v1=$2 AND v2='GET'", + ) + .bind(role) + .bind(path) + .fetch_one(&app.db_pool) + .await + .unwrap(); + assert_eq!(n, 1, "{role} {path}"); + } +} diff --git a/tests/contracts/quick-deploy-server-type.contract.json b/tests/contracts/quick-deploy-server-type.contract.json new file mode 100644 index 00000000..5ee2a955 --- /dev/null +++ b/tests/contracts/quick-deploy-server-type.contract.json @@ -0,0 +1,57 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "quick-deploy-server-type", + "description": "The server type a Quick Deploy of a stack provisions, and the rates it is priced at. Stacker's part of the contract.", + "_notes": [ + "A Quick Deploy clones the stack's baked snapshot, whose disk is the disk of the build box it was baked on, so the snapshot's server type is the type a clone of that stack uses.", + "The bake records the build box's server type on the snapshot. A snapshot baked before that has none, and the caller's default type applies.", + "POST /api/v1/deploy/validate returns quick_deploy: that server type and the daily rate and monthly cap of every server type - the rates the clone bills." + ], + "version": "v1", + "rates": { + "cpx32": { + "daily_rate": 0.86, + "monthly_cap": 26.0 + }, + "cpx42": { + "daily_rate": 1.67, + "monthly_cap": 50.0 + } + }, + "stacker_validate": { + "quick_deploy": { + "type": "object", + "required": [ + "server_type", + "rates" + ], + "properties": { + "server_type": { + "type": [ + "string", + "null" + ], + "description": "server type the stack's latest healthy snapshot was baked on; null when unknown" + }, + "rates": { + "type": "object", + "additionalProperties": { + "type": "object", + "required": [ + "daily_rate", + "monthly_cap" + ], + "properties": { + "daily_rate": { + "type": "number" + }, + "monthly_cap": { + "type": "number" + } + } + } + } + } + } + } +} diff --git a/tests/contracts/stack-service-runtime-fields.contract.json b/tests/contracts/stack-service-runtime-fields.contract.json new file mode 100644 index 00000000..b1fbd142 --- /dev/null +++ b/tests/contracts/stack-service-runtime-fields.contract.json @@ -0,0 +1,151 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "stack-service-runtime-fields", + "description": "Runtime settings of a service declared under `services:` in stacker.yml, and the Docker Compose they must produce on every path that renders a compose file from it.", + "_owner": "trydirect/config", + "_consumers": [ + "stacker" + ], + "_notes": [ + "A stacker.yml service reaches a compose file along four paths. cli_compose: the CLI generates the compose it deploys (local, and uploaded to the server for cloud/server targets). compose_sync: with deploy.compose_file set, the CLI writes the service into the author's own compose file. project_app: the CLI registers the service as a project app (POST /project), from which the server renders the compose for deploys started outside the CLI. agent_render: the project_app row the server keeps per app, from which the config renderer writes the compose the status-panel agent applies.", + "A field declared in stacker.yml must reach the compose on every path, or be refused before it is lost. A silently dropped field deploys a different stack from the one the author wrote: a one-shot job without restart: \"no\" restarts for ever, a GPU service without its device reservation runs on the CPU.", + "restart applies only when set. An unset restart keeps each path's existing default: unless-stopped from the CLI generator, always for a registered project app.", + "aliases are added on every network the service joins, so other services reach it by the alias whichever network they share with it. A service that names no network joins `default`, and carries its aliases there.", + "deploy.replicas: 0 keeps a service defined but not started. Together with profiles it lets one stack carry CPU and GPU variants of the same service: the CPU variant runs by default, the GPU variants only under their profile.", + "deploy.replicas and a device count may name a variable (${VAR}, ${VAR:-default}). When stacker.yml is read without filling variables - deploy validation, CLI commands that rewrite stacker.yml - the reference is kept and reaches the compose as written; filled, its value must be a whole number. A server-rendered compose (project_app, agent_render) only carries a known number.", + "Each case gives a service as written in stacker.yml (as JSON) and the compose service it must produce. The compose subset is what a consumer must contain; other keys (labels, networks without aliases, environment) are left to each path." + ], + "version": "v1", + "paths": [ + "cli_compose", + "compose_sync", + "project_app", + "agent_render" + ], + "fields": { + "restart": { + "status": "new", + "type": "string", + "enum": ["no", "always", "unless-stopped", "on-failure"] + }, + "entrypoint": { + "status": "new", + "type": "string" + }, + "profiles": { + "status": "new", + "type": "array", + "items": {"type": "string"} + }, + "aliases": { + "status": "new", + "type": "array", + "items": {"type": "string"}, + "compose": "networks..aliases" + }, + "deploy.replicas": { + "status": "new", + "oneOf": [ + {"type": "integer", "minimum": 0}, + {"type": "string", "description": "${VAR} reference, e.g. ${OLLAMA_CPU_REPLICAS} or ${OLLAMA_CPU_REPLICAS:-1}"} + ] + }, + "deploy.resources.reservations.devices": { + "status": "new", + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "properties": { + "driver": {"type": "string"}, + "count": {"oneOf": [{"type": "integer", "minimum": 1}, {"const": "all"}, {"type": "string", "description": "${VAR} reference"}]}, + "device_ids": {"type": "array", "items": {"type": "string"}}, + "capabilities": {"type": "array", "items": {"type": "string"}, "minItems": 1} + }, + "required": ["capabilities"] + } + }, + "depends_on": {"status": "existing", "type": "array", "items": {"type": "string"}}, + "healthcheck": {"status": "existing", "type": "object"}, + "cap_add": {"status": "existing", "type": "array", "items": {"type": "string"}}, + "cap_drop": {"status": "existing", "type": "array", "items": {"type": "string"}}, + "devices": {"status": "existing", "type": "array", "items": {"type": "string"}} + }, + "cases": [ + { + "name": "one-shot job that must not restart", + "stacker_yml": { + "name": "model-pull", + "image": "ollama/ollama:0.40.4", + "restart": "no", + "entrypoint": "/bin/sh", + "command": "-c 'ollama pull nomic-embed-text'", + "depends_on": ["ollama-cpu"] + }, + "compose": { + "restart": "no", + "entrypoint": "/bin/sh", + "depends_on": ["ollama-cpu"] + } + }, + { + "name": "CPU variant: runs by default, answers as ollama", + "stacker_yml": { + "name": "ollama-cpu", + "image": "ollama/ollama:0.40.4", + "aliases": ["ollama"], + "deploy": {"replicas": 1} + }, + "compose": { + "networks_aliases": ["ollama"], + "deploy": {"replicas": 1} + } + }, + { + "name": "NVIDIA variant behind a profile", + "stacker_yml": { + "name": "ollama-gpu", + "image": "ollama/ollama:0.40.4", + "profiles": ["gpu-nvidia"], + "aliases": ["ollama"], + "deploy": { + "resources": { + "reservations": { + "devices": [{"driver": "nvidia", "count": 1, "capabilities": ["gpu"]}] + } + } + } + }, + "compose": { + "profiles": ["gpu-nvidia"], + "networks_aliases": ["ollama"], + "deploy": { + "resources": { + "reservations": { + "devices": [{"driver": "nvidia", "count": 1, "capabilities": ["gpu"]}] + } + } + } + } + }, + { + "name": "AMD variant: devices, capabilities and healthcheck carried", + "stacker_yml": { + "name": "ollama-gpu-amd", + "image": "ollama/ollama:0.40.4-rocm", + "profiles": ["gpu-amd"], + "devices": ["/dev/kfd", "/dev/dri"], + "cap_add": ["SYS_PTRACE"], + "cap_drop": ["NET_RAW"], + "healthcheck": {"test": "CMD ollama list", "interval": "30s", "timeout": "5s", "retries": 3} + }, + "compose": { + "profiles": ["gpu-amd"], + "devices": ["/dev/kfd", "/dev/dri"], + "cap_add": ["SYS_PTRACE"], + "cap_drop": ["NET_RAW"], + "healthcheck": {"interval": "30s", "timeout": "5s", "retries": 3} + } + } + ] +} diff --git a/tests/deployment_member_access.rs b/tests/deployment_member_access.rs new file mode 100644 index 00000000..97453a1d --- /dev/null +++ b/tests/deployment_member_access.rs @@ -0,0 +1,107 @@ +//! The User Service asks Stacker which role a user holds on a deployment, so +//! a project member can operate the server (agent, apps, redeploy) through the +//! User Service installation endpoints, which only know the owner. +//! +//! The route is service-to-service: it answers only with STACKER_SERVICE_TOKEN +//! in `x-stacker-service-token`. + +mod common; + +use serde_json::Value; + +const SERVICE_TOKEN: &str = "test-stacker-service-token"; + +async fn role_of(address: &str, hash: &str, user_id: &str, token: Option<&str>) -> (u16, Value) { + let mut req = reqwest::Client::new().get(format!( + "{address}/api/v1/deployments/{hash}/member-access?user_id={user_id}" + )); + if let Some(token) = token { + req = req.header("x-stacker-service-token", token); + } + let resp = req.send().await.expect("request failed"); + let status = resp.status().as_u16(); + let text = resp.text().await.unwrap_or_default(); + ( + status, + serde_json::from_str(&text).unwrap_or(Value::String(text)), + ) +} + +async fn add_member(pool: &sqlx::PgPool, project_id: i32, user_id: &str, role: &str) { + sqlx::query( + "INSERT INTO project_member (project_id, user_id, role, created_by) VALUES ($1, $2, $3, $4)", + ) + .bind(project_id) + .bind(user_id) + .bind(role) + .bind(common::USER_A_ID) + .execute(pool) + .await + .unwrap(); +} + +#[tokio::test] +async fn the_user_service_learns_each_users_role_on_a_deployment() { + std::env::set_var("STACKER_SERVICE_TOKEN", SERVICE_TOKEN); + let Some(app) = common::spawn_app_two_users().await else { + return; + }; + let pool = &app.db_pool; + let project = common::create_test_project(pool, common::USER_A_ID).await; + let hash = format!("member-access-{}", uuid::Uuid::new_v4().simple()); + common::create_test_deployment(pool, common::USER_A_ID, project, &hash).await; + add_member(pool, project, "member_user", "member").await; + add_member(pool, project, "viewer_user", "viewer").await; + + for (user, expected) in [ + (common::USER_A_ID, Value::from("owner")), + ("member_user", Value::from("member")), + ("viewer_user", Value::from("viewer")), + ("someone_else", Value::Null), + ] { + let (status, body) = role_of(&app.address, &hash, user, Some(SERVICE_TOKEN)).await; + assert_eq!(status, 200, "{user}: {body}"); + assert_eq!(body["item"]["role"], expected, "{user}: {body}"); + } + + // An unknown deployment gives no role rather than an error. + let (status, body) = role_of( + &app.address, + "no-such-hash", + "member_user", + Some(SERVICE_TOKEN), + ) + .await; + assert_eq!(status, 200, "{body}"); + assert_eq!(body["item"]["role"], Value::Null); +} + +#[tokio::test] +async fn only_the_user_service_may_ask() { + std::env::set_var("STACKER_SERVICE_TOKEN", SERVICE_TOKEN); + let Some(app) = common::spawn_app_two_users().await else { + return; + }; + let pool = &app.db_pool; + let project = common::create_test_project(pool, common::USER_A_ID).await; + let hash = format!("member-access-{}", uuid::Uuid::new_v4().simple()); + common::create_test_deployment(pool, common::USER_A_ID, project, &hash).await; + + let (status, _) = role_of(&app.address, &hash, common::USER_A_ID, None).await; + assert_eq!(status, 403, "no service token"); + let (status, _) = role_of(&app.address, &hash, common::USER_A_ID, Some("wrong")).await; + assert_eq!(status, 403, "wrong service token"); + + // A signed-in user cannot ask about anyone either. + let resp = reqwest::Client::new() + .get(format!( + "{}/api/v1/deployments/{hash}/member-access?user_id={}", + app.address, + common::USER_A_ID + )) + .header("Authorization", format!("Bearer {}", common::USER_A_TOKEN)) + .send() + .await + .unwrap(); + assert_eq!(resp.status().as_u16(), 403, "user session"); +} diff --git a/tests/image_binaries_contract.rs b/tests/image_binaries_contract.rs new file mode 100644 index 00000000..0b90f4d4 --- /dev/null +++ b/tests/image_binaries_contract.rs @@ -0,0 +1,87 @@ +//! Every binary the stacker image ships is built and copied on both paths into +//! the image: the CI job (`.github/workflows/docker.yml` compiles, archives, +//! unpacks, and the Dockerfile copies it in as `prebuilt`) and a local +//! `docker build` (the Dockerfile's own `builder` stage). +//! +//! `bake` turns a deployed build box into a Quick Deploy snapshot. It was left +//! out of the image, so every bake meant compiling it by hand and `docker cp` +//! into the running production container. + +use std::fs; +use std::path::PathBuf; + +/// The binaries in /app of the runtime image. +const IMAGE_BINARIES: &[&str] = &[ + "server", + "console", + "cleanup-notify", + "backfill_field_policy", + "bake", +]; + +fn read(file: &str) -> String { + let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(file); + fs::read_to_string(&path).unwrap_or_else(|err| panic!("read {file}: {err}")) +} + +/// Lines that are not comments, so commented-out history does not count. +fn active(text: &str) -> Vec { + text.lines() + .filter(|line| !line.trim_start().starts_with('#')) + .map(str::to_string) + .collect() +} + +#[test] +fn the_dockerfile_builds_and_ships_every_image_binary() { + let dockerfile = active(&read("Dockerfile")); + for bin in IMAGE_BINARIES { + assert!( + dockerfile + .iter() + .any(|l| l.contains("cargo build --release") && l.contains(&format!("--bin {bin}"))), + "builder stage must compile `{bin}`" + ); + assert!( + dockerfile.iter().any( + |l| l.trim() == format!("COPY --from=builder /app/target/release/{bin} /{bin}") + ), + "builder-source must take `{bin}` from the builder stage" + ); + assert!( + dockerfile + .iter() + .any(|l| l.trim() == format!("COPY --from=binaries /{bin} .")), + "the runtime image must contain `{bin}`" + ); + } +} + +#[test] +fn ci_builds_archives_and_unpacks_every_image_binary() { + let workflow = active(&read(".github/workflows/docker.yml")).join("\n"); + for bin in IMAGE_BINARIES { + assert!( + workflow.contains(&format!("--bin {bin}")), + "CI must compile `{bin}`" + ); + assert!( + workflow.contains(&format!("cp target/release/{bin} app/stacker/{bin}")), + "CI must put `{bin}` in the artifact" + ); + assert!( + workflow.contains(&format!("stacker/{bin}")), + "the docker job must unpack `{bin}` from the artifact" + ); + } + let chmod = workflow + .lines() + .find(|l| l.trim_start().starts_with("chmod +x")) + .expect("the docker job makes the binaries executable"); + for bin in IMAGE_BINARIES { + assert!( + chmod.split_whitespace().any(|word| word == *bin), + "`{bin}` must be made executable: {chmod}" + ); + } +} diff --git a/tests/managed_usage_db.rs b/tests/managed_usage_db.rs new file mode 100644 index 00000000..cff95a6f --- /dev/null +++ b/tests/managed_usage_db.rs @@ -0,0 +1,262 @@ +//! Persistence of the Managed server usage behind a Quick Deploy. +//! Scenarios: config/shared-fixtures/features/quick-deploy-managed-server-billing.feature + +mod common; + +use chrono::{Datelike, Duration, NaiveDate, Utc}; +use stacker::db::managed_usage::{self, NewManagedUsage}; + +fn new_usage(hash: &str, user_id: &str) -> NewManagedUsage { + NewManagedUsage { + deployment_hash: hash.to_string(), + user_id: user_id.to_string(), + stack: "local-ai-packaged".to_string(), + server_type: "cpx32".to_string(), + daily_rate_minor: 86, + monthly_cap_minor: 2600, + } +} + +fn today() -> NaiveDate { + Utc::now().date_naive() +} + +#[tokio::test] +async fn a_deploy_records_pending_usage_and_success_makes_it_active() { + let Some(app) = common::spawn_app().await else { + return; + }; + let row = managed_usage::insert_pending(&app.db_pool, &new_usage("dep-start", "u-1")) + .await + .expect("insert"); + assert_eq!(row.status, "pending"); + assert!(row.started_at.is_none()); + + let started = managed_usage::start(&app.db_pool, "dep-start") + .await + .expect("start") + .expect("pending row started"); + assert_eq!(started.status, "active"); + assert!(started.started_at.is_some()); + + // Starting twice changes nothing. + assert!(managed_usage::start(&app.db_pool, "dep-start") + .await + .expect("start again") + .is_none()); +} + +#[tokio::test] +async fn a_failed_deploy_closes_the_usage_without_starting_it() { + let Some(app) = common::spawn_app().await else { + return; + }; + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-fail", "u-1")) + .await + .expect("insert"); + let closed = managed_usage::close(&app.db_pool, "dep-fail", "deploy_failed") + .await + .expect("close") + .expect("pending row closed"); + assert_eq!(closed.status, "closed"); + assert_eq!(closed.outcome.as_deref(), Some("deploy_failed")); + assert!(closed.started_at.is_none()); + assert!(managed_usage::list_due(&app.db_pool, today(), 100) + .await + .expect("due") + .iter() + .all(|r| r.deployment_hash != "dep-fail")); +} + +#[tokio::test] +async fn running_servers_are_due_until_their_day_is_recorded() { + let Some(app) = common::spawn_app().await else { + return; + }; + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-due", "u-1")) + .await + .expect("insert"); + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-pending", "u-1")) + .await + .expect("insert"); + let row = managed_usage::start(&app.db_pool, "dep-due") + .await + .expect("start") + .unwrap(); + + let due = managed_usage::list_due(&app.db_pool, today(), 100) + .await + .expect("due"); + assert!(due.iter().any(|r| r.deployment_hash == "dep-due")); + assert!( + due.iter().all(|r| r.deployment_hash != "dep-pending"), + "a pending deploy is not billed" + ); + + let period_start = today().with_day0(0).unwrap(); + assert!( + managed_usage::record_day(&app.db_pool, row.id, period_start, 86, today()) + .await + .expect("record") + ); + // The same day again is refused, so a retry cannot count it twice. + assert!( + !managed_usage::record_day(&app.db_pool, row.id, period_start, 172, today()) + .await + .expect("record again") + ); + + let after = managed_usage::fetch_by_hash(&app.db_pool, "dep-due") + .await + .expect("fetch") + .unwrap(); + assert_eq!(after.period_reported_minor, 86); + assert_eq!(after.last_reported_on, Some(today())); + assert!(managed_usage::list_due(&app.db_pool, today(), 100) + .await + .expect("due") + .iter() + .all(|r| r.deployment_hash != "dep-due")); +} + +#[tokio::test] +async fn a_deleted_server_stays_due_only_up_to_its_deletion_day() { + let Some(app) = common::spawn_app().await else { + return; + }; + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-stop", "u-1")) + .await + .expect("insert"); + managed_usage::start(&app.db_pool, "dep-stop") + .await + .expect("start"); + let stopped = managed_usage::stop(&app.db_pool, "dep-stop") + .await + .expect("stop") + .expect("active row stopped"); + assert_eq!(stopped.status, "stopped"); + assert!(stopped.stopped_at.is_some()); + + // Not yet reported for the deletion day: still due, until that day. + let due = managed_usage::list_due(&app.db_pool, today() + Duration::days(5), 100) + .await + .expect("due"); + let row = due + .iter() + .find(|r| r.deployment_hash == "dep-stop") + .expect("due for its last day"); + assert_eq!(row.report_until(today() + Duration::days(5)), today()); + + let period_start = today().with_day0(0).unwrap(); + managed_usage::record_day(&app.db_pool, row.id, period_start, 86, today()) + .await + .expect("record"); + assert!( + managed_usage::list_due(&app.db_pool, today() + Duration::days(5), 100) + .await + .expect("due") + .iter() + .all(|r| r.deployment_hash != "dep-stop") + ); +} + +#[tokio::test] +async fn an_unpaid_invoice_suspends_every_active_server_of_the_buyer() { + let Some(app) = common::spawn_app().await else { + return; + }; + for hash in ["dep-s1", "dep-s2"] { + managed_usage::insert_pending(&app.db_pool, &new_usage(hash, "u-owes")) + .await + .expect("insert"); + managed_usage::start(&app.db_pool, hash) + .await + .expect("start"); + } + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-other", "u-pays")) + .await + .expect("insert"); + managed_usage::start(&app.db_pool, "dep-other") + .await + .expect("start"); + + let mut suspended: Vec = managed_usage::suspend_for_user(&app.db_pool, "u-owes") + .await + .expect("suspend") + .into_iter() + .map(|r| r.deployment_hash) + .collect(); + suspended.sort(); + assert_eq!(suspended, vec!["dep-s1", "dep-s2"]); + + let other = managed_usage::fetch_by_hash(&app.db_pool, "dep-other") + .await + .expect("fetch") + .unwrap(); + assert_eq!(other.status, "active"); +} + +#[tokio::test] +async fn servers_suspended_for_3_days_are_listed_for_deletion() { + let Some(app) = common::spawn_app().await else { + return; + }; + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-old", "u-late")) + .await + .expect("insert"); + managed_usage::start(&app.db_pool, "dep-old") + .await + .expect("start"); + managed_usage::suspend_for_user(&app.db_pool, "u-late") + .await + .expect("suspend"); + + let now = Utc::now(); + assert!( + managed_usage::list_suspended_before(&app.db_pool, now - Duration::days(3)) + .await + .expect("list") + .iter() + .all(|r| r.deployment_hash != "dep-old") + ); + assert!( + managed_usage::list_suspended_before(&app.db_pool, now + Duration::seconds(1)) + .await + .expect("list") + .iter() + .any(|r| r.deployment_hash == "dep-old") + ); + + let closed = managed_usage::mark_delete_due(&app.db_pool, "dep-old") + .await + .expect("close") + .expect("suspended row closed"); + assert_eq!(closed.status, "closed"); + assert_eq!(closed.outcome.as_deref(), Some("server_delete_due")); + // Handed to admins once: not listed again. + assert!( + managed_usage::list_suspended_before(&app.db_pool, now + Duration::seconds(1)) + .await + .expect("list") + .iter() + .all(|r| r.deployment_hash != "dep-old") + ); +} + +#[tokio::test] +async fn the_hetzner_server_id_is_kept_for_the_admins() { + let Some(app) = common::spawn_app().await else { + return; + }; + managed_usage::insert_pending(&app.db_pool, &new_usage("dep-vm", "u-1")) + .await + .expect("insert"); + managed_usage::set_server_id(&app.db_pool, "dep-vm", 444002406) + .await + .expect("set server id"); + let row = managed_usage::fetch_by_hash(&app.db_pool, "dep-vm") + .await + .unwrap() + .unwrap(); + assert_eq!(row.server_id, Some(444002406)); +} diff --git a/tests/managed_usage_internal.rs b/tests/managed_usage_internal.rs new file mode 100644 index 00000000..a1ffe2cd --- /dev/null +++ b/tests/managed_usage_internal.rs @@ -0,0 +1,206 @@ +//! Internal endpoints the User Service calls for the Managed server of a +//! Quick Deploy. Contract: shared-fixtures/api-contracts/managed-server-billing.json + +mod common; + +use serde_json::{json, Value}; +use stacker::db::managed_usage::{self, NewManagedUsage}; + +const KEY: &str = "test-internal-key"; + +fn set_key() { + std::env::set_var("INTERNAL_SERVICES_ACCESS_KEY", KEY); +} + +async fn pending(app: &common::TestApp, hash: &str, user_id: &str) { + managed_usage::insert_pending( + &app.db_pool, + &NewManagedUsage { + deployment_hash: hash.to_string(), + user_id: user_id.to_string(), + stack: "local-ai-packaged".to_string(), + server_type: "cpx32".to_string(), + daily_rate_minor: 86, + monthly_cap_minor: 2600, + }, + ) + .await + .expect("insert"); +} + +async fn post(app: &common::TestApp, path: &str, key: Option<&str>, body: Value) -> (u16, Value) { + let mut req = reqwest::Client::new() + .post(format!( + "{}/api/internal/managed-usage/{}", + app.address, path + )) + .json(&body); + if let Some(key) = key { + req = req.header("x-internal-key", key); + } + let resp = req.send().await.expect("request"); + let status = resp.status().as_u16(); + let body = resp.json().await.unwrap_or(Value::Null); + (status, body) +} + +#[tokio::test] +async fn internal_calls_without_the_key_are_refused() { + let Some(app) = common::spawn_app().await else { + return; + }; + set_key(); + pending(&app, "dep-nokey", "u-1").await; + for path in [ + "dep-nokey/start", + "dep-nokey/fail", + "dep-nokey/stop", + "suspend", + ] { + let (status, _) = post(&app, path, None, json!({"user_id": "u-1"})).await; + assert_eq!(status, 401, "{path} without a key"); + let (status, _) = post(&app, path, Some("wrong"), json!({"user_id": "u-1"})).await; + assert_eq!(status, 401, "{path} with a wrong key"); + } + let row = managed_usage::fetch_by_hash(&app.db_pool, "dep-nokey") + .await + .unwrap() + .unwrap(); + assert_eq!(row.status, "pending"); +} + +#[tokio::test] +async fn start_makes_a_pending_deploy_billed() { + let Some(app) = common::spawn_app().await else { + return; + }; + set_key(); + pending(&app, "dep-go", "u-1").await; + + let (status, body) = post(&app, "dep-go/start", Some(KEY), json!({})).await; + assert_eq!(status, 200, "{body}"); + assert_eq!(body["status"], "active"); + + let (status, _) = post(&app, "dep-go/start", Some(KEY), json!({})).await; + assert_eq!(status, 409, "starting twice"); + let (status, _) = post(&app, "dep-unknown/start", Some(KEY), json!({})).await; + assert_eq!(status, 404); +} + +#[tokio::test] +async fn fail_closes_a_pending_deploy() { + let Some(app) = common::spawn_app().await else { + return; + }; + set_key(); + pending(&app, "dep-broke", "u-1").await; + + let (status, body) = post( + &app, + "dep-broke/fail", + Some(KEY), + json!({"error": "post-deploy setup failed"}), + ) + .await; + assert_eq!(status, 200, "{body}"); + assert_eq!(body["status"], "closed"); + assert_eq!(body["outcome"], "deploy_failed"); +} + +#[tokio::test] +async fn suspend_stops_billing_every_active_server_of_the_buyer() { + let Some(app) = common::spawn_app().await else { + return; + }; + set_key(); + for hash in ["dep-a", "dep-b"] { + pending(&app, hash, "u-owes").await; + managed_usage::start(&app.db_pool, hash).await.unwrap(); + } + pending(&app, "dep-c", "u-owes").await; // still deploying: not suspended + pending(&app, "dep-d", "u-pays").await; + managed_usage::start(&app.db_pool, "dep-d").await.unwrap(); + + let (status, body) = post(&app, "suspend", Some(KEY), json!({"user_id": "u-owes"})).await; + assert_eq!(status, 200, "{body}"); + let mut suspended: Vec = body["suspended"] + .as_array() + .unwrap() + .iter() + .map(|v| v.as_str().unwrap().to_string()) + .collect(); + suspended.sort(); + assert_eq!(suspended, vec!["dep-a", "dep-b"]); + + let other = managed_usage::fetch_by_hash(&app.db_pool, "dep-d") + .await + .unwrap() + .unwrap(); + assert_eq!(other.status, "active"); + + let (status, _) = post(&app, "suspend", Some(KEY), json!({})).await; + assert_eq!(status, 400, "user_id is required"); +} + +#[tokio::test] +async fn stop_ends_the_usage_of_a_deleted_deployment_and_returns_the_vm() { + let Some(app) = common::spawn_app().await else { + return; + }; + set_key(); + pending(&app, "dep-del", "u-1").await; + managed_usage::start(&app.db_pool, "dep-del").await.unwrap(); + managed_usage::set_server_id(&app.db_pool, "dep-del", 777) + .await + .unwrap(); + + let (status, body) = post(&app, "dep-del/stop", Some(KEY), json!({})).await; + assert_eq!(status, 200, "{body}"); + assert_eq!(body["status"], "stopped"); + assert_eq!(body["server_id"], 777); + + let (status, _) = post(&app, "dep-del/stop", Some(KEY), json!({})).await; + assert_eq!(status, 409, "stopping twice"); + let (status, _) = post(&app, "dep-none/stop", Some(KEY), json!({})).await; + assert_eq!(status, 404); +} + +/// Scenario: an unpaid invoice suspends the server: containers stopped, data kept. +#[tokio::test] +async fn suspend_queues_a_stop_for_every_container_of_the_deployment() { + let Some(app) = common::spawn_app().await else { + return; + }; + set_key(); + let pool = &app.db_pool; + let project_id = common::create_test_project(pool, "u-stop").await; + for code in ["n8n", "postgres"] { + sqlx::query( + "INSERT INTO project_app (project_id, code, name, image, created_at, updated_at) + VALUES ($1, $2, $2, 'nginx:alpine', NOW(), NOW())", + ) + .bind(project_id) + .bind(code) + .execute(pool) + .await + .expect("project_app"); + } + common::create_test_deployment(pool, "u-stop", project_id, "dep-stopme").await; + pending(&app, "dep-stopme", "u-stop").await; + managed_usage::start(pool, "dep-stopme").await.unwrap(); + + let (status, body) = post(&app, "suspend", Some(KEY), json!({"user_id": "u-stop"})).await; + assert_eq!(status, 200, "{body}"); + + let rows: Vec<(String, Value)> = sqlx::query_as( + "SELECT type, parameters FROM commands WHERE deployment_hash = $1 ORDER BY parameters->'params'->>'app_code'", + ) + .bind("dep-stopme") + .fetch_all(pool) + .await + .expect("commands"); + assert_eq!(rows.len(), 2, "{rows:?}"); + assert!(rows.iter().all(|(t, _)| t == "stop")); + assert_eq!(rows[0].1["params"]["app_code"], "n8n"); + assert_eq!(rows[1].1["params"]["app_code"], "postgres"); +} diff --git a/tests/managed_usage_sweeper.rs b/tests/managed_usage_sweeper.rs new file mode 100644 index 00000000..4fcb0937 --- /dev/null +++ b/tests/managed_usage_sweeper.rs @@ -0,0 +1,247 @@ +//! The daily report of the Managed server usage behind a Quick Deploy. +//! Scenarios: config/shared-fixtures/features/quick-deploy-managed-server-billing.feature + +mod common; + +use std::sync::Mutex; + +use async_trait::async_trait; +use chrono::{DateTime, Duration, NaiveDate, TimeZone, Utc}; +use sqlx::PgPool; +use stacker::connectors::user_service::{ManagedBillingEvent, ManagedUsageReport}; +use stacker::db::managed_usage::{self, NewManagedUsage}; +use stacker::services::managed_usage_sweeper::{tick_once, UsageSink}; + +#[derive(Default)] +struct Recorder { + reports: Mutex>, + events: Mutex>, + fail_reports: bool, +} + +#[async_trait] +impl UsageSink for Recorder { + async fn report(&self, usage: &ManagedUsageReport) -> Result<(), String> { + if self.fail_reports { + return Err("payment unreachable".to_string()); + } + self.reports.lock().unwrap().push(usage.clone()); + Ok(()) + } + + async fn event(&self, event: &ManagedBillingEvent) -> Result<(), String> { + self.events.lock().unwrap().push(event.clone()); + Ok(()) + } +} + +impl Recorder { + fn reports_for(&self, hash: &str) -> Vec<(String, i64)> { + self.reports + .lock() + .unwrap() + .iter() + .filter(|r| r.deployment_hash == hash) + .map(|r| (r.identifier.clone(), r.value_minor)) + .collect() + } +} + +fn day(y: i32, m: u32, d: u32) -> NaiveDate { + NaiveDate::from_ymd_opt(y, m, d).unwrap() +} + +fn noon(d: NaiveDate) -> DateTime { + Utc.from_utc_datetime(&d.and_hms_opt(12, 0, 0).unwrap()) +} + +/// A running deployment whose usage started at noon on `started`. +async fn running(pool: &PgPool, hash: &str, rate: i64, cap: i64, started: NaiveDate) { + managed_usage::insert_pending( + pool, + &NewManagedUsage { + deployment_hash: hash.to_string(), + user_id: "u-1".to_string(), + stack: "local-ai-packaged".to_string(), + server_type: "cpx32".to_string(), + daily_rate_minor: rate, + monthly_cap_minor: cap, + }, + ) + .await + .unwrap(); + managed_usage::start(pool, hash).await.unwrap(); + sqlx::query("UPDATE managed_usage SET started_at = $2 WHERE deployment_hash = $1") + .bind(hash) + .bind(noon(started)) + .execute(pool) + .await + .unwrap(); +} + +#[tokio::test] +async fn each_running_day_is_sent_once() { + let Some(app) = common::spawn_app().await else { + return; + }; + let today = day(2026, 10, 16); + running(&app.db_pool, "sw-once", 86, 2600, today).await; + let sink = Recorder::default(); + + tick_once(&app.db_pool, &sink, today, noon(today)) + .await + .unwrap(); + tick_once(&app.db_pool, &sink, today, noon(today)) + .await + .unwrap(); + + assert_eq!( + sink.reports_for("sw-once"), + vec![("sw-once:2026-10-16".to_string(), 86)] + ); +} + +#[tokio::test] +async fn missed_days_are_caught_up_in_order() { + let Some(app) = common::spawn_app().await else { + return; + }; + let today = day(2026, 10, 16); + running(&app.db_pool, "sw-catch", 86, 2600, day(2026, 10, 13)).await; + let sink = Recorder::default(); + + tick_once(&app.db_pool, &sink, today, noon(today)) + .await + .unwrap(); + + assert_eq!( + sink.reports_for("sw-catch"), + vec![ + ("sw-catch:2026-10-13".to_string(), 86), + ("sw-catch:2026-10-14".to_string(), 86), + ("sw-catch:2026-10-15".to_string(), 86), + ("sw-catch:2026-10-16".to_string(), 86), + ] + ); + // Past days are stamped within their own day; today is stamped now. + let reports = sink.reports.lock().unwrap(); + let stamp = |id: &str| { + reports + .iter() + .find(|r| r.identifier == id) + .and_then(|r| r.timestamp) + .unwrap() + }; + let d13 = stamp("sw-catch:2026-10-13"); + assert!(d13 > noon(day(2026, 10, 13)).timestamp() && d13 < noon(day(2026, 10, 14)).timestamp()); + assert_eq!(stamp("sw-catch:2026-10-16"), noon(today).timestamp()); +} + +#[tokio::test] +async fn the_month_never_goes_over_the_cap() { + let Some(app) = common::spawn_app().await else { + return; + }; + let today = day(2026, 10, 3); + running(&app.db_pool, "sw-cap", 86, 200, day(2026, 10, 1)).await; + let sink = Recorder::default(); + + tick_once(&app.db_pool, &sink, today, noon(today)) + .await + .unwrap(); + + assert_eq!( + sink.reports_for("sw-cap") + .iter() + .map(|(_, v)| *v) + .collect::>(), + vec![86, 86, 28] + ); +} + +#[tokio::test] +async fn a_failed_send_is_retried_on_the_next_tick() { + let Some(app) = common::spawn_app().await else { + return; + }; + let today = day(2026, 10, 16); + running(&app.db_pool, "sw-retry", 86, 2600, today).await; + + let failing = Recorder { + fail_reports: true, + ..Recorder::default() + }; + tick_once(&app.db_pool, &failing, today, noon(today)) + .await + .unwrap(); + let row = managed_usage::fetch_by_hash(&app.db_pool, "sw-retry") + .await + .unwrap() + .unwrap(); + assert_eq!(row.last_reported_on, None, "an unsent day is not marked"); + + let sink = Recorder::default(); + tick_once(&app.db_pool, &sink, today, noon(today)) + .await + .unwrap(); + assert_eq!( + sink.reports_for("sw-retry"), + vec![("sw-retry:2026-10-16".to_string(), 86)] + ); +} + +#[tokio::test] +async fn a_server_suspended_for_3_days_is_handed_to_the_admins() { + let Some(app) = common::spawn_app().await else { + return; + }; + let today = day(2026, 10, 16); + for (hash, suspended_days_ago) in [("sw-late", 4), ("sw-recent", 1)] { + running(&app.db_pool, hash, 86, 2600, day(2026, 10, 1)).await; + managed_usage::set_server_id(&app.db_pool, hash, 900 + suspended_days_ago) + .await + .unwrap(); + sqlx::query( + "UPDATE managed_usage SET status = 'suspended', suspended_at = $2 WHERE deployment_hash = $1", + ) + .bind(hash) + .bind(noon(today) - Duration::days(suspended_days_ago)) + .execute(&app.db_pool) + .await + .unwrap(); + } + let sink = Recorder::default(); + + tick_once(&app.db_pool, &sink, today, noon(today)) + .await + .unwrap(); + + let late = managed_usage::fetch_by_hash(&app.db_pool, "sw-late") + .await + .unwrap() + .unwrap(); + assert_eq!(late.status, "closed"); + assert_eq!(late.outcome.as_deref(), Some("server_delete_due")); + let recent = managed_usage::fetch_by_hash(&app.db_pool, "sw-recent") + .await + .unwrap() + .unwrap(); + assert_eq!(recent.status, "suspended"); + + let events = sink.events.lock().unwrap(); + let due: Vec<_> = events + .iter() + .filter(|e| e.event == "server_delete_due") + .collect(); + assert_eq!(due.len(), 1); + assert_eq!(due[0].deployment_hash, "sw-late"); + assert_eq!(due[0].metadata.as_ref().unwrap()["server_id"], 904); + + // Days are reported only up to the suspension day. + let last = sink + .reports_for("sw-late") + .last() + .map(|(id, _)| id.clone()) + .unwrap(); + assert_eq!(last, "sw-late:2026-10-12"); +} diff --git a/tests/marketplace_admin_template_edit.rs b/tests/marketplace_admin_template_edit.rs new file mode 100644 index 00000000..3e143bab --- /dev/null +++ b/tests/marketplace_admin_template_edit.rs @@ -0,0 +1,214 @@ +mod common; + +use chrono::{Duration, Utc}; +use reqwest::StatusCode; +use serde_json::json; +use sqlx::Row; +use tokio::sync::OnceCell; + +/// `PUT /api/admin/templates/{id}` - an admin correcting a published listing. +/// +/// The author's own `PUT /api/templates/{id}` refuses anything outside +/// draft/rejected/needs_changes, because it models the review workflow. An +/// admin fixing the copy on an approved listing is not in that workflow, so +/// the admin route lifts the status gate while reusing the same write path +/// (`creator::apply_template_update`) for validation and secret stripping. +/// +/// Who may call the route is covered by `security_route_matrix.rs`, which +/// classifies every `/api/admin/*` path as admin-only and checks live that a +/// group_user is refused. + +// A fresh `PgPool` per test, bound to that test's own runtime - see the note in +// tests/marketplace_admin_review.rs. +static APP_CONFIG: OnceCell = OnceCell::const_new(); + +async fn app() -> common::TestApp { + common::get_or_init_app_fresh(&APP_CONFIG) + .await + .expect("Failed to start test app") +} + +fn create_admin_jwt() -> String { + use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine}; + + let header = json!({"alg": "HS256", "typ": "JWT"}); + let payload = json!({ + "role": "admin_service", + "email": "ops@test.com", + "exp": (Utc::now() + Duration::minutes(30)).timestamp(), + }); + + let header_b64 = URL_SAFE_NO_PAD.encode(header.to_string()); + let payload_b64 = URL_SAFE_NO_PAD.encode(payload.to_string()); + let signature = common::sign_test_jwt(&header_b64, &payload_b64); + format!("{}.{}.{}", header_b64, payload_b64, signature) +} + +async fn insert_template(pool: &sqlx::PgPool, slug: &str, status: &str) -> String { + sqlx::query( + r#"INSERT INTO stack_template ( + creator_user_id, creator_name, name, slug, status, + short_description, long_description, tags, tech_stack + ) + VALUES ($1, 'Creator Example', 'Original Name', $2, $3, + 'original short', 'original long', '[]'::jsonb, '{}'::jsonb) + RETURNING id"#, + ) + .bind(common::USER_A_ID) + .bind(slug) + .bind(status) + .fetch_one(pool) + .await + .expect("Failed to insert template") + .get::("id") + .to_string() +} + +struct StoredTemplate { + name: String, + short_description: Option, + long_description: Option, + status: String, +} + +async fn fetch_template(pool: &sqlx::PgPool, id: &str) -> StoredTemplate { + let row = sqlx::query( + r#"SELECT name, short_description, long_description, status + FROM stack_template WHERE id = $1::uuid"#, + ) + .bind(uuid::Uuid::parse_str(id).expect("template id should be a uuid")) + .fetch_one(pool) + .await + .expect("Failed to fetch template"); + + StoredTemplate { + name: row.get("name"), + short_description: row.get("short_description"), + long_description: row.get("long_description"), + status: row.get("status"), + } +} + +/// The long description is authored in the admin UI's Draft editor, so what +/// arrives is raw Draft JSON serialized into a text column. It has to survive +/// the round trip byte for byte: the storefront decides how to render by +/// parsing it, and a mangled payload silently degrades to markdown. +const DRAFT_JSON: &str = r#"{"blocks":[{"key":"a1","text":"Overview","type":"header-two","depth":0,"inlineStyleRanges":[],"entityRanges":[],"data":{}},{"key":"a2","text":"Runs anywhere.","type":"unstyled","depth":0,"inlineStyleRanges":[{"offset":0,"length":4,"style":"BOLD"}],"entityRanges":[],"data":{}}],"entityMap":{}}"#; + +#[tokio::test] +async fn admin_can_edit_an_approved_template() { + let app = app().await; + let template_id = insert_template(&app.db_pool, "admin-edit-approved", "approved").await; + + let response = reqwest::Client::new() + .put(format!( + "{}/api/admin/templates/{}", + app.address, template_id + )) + .header("Authorization", format!("Bearer {}", create_admin_jwt())) + .json(&json!({ + "name": "Corrected Name", + "short_description": "corrected short", + "long_description": DRAFT_JSON, + })) + .send() + .await + .expect("Failed to send admin edit request"); + + assert_eq!(StatusCode::OK, response.status()); + + let stored = fetch_template(&app.db_pool, &template_id).await; + assert_eq!("Corrected Name", stored.name); + assert_eq!(Some("corrected short".to_string()), stored.short_description); + assert_eq!(Some(DRAFT_JSON.to_string()), stored.long_description); + // An edit is not a review decision. + assert_eq!("approved", stored.status); +} + +#[tokio::test] +async fn the_author_cannot_edit_an_approved_template_through_their_own_route() { + // The contrast that justifies the admin route: same fixture, same payload, + // refused on the creator path because the review workflow owns the status. + let app = app().await; + let template_id = insert_template(&app.db_pool, "author-edit-approved", "approved").await; + + let response = reqwest::Client::new() + .put(format!("{}/api/templates/{}", app.address, template_id)) + .header("Authorization", format!("Bearer {}", common::USER_A_TOKEN)) + .json(&json!({"name": "Author Rename"})) + .send() + .await + .expect("Failed to send creator edit request"); + + assert_ne!(StatusCode::OK, response.status()); + + let stored = fetch_template(&app.db_pool, &template_id).await; + assert_eq!("Original Name", stored.name); +} + +#[tokio::test] +async fn admin_edit_leaves_omitted_fields_alone() { + let app = app().await; + let template_id = insert_template(&app.db_pool, "admin-edit-partial", "approved").await; + + let response = reqwest::Client::new() + .put(format!( + "{}/api/admin/templates/{}", + app.address, template_id + )) + .header("Authorization", format!("Bearer {}", create_admin_jwt())) + .json(&json!({"short_description": "only this one changes"})) + .send() + .await + .expect("Failed to send admin edit request"); + + assert_eq!(StatusCode::OK, response.status()); + + let stored = fetch_template(&app.db_pool, &template_id).await; + assert_eq!( + Some("only this one changes".to_string()), + stored.short_description + ); + assert_eq!("Original Name", stored.name); + assert_eq!(Some("original long".to_string()), stored.long_description); +} + +#[tokio::test] +async fn admin_edit_of_a_missing_template_is_not_found() { + let app = app().await; + let missing = uuid::Uuid::new_v4(); + + let response = reqwest::Client::new() + .put(format!("{}/api/admin/templates/{}", app.address, missing)) + .header("Authorization", format!("Bearer {}", create_admin_jwt())) + .json(&json!({"name": "Nobody"})) + .send() + .await + .expect("Failed to send admin edit request"); + + assert_eq!(StatusCode::NOT_FOUND, response.status()); +} + +#[tokio::test] +async fn admin_can_edit_a_draft_template_too() { + // The status gate is lifted, not inverted: the statuses the author may + // edit must stay editable for an admin as well. + let app = app().await; + let template_id = insert_template(&app.db_pool, "admin-edit-draft", "draft").await; + + let response = reqwest::Client::new() + .put(format!( + "{}/api/admin/templates/{}", + app.address, template_id + )) + .header("Authorization", format!("Bearer {}", create_admin_jwt())) + .json(&json!({"name": "Draft Corrected"})) + .send() + .await + .expect("Failed to send admin edit request"); + + assert_eq!(StatusCode::OK, response.status()); + let stored = fetch_template(&app.db_pool, &template_id).await; + assert_eq!("Draft Corrected", stored.name); + assert_eq!("draft", stored.status); +} diff --git a/tests/marketplace_event_wiring.rs b/tests/marketplace_event_wiring.rs new file mode 100644 index 00000000..337cd57d --- /dev/null +++ b/tests/marketplace_event_wiring.rs @@ -0,0 +1,88 @@ +//! Regression: the view/deploy counter endpoints must also write event rows, +//! otherwise `marketplace_event` / `marketplace_template_event` stay empty and +//! creator analytics always reads zero. +mod common; + +use sqlx::Row; +use uuid::Uuid; + +async fn insert_template(pool: &sqlx::PgPool, id: Uuid) { + sqlx::query( + r#"INSERT INTO stack_template (id, creator_user_id, creator_name, name, slug, + short_description, status, tags, tech_stack, view_count, deploy_count, created_at, updated_at) + VALUES ($1, 'vendor-x', 'Vendor X', $2, $3, 'd', 'approved', '[]'::jsonb, '{}'::jsonb, 0, 0, NOW(), NOW())"#, + ) + .bind(id) + .bind(format!("T {id}")) + .bind(format!("t-{id}")) + .execute(pool) + .await + .expect("insert template"); +} + +async fn count(pool: &sqlx::PgPool, table: &str, id: Uuid, kind: &str) -> i64 { + sqlx::query(&format!( + "SELECT COUNT(*)::bigint AS c FROM {table} WHERE template_id = $1 AND event_type = $2" + )) + .bind(id) + .bind(kind) + .fetch_one(pool) + .await + .expect("count") + .get("c") +} + +#[tokio::test] +async fn detail_view_records_events_but_deploy_counter_does_not() { + let Some(app) = common::spawn_app().await else { + eprintln!("Skipping test: database not available"); + return; + }; + let id = Uuid::new_v4(); + insert_template(&app.db_pool, id).await; + + let resp = reqwest::get(format!("{}/api/templates/t-{id}", app.address)) + .await + .expect("request"); + assert!(resp.status().is_success(), "detail: {}", resp.status()); + assert_eq!(count(&app.db_pool, "marketplace_template_event", id, "view").await, 1); + assert_eq!(count(&app.db_pool, "marketplace_event", id, "view").await, 1); + + stacker::db::marketplace::increment_deploy_count(&app.db_pool, &id) + .await + .expect("deploy count"); + // The unauthenticated counter must not forge deploy analytics. + assert_eq!(count(&app.db_pool, "marketplace_template_event", id, "deploy").await, 0); + assert_eq!(count(&app.db_pool, "marketplace_event", id, "deploy").await, 0); +} + +#[tokio::test] +async fn unknown_template_records_no_event() { + let Some(app) = common::spawn_app().await else { + return; + }; + let id = Uuid::new_v4(); + let _ = stacker::db::marketplace::increment_view_count(&app.db_pool, &id).await; + assert_eq!(count(&app.db_pool, "marketplace_template_event", id, "view").await, 0); + assert_eq!(count(&app.db_pool, "marketplace_event", id, "view").await, 0); +} + +#[tokio::test] +async fn insert_helpers_write_creator_analytics_source() { + let Some(app) = common::spawn_app().await else { + return; + }; + let id = Uuid::new_v4(); + insert_template(&app.db_pool, id).await; + stacker::db::marketplace::insert_view_event(&app.db_pool, id, "u1", serde_json::json!({})) + .await + .expect("view"); + stacker::db::marketplace::insert_deploy_event(&app.db_pool, id, "u2", "hetzner", serde_json::json!({})) + .await + .expect("deploy"); + let a = stacker::db::marketplace::get_vendor_analytics(&app.db_pool, "vendor-x", Some("all")) + .await + .expect("analytics"); + let t = a.templates.iter().find(|t| t.template_id == id).expect("template in analytics"); + assert_eq!((t.views, t.deployments), (1, 1)); +} diff --git a/tests/migration_boot_contract.rs b/tests/migration_boot_contract.rs index b5548f37..a9e038c0 100644 --- a/tests/migration_boot_contract.rs +++ b/tests/migration_boot_contract.rs @@ -53,11 +53,7 @@ fn stub_sqlx(bin_dir: &Path, log: &Path) -> PathBuf { } /// Run the entrypoint with the given command, returning (exit code, stub log). -fn run_entrypoint( - workdir: &Path, - env: &[(&str, &str)], - args: &[&str], -) -> (Option, String) { +fn run_entrypoint(workdir: &Path, env: &[(&str, &str)], args: &[&str]) -> (Option, String) { let bin_dir = workdir.join("bin"); let log = workdir.join("sqlx-invocations.log"); stub_sqlx(&bin_dir, &log); @@ -110,7 +106,8 @@ fn write_database_config_with_password(dir: &Path, password: &str) { } fn temp_dir(name: &str) -> PathBuf { - let dir = std::env::temp_dir().join(format!("stacker-entrypoint-{name}-{}", std::process::id())); + let dir = + std::env::temp_dir().join(format!("stacker-entrypoint-{name}-{}", std::process::id())); let _ = fs::remove_dir_all(&dir); fs::create_dir_all(&dir).expect("temp dir"); dir @@ -299,3 +296,33 @@ fn compose_files_do_not_boot_around_the_migration_entrypoint() { } } } + +/// sqlx keys `_sqlx_migrations` by version: two migrations with one version +/// make every fresh database fail with a duplicate key on +/// `_sqlx_migrations_pkey`, and a database that applied one of them refuses the +/// other at boot. 20261016120000 was used twice (baked_snapshot_server_type and +/// casbin_admin_template_update). +#[test] +fn every_migration_version_is_used_once() { + let mut owners: std::collections::BTreeMap> = + Default::default(); + for entry in std::fs::read_dir(manifest_path("migrations")).expect("migrations dir") { + let name = entry.unwrap().file_name().to_string_lossy().to_string(); + let Some(stem) = name + .strip_suffix(".up.sql") + .or_else(|| name.strip_suffix(".down.sql")) + else { + continue; + }; + let (version, description) = stem.split_once('_').unwrap_or((stem, "")); + owners + .entry(version.to_string()) + .or_default() + .insert(description.to_string()); + } + let duplicates: Vec<_> = owners.iter().filter(|(_, d)| d.len() > 1).collect(); + assert!( + duplicates.is_empty(), + "migration versions used more than once: {duplicates:?}" + ); +} diff --git a/tests/project_team_seats.rs b/tests/project_team_seats.rs index 6e36d592..ad275b85 100644 --- a/tests/project_team_seats.rs +++ b/tests/project_team_seats.rs @@ -224,6 +224,7 @@ impl App { "GET" => self.client.get(&url), "POST" => self.client.post(&url), "PUT" => self.client.put(&url), + "PATCH" => self.client.patch(&url), "DELETE" => self.client.delete(&url), other => panic!("unsupported verb {other}"), }; @@ -1281,3 +1282,199 @@ async fn the_owner_sees_who_the_members_are() { assert_eq!(email_of(COLLEAGUE.id), json!("colleague@try.direct")); assert_eq!(email_of("legacy_viewer_id"), Value::Null); } + +// ── The project's servers ── + +#[tokio::test] +async fn a_member_sees_the_projects_servers_to_deploy_to() { + let Some(app) = App::spawn().await else { + return; + }; + let project = app.project(TEAM).await; + let server = common::create_test_server(&app.pool, TEAM.id, project, "ssh", None).await; + app.add_member(project, COLLEAGUE.id, "member").await; + + let (status, body) = app + .call(Some(COLLEAGUE), "GET", &format!("/server/project/{project}"), None) + .await; + assert_eq!(status, 200, "member: {body}"); + let ids: Vec = body["list"] + .as_array() + .unwrap() + .iter() + .filter_map(|s| s["id"].as_i64()) + .collect(); + assert_eq!(ids, vec![server as i64]); +} + +#[tokio::test] +async fn a_viewer_and_an_outsider_do_not_get_the_projects_servers() { + let Some(app) = App::spawn().await else { + return; + }; + let project = app.project(TEAM).await; + common::create_test_server(&app.pool, TEAM.id, project, "ssh", None).await; + app.add_member(project, COLLEAGUE.id, "viewer").await; + + let (status, _) = app + .call(Some(COLLEAGUE), "GET", &format!("/server/project/{project}"), None) + .await; + assert_eq!(status, 403, "viewer"); + let (status, _) = app + .call(Some(PRO), "GET", &format!("/server/project/{project}"), None) + .await; + assert_eq!(status, 404, "outsider"); +} + +// ── The owner's saved clouds ── + +#[tokio::test] +async fn a_member_picks_from_the_owners_saved_clouds_without_seeing_credentials() { + let Some(app) = App::spawn().await else { + return; + }; + let project = app.project(TEAM).await; + let cloud = common::create_test_cloud(&app.pool, TEAM.id, "acme-htz", "htz").await; + common::create_test_cloud(&app.pool, COLLEAGUE.id, "colleague-own", "htz").await; + app.add_member(project, COLLEAGUE.id, "member").await; + + for who in [COLLEAGUE, TEAM] { + let (status, body) = app + .call(Some(who), "GET", &format!("/project/{project}/clouds"), None) + .await; + assert_eq!(status, 200, "{}: {body}", who.email); + let list = body["list"].as_array().unwrap(); + assert_eq!(list.len(), 1, "only the owner's clouds: {body}"); + assert_eq!(list[0]["id"], cloud); + assert_eq!(list[0]["name"], "acme-htz"); + assert_eq!(list[0]["provider"], "htz"); + let text = body.to_string(); + for field in ["cloud_token", "cloud_key", "cloud_secret", "test-cloud-token"] { + assert!(!text.contains(field), "{field} leaked: {text}"); + } + } +} + +#[tokio::test] +async fn a_viewer_and_an_outsider_do_not_get_the_owners_clouds() { + let Some(app) = App::spawn().await else { + return; + }; + let project = app.project(TEAM).await; + common::create_test_cloud(&app.pool, TEAM.id, "acme-htz", "htz").await; + app.add_member(project, COLLEAGUE.id, "viewer").await; + + let (status, _) = app + .call(Some(COLLEAGUE), "GET", &format!("/project/{project}/clouds"), None) + .await; + assert_eq!(status, 403, "viewer"); + let (status, _) = app + .call(Some(PRO), "GET", &format!("/project/{project}/clouds"), None) + .await; + assert_eq!(status, 404, "outsider"); +} + +// ── Server access for members ── +// +// The deployment report carries the server's SSH key and passwords. A member +// who deploys gets it only when the owner turned on server access for members. + +async fn server_access(app: &App, project: i32) -> bool { + let (status, body) = app + .call( + Some(TEAM), + "GET", + &format!("/project/{project}/seats"), + None, + ) + .await; + assert_eq!(status, 200, "seats: {body}"); + body["item"]["members_server_access"].as_bool().unwrap() +} + +#[tokio::test] +async fn the_owner_decides_whether_members_get_server_access() { + let Some(app) = App::spawn().await else { + return; + }; + let project = app.project(TEAM).await; + app.add_member(project, COLLEAGUE.id, "member").await; + assert!(!server_access(&app, project).await, "off by default"); + + let route = format!("/project/{project}/server-access"); + let (status, body) = app + .call( + Some(TEAM), + "PATCH", + &route, + Some(json!({ "enabled": true })), + ) + .await; + assert_eq!(status, 200, "{body}"); + assert!(server_access(&app, project).await); + + let (status, _) = app + .call( + Some(COLLEAGUE), + "PATCH", + &route, + Some(json!({ "enabled": false })), + ) + .await; + assert_eq!(status, 403, "a member cannot change it"); + assert!(server_access(&app, project).await); + + let (status, _) = app + .call( + Some(TEAM), + "PATCH", + &route, + Some(json!({ "enabled": false })), + ) + .await; + assert_eq!(status, 200); + assert!(!server_access(&app, project).await); +} + +#[tokio::test] +async fn a_member_gets_the_deployment_report_only_with_server_access() { + use stacker::routes::access::{report_initiator, Access}; + + let Some(app) = App::spawn().await else { + return; + }; + let project_id = app.project(TEAM).await; + let project = stacker::db::project::fetch(&app.pool, project_id) + .await + .unwrap() + .unwrap(); + let member = stacker::models::User { + id: COLLEAGUE.id.to_string(), + first_name: String::new(), + last_name: String::new(), + email: COLLEAGUE.email.to_string(), + role: "group_user".to_string(), + email_confirmed: true, + mfa_verified: false, + access_token: None, + }; + + assert_eq!( + report_initiator(&app.pool, &project, &member, Access::Member).await, + None + ); + + stacker::db::project::set_members_server_access(&app.pool, project_id, true) + .await + .unwrap(); + assert_eq!( + report_initiator(&app.pool, &project, &member, Access::Member) + .await + .as_deref(), + Some(COLLEAGUE.email) + ); + assert_eq!( + report_initiator(&app.pool, &project, &member, Access::Owner).await, + None + ); +} diff --git a/tests/quick_deploy_server_type.rs b/tests/quick_deploy_server_type.rs new file mode 100644 index 00000000..34e9aee0 --- /dev/null +++ b/tests/quick_deploy_server_type.rs @@ -0,0 +1,126 @@ +//! Quick Deploy provisions the server type a stack was baked on, at that +//! type's price. The contract is mirrored from +//! ../config/shared-fixtures/api-contracts/quick-deploy-server-type.json to +//! tests/contracts/quick-deploy-server-type.contract.json. + +mod common; + +use serde_json::Value; + +fn contract() -> Value { + serde_json::from_str(include_str!( + "contracts/quick-deploy-server-type.contract.json" + )) + .expect("contract JSON") +} + +async fn validate(app: &common::TestApp, stack: &str) -> Value { + let yml = format!("name: {stack}\napp:\n type: custom\n image: nginx:1.27\n"); + let resp = reqwest::Client::new() + .post(format!("{}/api/v1/deploy/validate", app.address)) + .header("Content-Type", "text/plain") + .body(yml) + .send() + .await + .expect("validate request"); + assert_eq!(resp.status().as_u16(), 200); + resp.json().await.expect("validate JSON") +} + +#[tokio::test] +async fn validate_names_the_snapshot_server_type_and_lists_the_rates() { + let Some(app) = common::spawn_app().await else { + return; + }; + let contract = contract(); + stacker::db::baked_snapshot::record( + &app.db_pool, + "qd-baked-on-cpx42", + "1.0.0", + "hetzner", + 444002406, + true, + None, + None, + None, + Some("cpx42"), + ) + .await + .expect("record snapshot"); + + let body = validate(&app, "qd-baked-on-cpx42").await; + let quick = &body["quick_deploy"]; + assert_eq!(quick["server_type"], "cpx42", "{body}"); + for (server_type, rate) in contract["rates"].as_object().unwrap() { + assert_eq!( + quick["rates"][server_type]["daily_rate"].as_f64(), + rate["daily_rate"].as_f64(), + "{server_type}: {quick}" + ); + assert_eq!( + quick["rates"][server_type]["monthly_cap"].as_f64(), + rate["monthly_cap"].as_f64(), + "{server_type}: {quick}" + ); + } + + // A stacker.yml that pins a version gets that version's snapshot - the + // one the clone uses - not the latest. + stacker::db::baked_snapshot::record( + &app.db_pool, + "qd-baked-on-cpx42", + "0.9.0", + "hetzner", + 1, + true, + None, + None, + None, + Some("cpx32"), + ) + .await + .expect("record older snapshot"); + let yml = + "name: qd-baked-on-cpx42\nversion: \"0.9.0\"\napp:\n type: custom\n image: nginx:1.27\n"; + let body: Value = reqwest::Client::new() + .post(format!("{}/api/v1/deploy/validate", app.address)) + .body(yml) + .send() + .await + .expect("validate request") + .json() + .await + .expect("validate JSON"); + assert_eq!(body["quick_deploy"]["server_type"], "cpx32", "{body}"); + + // No snapshot - or one baked before the type was recorded - names none, + // and the caller's default type applies. + let body = validate(&app, "qd-never-baked").await; + assert!(body["quick_deploy"]["server_type"].is_null(), "{body}"); + assert!( + body["quick_deploy"]["rates"].get("cpx32").is_some(), + "{body}" + ); +} + +#[tokio::test] +async fn a_recorded_snapshot_keeps_its_server_type() { + let Some(app) = common::spawn_app().await else { + return; + }; + let row = stacker::db::baked_snapshot::record( + &app.db_pool, + "qd-record", + "1.0.0", + "hetzner", + 1, + true, + None, + None, + None, + Some("cpx42"), + ) + .await + .expect("record snapshot"); + assert_eq!(row.server_type.as_deref(), Some("cpx42")); +} diff --git a/tests/security_project_routes.rs b/tests/security_project_routes.rs index 6b7fb152..e56b01ca 100644 --- a/tests/security_project_routes.rs +++ b/tests/security_project_routes.rs @@ -57,6 +57,8 @@ const ROUTES: &[(&str, &str)] = &[ ("GET", "/{id}/invitations"), ("POST", "/{id}/invitations"), ("GET", "/{id}/seats"), + ("PATCH", "/{id}/server-access"), + ("GET", "/{id}/clouds"), // Destructive ones last, so a missing check cannot hide the others. ("DELETE", "/{id}/apps/{code}"), ("DELETE", "/{id}/invitations/{invitation_id}"), @@ -106,6 +108,7 @@ fn body_for(method: &str, path: &str) -> Option { json!({ "custom": { "custom_stack_code": "intruder" } }) } ("PATCH", "/{id}/protection") => json!({ "is_protected": false }), + ("PATCH", "/{id}/server-access") => json!({ "enabled": true }), _ => return None, }; Some(body) @@ -396,6 +399,8 @@ const NEEDS: &[(&str, &str, Need)] = &[ ("GET", "/{id}/invitations", Need::Owner), ("POST", "/{id}/invitations", Need::Owner), ("GET", "/{id}/seats", Need::Owner), + ("PATCH", "/{id}/server-access", Need::Owner), + ("GET", "/{id}/clouds", Need::Manage), ("DELETE", "/{id}/apps/{code}", Need::Manage), ("DELETE", "/{id}/invitations/{invitation_id}", Need::Owner), ("DELETE", "/{id}", Need::Owner), diff --git a/tests/security_route_matrix.rs b/tests/security_route_matrix.rs index c1931675..b06f2f14 100644 --- a/tests/security_route_matrix.rs +++ b/tests/security_route_matrix.rs @@ -88,6 +88,7 @@ const ROUTES: &[(&str, &str, Kind, &str)] = &[ ("GET", "/rating/{id}", Public, "visible rating"), ("GET", "/agreement/{id}", Public, "agreement text"), ("GET", "/api/agreement/{id}", Public, "agreement text"), + ("GET", "/api/agreement/name/{name}", Public, "agreement text"), ( "GET", "/dockerhub/namespaces", @@ -263,6 +264,12 @@ const ROUTES: &[(&str, &str, Kind, &str)] = &[ Token, "internal service key", ), + ( + "GET", + "/api/v1/deployments/{deployment_hash}/member-access", + Token, + "internal service key; a user's role for the User Service", + ), ( "POST", "/api/v1/marketplace/payouts/webhook", @@ -281,6 +288,31 @@ const ROUTES: &[(&str, &str, Kind, &str)] = &[ Token, "internal service key", ), + // Managed server usage of a Quick Deploy, called by the User Service. + ( + "POST", + "/api/internal/managed-usage/suspend", + Token, + "internal service key", + ), + ( + "POST", + "/api/internal/managed-usage/{deployment_hash}/start", + Token, + "internal service key", + ), + ( + "POST", + "/api/internal/managed-usage/{deployment_hash}/fail", + Token, + "internal service key", + ), + ( + "POST", + "/api/internal/managed-usage/{deployment_hash}/stop", + Token, + "internal service key", + ), ( "POST", "/api/v1/agent/login", diff --git a/tests/stack_service_runtime_fields_contract.rs b/tests/stack_service_runtime_fields_contract.rs new file mode 100644 index 00000000..5be036ee --- /dev/null +++ b/tests/stack_service_runtime_fields_contract.rs @@ -0,0 +1,252 @@ +//! Runtime settings of a `services:` entry in stacker.yml reach the compose +//! file on every path that renders one. +//! +//! The contract is mirrored from +//! ../config/shared-fixtures/api-contracts/stack-service-runtime-fields.json +//! to tests/contracts/stack-service-runtime-fields.contract.json. +//! +//! cli_compose: the compose the CLI generates and deploys. +//! compose_sync: the service written into the author's own compose file +//! (`deploy.compose_file`). +//! project_app: the project app the CLI registers (POST /project), from which +//! the server renders the compose for deploys started outside the CLI. + +use serde_json::{json, Value}; +use stacker::cli::config_parser::StackerConfig; +use stacker::cli::generator::compose::ComposeDefinition; +use stacker::cli::stacker_client::build_project_body; +use stacker::forms::project::ProjectForm; + +fn contract() -> Value { + serde_json::from_str(include_str!( + "contracts/stack-service-runtime-fields.contract.json" + )) + .expect("contract JSON") +} + +fn config_with(service: &Value) -> StackerConfig { + let doc = json!({ + "name": "runtime-fields", + "app": { "type": "custom", "image": "nginx:1.27" }, + "services": [service], + }); + StackerConfig::from_str(&serde_yaml::to_string(&doc).unwrap()) + .unwrap_or_else(|e| panic!("stacker.yml with {service} must parse: {e}")) +} + +/// Every key of `expected` is present in `actual` with the same value; +/// `networks_aliases` means: every network the service joins carries them. +fn assert_contains(path: &str, case: &str, actual: &Value, expected: &Value) { + for (key, want) in expected.as_object().unwrap() { + if key == "networks_aliases" { + let networks = actual["networks"].as_object().unwrap_or_else(|| { + panic!( + "{path}/{case}: networks with aliases expected, got {}", + actual["networks"] + ) + }); + assert!(!networks.is_empty(), "{path}/{case}: no network"); + for (name, net) in networks { + assert_eq!( + &net["aliases"], want, + "{path}/{case}: aliases on network {name}" + ); + } + continue; + } + let got = &actual[key]; + if want.is_object() { + assert_contains(path, case, got, want); + } else { + assert_eq!(got, want, "{path}/{case}: `{key}`\nservice: {actual}"); + } + } +} + +fn cases() -> Vec<(String, Value, Value)> { + contract()["cases"] + .as_array() + .unwrap() + .iter() + .map(|c| { + ( + c["name"].as_str().unwrap().to_string(), + c["stacker_yml"].clone(), + c["compose"].clone(), + ) + }) + .collect() +} + +#[test] +fn cli_compose_carries_every_declared_setting() { + for (case, service, expected) in cases() { + let config = config_with(&service); + let compose = ComposeDefinition::try_from(&config) + .expect("compose") + .render(); + let doc: Value = serde_yaml::from_str(&compose).expect("rendered compose is YAML"); + let name = service["name"].as_str().unwrap(); + assert_contains("cli_compose", &case, &doc["services"][name], &expected); + } +} + +#[test] +fn compose_sync_carries_every_declared_setting() { + for (case, service, expected) in cases() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("compose.yml"), "services: {}\n").unwrap(); + let mut config = config_with(&service); + config.deploy.compose_file = Some("compose.yml".into()); + let name = service["name"].as_str().unwrap().to_string(); + stacker::cli::compose_service_sync::sync_configured_compose_services( + dir.path(), + &config, + &[name.clone()], + ) + .unwrap_or_else(|e| panic!("compose_sync/{case}: {e}")); + let written = std::fs::read_to_string(dir.path().join("compose.yml")).unwrap(); + let doc: Value = serde_yaml::from_str(&written).unwrap(); + assert_contains("compose_sync", &case, &doc["services"][&name], &expected); + } +} + +#[test] +fn project_app_carries_every_declared_setting() { + for (case, service, expected) in cases() { + let body = build_project_body(&config_with(&service)); + let form: ProjectForm = serde_json::from_value(json!({ "custom": body["custom"] })) + .unwrap_or_else(|e| { + panic!("project_app/{case}: the server must accept the registered app: {e}") + }); + let services = form.custom.services().expect("server renders services"); + let name = service["name"].as_str().unwrap(); + let rendered = services + .get(name) + .cloned() + .flatten() + .unwrap_or_else(|| panic!("project_app/{case}: service `{name}` missing")); + let actual = serde_json::to_value( + serde_yaml::from_str::(&serde_yaml::to_string(&rendered).unwrap()).unwrap(), + ) + .unwrap(); + assert_contains("project_app", &case, &actual, &expected); + } +} + +#[test] +fn an_unset_restart_keeps_each_paths_default() { + let service = json!({ "name": "plain", "image": "redis:7.4.11-alpine" }); + let config = config_with(&service); + let compose: Value = + serde_yaml::from_str(&ComposeDefinition::try_from(&config).unwrap().render()).unwrap(); + assert_eq!(compose["services"]["plain"]["restart"], "unless-stopped"); + + let body = build_project_body(&config); + let apps = body["custom"]["service"] + .as_array() + .cloned() + .unwrap_or_default(); + let app = apps + .iter() + .chain(body["custom"]["web"].as_array().into_iter().flatten()) + .find(|a| a["code"] == "plain") + .expect("registered app"); + assert_eq!(app["restart"], "always"); +} + +#[test] +fn an_unknown_restart_value_is_refused() { + let doc = json!({ + "name": "runtime-fields", + "app": { "type": "custom", "image": "nginx:1.27" }, + "services": [{ "name": "x", "image": "redis:7", "restart": "sometimes" }], + }); + let err = StackerConfig::from_str(&serde_yaml::to_string(&doc).unwrap()) + .expect_err("restart: sometimes must be refused"); + assert!( + format!("{err}").contains("unless-stopped"), + "error names the allowed values: {err}" + ); +} + +fn ollama_with_counts(count: &str) -> String { + format!( + r#"name: runtime-fields +app: + type: custom + image: nginx:1.27 +services: + - name: ollama-cpu + image: ollama/ollama:0.40.4 + deploy: + replicas: {count} + resources: + reservations: + devices: + - driver: nvidia + count: {count} + capabilities: [gpu] +"# + ) +} + +/// local-ai-packaged writes `replicas: ${OLLAMA_CPU_REPLICAS}`. Read without +/// filling variables (Quick Deploy validation, CLI commands that rewrite +/// stacker.yml), the reference is kept and reaches the compose as written. +#[test] +fn a_count_written_as_a_variable_reference_is_kept() { + for reference in ["${OLLAMA_CPU_REPLICAS}", "${OLLAMA_CPU_REPLICAS:-1}"] { + let config = StackerConfig::from_str_raw(&ollama_with_counts(reference)) + .unwrap_or_else(|e| panic!("{reference} must parse: {e}")); + let compose = ComposeDefinition::try_from(&config) + .expect("compose") + .render(); + let doc: Value = serde_yaml::from_str(&compose).expect("rendered compose is YAML"); + let deploy = &doc["services"]["ollama-cpu"]["deploy"]; + assert_eq!(deploy["replicas"], json!(reference), "{compose}"); + assert_eq!( + deploy["resources"]["reservations"]["devices"][0]["count"], + json!(reference), + "{compose}" + ); + + let written = serde_yaml::to_string(&config).expect("config serializes"); + let reread: serde_yaml::Value = serde_yaml::from_str(&written).unwrap(); + assert_eq!( + reread["services"][0]["deploy"]["replicas"], + serde_yaml::Value::String(reference.to_string()), + "{written}" + ); + } +} + +/// Filled from the environment, the value arrives as the string "0". +#[test] +fn a_filled_variable_reference_is_a_count() { + std::env::set_var("STACKER_TEST_OLLAMA_CPU_REPLICAS", "0"); + let config = + StackerConfig::from_str(&ollama_with_counts("${STACKER_TEST_OLLAMA_CPU_REPLICAS}")) + .expect("filled reference parses"); + let compose = ComposeDefinition::try_from(&config) + .expect("compose") + .render(); + let doc: Value = serde_yaml::from_str(&compose).expect("rendered compose is YAML"); + let deploy = &doc["services"]["ollama-cpu"]["deploy"]; + assert_eq!(deploy["replicas"], json!(0), "{compose}"); + assert_eq!( + deploy["resources"]["reservations"]["devices"][0]["count"], + json!(0), + "{compose}" + ); +} + +#[test] +fn a_count_that_is_neither_a_number_nor_a_reference_is_refused() { + for bad in ["lots", "\"-1\"", "\"${}\"", "\"$1\""] { + let err = StackerConfig::from_str_raw(&ollama_with_counts(bad)) + .err() + .unwrap_or_else(|| panic!("{bad} must be refused")); + assert!(err.to_string().contains("replicas must be"), "{bad}: {err}"); + } +} diff --git a/tests/stacker_yml_variable_references.rs b/tests/stacker_yml_variable_references.rs new file mode 100644 index 00000000..e33aaf3f --- /dev/null +++ b/tests/stacker_yml_variable_references.rs @@ -0,0 +1,173 @@ +//! A stacker.yml value can name a variable (`${VAR}`). Filled, the value +//! arrives as a string; read without filling variables (deploy validation), +//! the reference is still there. Neither may break a field that is not a +//! string. + +use stacker::cli::config_parser::StackerConfig; +use stacker::cli::error::Severity; + +fn errors(config: &StackerConfig) -> Vec { + config + .validate_semantics() + .iter() + .filter(|i| matches!(i.severity, Severity::Error)) + .map(|i| i.message.clone()) + .collect() +} + +/// Every boolean and number of stacker.yml written as `${VAR}`, with the +/// value the deploy fills in. +fn filled_config() -> String { + r#" +name: refs +app: + type: node + privileged: ${STACKER_REF_FALSE} + healthcheck: + test: "CMD true" + retries: ${STACKER_REF_FOUR} +services: + - name: db + image: postgres:16 + privileged: ${STACKER_REF_TRUE} + healthcheck: + test: "CMD pg_isready" + retries: ${STACKER_REF_FOUR} +proxy: + type: nginx + auto_detect: ${STACKER_REF_FALSE} +deploy: + target: server + server: + host: 203.0.113.10 + port: ${STACKER_REF_PORT} +ai: + enabled: ${STACKER_REF_TRUE} + timeout: ${STACKER_REF_FOUR} +monitoring: + status_panel: ${STACKER_REF_TRUE} + metrics: + enabled: ${STACKER_REF_TRUE} + telegraf: ${STACKER_REF_TRUE} + alerts: + interval: ${STACKER_REF_FOUR} + on_recovery: ${STACKER_REF_FALSE} + target: { pipe: oncall } +hooks: + notify: ${STACKER_REF_FALSE} +pipes: + - name: p + source: app + target: db + source_endpoint: GET /a + target_endpoint: POST /b + poll_interval: ${STACKER_REF_FOUR} + retry: ${STACKER_REF_FOUR} + retry_backoff_ms: ${STACKER_REF_FOUR} + retry_backoff_max_ms: ${STACKER_REF_FOUR} +"# + .to_string() +} + +#[test] +fn filled_booleans_and_numbers_are_read_as_their_type() { + std::env::set_var("STACKER_REF_TRUE", "true"); + std::env::set_var("STACKER_REF_FALSE", "false"); + std::env::set_var("STACKER_REF_FOUR", "4"); + std::env::set_var("STACKER_REF_PORT", "2222"); + let config = StackerConfig::from_str(&filled_config()).unwrap_or_else(|e| panic!("{e}")); + + assert!(!config.app.privileged); + assert_eq!(config.app.healthcheck.as_ref().unwrap().retries, 4); + assert!(config.services[0].privileged); + assert_eq!(config.services[0].healthcheck.as_ref().unwrap().retries, 4); + assert!(!config.proxy.auto_detect); + assert_eq!(config.deploy.server.as_ref().unwrap().port, 2222); + assert!(config.ai.enabled); + assert_eq!(config.ai.timeout, 4); + assert!(config.monitoring.status_panel); + let metrics = config.monitoring.metrics.as_ref().unwrap(); + assert!(metrics.enabled && metrics.telegraf); + let alerts = config.monitoring.alerts.as_ref().unwrap(); + assert_eq!(alerts.interval, 4); + assert!(!alerts.on_recovery); + assert!(!config.hooks.notify); + let pipe = &config.pipes[0]; + assert_eq!(pipe.poll_interval, Some(4)); + assert_eq!(pipe.retry, Some(4)); + assert_eq!(pipe.retry_backoff_ms, Some(4)); + assert_eq!(pipe.retry_backoff_max_ms, Some(4)); +} + +#[test] +fn a_boolean_or_number_that_is_not_one_is_still_refused() { + for (field, bad) in [ + ("privileged", "\"yes please\""), + ("privileged", "\"1\""), + ( + "healthcheck:\n test: \"CMD true\"\n retries", + "\"four\"", + ), + ( + "healthcheck:\n test: \"CMD true\"\n retries", + "\"-1\"", + ), + ] { + let yaml = format!("name: refs\napp:\n type: node\n {field}: {bad}\n"); + assert!( + StackerConfig::from_str_raw(&yaml).is_err(), + "{field}: {bad} must be refused" + ); + } +} + +/// Compose allows a variable for any part of a port mapping; validation +/// without filling variables checks the rest and lets the variable through. +#[test] +fn port_mappings_with_variables_pass_validation() { + let yaml = r#" +name: refs +app: + type: static + ports: ["${APP_PORT:-8080}:80", "${APP_PORT}"] +services: + - name: db + image: postgres:16 + ports: ["127.0.0.1:${DB_PORT}:5432", "${BIND_IP}:5433:5432", "${LOW}-${HIGH}:9000-9010/udp"] +deploy: + target: cloud + cloud: + provider: hetzner + public_ports: ["${PUBLIC_PORT}", "${PUBLIC_PORT}/tcp", "443"] +"#; + let config = StackerConfig::from_str_raw(yaml).expect("parses"); + assert_eq!(errors(&config), Vec::::new()); +} + +#[test] +fn the_rest_of_a_port_mapping_with_a_variable_is_still_checked() { + for bad in [ + "${APP_PORT}:133342", + "${APP_PORT}:80/http", + "abc:${APP_PORT}", + ] { + let yaml = format!("name: refs\napp:\n type: static\n ports: [\"{bad}\"]\n"); + let config = StackerConfig::from_str_raw(&yaml).expect("parses"); + assert!(!errors(&config).is_empty(), "{bad} must be refused"); + } + let yaml = r#" +name: refs +app: + type: static +deploy: + target: cloud + cloud: + provider: hetzner + public_ports: ["${PUBLIC_PORT}/http"] +"#; + let config = StackerConfig::from_str_raw(yaml).expect("parses"); + assert!( + !errors(&config).is_empty(), + "public port with a bad protocol must be refused" + ); +} diff --git a/tests/vault_health_tls.rs b/tests/vault_health_tls.rs new file mode 100644 index 00000000..56c8829d --- /dev/null +++ b/tests/vault_health_tls.rs @@ -0,0 +1,179 @@ +//! The Vault health check must trust the same CA as the Vault client. +//! +//! Vault behind a private CA works for every real call (the client loads +//! VAULT_CACERT), yet the health check used a bare HTTP client and reported +//! "certificate verify failed" forever, with nothing misconfigured. +//! +//! The test makes a private CA and a server certificate with the `openssl` +//! binary and serves HTTPS with `openssl s_server`; it is skipped where +//! `openssl` is not installed. + +use stacker::health::{ComponentStatus, HealthChecker}; +use std::path::Path; +use std::process::{Child, Command, Stdio}; +use std::sync::Arc; +use std::time::Duration; + +fn openssl(dir: &Path, args: &[&str]) -> bool { + Command::new("openssl") + .args(args) + .current_dir(dir) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .map(|s| s.success()) + .unwrap_or(false) +} + +/// A private CA and a server certificate for localhost signed by it. +fn make_certs(dir: &Path) -> bool { + std::fs::write( + dir.join("san.ext"), + "subjectAltName=DNS:localhost,IP:127.0.0.1\nextendedKeyUsage=serverAuth\nkeyUsage=digitalSignature,keyEncipherment\n", + ) + .unwrap(); + openssl( + dir, + &[ + "req", + "-x509", + "-newkey", + "rsa:2048", + "-nodes", + "-keyout", + "ca.key", + "-out", + "ca.pem", + "-days", + "2", + "-subj", + "/CN=Test Private CA", + ], + ) && openssl( + dir, + &[ + "req", + "-newkey", + "rsa:2048", + "-nodes", + "-keyout", + "server.key", + "-out", + "server.csr", + "-subj", + "/CN=localhost", + ], + ) && openssl( + dir, + &[ + "x509", + "-req", + "-in", + "server.csr", + "-CA", + "ca.pem", + "-CAkey", + "ca.key", + "-CAcreateserial", + "-out", + "server.pem", + "-days", + "2", + "-extfile", + "san.ext", + ], + ) +} + +struct Server(Child); + +impl Drop for Server { + fn drop(&mut self) { + let _ = self.0.kill(); + } +} + +fn serve_https(dir: &Path) -> Option<(Server, u16)> { + let port = std::net::TcpListener::bind("127.0.0.1:0") + .ok()? + .local_addr() + .ok()? + .port(); + let child = Command::new("openssl") + .args([ + "s_server", + "-accept", + &port.to_string(), + "-cert", + "server.pem", + "-key", + "server.key", + "-www", + "-quiet", + ]) + .current_dir(dir) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .ok()?; + let server = Server(child); + for _ in 0..50 { + if std::net::TcpStream::connect(("127.0.0.1", port)).is_ok() { + return Some((server, port)); + } + std::thread::sleep(Duration::from_millis(100)); + } + None +} + +async fn vault_status(address: &str, ca_cert: Option) -> (ComponentStatus, String) { + let mut settings = stacker::configuration::get_configuration().expect("configuration"); + settings.vault.address = address.to_string(); + settings.vault.ca_cert = ca_cert; + settings.vault.client_cert = None; + settings.vault.client_key = None; + let pool = sqlx::PgPool::connect_lazy("postgres://unused@127.0.0.1:1/unused").unwrap(); + let checker = HealthChecker::new(Arc::new(pool), Arc::new(settings)); + let response = checker.check_all().await; + let vault = response + .components + .get("vault") + .expect("vault component") + .clone(); + (vault.status, vault.message.unwrap_or_default()) +} + +#[tokio::test] +async fn the_vault_health_check_trusts_the_configured_private_ca() { + let dir = tempfile::tempdir().unwrap(); + if !make_certs(dir.path()) { + eprintln!("Skipping: openssl is not available"); + return; + } + let Some((_server, port)) = serve_https(dir.path()) else { + eprintln!("Skipping: could not start openssl s_server"); + return; + }; + let address = format!("https://localhost:{port}"); + let ca = std::fs::read_to_string(dir.path().join("ca.pem")).unwrap(); + + let (status, message) = vault_status(&address, Some(ca)).await; + // TLS wording differs by platform (OpenSSL, macOS Security), so check + // for the connection failure itself rather than its text. + assert!( + !message.contains("error sending request"), + "with the CA configured the connection must succeed: {message}" + ); + assert!( + status == ComponentStatus::Healthy || message.contains("slowly"), + "with the CA configured Vault must read healthy: {status:?} {message}" + ); + + // Without the CA it still reports the TLS failure, so a real + // misconfiguration stays visible. + let (_, message) = vault_status(&address, None).await; + assert!( + message.contains("error sending request"), + "without the CA the TLS failure must be reported: {message}" + ); +}