Repository navigation
Conversation
…s index (LeanerCloud#408) Migration 000140 adds a generated az_config column, read verbatim from payload->'details'->>'az_config' (RDS single-az/multi-az, Azure SQL zone redundancy, '' otherwise), and two unique indexes that include it: the 11-column natural key plus az_config (the new upsert arbiter) and the 000120 source-account key plus az_config. The existing 11-column and 000120 source indexes are kept, so the current upsert keeps working and this can be applied before the store switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tency key (LeanerCloud#408) Single-AZ and Multi-AZ RDS recommendations of the same class, engine, term and payment shared one rec ID and one identity key. Read az_config verbatim from Details (RecommendationRecord.DetailsAZConfig, pinned to the 000140 generated column by shared vectors) and add it to RecIdentityKey, the rec ID (appended only when set, so every other rec keeps its ID), the purchase idempotency key and describeRec. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nfig (LeanerCloud#408) Requires migration 000140. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oud#408) Migration 000141 drops the 11-column natural key and the 000120 source index, both superseded by the 000140 indexes that add az_config. While either exists, a collection batch carrying a Single-AZ and a Multi-AZ RDS recommendation of the same class fails as a whole. The rollback collapses az_config-only duplicates to the newest row and recreates both. Apply only after the 12-column upsert is live everywhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
|
Independent review 2 (claude-fleet, cold): REQUEST_CHANGES a035c5d Findings:
Attacked:
Verified locally on the merged tree (origin/main 6af022f plus head, with merge-tree clean), with GOTOOLCHAIN=go1.26.10 and no cloud credentials, under the heavy lock, with testcontainers Postgres:
Reviewed by an independent agent with no author context (second reviewer). Review only: nothing was pushed, and the GitHub review state is unchanged. |
Bring the branch up to date with main. Conflicted files: - internal/config/store_postgres_recommendations.go: main (LeanerCloud#1209) removed ReplaceRecommendations and the onConflict flag, making the natural-key ON CONFLICT clause unconditional. The branch had switched that clause to the 12-column key with az_config. Kept main's unconditional upsert and applied the branch's az_config arbiter to it, plus the branch's doc comment on the 12-column index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged origin/main (1c5787a) into this branch. New head: 1fa7665. Conflict resolution:
I found no semantic conflicts. Nothing on the branch calls Checks on the merged head:
|
|
Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: REQUEST_CHANGES Blocking:
Non-blocking:
Attacked:
Body-stale:
Reviewer: rB22. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths. |
…ation 000145 -> 000141 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (LeanerCloud#1239, LeanerCloud#1241, LeanerCloud#1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…000140 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140, 000141, 000143 and 000144 belong to PRs held on owner decisions (LeanerCloud#1239, LeanerCloud#1241, LeanerCloud#1253); this PR takes 000140 so it can merge ahead of them, and they will take the next free numbers when decided. Rename only: the SQL is unchanged, and the test's version constant and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…7 -> 000143 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (LeanerCloud#1239, LeanerCloud#1241, LeanerCloud#1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-> 000145 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (LeanerCloud#1239, LeanerCloud#1241, LeanerCloud#1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…148 -> 000144 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (LeanerCloud#1239, LeanerCloud#1241, LeanerCloud#1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Migration renumbering needed before this merges. The reviewed, ready PRs have taken 000140-000145 so they could merge without waiting on owner decisions: #1240 (000140), #1302 (000141), #1268 (000142), #1287 (000143), #1289 (000144), #1116 (000145). This PR's migration(s) (000140, 000141) must be renumbered to start at 000146, or the next free number at that time, when it's ready to merge (for example 000140 -> 000146 and 000141 -> 000147, keeping the expand/contract pair adjacent and in order). golang-migrate skips a version below the database's current one, so a lower number merged after those would never apply. Dependency check: none of the six ready migrations depends on this PR's schema. #1240's |
…n 000146 -> 000142 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (LeanerCloud#1239, LeanerCloud#1241, LeanerCloud#1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d of the full list (#1240) * fix(api): add opt-in keyset pagination to GET /api/recommendations The list endpoint returned the whole filtered recommendations cache in a single body, so a tenant with a few thousand cached rows pushed the response past the 6 MB Lambda payload cap and the page failed with 502. Gzip (#775) raised that ceiling but did not bound it. Callers can now send limit (1-1000) and the returned next_cursor to read the list in pages ordered by recommendation ID. The cursor is an opaque base64url encoding of the last ID seen, so a row deleted between requests does not shift later pages. summary and regions still describe the full filtered set on every page, and compute capacity is only decoded for the returned rows. Requests without limit or cursor keep the existing unpaginated response, so the current frontend is unaffected until it adopts the parameters. openapi.yaml documents the parameters and the actual response shape. Refs #777 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(api): cap recommendation pages by serialized bytes and reject empty paging params A page was bounded only by row count, but the opaque Details blob makes a row's size data-dependent, so 1000 large rows could still pass the 6 MiB Lambda response cap. Pages now also stop once the rows, measured as they appear inside the Lambda response envelope (the body re-encoded as a JSON string, so quotes and backslashes count twice), plus the summary, regions, keys and next_cursor would pass a 4 MiB budget. The budget is checked against uncompressed bytes because the #775 gzip only runs when the client accepts it. A page always holds at least one row, so a row that alone is over the budget is returned on its own page with a logged warning instead of stalling the cursor walk. Legacy unpaginated mode is now selected only when neither limit nor cursor is present. A present but empty ?limit= or ?cursor= returns 400 instead of falling back to the unbounded response. Refs #723 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(api): reject duplicate recommendation IDs when paging and document 400s - Correct the ordering comment: the paging key is the application recommendation ID built from the natural-key columns, not the cache table's UUID primary key. A paginated request now fails if two rows share an ID, since the strict "after" comparison would skip one of them at a page boundary. - Describe the feature as opt-in in-memory cursor slicing, not a SQL keyset (store-side paging is #981). - TestGetRecommendations_UnpaginatedShapeUnchanged copies the expected IDs before the call so an in-place reorder in legacy mode fails it. - Add TestGetRecommendations_PaginationRejectsDuplicateIDs. - openapi: document the 400 response for bad limit/cursor/filter values, and say next_cursor's absence is the only end-of-list signal. Refs #723 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(db): index recommendations by byte-wise payload id for keyset scans Migration 000142 adds recommendations_scan_key_idx on (COALESCE(payload->>'id', '') COLLATE "C", id). The paginated GET /api/recommendations scan (issue #981) walks the cache in chunks in that order and resumes each chunk after the last (id, primary key) it read; without the index every chunk is a full scan plus a sort. COLLATE "C" is byte order, the same order Go's string comparison uses, so SQL paging keeps the order the API already returns. Refs #981 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(config): read stored recommendations in keyset chunks ListStoredRecommendationsChunk returns at most limit rows matching a RecommendationFilter, ordered by the byte-wise recommendation ID and then the table's UUID primary key, resuming strictly after a given position. The primary key keeps rows that share an ID in a total order, so a chunk boundary inside such a run neither skips nor repeats a row. The MinSavingsPct floor stays in-process (shared with ListStoredRecommendations via decodeStoredRecommendation); rows it drops still advance the position. Integration tests pin the order to Go's sort.Strings on IDs that a linguistic collation orders differently, cover duplicate IDs across every chunk size, the pct floor emptying a whole chunk, the empty table, an EXPLAIN check that recommendations_scan_key_idx serves the query without a Sort node, and walks racing concurrent upserts. Refs #981 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scheduler): scan cached recommendations chunk by chunk ScanRecommendations walks the recommendations cache through ListStoredRecommendationsChunk in byte-wise ID order and applies the same suppressions and account overrides as ListRecommendations to each chunk, handing the survivors to a callback. It holds one chunk at a time instead of the whole filtered set. Suppressions are read once per scan and each (account, provider, service) override is resolved once per scan, not once per chunk. Read failures over-show exactly like ListRecommendations. The freshness read, cold-start kick and stale-while-revalidate kick are pulled out of ListRecommendations into startCacheRead/finishCacheRead so both reads share them unchanged. Tests pin the scan to ListRecommendations' output (sorted by ID) for chunk sizes 1 to 31 with suppressions and overrides active. Refs #981 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(api): page GET /api/recommendations from the chunked scan A paginated request (limit or cursor sent) no longer loads the whole filtered set through ListRecommendations and sorts it in memory. It runs Scheduler.ScanRecommendations, applies the enabled-providers and account scope filters to each chunk (their inputs read once per request), and feeds the survivors to recommendationsPageCollector. The collector totals every row into the summary and keeps only the rows after the cursor up to limit+1 (the page plus one row proving a next page exists), then cuts the page with the existing byte budget. Peak memory is one scan chunk plus one page instead of the whole tenant. Output is unchanged: rows arrive in byte-wise ID order (Go string order), duplicate IDs still fail the request, and the summary still covers the full filtered set. The unpaginated path is untouched. The in-memory sortRecommendationsAfter/buildBudgetedPage are removed; the existing #723 pagination tests now run against the scan path, with buildBudgetedPage kept as a test helper over the collector. Refs #981 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): pin paginated recommendations order, ties and last page - PaginatedRequestDoesNotLoadFullList: the #981 regression; a paged request must use ScanRecommendations, never ListRecommendations. - PagedWalkMatchesLegacyOrder: for an unrestricted and a scoped reader, every chunk size and page size (including the exact-fit last page), the walk returns the legacy rows sorted by Go string order in consecutive pages, next_cursor on all but the last page, and the legacy summary on every page. IDs mix case, "|", "-", "/", spaces and accents. - PagedDuplicateIDs: ties fail every paged request at any chunk/page boundary; a twin removed by the scope filter is not a tie. - Background-collection scope (#627), the one-page memory bound, the out-of-order guard and the empty cache. - An integration test walks a real Postgres cache spanning several scan chunks with the min_savings_pct floor and matches the legacy output. Refs #981 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(scheduler): scan with main's variant-aware suppression index Merging main brought #1110 (issue #816), which changed aggregateSuppressions to return the raw rows grouped by 6-tuple (map[suppressionKey][]config.PurchaseSuppression) and moved the per-rec aggregate into applySuppressionIndex, because which rows apply now depends on the rec's variant (AZ, platform, tenancy). ScanRecommendations still held the old map[suppressionKey]*suppressionAgg, so the merged tree did not build. loadSuppressionIndex now returns main's grouped index and the scan passes it to the same applySuppressionIndex as ListRecommendations. Matching is per rec, so applying it chunk by chunk suppresses exactly what the non-scan path does. A new test pins the scan to ListRecommendations at every chunk size on an EC2 fixture with a variant purchase, a fully covered variant, a legacy wildcard row and a rec with unreadable Details, and checks the expected per-variant counts. Stripping variants from the scan's index fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber recommendations_scan_key_idx migration 000142 -> 000140 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140, 000141, 000143 and 000144 belong to PRs held on owner decisions (#1239, #1241, #1253); this PR takes 000140 so it can merge ahead of them, and they will take the next free numbers when decided. Rename only: the SQL is unchanged, and the test's version constant and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Cristian Magherusan-Stanciu <cristim@users.noreply.github.com>
…vice (#1302) * fix(purchase,server): skip the $0 purchase confirmation and report a failed lazy DB init on /health An execution with nothing selected never reaches a cloud API, but both the single-account and the per-account fan-out paths still sent a "purchase confirmed" email listing zero commitments. Gate the confirmation on anyRecPurchased / committed, so it goes out only when a real commitment was created (full or partial success). When reinitializeAfterConnect fails after setting app.Config, ensureDB closes the pool and clears app.DB but leaves app.Config, and the config_store health check reported healthy over the closed pool. Snapshot dbConnected with the store pointers under dbMu and report unhealthy while lazy PostgreSQL init has not completed. Audit findings A05-015 and A06-009. A06-019 (commitment_type labels) follows separately. Refs #339 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(analytics): label snapshot commitment types from provider and service commitmentTypeFor matched only a service literally named "SavingsPlans", which purchase_history never stores, so every commitment was stored as "RI": AWS Savings Plans ("savingsplans" / "savings-plans-<type>") and GCP committed-use discounts included. Derive the type from (provider, service) in the pkg/common CommitmentType vocabulary, and widen the savings_snapshots CHECK in migration 000145 to match. The CHECK still admits the legacy 'RI' / 'SavingsPlan' literals so an old replica can write during a rolling deploy; existing rows are relabeled. Audit finding A06-019. Refs #339 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(purchase): make the #339 no-confirmation assertions checkable by the mock guard TestNoUnfailableMockAssertions could not resolve the receiver of two AssertNotCalled sites (a mock returned by a helper). Declare the mock directly in the new test, and in the approve test rely on the missing SendPurchaseConfirmation expectation, which fails the test on any send. Refs #339 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(analytics): restore the pkg/common import after #1254 #1254 (on main) replaced the hand-rolled config-store mock in collector_test.go and dropped the now-unused pkg/common import. This branch's new commitment-type assertion uses common.CommitmentType, so the analytics test build broke after main was merged in although the merge was textually clean. Same assertion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber savings_snapshots_commitment_type_vocabulary migration 000145 -> 000141 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (#1239, #1241, #1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ail-closed plan providers, revocation_in_flight clear (#1268) * fix(config/db): audit findings A04-010/011/012/013 in migrations and config store - 000059 down: only remove ...0005 when it really is Purchaser. 000057 gave that UUID to Standard Users, so the old down deleted Standard Users and stripped it from every member. Refuse with a clear error rather than leave a Purchaser-only user with no group. 000096 down gets the same up-front refusal instead of a bare users_min_one_group violation. - 000027: dedupe savings_snapshots on (tableoid, ctid). ctid is per partition, so matching on it alone deleted unrelated rows of other partitions on a forced re-run. - DerivePlanProviders: accept the "provider:service" keys 000040 wrote, and return an error instead of an empty provider set when no key names a provider, so both plan-account guards fail closed. - MarkPurchaseRevoked clears revocation_in_flight, as 000072 documents; migration 000146 clears it on rows already revoked. Real-Postgres tests for each, using the shared container. Normalizing the plan-services key delimiter is deferred (see the PR). Refs #316 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(config): identify legacy-keyed plan services by their provider field The first #316 commit made DerivePlanProviders fail when no service key had a provider prefix. That refused supported plans: a legacy alias kept under its original key (#1123's singleton-alias edit path, exercised by TestLegacyServiceB1S1*) has a key like "original-key", and so do fixtures keyed "ec2" with Provider "aws". Each entry's provider now comes from its key prefix ("provider/service" or "provider:service"), falling back to the entry's own Provider field, which is what mergeRequestedServices and plan step resolution use. The check fails closed only when no entry identifies a provider either way. Refs #316 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(migrations): drop the duplicate TestMain from helpers_test.go main's #1400 added main_integration_test.go with a TestMain that stops the shared PostgreSQL container after the run. This branch added an identical TestMain to helpers_test.go, so after merging main the integration build of the package declared TestMain twice and failed to compile. Keep main's TestMain, which does the same teardown, and restore helpers_test.go to main's version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber purchase_history_clear_revoked_in_flight migration 000146 -> 000142 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (#1239, #1241, #1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ngs rollups (#1287) * fix(analytics): keep unknown usage and commitment unknown in the savings rollups savings_snapshots.total_usage and total_commitment are nullable on purpose: NULL means the provider gave no figure. monthly_savings_summary and provider_savings_summary rolled them up with SUM and AVG, which skip NULL, so "us-east-1 $4000 + eu-west-1 unknown" became a definite $4000. Migration 000147 recreates both views so a timestamp's total is NULL when any of its rows is unknown, and the period average is NULL when any of its timestamps is. A known 0 stays 0. Column names and types, the unique indexes and the CONCURRENTLY refresh path are unchanged; the down migration restores the previous definitions. Integration tests on real Postgres cover unknown, known zero, fully known, unknown-at-one-instant, down/up, and a refresh_savings_materialized_views() call queued behind the migration's locks. Refs #151 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber savings_rollups_propagate_unknown migration 000147 -> 000143 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (#1239, #1241, #1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…nd AWS fee (backend slice of #27) (#1289) * feat(marketplace): add listing settlement columns and guarded poller writes Migration 000148 adds the purchase_history settlement columns that 000087 deferred (listed_at, listing_proceeds_received, listing_fee_paid). listing_price_schedule already exists via 000105 (#525) and is left alone, including on rollback. The store gains the three statements the RI Marketplace status poller needs: the scan of listings to poll and two status-guarded UPDATEs. The guards (same listing id, active or same terminal state, proceeds not yet recorded) decide whether a poll result applies. Concurrent polls therefore record proceeds once, and a cancel recorded by the cancel handler is never revived or turned into a sale. Real-Postgres tests cover the races. Refs #27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(marketplace): poll RI Marketplace listings for sales and fees PollMarketplaceListings reads each listing due for a poll from AWS (DescribeReservedInstancesListings through the existing listing-finder interface, signed as the purchase's own account like list and cancel). While a listing is active it records its CreateDate as listed_at. When the listing closes, or is canceled after a partial sale, it records the terminal state with the seller's net proceeds and the AWS fee: the instances AWS reports sold, times the active price tier, minus the 12% fee, in whole cents. When instances sold but no single active USD tier exists, the state is recorded and the proceeds are left unset for a later poll instead of guessed. A failure on one listing is counted and the run moves on to the next. Refs #27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): check poller non-writes through MockConfigStore's call log The vacuous-assertion guard cannot resolve f.store as a receiver. Give the two tests their own *MockConfigStore and use the name-only AssertNotCalled, which reads the call log and fails on any call. Refs #27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(server): run the marketplace listing poll in the finalize_revocations sweep The finalize_revocations task now also runs the RI Marketplace status poller after the revocation repair. Both reconcile purchase_history with the provider on the same hourly schedule, so no new schedule or infrastructure is needed. The two steps are independent: one failing does not skip the other, and the task fails if either did. The task's result keeps the revocation counts under their existing keys and adds the poll counts under marketplace_listings. Refs #27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber purchase_history_listing_settlement migration 000148 -> 000144 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (#1239, #1241, #1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(config): drop the previous listing's settlement when a relist is recorded Repricing an RI on the Marketplace means cancel and relist, since AWS has no price edit. A relisted row kept the earlier listing's listed_at, proceeds and fee, and only RecordMarketplaceListingActive cleared them, which needs a poll to see the new listing active first. A relist that sold out (or was canceled) before the next hourly poll was therefore never settled: the settle guard refuses rows whose proceeds are set, so the row stayed active with the old 0.00 and was polled and skipped every hour, or showed the old listing's settlement as the new one's. UpdatePurchaseHistoryListing and ResolveMarketplaceListingAttempt, the two statements that write a listing id, now null the three columns in the same UPDATE when the id changes. Re-recording the id the row already holds (a cancel or its retry) keeps them. The new real-Postgres test runs the review's exact sequence plus the canceled-within-the-hour and resolved-attempt variants. Refs #27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(auth): bind group account scope entries to account IDs A group's allowed_accounts could hold an account's display name. The name was not bound to any account, so once that account was deleted or renamed the entry matched whichever account was later created or renamed to the same name. #1032 limited name matching to unique names, which does not help here: the newcomer is the only account carrying the name. Scope entries now match by account identifier only: - AccountScope.Allows never compares an entry with a display name. A caller that knows an account only by name (legacy rows keyed by external ID) still resolves through WithRegisteredAccounts to the one account carrying that name, and that account's ID must be in the scope. - Group create/update bind allowed_accounts at grant time: "*", account IDs and external IDs are kept; a name exactly one registered account carries, inside the actor's own scope, is stored as that account's ID; anything else is a 400. A failed account or scope load refuses the write. - Migration 000135 rewrites existing name entries to IDs when exactly one account carries the name and it predates the group's last write. Other entries are left in place and no longer grant access. Down is a no-op. - The name-only MatchesAccount and AuthContext.CanAccessAccount helpers, which had no production callers, are removed. Closes #1034 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): scope the history-summary fixture by account ID The history-summary integration tests from #1200 scoped their user with the tenant's display name. This branch binds allowed_accounts entries to account IDs and makes a bare name entry match nothing (#1034), so the scoped user saw no tenant A rows and the totals and listed-rows checks failed. Grant the scope by tenant A's account ID, the form group writes now store. The assertions are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber the group scope binding migration to 000149 main now ships migrations up to 000139, so 000135 would sit below versions a deployed database has already applied and golang-migrate would never run it. Open PRs hold 000140-000148, so this takes 000149 and must merge after that chain. The SQL is unchanged; only the file names, header comments, test version and test fixture IDs move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(db): renumber bind_group_account_names_to_ids migration 000149 -> 000145 golang-migrate only applies versions above the database's current one, so PRs must merge in migration-number order. 000140-000144 had been taken by PRs that are now held on owner decisions (#1239, #1241, #1253). The ready PRs move to 000140-000145 so they can merge ahead of them, and the held ones will take the next free numbers when decided. Rename only: the SQL statements are unchanged, and the test version constants and the comments naming the migration follow the new number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): register the wire-format group account for grant-time binding TestUserGroupWrites_WireFormat came in from main (#1261) and builds a Handler with no config store. Since #1034, group writes bind every allowed_accounts entry to a registered account and refuse the write when the account list cannot be loaded, so POST /api/groups answered 500. The test now registers its account ID in a store mock, as the binding tests do; the wire format it pins is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): scope the History paging user by account ID, not name TestIntegration_HistoryPaging_ScopedUserNeverPagesIntoAnotherTenant came in from main (#1277) and grants the scoped user tenant A by its display name. Since #1034 an allowed_accounts entry matches by cloud account ID only, so that user saw none of tenant A's rows and the walk returned only its own unscoped execution. The grant now names tenant A's ID, as the other scope tests on this branch do; what the test asserts is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(analytics): scope account_ids fixtures by account ID The account_ids analytics tests that arrived with the main merge (#1448) grant scope by account name ("a", "b", "c"). This branch binds group scope entries to account IDs and no longer matches an entry against a display name (#1034), so those name entries authorize nothing and the scoped cases got 404. List the account IDs in the scope instead, as the other analytics scope tests on this branch already do. The assertions are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(analytics): scope the account_ids DB test by account ID TestAnalyticsAccountIDs_SumsSelectedAccountsOnly_DB (from #1448) grants the scoped user ["a", "b"], which are account names. Scope entries are account IDs on this branch (#1034), so the name entries allowed nothing and the in-scope query returned 404. Use the created accounts' IDs. The foreign-id refusal assertion is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(history): scope the plan_id filter DB test by account ID TestIntegration_HistoryPlanFilter_CrossTenant (from #1445) grants the scoped user tenant A by account name. Scope entries are account IDs on this branch (#1034), so the name allowed nothing and tenant A's own plan came back empty. Grant tenant A's ID, as the other history integration tests on this branch do. The cross-tenant assertions are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api): scope #1464's actionable History tests by account ID #1464 landed on main with tests that scope the user by the display name "dev" and with an executionScope case built on WithRegisteredNames. This branch (#1034) makes name entries grant nothing and replaces that constructor, so after merging main those tests no longer compiled or no longer exercised the scoped path. Scope them by the account's ID, which is what a group stores now, and pin that a name entry contributes only its literal value to the SQL scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth,api): bind account names to the identifier in scope checks Review (rE7) found that an ID-scoped user could pass the purchase gate for an account outside their scope. resolveAccountNamesByID keyed names by both account ID and the free-text external ID in one map, so an editor of in-scope account Z who set Z's external ID to victim Y's ID got Z's name back for Y. AccountScope.Allows then resolved that name to Z, which is in scope, and allowed a purchase in Y. The same map fed the dashboard, ladder and RI reshape filters. Decide those gates on the record's cloud_account_id itself, the ID the purchase resolves its credentials from, and delete the dual-keyed map. Bind the name path in Allows as well, so a name only identifies an account through that account's own external ID and never through a value that is some registered account's ID. Callers that pass a name for an external ID already resolve it within the record's provider or require exactly one owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
VERDICT: REQUEST_CHANGES 1fa7665 Re-review of #1239. The previous verdict was REQUEST_CHANGES at a035c5d. The only change since then is the merge 1fa7665 of origin/main 1c5787a ( Findings:
Attacked:
|
Summary
#806 and #792 already added the zonal EC2 AZ, platform and tenancy to the recommendations natural key, rec ID, identity key and idempotency key. What #408 still lists as missing is EC2
scopeand RDSaz_config.az_config(fixed here). Cost Explorer reports Single-AZ and Multi-AZ RDS reservations of the same class, engine, term and payment as two separate recommendations (parseRDSDetailsmapsDeploymentOptiontosingle-az/multi-az). They are different offerings, but both got the same natural key, rec ID and identity key. That breaks three things on main:UpsertRecommendationswrites the whole multi-provider batch in one statement. When the batch has both variants it fails withSQLSTATE 21000("ON CONFLICT DO UPDATE command cannot affect row a second time"), so the whole collection write is lost, not just one rec.loadStoredRecommendationIndexrefuses every purchase for that provider with "more than one row for identity key".scope(no change needed). The AWS parser derives scope from the AZ (resolveEC2Scope: zonal exactly when CE gives an AZ). The AZ is already in every identity, so two recs cannot differ only by scope. Azure collects Shared-scope recs only (properties/scope eq 'Shared') and GCP does not set scope. The purchase guardcomputeDetailMismatchstill compares scope as a backstop.findOfferingIDalready filters byAZConfigand refuses an empty one, and EC2 filters already use platform, tenancy, scope and AZ, so no provider change is needed.Fix (four commits, expand/contract like the #792/#806 chains)
feat(db)migration 000140 (expand). Adds:az_configcolumn,COALESCE(payload->'details'->>'az_config', ''), verbatim. The table rewrite backfills it, so no data migration is needed.recommendations_natural_key_az_tp_azc_idx: the 11 columns plusaz_config.recommendations_natural_key_source_azc_idx: 000120's source-account key plusaz_config. This is needed because any unique index withoutaz_configstill rejects the pair. See the feat(insurance): per-recommendation Archera quotes (blocked on an Archera API mapping) #799 note below.The old indexes stay, so the current upsert keeps working.
fix(purchase): newRecommendationRecord.DetailsAZConfig(), pinned to the generated column by shared vectors.az_confignow goes into:RecIdentityKey. This covers the pricing index, planstep and the ramp-baseline netting.|<az_config>only when set, so EC2, cache and other recs keep their IDs.purchaseIdempotencyKey.describeRec.fix(config): the upsertON CONFLICTtargets the 12-column index. This needs 000140.fix(db)migration 000141 (contract). Drops the 11-column index and 000120'srecommendations_natural_key_source_idx. Each has a strictly finer replacement from 000140, so no existing row can violate the new indexes. The down migration collapsesaz_config-only duplicates to the newest row (the table is a regenerable cache) and recreates both indexes exactly.Prod deploy order (auto_migrate=false)
database-migration.ymlfor 000140 only (stepscounted from the live prod version).ON CONFLICTneeds the index that 000141 drops, so never runsteps=0before the release.Dev auto-migrates at startup, so the window there is one deploy. If a maintainer prefers separate PRs as in #808/#810/#811, the four commits split cleanly.
Behavior changes to know about
az_configchange once, on the next collection. A stale browser tab gets the usual 409 ("not in the current recommendation set").az_config, so it gets a 409. This is the same rule fix(purchase): carry EC2 platform and tenancy in rec ID, identity key and idempotency key (#806) #817 introduced for EC2 platform/tenancy, and the web modal echoes the listed rec's Details. I did not re-audit external API clients beyond what fix(purchase): carry EC2 platform and tenancy in rec ID, identity key and idempotency key (#806) #817 recorded.purchase_executionsrows: the identity and idempotency keys are recomputed from each row's persisted Details on both sides, so in-flight rows still hash and match consistently. Rows from before fix(db): redact bound query arguments at every log level #453, which have no Details, readaz_configas"", the same as fix(purchase): carry EC2 platform and tenancy in rec ID, identity key and idempotency key (#806) #817.Interaction with the #799 source-account chain
Migration 000120 (#881) added
recommendations_natural_key_source_idxas a prerequisite and did not switch the upsert to it. That index lacksaz_config, so it would keep rejecting the Single-AZ/Multi-AZ pair. This PR replaces it withrecommendations_natural_key_source_azc_idx, the same key plusaz_config. When #799 later switches to a source-aware upsert, it should target that index. 000120's down-migration guard still works, because rolling back past 000141 recreates both old indexes first.Verification
All runs were on macOS with Go 1.26 and real Postgres 17 through testcontainers. This is local unit, mock and integration evidence on synthetic fixtures. I did no real-cloud or real-account verification: a live CE response with both RDS deployment options was not exercised. Tests ran with
AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/nullbecause this machine's default AWS profile is invalid.Fail before / pass after
go test -tags integration ./internal/config/ -run 'AZConfig(Batch|Concurrent)'on the main schema and store code, with the index precondition disabled, fails in every batch case withfailed to insert recommendations batch: ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time (SQLSTATE 21000). It passes after the fix, six repeated runs.az_configremoved fromRecIdentityKeyandpurchaseIdempotencyKey(pre-fix behavior):TestPriceRecommendationsFromStore_PricesFromRequestedAZConfigfails withstored recommendations for aws contain more than one row for identity key ....TestPriceRecommendationsFromStore_RequestWithoutAZConfigRefusedForRDS,TestPurchaseIdempotencyKey_DiffersByAZConfigandTestRecIdentityKey_DiffersByAZConfigfail.TestConvertRecommendations_RDSAZConfigInIDshows both variants gettingaws||rds|us-east-1|db.r5.large|mysql|1|all-upfront, andTestRecommendationID_MalformedAZConfigErrorsfails. Both pass after the fix.Migrations (
go test -tags integration ./internal/database/postgres/migrations/ -run 'TestMigration_(RecommendationsAZConfig|DropRecommendations11ColNaturalKey|RecommendationsSourceAccount|DropRecommendations9ColNaturalKey|RecommendationsPlatformTenancy)$'): all pass. They cover:Concurrency.
TestPostgresStore_UpsertRecommendations_AZConfigConcurrentruns 8 concurrent collections re-upserting the stored pair. All succeed and exactly one row peraz_configremains.Suites and checks
go build ./...go veton the touched packages, with and without-tags integrationgo test ./internal/mocks/ ./internal/config/ ./internal/scheduler/ ./internal/api/ ./internal/purchase/ ./internal/planstep/ ./internal/database/... -count=1go test -tags integrationfor./internal/config/,./internal/scheduler/,./internal/api/,./internal/purchase/and./internal/database/postgres/migrations/. One test needed a fixture bump:TestApproveAzurePricingRequiresMigrationAndRecollectionnow migrates to 140, because the upsert needs the new column.golangci-lintv2.10.1 with--allow-parallel-runners, with and without--build-tags integration: 0 issuesgocyclo -over 10on the changed non-test files: silentscripts/gosec-hook.sh: cleangit secrets --scanin an isolated temp repo with the changed files tracked: clean. A planted fake key was flagged.scripts/test-git-secrets-allowlist.sh: 137 PASSgit diff --check: cleanOut of scope / noted
23505on that index. Postgres only routes conflicts on the arbiter index toDO UPDATE. I reproduced this on unmodifiedorigin/mainwith 8 concurrent single-row upserts: it failed 1 run in 4 onrecommendations_natural_key_source_idx. This PR keeps an equivalent index (..._source_azc_idx), so the race is unchanged. The concurrency test seeds the rows first so that it covers the steady-state re-collection path. The collection single-flight marker makes this rare. It could be fixed by making the source index non-unique until feat(insurance): per-recommendation Archera quotes (blocked on an Archera API mapping) #799 switches the arbiter, which is the owner's decision.TestMigration_ExecutionsDetachKeepsUpdatedAt(000128, purchase_executions) failed once during a full migrations-suite run and passed 3/3 in isolation. It is time-based and unrelated to this change.az_config, which is display-only.Frontend follow-up for the UI owner
cellKey(frontend/src/recommendations.ts) groups recs by physical cell. It includes the AZ and the platform/tenancy pair, but notaz_config. Single-AZ and Multi-AZ RDS rows of the same class therefore render as variants of one cell. Purchases are still correct, becausesamePurchaseVariantIdentityalready comparesaz_configfor RDS, but they should be separate cells, the same way fix(frontend): include platform and tenancy in EC2 recommendation cell key #818 did it for EC2 platform/tenancy.Closes #408
Maintainer: please mirror the issue's triage labels on this PR:
priority/p2,type/chore,triaged,severity/low,urgency/this-quarter,impact/few,effort/m.🤖 Generated with Claude Code