test(cli): collocate live e2e coverage - #6294
Conversation
jgoux
changed the title
Supabase CLI previewnpx --yes https://pkg.pr.new/supabase/cli/supabase@08aadb12637841861ea155f133e66ec7d623bf4aPreview package for commit |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: abcbb474a6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 980bb111ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f7962cb709
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e08e3ec85
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Final automated-review triage (capped batch): I reproduced all three new findings against the current tree. I will fix them together: derive outer budgets from the sequential subprocess guards, use the valid legacy logflare exclusion key, and register cleanup that retries the exact legacy stop before deleting the owned temp project. No findings are being deferred or rejected in this batch. After the replacement commit and focused CI, I will stop automated-review churn and request human review; later bot-only suggestions will be documented unless they demonstrate a release-blocking regression introduced by this final batch. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 08aadb1263
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Human review requested: the automated-review loop is capped after the final verified batch.\n\nExact head 08aadb1 is green in the full Test workflow and the standalone 24-test live workflow. The following later bot-only signals have been triaged as a group:\n\nDeferred for a human scope decision or separately tracked follow-up:\n1. Migrate the remaining legacy start Docker scenarios to registered, retryable cleanup.\n2. Redesign db start port allocation so reservations are owned through startup, and pair it with registered legacy cleanup.\n3. Replace the shared 240-second live subprocess default with phase-specific budgets whose sequential sums fit each outer test deadline.\n4. Add an independent remote-state postcondition to the functions delete golden path.\n\nRejected for this PR:\n- Accepting or skipping the preview-branch plan gate. The agreed suite contract has no capability skips: every configured target must support the commands under test, and a target without branch entitlement is an invalid environment for the full golden-path suite.\n\nThis comment is the follow-up record for the deferred signals; I did not create separate tracker items because this task did not authorize new tracker writes. Please decide in human review whether any deferred item should block merge or be filed separately. No further bot-only suggestion will reopen the implementation loop absent a demonstrated release-blocking regression introduced by the final batch. |
1 participant
Summary
Collocate remote-platform golden-path tests beside the commands they cover in apps/cli, while keeping Docker-stack-only scenarios as ordinary e2e tests and leaving the apps/cli-e2e replay and recording suite unchanged.
Use one serial live Vitest project with a single extended test fixture. Global setup requires a Management API URL and access token, provisions one disposable project through the typed Effect API client, waits with bounded retry semantics, derives the platform tenant host from project metadata, and shares cli plus project fixtures with every test. Explicit live runs fail fast when configuration is missing; there are no capability gates or runtime skips.
Keep live coverage intentionally narrow: one representative golden path per covered command. Setup and teardown may invoke other commands, but assertions stay focused on the command under test. Exact owned resources are cleaned even after ambiguous command results, and target plus cleanup failures are preserved.
Move the eight local Docker-stack suites to e2e naming, including functions dev synchronization on observable reload completion. The standalone live workflow retains Docker preflight, serial execution, one attempt, a 20-minute limit, and scoped project sweeping.