Skip to content

ci: shorten the Test workflow e2e critical path - #7159

Merged
avallete merged 29 commits into
developfrom
avallete/ci-faster-e2e
Oct 15, 2026
Merged

avallete merged 29 commits into
developfrom
avallete/ci-faster-e2e

Conversation

@avallete

@avallete avallete commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #7129. CLI e2e jobs now set the Test workflow's wall time.

  • Split the @supabase/cli-e2e suite into two shards; it was the single longest job.
  • The e2e sequencer packs shards by estimated wall time: concurrent files share half the e2e project's workers, because they contend for one Docker daemon, and exclusive files add their full duration.
  • Drop the actions/cache + go build steps in e2e jobs; supabase#build already builds or restores @supabase/cli-go through turbo.
  • The gen types e2e readiness probe connects over TCP. The Postgres entrypoint's init server listens only on the Unix socket, so a socket probe could pass before the final server started and the seed then failed mid-restart.
  • macOS port tests run on Blacksmith macOS runners; GitHub-hosted macOS runs took up to several times as long.

Measured over repeated CI runs against develop after #7129:

develop after #7129 This PR Change
Test workflow wall time (median) 3.9 min 2.75 min about 30% faster
Slowest CLI e2e job 3.3 min about 2.5 min about 25% faster
macOS port tests job 2.1–8.5 min 0.7–0.9 min 3–10x faster
Files on the Docker-heavy CLI shard 32 10

Required check names are unchanged.

🤖 Generated with Claude Code

…edundant Go build

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avallete
avallete requested review from a team and supabase-oss as code owners October 14, 2026 16:35
@avallete

Copy link
Copy Markdown
Member Author

/ai-review

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Both independent reviews were available: Claude reported three findings and Codex reported none. Code inspection confirms one minor performance advisory and two nits concerning test coverage and documentation. Actual cli-e2e shard imbalance is unmeasured. No major or critical defect was substantiated.

Findings

Severity Location Category Sources Claim
🟡 MINOR .github/workflows/test.yml:415 ci-performance claude The new cli-e2e shards inherit Vitest's default sharding rather than the timing-aware packing used by apps/cli. Because cli-e2e files run serially, unequal file durations can limit the wall-time benefit.
⚪ NIT apps/cli/tests/e2e-sequencer.ts:153 test-coverage claude The new E2eSequencer.shard wiring for project classification and configured worker count has no direct test coverage.
⚪ NIT apps/cli/tests/e2e-sequencer.ts:63 documentation claude The determinism comment incorrectly says the partition depends only on the file set; worker count also affects assignment, so shard processes need matching packing inputs.

Stats

Claude findings: 3 · Codex findings: 0 · Confirmed: 3 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread .github/workflows/test.yml
Comment thread apps/cli/tests/e2e-sequencer.ts
Comment thread apps/cli/tests/e2e-sequencer.ts

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Verified both reviews. Confirmed one non-blocking CI performance nit and refuted the missing-Go-binary regression: the existing Turbo build builds or restores the referenced binary. No confirmed correctness bugs.

Findings

Severity Location Category Sources Claim
⚪ NIT .github/workflows/test.yml:415 ci-performance claude The new cli-e2e shards use Vitest's default path-hash partition rather than measured durations, so uneven runtimes may limit the speedup.
Refuted findings (kept for transparency, not posted as review comments)
  • .github/workflows/test.yml:417 (test-infrastructure): Removing the explicit Go binary cache and build steps leaves SUPABASE_GO_BINARY pointing at a missing binary, causing delegated-command tests to fail on fresh runners.
    Refuted: .github/workflows/test.yml:403-404 still runs turbo run supabase#build before testing. turbo.json:64-66 explicitly makes that task depend on @supabase/cli-go#build. apps/cli-go/package.json:5 builds the exact referenced file with go build -o supabase-go ., and turbo.json:58-62 declares supabase-go as a cached output, so cache hits restore it. apps/cli/package.json:28 then copies this binary into dist. The removed workflow steps were redundant.

Stats

Claude findings: 1 · Codex findings: 1 · Confirmed: 1 · Refuted: 1 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread .github/workflows/test.yml
avallete and others added 24 commits October 14, 2026 19:20
…ndows test jobs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the macOS store archive

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fewer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…load

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avallete
avallete enabled auto-merge October 15, 2026 08:05
@avallete
avallete added this pull request to the merge queue Oct 15, 2026
Merged via the queue into develop with commit 1c01964 Oct 15, 2026
47 checks passed
@avallete
avallete deleted the avallete/ci-faster-e2e branch October 15, 2026 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants