Skip to content

perf(ci): overlap iOS simulator boot with the native build - #3836

Merged
thymikee merged 1 commit into
mainfrom
perf/ios-boot-build-overlap
Oct 20, 2026
Merged

thymikee merged 1 commit into
mainfrom
perf/ios-boot-build-overlap

Conversation

@thymikee

@thymikee thymikee commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Start iOS simulator boot before restoring/building the generic XCTest runner. Require successful boot readiness before XCTest or CLI preflight. Follow-up to test(ios): speed up fixtures and run native regressions nightly #3824.
  • Existing boot-action callers keep blocking behavior. The bounded wait and native failure exits have one shared owner. Test selection and runtime behavior stay unchanged.
  • Five files, +260/-16 (276 gross): CI/tooling +48/-16, tests +155/-0, evidence +57/-0; production +0/-0.

Validation

  • Tested 7606f85d717f32011c92f981008be05754d1a175: pnpm check:affected --run --head HEAD passed (36 checks; 25 skipped; 2 deduplicated), exactCommit=true, checkoutChanged=false. Independent review: no findings.
  • Focused workflow/process tests: 40 passed, covering deferred/default behavior, mandatory readiness, native failure and alarm propagation.
  • Four fresh Xcode 26.2/iOS 26.2 native build/boot comparisons passed. Mean local preparation: 40.739s serial → 29.653s overlapping, 27.2% lower. Builds slow under contention; combined preparation still improves in both comparisons. All owned simulators removed. Commands, timings and limitations.
  • Required hosted CI passed at this head, including Apple/Android/Linux smoke, Coverage and Integration. Apple completed in 20m52s, versus the earlier 23m49s. Cache and test timing differences prevent attributing the entire improvement to overlap. Cache matrix skipped; automated review neutral.

View guided diff Turn on auto-fix

@github-actions

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.45 MB 5.44 MB -13.8 kB
Package (unpacked) 5.45 MB 5.44 MB -13.8 kB
Package (download) 1.65 MB 1.64 MB -9.5 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 31.8 ms 30.0 ms -1.7 ms
CLI --help 60.3 ms 56.7 ms -3.6 ms

@thymikee

Copy link
Copy Markdown
Member Author

Hosted validation passed at 7606f85d717f32011c92f981008be05754d1a175, including Apple, Android and Linux smoke, Coverage and Integration. No unresolved review threads.

The Apple job completed in 20m52s, versus 23m49s in the earlier baseline. iOS took 15m02s and macOS 4m40s. The readiness barrier succeeded, all four selected iOS regressions ran, and the macOS runner suite executed 338 tests with zero failures.

This is an observed 2m57s shorter job, not a controlled measurement of the overlap alone: runner restore/build took about 5m24s, while fixture E2E took 3m37s versus the previous 6m52s. The four controlled local preparation samples remain the evidence for the overlap benefit (mean 40.739s serial → 29.653s overlapping). No test coverage was removed by this PR.

@thymikee

Copy link
Copy Markdown
Member Author

This PR is ready at 7606f85. All 16 checks pass at that commit, including the hosted Apple job, which ran the changed smoke-ios-run route through deferred boot, build, the readiness barrier, XCTest and the CLI preflight. There are no conflicts, and nothing else must happen before merge. I read the diff but did not run the tests or workflows, and I could not verify the local benchmark numbers or the hosted timing comparison, which the PR notes is affected by cache and test timing differences.

Not blocking, and fine to take or leave: the boot-timeout-seconds output in .github/actions/boot-ios-test-simulator/action.yml only echoes its input, so naming it as the deferred-wait budget or passing it explicitly from the caller would make it clearer. The boot action and scripts/wait-for-ios-simulator.sh tests sit in apple-ci-impact.test.ts and stub the alarm path with exit code 142, so a test file named for the script would cover it better. The evidence doc points to a /private/tmp path that others cannot open, and the PR body is over the 250-word limit in docs/agents/pull-requests.md.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 19, 2026
@thymikee
thymikee merged commit 89243e1 into main Oct 20, 2026
16 checks passed
@thymikee
thymikee deleted the perf/ios-boot-build-overlap branch October 20, 2026 07:14
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-20 07:15 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant