ci(tests): upload Vitest and Playwright JUnit reports to Trunk Flaky Tests#2941
Open
gantoine wants to merge 5 commits into
Open
ci(tests): upload Vitest and Playwright JUnit reports to Trunk Flaky Tests#2941gantoine wants to merge 5 commits into
gantoine wants to merge 5 commits into
Conversation
Configure Trunk Flaky Tests test uploads for the posthog-inc org. - Add the junit reporter (outputFile ./junit.xml, addFileAttribute: true) alongside the default reporter in all 8 Vitest configs, and set an explicit retry: 0 so flaky detection sees raw pass/fail results. - Add an "Upload test results to Trunk" step to the unit-test job in test.yml using trunk-io/analytics-uploader (pinned), scoped to apps/*/junit.xml,packages/*/junit.xml to avoid node_modules dupes. - Gitignore the generated junit.xml reports. Validated locally with trunk-analytics-cli validate: 8 valid files, 0 warnings, 0 errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
Contributor
|
Reviews (1): Last reviewed commit: "ci(tests): upload JUnit reports to Trunk..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Extend the Trunk Flaky Tests setup to the Playwright E2E suite. - Add the junit reporter (outputFile junit.xml, resolved next to the config => apps/code/tests/e2e/junit.xml) to playwright.config.ts, in both the CI and local reporter sets, and set retries: 0 (was isCI ? 2 : 0) so flaky detection sees raw pass/fail results. - Add an "Upload test results to Trunk" step to the integration-test job in test.yml, after the E2E run, using the pinned trunk-io/analytics-uploader with junit-paths apps/code/tests/e2e/junit.xml. Validated locally with trunk-analytics-cli validate: 0 errors. (One non-blocking warning remains -- "missing file or filepath" -- which is inherent to Playwright's built-in JUnit reporter, which has no option to emit a per-testcase file attribute; it matches Trunk's documented Playwright config.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gantoine
changed the title
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The retry: 0 setting and junit reporter array were pasted verbatim into all 8 vitest.config.ts files, so any change (outputFile path, a new Trunk reporter option) had to be made in lockstep across every file. Extract them to a root-level vitest.config.base.ts as `trunkTestOptions` and spread it into each config's `test` block. The constant is explicitly typed so the reporter tuple stays assignable across the Vitest 2.x and 4.x versions used in the workspace. Verified at runtime in a v4 package (shared), a v2 package (git), and the vite-augmented config (electron-trpc): junit.xml is still emitted and trunk-analytics-cli validate reports 0 warnings, 0 errors for all three. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trunk-io/analytics-uploader action maps the token: input to the CLI --token flag, so the separately-set env var was redundant and exposed the secret in the step environment unnecessarily. Keep only the token: input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gantoine
requested review from
a team and
joshsny
and removed request for
joshsny
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Sets up Trunk Flaky Tests test uploads for the
posthog-incorg. CI provider is GitHub Actions; covers both test frameworks in the repo — Vitest (unit/integration) and Playwright (E2E).Vitest
Config (all 8 configs)
apps/code,apps/mobile,packages/{ui,agent,git,shared,workspace-server,electron-trpc}:junitreporter (outputFile: ./junit.xml,addFileAttribute: true— Trunk needs the file attribute to map results to source) alongside the existingdefaultreporter, so console output is preserved.retry: 0so flaky detection sees raw pass/fail results. (0is already Vitest's default; this makes it explicit.)CI (
unit-testjob in.github/workflows/test.yml)New
Upload test results to Trunkstep afterpnpm test:junit-paths: "apps/*/junit.xml,packages/*/junit.xml"— scoped to package roots. A bare**/junit.xmlglob descends intonode_modules(pnpm symlinks workspace packages) and uploads each report dozens of times; the scoped glob matches exactly the 8 reports.Playwright
Config (
apps/code/tests/e2e/playwright.config.ts)junitreporter (outputFile: junit.xml, resolved next to the config →apps/code/tests/e2e/junit.xml) to both the CI and local reporter sets.retries: 0(wasisCI ? 2 : 0) so flaky detection sees raw pass/fail results.CI (
integration-testjob)New
Upload test results to Trunkstep after the E2E run:junit-paths: "apps/code/tests/e2e/junit.xml".Shared CI details
Both upload steps:
trunk-io/analytics-uploaderpinned tov2.1.2(SHA), per the repo's action-pinning convention.if: ${{ !cancelled() }}so failed/flaky results are still reported;continue-on-error: trueso an upload hiccup can't fail the job.org-slug: posthog-inc;TRUNK_API_TOKENwired as both thetokeninput and a step env var..gitignoreignores the generatedjunit.xmlreports.Local validation (
trunk-analytics-cli validate)"report has test cases with missing file or filepath". This is inherent to Playwright's built-in JUnit reporter (no option to emit a per-testcasefileattribute, confirmed inplaywright@1.58.1source) and matches Trunk's documented Playwright config. Clearing it would require a custom reporter; happy to add one if we want absolute 0 warnings (low value today — there's noCODEOWNERSfile for Trunk to map against).Follow-ups (not in this PR)
TRUNK_API_TOKENsecret to the repo's GitHub Actions secrets (from Trunk org settings). Confirmposthog-incis the exact Trunk org URL slug. ✅🤖 Generated with Claude Code