Skip to content

feat: score rendered-test interactions by main-thread work to commit - #901

Open
danielroe wants to merge 2 commits into
e18e:mainfrom
danielroe:feat/interaction-work-to-commit
Open

danielroe wants to merge 2 commits into
e18e:mainfrom
danielroe:feat/interaction-work-to-commit

Conversation

@danielroe

@danielroe danielroe commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

resolves #573

Note

nuxt is already quite high up on this measurement (in second place), so this isn't motivated by a feeling that we should be in a different position

while running some benchmarks locally, I noticed that most of the latency that frameworks are showing in the current numbers is the wait for the next frame tick after processing, which makes it very dependent on where the click lands in the 16.7 ms frame - rather than on the specific framework.

so this is an alternative proposal: score each click as work to commit instead

the idea is to measure input delay + processing duration, plus the renderer main thread's busy time from the end of processing until the Event Timing commitFinishTime

it skips idle time before the frame, and everything after commit (raster, draw, present, browser UI)

👉 update: this now keeps the Lighthouse INP breakdown, and records work to commit as an additional metric. it's not totally separate, in that we use the same trace/click event to produce both the traditional INP numbers and this rescore. we also now sort by the new metric, as I think it's more accurate, but 🤷

a chart rendered from the numbers in the CI run for the latest commit

interaction-scores-ci-head-sf

Note

Astro's full-document navigation mostly stops at unload under both metrics, so it isn't directly comparable.

Checklist

  • If updating UI/UX components ensure you have added screenshots or a gif or video of the changes to the PR description
  • If this is a dependabot PR that bumps a metaframework version, ensure that all package versions in the package match the versions installed by the metaframework version being bumped
  • If this is a dependabot PR that bumps a metaframework version in a starter-* package, ensure that the deps, and file content matches the output of the metaframeworks recommended starter/default project

@dreyfus92

Copy link
Copy Markdown
Member

hey @danielroe can you resolve conflicts?

@AlexanderKaran

Copy link
Copy Markdown
Collaborator

@danielroe Thank you so much for the PR. Just testing a few things of this locally and against my other in the works test and will get back to you

@AlexanderKaran

Copy link
Copy Markdown
Collaborator

@danielroe Ok, I have gone through the PR and would love to add this; however, I am leaning towards keeping both the Lighthouse INP and this new version. I think having both results to compare would be beneficial. The Lighthouse default INP breakdown is used in lots of places, so I think we should keep it.

The next question is do we extend the two lighthouse schemas to include this new metric:

export const CSRLighthouseBenchmark = BaseBenchmark.extend({
  project: z.literal('app'),
  name: z.literal('csrLighthouse'),
  iterations: createIteration(
    z.object({
      firstPaint: createValueWithUnit('ms', z.number().positive()),
      firstContentfulPaint: createValueWithUnit('ms', z.number().positive()),
      interaction: LighthouseInteractionTimingIteration,
    }),
  ),
  firstPaint: createIterationSummary('ms'),
  firstContentfulPaint: createIterationSummary('ms'),
  interaction: LighthouseInteractionTimingSummary,
})

export const SSRLighthouseBenchmark = BaseBenchmark.extend({
  project: z.literal('app'),
  name: z.literal('ssrLighthouse'),
  iterations: createIteration(
    z.object({
      firstPaint: createValueWithUnit('ms', z.number().positive()),
      firstContentfulPaint: createValueWithUnit('ms', z.number().positive()),
      interaction: LighthouseInteractionTimingIteration,
    }),
  ),
  firstPaint: createIterationSummary('ms'),
  firstContentfulPaint: createIterationSummary('ms'),
  interaction: LighthouseInteractionTimingSummary,
})

Or add this in as its own benchmark for example CSRBenchmarkName and SSRBenchmarkName

Let me know what you think.

@danielroe
danielroe force-pushed the feat/interaction-work-to-commit branch from 812797d to ad09f3b Compare November 20, 2026 17:30
@danielroe

Copy link
Copy Markdown
Contributor Author

done! ✅

it was a hefty rebase, so I pretty much reimplemented this instead, as an alternative measure made from the same trace/click event (so they are directly comparable).

some of the data is just therefore exactly the same between the two INP measures, but I thought that was better than having one implicitly depending on the other.

I've also chosen to rank by this new metric, but feel free to revert that if you prefer, of course 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

INP discrepancy due to testing harness

3 participants