Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
These CSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction.
These CSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction. The interaction is also scored as [work to commit](/methodology/#interaction-work-to-commit) from the same trace.

Methodology: [Client Side Rendered Tests (Lighthouse)](/methodology/#client-side-rendered-tests-lighthouse).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ import {
prepareLighthouseClientSideRenderedTestsFcpMsHistory,
prepareLighthouseClientSideRenderedTestsFirstPaintMsHistory,
prepareLighthouseClientSideRenderedTestsInteractionHistory,
prepareClientSideRenderedInteractionWorkHistory,
} from '../../lib/prepare-chart'
import { INTERACTION_WORK } from '../../lib/interaction-phases'
import VersionLineChart from '../../components/VersionLineChart.astro'
import InteractionHistoryChart from '../shared/InteractionHistoryChart.astro'

Expand All @@ -16,6 +18,8 @@ const { versions } = Astro.props

const interactionData =
prepareLighthouseClientSideRenderedTestsInteractionHistory(versions)
const interactionWorkData =
prepareClientSideRenderedInteractionWorkHistory(versions)
---

<VersionLineChart
Expand All @@ -24,6 +28,15 @@ const interactionData =
<VersionLineChart
{...prepareLighthouseClientSideRenderedTestsFcpMsHistory(versions)}
/>
{
interactionWorkData.length > 0 ? (
<InteractionHistoryChart
title="Client Interaction Work to Commit by Version"
data={interactionWorkData}
metric={INTERACTION_WORK}
/>
) : null
}
{
interactionData.length > 0 ? (
<InteractionHistoryChart
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
These SSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction.
These SSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction. The interaction is also scored as [work to commit](/methodology/#interaction-work-to-commit) from the same trace.

Methodology: [Server Side Rendered Tests (Lighthouse)](/methodology/#server-side-rendered-tests-lighthouse).
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ import {
prepareLighthouseServerSideRenderedTestsFcpMsHistory,
prepareLighthouseServerSideRenderedTestsFirstPaintMsHistory,
prepareLighthouseServerSideRenderedTestsInteractionHistory,
prepareServerSideRenderedInteractionWorkHistory,
} from '../../lib/prepare-chart'
import { INTERACTION_WORK } from '../../lib/interaction-phases'
import VersionLineChart from '../../components/VersionLineChart.astro'
import InteractionHistoryChart from '../shared/InteractionHistoryChart.astro'

Expand All @@ -16,6 +18,8 @@ const { versions } = Astro.props

const interactionData =
prepareLighthouseServerSideRenderedTestsInteractionHistory(versions)
const interactionWorkData =
prepareServerSideRenderedInteractionWorkHistory(versions)
---

<VersionLineChart
Expand All @@ -24,6 +28,15 @@ const interactionData =
<VersionLineChart
{...prepareLighthouseServerSideRenderedTestsFcpMsHistory(versions)}
/>
{
interactionWorkData.length > 0 ? (
<InteractionHistoryChart
title="Server Interaction Work to Commit by Version"
data={interactionWorkData}
metric={INTERACTION_WORK}
/>
) : null
}
{
interactionData.length > 0 ? (
<InteractionHistoryChart
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
import { prepareInteractionBreakdownTable } from '../../lib/prepare-table'
import StatsTable from '../../components/StatsTable.astro'
import ViewSwitcher from '../../components/ViewSwitcher.astro'
import { INTERACTION_PHASES } from '../../lib/interaction-phases'
import {
LIGHTHOUSE_INTERACTION,
type InteractionMetric,
} from '../../lib/interaction-phases'
import InteractionChartCaption from './InteractionChartCaption.astro'
import StackedBarChart from '../../components/StackedBarChart.astro'

Expand All @@ -14,10 +17,11 @@ interface InteractionBreakdownDatum {
interface Props {
title: string
data: InteractionBreakdownDatum[]
metric?: InteractionMetric
}

const { title, data } = Astro.props
const series = INTERACTION_PHASES.map(({ key, label, color }) => ({
const { title, data, metric = LIGHTHOUSE_INTERACTION } = Astro.props
const series = metric.parts.map(({ key, label, color }) => ({
key,
label,
color,
Expand All @@ -37,7 +41,7 @@ const chart1Table = prepareInteractionBreakdownTable(chart1)
defaultValue="chart"
>
<StackedBarChart slot="chart" {...chart1}>
<InteractionChartCaption slot="caption" />
<InteractionChartCaption slot="caption" {metric} />
</StackedBarChart>
<StatsTable slot="table" {...chart1Table} />
</ViewSwitcher>
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
---
import { INTERACTION_PHASES } from '../../lib/interaction-phases'
import {
LIGHTHOUSE_INTERACTION,
type InteractionMetric,
} from '../../lib/interaction-phases'

interface Props {
title?: string
metric?: InteractionMetric
}

const { title } = Astro.props
const { title, metric = LIGHTHOUSE_INTERACTION } = Astro.props
---

<figcaption>
{title && <p class="interaction-chart-title">{title}</p>}
<p>
Total latency is split into the three consecutive phases reported by
Lighthouse.
</p>
<p>{metric.summary}</p>
<ul class="phase-descriptions">
{
INTERACTION_PHASES.map(({ label, description }) => (
metric.parts.map(({ label, description }) => (
<li>
The <strong>{label.toLowerCase()}</strong>, {description}
</li>
Expand All @@ -25,7 +26,7 @@ const { title } = Astro.props
</ul>
<ul class="legend" aria-hidden="true">
{
INTERACTION_PHASES.map(({ label, color }) => (
metric.parts.map(({ label, color }) => (
<li>
<span class="swatch" style={`background: ${color}`} />
{label}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,33 +1,42 @@
---
import StackedAreaChart from '../../components/StackedAreaChart.astro'
import { INTERACTION_PHASES } from '../../lib/interaction-phases'
import type { InteractionHistoryDatum } from '../../lib/types'
import {
LIGHTHOUSE_INTERACTION,
type InteractionMetric,
} from '../../lib/interaction-phases'
import type {
InteractionHistoryDatum,
InteractionWorkHistoryDatum,
} from '../../lib/types'
import InteractionChartCaption from './InteractionChartCaption.astro'

interface Props {
title: string
data: InteractionHistoryDatum[]
data: (InteractionHistoryDatum | InteractionWorkHistoryDatum)[]
metric?: InteractionMetric
}

const { title, data } = Astro.props
const { title, data, metric = LIGHTHOUSE_INTERACTION } = Astro.props
const valueOf = (datum: Props['data'][number], key: string) =>
(datum as unknown as Record<string, number>)[key]!
---

<StackedAreaChart
{title}
data={data.map((datum) => ({
name: datum.name,
total: datum.interactionLatency,
total: valueOf(datum, metric.totalKey),
values: Object.fromEntries(
INTERACTION_PHASES.map(({ key }) => [key, datum[key]]),
metric.parts.map(({ key }) => [key, valueOf(datum, key)]),
),
}))}
series={INTERACTION_PHASES.map(({ key, label, color }) => ({
series={metric.parts.map(({ key, label, color }) => ({
key,
label,
color,
}))}
valueFormat="ms"
yAxisLabel="Interaction latency (ms)"
yAxisLabel={metric.yAxisLabel}
>
<InteractionChartCaption slot="caption" {title} />
<InteractionChartCaption slot="caption" {title} {metric} />
</StackedAreaChart>
Original file line number Diff line number Diff line change
@@ -1,18 +1,41 @@
---
import { getRenderedInteractionBreakdownData } from '../../lib/prepare-chart'
import {
getRenderedInteractionBreakdownData,
getRenderedInteractionWorkBreakdownData,
} from '../../lib/prepare-chart'
import { INTERACTION_WORK } from '../../lib/interaction-phases'
import InteractionBreakdownChart from './InteractionBreakdownChart.astro'

interface Props {
testsKey: 'clientSideRendered' | 'serverSideRendered'
}

const { testsKey } = Astro.props
const data = getRenderedInteractionBreakdownData(testsKey).sort(
(a, b) =>
a.interactionLatency - b.interactionLatency || a.name.localeCompare(b.name),
const work = getRenderedInteractionWorkBreakdownData(testsKey)
const workToCommit = new Map(
work.map((datum) => [datum.name, datum.workToCommit]),
)
const byWorkToCommit = (
a: { name: string; interactionLatency?: number },
b: { name: string; interactionLatency?: number },
) =>
(workToCommit.get(a.name) ?? Infinity) -
(workToCommit.get(b.name) ?? Infinity) ||
(a.interactionLatency ?? 0) - (b.interactionLatency ?? 0) ||
a.name.localeCompare(b.name)
const data = getRenderedInteractionBreakdownData(testsKey).sort(byWorkToCommit)
work.sort(byWorkToCommit)
---

{
work.length > 0 ? (
<InteractionBreakdownChart
title="Interaction Work to Commit"
data={work}
metric={INTERACTION_WORK}
/>
) : null
}
{
data.length > 0 ? (
<InteractionBreakdownChart
Expand Down
61 changes: 54 additions & 7 deletions packages/docs/src/content/docs/methodology.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ Test use larger, 16-CPU runners, as described in their sections below.
disables its screen emulation, with no simulated CPU or network throttling.
Requests use a local connection to a production server. The Lighthouse flow
runs only the audits required for the published First Paint, First Contentful
Paint, and interaction-latency measurements.
Paint, and interaction-latency measurements, and records the interaction's
trace for work to commit.
- **Recorded environment:** Results include runner details and the measured
framework version; browser results also include the Chrome version used. Any benchmark that runs multiple itterations record mean, median, minimum, and maximum.

Expand Down Expand Up @@ -385,14 +386,18 @@ Current pinned dependencies are listed in each project's `package.json`:
- Interaction latency sums the input delay, processing duration, and
presentation delay from Lighthouse's INP breakdown. It measures this
controlled interaction, not page-lifetime INP.
- The same interaction is also scored as
[work to commit](/methodology/#interaction-work-to-commit), from the trace
Lighthouse records for it. Charts rank frameworks by work to commit and show
the Lighthouse breakdown alongside it.
- CI builds once per framework and test, then keeps one production server running
for all five measurements. Each measurement launches a fresh Chrome process.
- Before all measurements, one HTTP request warms `/client-side-rendered`.
The response is fully consumed with a ten-second timeout. This warms the
server, not the measured browsers, and is excluded from results.
- Valid slow results are retained. Paint values and total interaction latency
must be finite and greater than zero; individual interaction components may
be zero.
- Valid slow results are retained. Paint values, total interaction latency, and
work to commit must be finite and greater than zero; individual interaction
components may be zero.
- Each framework uses its supported production routing and rendering controls.
- Next.js disables SSR for the measured table and detail content.
- Astro uses React client-only islands. Its detail link performs a full-document
Expand All @@ -414,15 +419,19 @@ Current pinned dependencies are listed in each project's `package.json`:
- Interaction latency sums the input delay, processing duration, and
presentation delay from Lighthouse's INP breakdown. It measures this
controlled interaction, not page-lifetime INP.
- The same interaction is also scored as
[work to commit](/methodology/#interaction-work-to-commit), from the trace
Lighthouse records for it. Charts rank frameworks by work to commit and show
the Lighthouse breakdown alongside it.
- CI builds once per framework and test, then keeps one production server running
for all five measurements. Each measurement launches a fresh Chrome process.
- Before all measurements, HTTP requests warm `/server-side-rendered` and a
detail URL with a random UUID. Both responses are fully consumed under one
ten-second timeout. This warms the server, not the measured browsers, and is
excluded from results.
- Valid slow results are retained. Paint values and total interaction latency
must be finite and greater than zero; individual interaction components may
be zero.
- Valid slow results are retained. Paint values, total interaction latency, and
work to commit must be finite and greater than zero; individual interaction
components may be zero.
- Links use each framework's production navigation component or ordinary
anchors, allowing default prefetching. The table is loaded and rendered on
demand without caching or prerendering its output. Static surrounding shells
Expand All @@ -432,6 +441,44 @@ Current pinned dependencies are listed in each project's `package.json`:
- Astro's table and detail routes use `export const prerender = false` for
request-time rendering.

### Interaction Work to Commit

The client and server side rendered tests also score the benchmark's click from
the Chrome trace that Lighthouse records for the interaction step.

- Work to commit is the sum of three parts:
- **Input delay:** from the input's timestamp to the start of the first event
callback for the interaction.
- **Processing duration:** from the start of the first event callback to the
end of the last, across the interaction's pointer and click events.
- **Rendering work:** time the renderer main thread is busy, from the end of
processing until the interaction's frame commits. This includes framework
work scheduled by the event, style, layout, and paint. Overlapping tasks are
counted once.
- Input delay and processing duration match Lighthouse's INP breakdown for
client-routed navigations. Rendering work takes the place of presentation
delay.
- Idle main-thread time while waiting for the next frame is excluded. That wait
depends on where the click lands within the frame interval rather than on the
framework.
- Work after commit is not measured. This includes raster, draw, presentation,
and browser UI frames, such as the frame software-rendered headless Chrome
waits for after `pushState`
([#573](https://github.com/e18e/framework-performance/issues/573)).
- The commit boundary is the Event Timing `commitFinishTime` for the
interaction, capped at the end of its Event Timing duration. A client-routed
run without a `commitFinishTime` fails the benchmark.
- Tables and breakdown charts show the run with the median work to commit, so
its parts add up to the total. Results also record the mean, median, minimum,
and maximum of each part.
- For full-document navigations (Astro), the document usually unloads before
it presents a frame. The Event Timing duration then ends with processing and
rendering work is zero. Astro's score stops at unload and does not include
loading or rendering the detail page, so it is not directly comparable with
the client-routed frameworks.
- Results recorded before work to commit was measured only have the Lighthouse
breakdown, so its version history starts later.

### Initial Load

- Each production app serves the same small, styled landing page at
Expand Down
Loading
Loading