FE-711: Add Metrics to Experiments

[kube]

🌟 What is the purpose of this PR?

🔗 Related links

• ...

🚫 Blocked by

• ...

🔍 What does this change?

• ...

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• does not modify any publishable blocks or libraries, or modifications do not need publishing
• modifies an npm-publishable library and I have added a changeset file(s)
• modifies a Cargo-publishable library and I have amended the version
• modifies a Cargo-publishable library, but it is not yet ready to publish
• modifies a block that will need publishing via GitHub action once merged
• I am unsure / need advice

📜 Does this require a change to the docs?

The changes in this PR:

• are internal and do not require a docs change
• are in a state where docs changes are not yet required but will be
  • this is tracked in: Insert Link Here
• require changes to docs which are made as part of this PR
• require changes to docs which are not made in this PR
  • Provide more detail here
• I am unsure / need advice

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph
• affected the execution graph, and the turbo.json's have been updated to reflect this
• I am unsure / need advice

⚠️ Known issues

🐾 Next steps

🛡 What tests cover this?

❓ How to test this?

1. Checkout the branch / view the deployment
2. Try X
3. Confirm that Y

📹 Demo

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview, Comment	May 29, 2026 2:01am
petrinaut	Ready	Preview, Comment	May 29, 2026 2:01am

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	May 29, 2026 2:01am

[cursor]

PR Summary

Medium Risk
Large refactor of Monte Carlo experiment outputs and public exports; breaking for consumers of place-token distribution types/messages, mitigated by tests and parallel local/worker paths.

Overview
Monte Carlo experiments no longer hard-code place token count distributions. They now support configurable user-defined metrics (expression code, place-token means, transition firing counts) with scalar or per-run distribution output, mergeable accumulators, and specs compiled in the worker or run locally.

The experiment API and worker protocol switch from distributions / distributionFrames to metrics / metricFrames keyed by metric id. The React layer requires at least one metric when creating an experiment, streams frames into experiment records, and replaces the old place-only timeline with a multi-metric uPlot UI (heatmaps, percentile bands, run/time aggregation).

The create-experiment drawer adds a Metrics section (built-in kinds, model metrics, custom code with LSP). Simulate view hides the standalone Metrics tab from the sidebar while keeping scenarios and experiments.

^{Reviewed by Cursor Bugbot for commit 09d211d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[augmentcode]

🤖 Augment PR Summary

Summary: This PR introduces first-class Monte Carlo “experiment metrics” for Petrinaut experiments, including runtime/user-defined metric evaluation and UI controls to configure and view metric outputs.

Changes:

• Exports new Monte Carlo metric APIs and types from @hashintel/petrinaut-core (metric specs, user-defined metrics, accumulators).
• Adds a SimulationFrameReader-based per-run frame reader so metrics can safely inspect place/transition state.
• Implements numeric and histogram accumulator utilities plus tests for merge/monoid behavior.
• Adds metric spec compilation (including expression metrics via compileMetric) into user-defined metric configs.
• Extends createMonteCarloExperiment() with a local execution path when metric callbacks/specs are provided (worker cannot receive executable code).
• Updates experiments React state/provider to store metric specs, metric frames, and latest-by-id.
• Adds UI for defining metrics when creating an experiment (including LSP diagnostics for expression metrics).
• Updates experiment viewing UI to optionally show token-count timeline and a new metrics summary section.
• Adds architecture/proposal/usage docs for Monte Carlo metrics direction.

Technical Notes: Worker-backed experiments remain distribution-only; experiments with expression/aggregated metrics run locally to avoid posting JS code across the worker boundary.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

[graphite-app]

Metric ID validation inconsistency allows duplicate IDs with different whitespace. The code trims the ID when checking if empty (line 136) but adds the untrimmed ID to the Set (line 142), and checks for duplicates using the untrimmed ID (line 139). This means metric IDs like " test" and "test" would both pass validation as unique IDs, causing potential conflicts.

for (const metricSpec of input.metricSpecs) {
  const trimmedId = metricSpec.id.trim();
  if (trimmedId === "") {
    throw new Error("Metric id is required");
  }
  if (metricIds.has(trimmedId)) {
    throw new Error(`Metric id "${trimmedId}" is duplicated`);
  }
  metricIds.add(trimmedId);
  // ...
}

Spotted by Graphite



Is this helpful? React 👍 or 👎 to let us know.

[cursor]

Distribution metric timeSampleCount is zero without time aggregation

Low Severity

When a distribution metric has no time aggregation (aggregateTime: "none"), timeSampleCount is initialized to 0 and never updated, so every distribution frame reports timeSampleCount: 0 regardless of how many run samples were collected. This is inconsistent with the scalar path which tracks frame count in scalarFrameCountState.count. Consumers displaying or reasoning about sample counts for distribution frames will see a misleading zero.

 

^{Reviewed by Cursor Bugbot for commit 823f376. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 09d211d. Configure here.}

[cursor]

Progress reports stale frame number from first run

Low Severity

Both progressFromResult (worker) and getProgressFromResult (local) derive frameNumber and time from simulator.getRunSummary(0), i.e. the first run's individual state. The old code used the distribution metric's latest frame, whose frameNumber came from the simulator-level context.frameNumber—the global counter incremented by every advanceAll() call. If the first run errors while others continue, the reported frameNumber/time in progress will be stale, because a completed or errored run's frame number stops advancing while the simulator-level frame count keeps going.

Additional Locations (1)

• libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/runtime/experiment.ts#L173-L186

 

^{Reviewed by Cursor Bugbot for commit 09d211d. Configure here.}

[cursor]

Local experiment recopies all metric frames every batch

Low Severity

getMetricsState rebuilds the entire frames array via metrics.flatMap(m => [...m.frames]) on every syncStores call, which runs after each batch in the local run loop. As frames accumulate over a long experiment, each batch copies an ever-growing history, making total work quadratic. The worker path's appendMetricFrames has similar spreading costs but only appends new frames, making intent clearer and avoiding re-reading from metric objects.

 

^{Reviewed by Cursor Bugbot for commit 09d211d. Configure here.}