Skip to content

Optimize a parameter sweep from its Parameters card - #9650

Draft
kube wants to merge 4 commits into
claude/opt-proto-shared-resultsfrom
claude/opt-proto-experiment-optimizer
Draft

Optimize a parameter sweep from its Parameters card#9650
kube wants to merge 4 commits into
claude/opt-proto-shared-resultsfrom
claude/opt-proto-experiment-optimizer

Conversation

@kube

@kube kube commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Important

Experimental
Behind the Parameter sweeps and In-browser optimization feature flags.

Summary

Before this PR, a parameter sweep computed on its own from the moment it was created: the selection ran over the full ranges, and the Surface card sampled an eleven-by-eleven grid of the two shown parameters in the background, coarse to fine, on a CPU lane of its own. Optimizing the same ranges meant leaving the Experiments tab, recreating the search in the Optimizations tab, and running a study that computed everything again.

This PR makes a sweep compute only what is selected, and lets its Parameters card hand the selection to the optimizer. A fresh sweep sits idle until a control moves, the plot is clicked, or Optimize is pressed. The Surface draws the points the sweep has computed, dots with the field interpolated between them and a ring at the point computing. Optimize asks for a metric, a direction and a step count, then starts an in-browser study whose trials are evaluated through the sweep's own compute: each step moves the sliders to the suggested point, computes eight runs there, and reads the metric. While the study runs the card is purple with a glow, its controls follow the steps disabled, and Stop ends it; a finished study parks the sweep on the best point, which then refines to the run budget.

Links

Changes

Sweep session

  • Sweeps start idle

    A session created with startComputing: false publishes once as idle. The first selection starts the ladder.

  • navigateTo with a runCap resolves once the point has its runs

    The ladder stops at the cap instead of the run count, so a trial never pipelines a rung the next trial would abort.
    Resolves with the point's per-metric values, or null when another navigation superseded it, a batch failed, or the session was disposed. A plain setSelection lifts the cap.

  • Every folded point is a visited cell

    SweepSessionUpdate.visited lists { position, runsCompleted, means } in first-visit order, a new array only when a batch folds.

  • Background sampling is gone

    sampleCells, cell-batch.ts, the surface and refine batch kinds, whenSelectionStreamed, the CPU lanes and initialMarkingKey are deleted; InstantiateSweepBatch keeps the foreground fields only.

  • Experiments context exposes navigateSweep and sweep.visited

    sampleSurfaceCells is removed from the context, the default value and every fake.

Surface

  • SweepSurface draws visited points

    visitedSurfaceField projects every visited point onto the shown axes as a dot with the shown metric's value, the selected one emphasized, a muted ring where the metric never reported.
    computingSurfaceField rings the point computing and feeds its running value into the field. describeVisitedSurface writes the caption.

  • surfaceGridCoordinate and mergeSurfaceFields move to shared/surface-field.ts

    Both surfaces map an axis position onto the contour's index space through the same helper; the optimization plot re-exports them.

  • Surface is display-only while a study drives the sweep

Optimizer

  • createOptimization with a sweep option evaluates trials through a sweep

    New layer react.optimizations.sweep-evaluator: each trial quantizes the suggestion onto the sweep's axes, calls navigateSweep with the trial cap and returns the metric's value at the point, or a pruned outcome naming why.
    The provider routes such a run's evaluateTrial in front of the channel, records origin: { kind: "sweep", experimentId }, creates no connected study, opens no drawer, and parks the sweep uncapped on the best point when the study settles, or on the point it was trying when it is stopped.

  • Optimizations list hides studies with an origin

Parameters card

  • SweepOptimizeControl in the card's header

    Idle: an Optimize button opening a popover with the metric, Maximize or Minimize, the step count and Start; an error from the manifest shows inline.
    Driving: a purple chip counting the steps and a Stop button.

  • useSweepOptimizer builds the manifest from the experiment

    Swept axes as linear optimize bindings, every other scenario parameter fixed at its default, the chosen metric as the sole Metric, seedsPerTrial at the trial's eight runs. Available when the in-browser optimizer is on and the sweep has a saved scenario and a metric.

  • SweepNavigator follows a study

    status.following disables the controls and reads Following step N of M; an idle sweep reads move a control or click the surface to compute a point.

  • optimizing chart card tone

    Purple border and ground, purple title, the petrinautOptimizingGlow halo, off under reduced motion. FrameCard and ResultsBand carry a tone.

  • Removing the experiment removes its studies

Known issues

  • A trial's point computes eight runs whatever the run count

    The cap is a constant; the popover offers no runs-per-step field yet.

  • Ad-hoc sweeps get no Optimize button

    Their scenario is never persisted, so the manifest has no scenario to bind.

  • Points on the Surface keep every visit regardless of the hidden axes

    Projected like a study's trials; a slice view is not offered.

Next steps

  • Runs per step in the prompt
  • Ad-hoc sweeps through a synthesized scenario
  • Continue and Pause for a sweep's study

Test coverage

  • sweep-session.test.ts:

    Idle start publishes once and waits; navigateTo stops at the cap, resolves the point's values, publishes it as visited, resumes from cache, and resolves null when superseded or disposed; the ladder resumes when a plain selection lifts the cap.

  • create-sweep-trial-evaluator.test.ts:

    Quantizing suggestions onto the axes, the cap passed to navigateSweep, pruned outcomes for a superseded point, a missing metric and a settled study, and the settle navigation to the best point.

  • visited-field.test.ts:

    Fractional grid coordinates, emphasis on the selected point, muted rings, the computing ring with its running value, the caption.

  • experiment-results.test.tsx:

    The card's tone, the control in trailing when the optimizer is available, the surface's following and tone while a study drives.

  • view-optimization-drawer.test.tsx, view-experiment-drawer.test.tsx, drawer-frame.test.tsx, chart-card.test.tsx:

    Existing suites over the shared cards and both drawers.

  • Storybook:

    Simulate / ViewExperimentDrawer gains Sweep, optimizer available and Sweep, optimizer driving; Simulate / SweepSurface shows visited points, an empty sweep and the following state; Simulate / SweepNavigator gains Following the optimizer.

How to test

  • Open Petrinaut preview on Vercel
  • Viewport controls > Settings > Simulation > Parameter sweeps, In-browser optimization
  • Menu > Load example > SIR Model
  • Simulate > Experiments > Create
  • Max runs per selection 100, Scenario Seasonal Flu, Sweep on population and infected_ratio
  • Add metric > Place tokens, Place Infected
  • Run

    Expect the drawer to open on Idle, empty charts, an empty Surface reading no points yet, the sliders spanning their ranges, Optimize on the Parameters card

  • Click the Surface

    Expect the point to compute, a ring then a dot on the Surface, the chart streaming, 100 of 100 runs — fully sampled

  • Parameters > Optimize > Minimize, Steps 12 > Start

    Expect the card purple with a glow, the sliders locked and moving step by step, Following step N of 12 — 0 of 8 runs, a dot per step on the Surface, Optimizing · step N of 12 in the card header

  • Wait for the last step

    Expect the card back to default, the sliders on the best point, that point refining from 8 to 100 runs

  • Parameters > Optimize > Start
  • Stop in the card header

    Expect the search to end where it stands, the controls freed, that point refining to 100 runs

  • Simulate > Optimizations

    Expect no row for the sweep's study

…d let an optimizer evaluate trials through it
… in purple and document the idle sweep and its visited-point surface
@kube kube self-assigned this Sep 10, 2026
@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 10, 2026 9:51pm UTC
petrinaut Ready Ready Preview Sep 10, 2026 9:51pm UTC
petrinaut-docs Ready Ready Preview Sep 10, 2026 9:51pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 10, 2026 9:51pm UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/apps > hash.design Affects the `hash.design` design site (app) labels Sep 10, 2026
@kube
kube added this pull request to stack #9549 September 10, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash.design Affects the `hash.design` design site (app) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

1 participant