Skip to content

Smooth the experiments and optimizer flow - #9653

Draft
kube wants to merge 10 commits into
claude/opt-proto-experiment-optimizerfrom
claude/opt-proto-smooth-flow
Draft

Smooth the experiments and optimizer flow#9653
kube wants to merge 10 commits into
claude/opt-proto-experiment-optimizerfrom
claude/opt-proto-smooth-flow

Conversation

@kube

@kube kube commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Important

Experimental
Behind the Parameter sweeps and In-browser optimization feature flags.
Drawer header, results card and study drawer fixes ship for everyone.

Summary

Before this PR, the results drawers carried the seams of their construction. The header condensed and re-expanded on every wheel tick when the body overflowed by less than the header gave back, and its title line cut while the strip folded. A 250 ms clock re-rendered the whole drawer. One failed rung marked the sweep failed for the rest of its life, and Cancel mid-batch left it computing. A sweep read Idle in its header while the optimizer drove its sliders, showed the run budget in its Runs column, and labelled the optimizer's batch Selection. Sweep studies were invisible to cancel, pause and remove. A remote study's Best parameters card and Steps table appeared mid-stream. The objective chart was destroyed on every width change. Around all of it, helpers, types and fixtures existed two or three times.

This PR is the outcome of an audit of the experiments and optimizer stack for smoothness: seven review lenses, each finding verified twice, and the confirmed set implemented as eight commits. The drawer header condenses with hysteresis and crossfades its title line. Tiles, clock and chart no longer churn the drawer. Failures scope to the selection that failed, Cancel leaves an idle sweep, and sweep studies are first-class in the optimizations provider. A sweep's header reads Optimizing with a progress bar counting steps, its strip shows Selection, one purple button toggles between Optimize and Stop, the navigator reports the settled outcome, and a surface pick always lands a point. Remote study cards reserve their place with dashes. Every duplicated helper, type and fixture has one home.

Links

Changes

Sweep session and optimizer bridge

  • Failure scoped to the selection that failed

    A failed rung marks its own generation; the next selection climbs the full ladder.
    navigateTo rejects with the batch's message when its own generation fails, and keeps resolving null for supersession, disposal and an empty snapshot.

  • Cancel leaves an idle sweep

    dispose() publishes the idle frame before it flips; the provider's onUpdate clears a stale error on the next healthy publish.

  • Batches ride on the session update

    The batch registry, onBatches and the one-variant batch kind are gone; every publish carries batches and selectionKey.

  • One trial run budget

    The evaluator reads seedsPerTrial from the manifest; SWEEP_TRIAL_RUNS is deleted. A rejected navigation fails the trial; settle is idempotent.

  • Sweep studies are connected studies

    Cancel, pause and remove reach the evaluator; the source teardown clears them.

  • Connected study

    A pause that lands after the last ask upgrades to complete. The followed trial's frames filter once per event. Stream-driven publishes throttle at 100 ms.

Drawer frame

  • Header condenses with hysteresis

    Condenses only when the body still overflows after the header shrinks; stays condensed until the overflow is gone.

  • Title line crossfades

    Headline and compact stats share one grid cell, always mounted, swapped by opacity under the animate switch. The folded strip stays reachable by keyboard and grows the header on focus.

  • One Fold

    Header strip and card footer share shared/drawer-frame/fold.tsx; FrameBand is deleted.

  • Overflow fade in CSS

    The strip's right-edge mask is a scroll-driven animation under @supports (animation-timeline: scroll()); use-overflow-end.ts is deleted.

  • Every transition gated on the animate switch

    FrameAnimateContext wraps header, body and footer; progress fills, the computing popover and the optimizing halo read it.

  • Optimizing halo on a pseudo-element

    The glow animates opacity on ::after; the card keeps its static ring and tint.

  • Layer and slots

    DrawerFrame lives in shared/drawer-frame/frame.tsx, so the layer covers its folder only. Kit slots accept null; shared/results.tsx is deleted and consumers import the view and model directly.

Results drawers

  • Tiles stable across publishes

    useExperimentMetricTiles caches on the two record fields it reads.

  • Clock in a leaf

    ElapsedStat owns its 250 ms interval; useNow is deleted.

  • Objective chart resizes in place

    setSize on width change; the plot is created once per height.

  • Remote study cards reserved

    Best parameters lists the optimized bindings with until the first step reports; the Steps table is always present.

  • Steps described once

    describeStep returns { verdict, head, detail }; the clearr and unconstrainedd typos are gone with the slice(-1) that produced them.

  • Shared study helpers

    partitionParameterBindings, study-labels.ts, describe-study-progress.ts, constraintNameIn in constraint-rates.ts, one infeasible colour, optimizationDisplayStatus with stopped and reconnecting.

  • Sensitivity rows in binding order
  • Surface axes survive the drawer to full-view swap

    OptimizationNavigation.surfaceAxes replaces the component state.

  • Footer secondary slot

    Open full view and Show in drawer sit at the left under space-between; nothing slides when the primary actions change.

Sweep drawer

  • Header driven by the study

    Status reads Optimizing, Cancel is offered and stops the study, the progress bar counts steps, the computing chip labels the step's batch Step N.

  • Honest columns

    A sweep shows Selection instead of Runs and no Elapsed; Time reads 0 before anything computed.

  • One purple button

    Optimize with sparkles becomes Stop with stop, same size and variant.

  • Settled outcome under the sliders

    Finished 30 steps · best step so far: step 12 (650.500) or Stopped after …, with the parked point's sampling while it refines.

  • A failed study reports in the note row
  • Surface read-only while a study drives

    The X, Y and Metric pickers lock and a purple Read-only mark sits beside them; the plot shows a not-allowed cursor. A cancelled sweep locks the same way in grey.

  • Surface picks land a point

    Hidden axes collapse to their midpoint. The computing ring is marker-only; captions read computing the selected point or sampling across the selected ranges.

  • List and create drawer

    A fresh idle sweep is grey in the list; the create button reads Create sweep when a parameter is swept.

Cleanup

  • One PetrinautOptimizationDirection exported from core replaces three spellings
  • EXPERIMENT_RUN_LADDER is as const
  • Story fixtures share the short study triplet; duplicated test helpers and one-line smoke tests are deleted
  • Dead exports, constants and the test-only layout probes are gone

Known issues

  • Safari clips the halo's pulse

    No overflow-clip-margin, so only the static ring and tint show there.

  • A paused sweep study cannot resume

    Pause is not offered for sweep studies in the UI; through the API it would settle the evaluator for good.

  • The metric timeline still recreates its uPlot on size

    Same pattern the objective chart left; the file is outside this stack's diff.

Next steps

  • Move experiment-metric-timeline under shared/
  • Keep volatile samples out of the study surface's accumulator
  • Inject optimization actions through the study results dependencies
  • Adopt Fold outside the drawer frame

Test coverage

  • sweep-session.test.ts:

    A failed selection followed by a healthy one climbs the full ladder; a failing generation rejects its own navigation; dispose mid-batch ends idle; batches on the update; the live merge cache hits on a progress tick.

  • create-sweep-trial-evaluator.test.ts, provider.test.tsx (optimizations):

    The manifest's seedsPerTrial as run cap, a rejected navigation failing the trial, cancel and remove reaching the evaluator, the sweep record created unselected.

  • connected-study.test.ts:

    Pause upgrading to complete, frame identity across progress ticks, the throttled publish.

  • drawer-frame.test.tsx, drawer-frame-test-helpers.ts:

    Hysteresis at 30 px and 50 px of overflow, the compact echo inert, the folded strip focusable, the computing popover closing with its last batch.

  • experiment-results.test.tsx, view-experiment-drawer.test.tsx:

    Optimizing status and Cancel while driving, Step N batch labels, the study error in the note, the settled outcome, Selection in place of Runs, Time 0 / 180 on a fresh sweep, tiles keeping their reference.

  • study-results.test.tsx, constraint-summary, importance-view.test.ts, optimization-status tests:

    Reserved remote cards, the structured step description, binding order, stopped and reconnecting labels.

  • visited-field.test.ts, surface-sampling tests:

    Marker-only computing field, range and following captions, surfaceColumnCount.

  • Storybook:

    Simulate / ViewExperimentDrawer sweep stories show the one-button control, the Optimizing header and the settled line; Simulate / ViewOptimizationDrawer remote story shows the reserved cards.

How to test

  • Open Petrinaut preview on Vercel
  • Viewport controls > Settings > Simulation > Parameter sweeps, In-browser optimization
  • Menu > Load example > SIR Model
  • Simulate > Experiments > Create
  • Max runs per selection 100, Scenario Seasonal Flu, Sweep on population and infected_ratio
  • Add metric > Place tokens, Place Infected
  • Create sweep

    Expect the row grey Idle, the drawer header Time 0 / 180, a Selection column and no Runs or Elapsed column

  • Click the Surface

    Expect one point computing, the chart streaming, the caption computing the selected point

  • Parameters > Optimize > Minimize, Steps 12 > Start

    Expect the button to read Stop, the header Optimizing with the bar counting steps, Following step N of 12, the computing chip listing Step N, the Surface pickers locked beside a Read-only mark

  • Stop

    Expect the sliders freed, the line under them reading Stopped after N of 12 steps · …, the point refining to 100 runs

  • Optimize > Start, then Cancel in the footer

    Expect the study stopped with the sweep, the header Cancelled, the sliders locked

  • Simulate > Optimizations

    Expect the row's objective in full, Minimize Infected

@kube kube self-assigned this Sep 10, 2026
@kube
kube added this pull request to stack #9549 September 10, 2026 21:44
@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 11, 2026 2:43am UTC
petrinaut Ready Ready Preview Sep 11, 2026 2:43am UTC
petrinaut-docs Ready Ready Preview Sep 11, 2026 2:43am UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 11, 2026 2:43am UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/apps > hash.design Affects the `hash.design` design site (app) labels Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash.design Affects the `hash.design` design site (app) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

1 participant