Skip to content

feat: bring desktop Studio UI and local playback to web - #2312

Open
richiemcilroy wants to merge 660 commits into
mainfrom
feature/web-editor-desktop-parity
Open

richiemcilroy wants to merge 660 commits into
mainfrom
feature/web-editor-desktop-parity

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

Brings the desktop Studio editor to the web as a pilot, with a browser recorder built around it. Recordings keep screen, camera, microphone and system audio as separate tracks, open straight into the editor, and can be styled, extended with more clips, exported locally or saved back to the same share link.

Studio is gated by CAP_WEB_EDITOR_STUDIO_ENABLED=enabled and the pilot account (isWebStudioEnabledForEmail). Other accounts keep the existing editor. No database migration.

Recorder

  • Editor-framed recorder (player, sources, tracks) that follows the system light/dark theme, with a system audio guide before the share picker and a "How does recording work?" explainer.
  • Draggable camera bubble (six anchors, size, shape, flip) whose layout carries into the editor; recording quality settings; desktop-style Start pill and recording bar; cancellable countdown.
  • While recording, the camera preview stays visible but dimmed (hidden when the capture can see this tab, so it isn't recorded twice). Restart asks first and reuses the shared screen, so cancelling never ends the recording. Pause and resume keep the timer right.
  • Stop opens the share link. A "Dashboard" back button and "How does recording work?" sit in the top bar.
  • Start, stop and upload finalisation run in parallel where they can; the next upload part is signed ahead of time.

Opening the editor

  • The editor opens as soon as the raw sources are uploaded instead of after share-video processing.
  • The editor bundle and renderer are prewarmed from the dashboard and cached as immutable assets; backgrounds decode with the browser's image decoder; timeline waveforms decode in a worker.
  • The share page video morphs into the editor's player through a view transition that waits for the editor page.
  • Older recordings stored without a duration have it measured once so they open.
  • Recordings are read through one shared ranged reader per file: head and tail are fetched in parallel from the size the bootstrap already gives, the probe and the preview share those reads, and a far seek in a long recording starts the decoder at a located fragment (a virtual file of the init segment plus everything from there) instead of reading every fragment before the target. Long mic and system WebM audio seeks the same way through MediaSource, falling back to the plain file. Storage that ignores ranges or errors falls back to plain reads instead of retrying.
  • Long recordings open without reading whole files: durations come from the last fragment or cluster, clip thumbnails decode from the reader the preview already has open instead of a <video> (which downloads a whole fragmented MP4), the waveform streams the mic once and is cached, and decoded frames go straight to the renderer. Sources up to 4K decode with WebCodecs.

Editing

  • Clip strip above the timeline: jump, reorder, rename, and add a recording, video or audio before, between or after clips. Record a clip shares the screen first and shows where the clip lands and the frame it follows on from.
  • Presets gallery (layouts, looks, social formats) previewed with the recording's own frames, next to saved presets and a one-click "Save current style". The card the project still matches shows as selected, and any card can be made the default in one click.
  • New recordings start full bleed (no background, padding or shadow) with the camera where it sat in the recorder, unless the owner chose a default.
  • Live audio waveform overlay drawn by the renderer (bars, mirrored, line, dots) with its own track, canvas handles and sidebar. Audio files import as-is and open as audio-only projects with the screen hidden.
  • Timeline for long recordings: opens showing the whole recording (no 10 minute cap), pinch, the slider and zoom buttons scale by factors (Mod+0 fits), the ruler steps up to hours, and timecodes show hours. Segments narrower than 2 px render as merged runs on one canvas per track instead of an element each, and pressing a run zooms in on it; per-frame work no longer scales with the number of segments (total duration is memoized, visibility snaps to coarse steps, the playhead restyles only when it moves a pixel, captions and transcript words update only on change).
  • Editor behaves like an app: no page zoom, swipe-back or stray text selection.

Save, export and import

  • Top bar, shared by the editor and the share page for Studio owners: the share link and Editor as one toggle. The link tab opens the share page, copies the link the way the share page does (rich HTML for email), and its green (public) or red (private) light opens the sharing settings. The editor header keeps Download (local export) and Save as the primary action.
  • Save is disabled while the share link shows the current version, and the save routes refuse to render a project the share link already shows.
  • Leaving with edits the share link doesn't show yet asks first.
  • Save uses the render farm first and falls back to rendering in the browser and replacing the share video when the farm or an editor worker isn't available, with Rendering and Uploading progress and specific error messages.
  • Browser exports stream to the origin private file system, compensate AAC priming, include waveforms, fall back to WebGL2, and estimate size from the previous export.
  • Imports: m4a works, long audio is about 3x faster, portrait videos keep their size, unreadable files are rejected before upload.

Rendering new recordings

  • A finished recording renders on the render farm in its default style, and keeps rendering when its editor is opened; saved edits replace it.
  • The share page plays the render as far as it has got, with a drifting fog over the frame until the first part is ready and while playback waits for the rest. The Cap card shows the same fog and progress.
  • Without the render farm, the share page the recorder opens publishes the recording by rendering it in a hidden editor, and a Save rendering in the browser reports its progress to the share page and Cap card. Following the share link during such a Save opens it beside the editor instead of stopping the render.
  • A Save rendering in the browser streams too: the encoders' packets also go into 2 s fragments that upload (in parallel, listed in order) as they're made, and the share page plays them as an HLS stream once two are up, then swaps to the final MP4 (index up front). Nothing is encoded or stored twice. Chunks stop streaming if the upload falls 64 MB behind, and are cleared an hour after they're uploaded.
  • Browser exports decode audio in 10 s blocks as they mix instead of whole tracks up front, so memory no longer grows with length, and an export that runs out of browser storage fails instead of publishing a truncated file.
  • After a render or reupload replaces the share video, its thumbnail and preview GIF are remade from it (new media server endpoint), under new keys so caches never serve the old ones.

Validation

  • Local (dev server, Apple silicon), medians of repeated runs: Stop to editor ready about 1.0 s (was 8.3 s); warm editor load 0.47 to 0.6 s for recordings, legacy MP4s, audio-only and wallpaper projects; scrubbing 32 to 37 fps with no multi-hundred-millisecond stalls; audio and video start within ±16 ms; a 147 s project exports 1080p30 in about 11 s.
  • Reliability replays: throttled and interrupted uploads, pause/resume, share ended from the browser, cancelled countdowns, cancelled and repeated exports, save while offline or with no farm, imports of corrupt, huge and unicode-named files.
  • 2 hour recording (1080p30 screen, 720p camera, mic), local: editor ready in 0.6 s (was 17.2 s) reading 1.8 MB of media (was about 3 GB); Full playback 60 fps with about 16% less CPU; a browser Save renders all 216,001 frames in about 10 minutes with the share page playing along the whole way and Chrome's largest process flat at about 2.6 GB; the published file is the full 7200 s. Audio from the new mixer is sample-identical to the old one.
  • Over an emulated slow link (150 ms to storage, 50 Mbps) the editor reaches its first frame in about 2.0 s for both 2 minute and 2 hour recordings (was 4.2 and 4.7 s), far seeks in the 2 hour recording take about 1 s (one previously took over 3 minutes), and audio starts about 1 s after a far seek (was 11.6 s). On the Vercel preview with the 2 hour recording, first frame is about 2.1 s in Chromium and 1.8 s in WebKit (was 7.0 and 4.4 s).
  • Timeline with a 2 hour project holding 2,400 captions: 135 timeline elements when zoomed out (was 20,931); pinching in or out has 0.25 to 0.36 s of long tasks in total (was 2.5 to 3.1 s) and scrolling stays under 10 ms per frame.
  • Chromium and WebKit (Safari's engine) both open, play and Save the 2 hour recording; the share page starts playing in under 0.5 s and seeks in under 50 ms in both.
  • apps/web unit vitest 3379 passing; editor-solid-web bun tests 79 passing; desktop editor vitest 377 passing; media-server video route tests 40 passing; tsc and Biome clean; cargo check for rendering, project, export, editor and the GPUI app.

Known gaps

  • Imports still wait for share-video processing before the editor opens.
  • GIF and file size optimisation exports need an editor worker.
  • A share page that is already open doesn't pick up a new Save until it reloads.
  • The media server's new preview-assets endpoint needs deploying, and videos already re-rendered in production need a one-off run of the preview refresh workflow.
  • On uplinks slower than the combined recording bitrate, long recordings can fall behind the upload.
  • Firefox is not covered by automated runs yet.

RetriggerConfidence Score: 2/5

The PR does not appear safe to merge while the previously reported private-image caching and concurrent image-deletion defects remain unresolved.

Findings

  1. P1 Preview editor is unreachable ▶
  2. P1 Security Private images remain cached ▶
  3. P1 Concurrent updates delete saved image ▶
  4. P2 Comment Narrates Selection Clearing ▶
  5. P2 Route bypasses required API pattern ▶
  6. P2 Comment repeats action behavior ▶
Fix with agent prompt
### Issue 1
apps/web/app/s/[videoId]/_components/link-preview/LinkPreviewDialog.tsx:undefined-100
The new dialog is exported but never rendered or opened by the share page. Although preview metadata and the save action are implemented, owners have no way to choose a title, description, or image through the UI, so they cannot use the feature.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 2
apps/web/app/api/video/link-preview/route.ts:104-108
When an anonymous visitor requests a public video's custom preview image, this response allows shared caches to keep it for 24 hours. If the owner then makes the video private, the image URL does not change, so a cache can continue serving the image without reaching the route's access check. **How this was verified:** The image route checks viewing permission at the origin but issues a 24-hour shared-cache response keyed only by the image version.

### Issue 3
apps/web/actions/videos/link-preview.ts:187-188
A reset and a text-only save can both read the same existing image. If the reset clears the preview first, the save can restore a reference to that image, after which the reset deletes it. The saved preview then points to a missing image because deletion does not check what the database currently references.

### Issue 4
apps/desktop/src/routes/editor/ConfigSidebar.tsx:774
“Clear any active selection first” only describes the check and update immediately below it. The repository requires comments to preserve non-obvious context, not narrate ordinary control flow. This requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 5
apps/web/app/api/video/link-preview/route.ts:41-61
This new API route defines an ad-hoc Next.js handler and calls `runPromise` inside it. The repository requires routes under `apps/web/app/api/*` to use the `HttpApi` class/group/endpoint pattern and export a handler through `apiToHandler(ApiLive)`. That requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 6
apps/web/actions/videos/link-preview.ts:94-98
This new JSDoc restates the save action's form fields and blank-field behavior. The repository requires comments to preserve non-obvious context rather than narrate adjacent code; the new image-card and preview-state descriptions follow the same pattern. This requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The latest changes connect the link-preview editor to the share page and adjust its mobile layout.

  • Owners can open the dialog from the Manage menu, with saved preview state supplied by the share page.
  • The mobile form section now retains its content height inside the scrollable dialog.

Reviews (268) · Last reviewed commit: "feat(web): open the link preview dialog ..."

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 2 security concern(s).

Comment thread .github/workflows/docker-build-editor-worker.yml
Comment thread apps/media-server/Dockerfile.editor Outdated
Comment thread apps/media-server/src/lib/editor-sessions.ts
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@greptile-apps

greptile-apps Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings could not be posted inline.

  • P2 Single-Quoted Warning String apps/web/app/(org)/dashboard/caps/components/web-recorder-dialog/useWebRecorder.ts:1020 ▶

    This warning uses a single-quoted TypeScript string, which violates the repository requirement that JavaScript and TypeScript string literals use double quotes. This requirement must be satisfied before merging.

    						? "System audio wasn't captured. Make sure \"Share tab audio\" is checked in the browser picker."
    

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread apps/media-server/src/lib/editor-signed-assets.ts
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread apps/web/lib/editor-worker-routing.ts
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread docker-compose.yml Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

Comment thread apps/chrome-extension/e2e/recording-upload.spec.ts Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread apps/desktop/src-tauri/src/import.rs
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

Comment thread apps/media-server/src/lib/editor-sessions.ts Outdated
Comment thread packages/editor-solid-web/src/browser-remote-media.ts Outdated
Comment thread apps/desktop/src/routes/editor/editor-skeleton.tsx Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

Comment thread packages/editor-solid-web/src/browser-local-playback.ts Outdated
Comment thread packages/editor-solid-web/src/browser-local-playback.ts Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

Comment thread apps/media-server/src/lib/editor-sessions.ts Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

}).pipe(runPromise);
} catch (error) {
console.error(
`[link-preview] Failed to delete old preview image for ${videoId}:`,
bytes = new Uint8Array(await response.arrayBuffer());
} catch (error) {
console.error(
`[video/link-preview] Failed to read preview image for ${rawVideoId}:`,
document.querySelector<HTMLVideoElement>("[data-edit-video] video") ??
document.querySelector<HTMLVideoElement>("video");

export function LinkPreviewDialog({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Preview editor is unreachable

The new dialog is exported but never rendered or opened by the share page. Although preview metadata and the save action are implemented, owners have no way to choose a title, description, or image through the UI, so they cannot use the feature.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/app/s/[videoId]/_components/link-preview/LinkPreviewDialog.tsx
Line: 100

Comment:
**Preview editor is unreachable**

The new dialog is exported but never rendered or opened by the share page. Although preview metadata and the save action are implemented, owners have no way to choose a title, description, or image through the UI, so they cannot use the feature.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +104 to +108
"Cache-Control": !anonymous
? "private, max-age=300"
: current
? "public, max-age=3600, s-maxage=86400, stale-while-revalidate=604800"
: "public, max-age=60, s-maxage=60",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Private images remain cached

When an anonymous visitor requests a public video's custom preview image, this response allows shared caches to keep it for 24 hours. If the owner then makes the video private, the image URL does not change, so a cache can continue serving the image without reaching the route's access check. How this was verified: The image route checks viewing permission at the origin but issues a 24-hour shared-cache response keyed only by the image version.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/app/api/video/link-preview/route.ts
Line: 104-108

Comment:
**Private images remain cached**

When an anonymous visitor requests a public video's custom preview image, this response allows shared caches to keep it for 24 hours. If the owner then makes the video private, the image URL does not change, so a cache can continue serving the image without reaching the route's access check. **How this was verified:** The image route checks viewing permission at the origin but issues a 24-hour shared-cache response keyed only by the image version.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +187 to +188
await writeLinkPreview(videoId, user.id, null);
if (existing?.image) await deleteImage(videoId, existing.image.key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Concurrent updates delete saved image

A reset and a text-only save can both read the same existing image. If the reset clears the preview first, the save can restore a reference to that image, after which the reset deletes it. The saved preview then points to a missing image because deletion does not check what the database currently references.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/actions/videos/link-preview.ts
Line: 187-188

Comment:
**Concurrent updates delete saved image**

A reset and a text-only save can both read the same existing image. If the reset clears the preview first, the save can restore a reference to that image, after which the reset deletes it. The saved preview then points to a missing image because deletion does not check what the database currently references.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +41 to +61
image = await Effect.gen(function* () {
const maybeVideo = yield* Effect.flatMap(Videos, (videos) =>
videos.getByIdForViewing(videoId),
);
if (Option.isNone(maybeVideo)) return null;
const [video] = maybeVideo.value;
const stored = readLinkPreview(
Option.getOrNull(video.metadata),
video.id,
)?.image;
if (!stored) return null;
// Paused while the owner doesn't have Cap Pro.
if (!(yield* Effect.promise(() => ownerServesLinkPreview(video.ownerId))))
return null;

const [bucket] = yield* S3Buckets.getBucketAccess(Option.none());
const url = yield* bucket.getInternalSignedObjectUrl(stored.key, {
expiresIn: 60,
});
return { url, key: stored.key, contentType: stored.contentType };
}).pipe(provideOptionalAuth, runPromise);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Route bypasses required API pattern

This new API route defines an ad-hoc Next.js handler and calls runPromise inside it. The repository requires routes under apps/web/app/api/* to use the HttpApi class/group/endpoint pattern and export a handler through apiToHandler(ApiLive). That requirement must be satisfied before merging.

Context Used: CLAUDE.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/app/api/video/link-preview/route.ts
Line: 41-61

Comment:
**Route bypasses required API pattern**

This new API route defines an ad-hoc Next.js handler and calls `runPromise` inside it. The repository requires routes under `apps/web/app/api/*` to use the `HttpApi` class/group/endpoint pattern and export a handler through `apiToHandler(ApiLive)`. That requirement must be satisfied before merging.

**Context Used:** CLAUDE.md ([source](https://github.com/capsoftware/cap/blob/main/CLAUDE.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +94 to +98
/**
* Sets a Cap's link preview. Form fields: `videoId`, `title`, `description`,
* an optional `image` file to replace the image, and `removeImage=1` to drop
* it. Blank text fields fall back to the defaults.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Comment repeats action behavior

This new JSDoc restates the save action's form fields and blank-field behavior. The repository requires comments to preserve non-obvious context rather than narrate adjacent code; the new image-card and preview-state descriptions follow the same pattern. This requirement must be satisfied before merging.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/actions/videos/link-preview.ts
Line: 94-98

Comment:
**Comment repeats action behavior**

This new JSDoc restates the save action's form fields and blank-field behavior. The repository requires comments to preserve non-obvious context rather than narrate adjacent code; the new image-card and preview-state descriptions follow the same pattern. This requirement must be satisfied before merging.

**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

editorState.timeline.audioPicker === null &&
editorState.timeline.audioReplace === null;
props.sheet?.setOpen(!showing);
// Clear any active selection first

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Comment Narrates Selection Clearing

“Clear any active selection first” only describes the check and update immediately below it. The repository requires comments to preserve non-obvious context, not narrate ordinary control flow. This requirement must be satisfied before merging.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop/src/routes/editor/ConfigSidebar.tsx
Line: 774

Comment:
**Comment Narrates Selection Clearing**

“Clear any active selection first” only describes the check and update immediately below it. The repository requires comments to preserve non-obvious context, not narrate ordinary control flow. This requirement must be satisfied before merging.

**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

This branch is being deployed

1 in progress and 1 active deployments
Preview — c7dd84b4 Deployed Oct 5, 2026 by vercel[bot]
Media Server / production — c7dd84b4 Deployed Oct 5, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants