Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
e5be96b
chore(quest): claim quest/m1/dual-format.md
kixelated Nov 17, 2026
8acb429
feat(scene): a portrait arrangement of each scene, drawn beside the l…
kixelated Nov 17, 2026
10512bb
feat(studio): BUILD edits a layer's place in the picture it shows
kixelated Nov 17, 2026
69b0df4
feat(studio): LIVE and BUILD show both pictures, pads show the pair, …
kixelated Nov 17, 2026
3b1021b
feat: the format picker, go-live checks, and docs for both formats at…
kixelated Nov 17, 2026
2ebc82d
chore: the dual-format views, for the PR body
kixelated Nov 17, 2026
697a46f
chore: drop the dual-format views, linked from the PR body
kixelated Nov 17, 2026
0d3fa77
Merge remote-tracking branch 'origin/main' into quest/m1/dual-format
kixelated Nov 17, 2026
32a39b4
quest: time two program encodes for real, a follow-up of both formats…
kixelated Nov 17, 2026
c9641f7
fix: the review's dual-format findings
kixelated Nov 17, 2026
286b726
fix(studio): a drop read while its layers stopped going out in both l…
kixelated Nov 17, 2026
0de4ccc
Merge remote-tracking branch 'origin/main' into quest/m1/dual-format
kixelated Nov 17, 2026
8a06dcb
fix: the PR review's dual-format findings
kixelated Nov 17, 2026
b56c4a6
fix(scene): a scene source shows its landscape picture in portrait
kixelated Nov 17, 2026
442bfc2
fix(scene): what a scene draws counts a nested scene's landscape layers
kixelated Nov 17, 2026
3ae6b21
fix: the CodeRabbit review's dual-format findings
kixelated Nov 17, 2026
def3a7a
chore: the compact dual-format views, for the PR body
kixelated Nov 17, 2026
23746cd
chore: drop the compact dual-format views, linked from the PR body
kixelated Nov 17, 2026
98f71a9
fix(studio): a canvas that turns Both off deselects the layer too
kixelated Nov 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 80 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,12 +383,15 @@ window, so there the desktop picks where it opens.

- **Where it goes:** the show's relay and its failover relay, each with a
switch and its state, such as *Connected, 18 ms away*. A relay switched off
is left out of this stream.
is left out of this stream. A show going out in both formats says each
relay carries both broadcasts.
- **Checks**, filled in as they finish: whether each camera has a picture,
whether a microphone's level moves, whether the encoder keeps up with the
canvas (timed as [setup](#first-launch) times it), and whether the
connection to the first relay switched on carries the program (measured as
setup measures it; published scenes and sources share what is left). A failed check is one sentence and its fix,
setup measures it; published scenes and sources share what is left). A show
going out in both formats needs about twice of each: two frames' encoding in
a frame's time, and two videos' bitrate. A failed check is one sentence and its fix,
such as **Send 720p instead**, which changes the canvas, or **Check again**.
- **GO LIVE TO** the number of destinations switched on. A failed check never
stops you; only having none switched on does.
Expand Down Expand Up @@ -436,7 +439,7 @@ setting costs. Escape or a click outside closes it.

| Pane | Use it to |
| --- | --- |
| Show | Pick the format, landscape or portrait, which swaps the canvas's sides; the canvas size as 720p, 1080p, 1440p, or 4K, with the upload each wants, or **Custom** for exact numbers; the frame rate as 30, 60, or **Custom**; and, under **Scene sizes**, each scene's own size (see [Published scenes](#published-scenes)). Saved in the scene file, and undone like any edit. |
| Show | Pick the format, landscape or portrait, which swaps the canvas's sides; **Both at once**, which sends a portrait picture beside a landscape one (see [Both formats at once](#both-formats-at-once)); the canvas size as 720p, 1080p, 1440p, or 4K, with the upload each wants, or **Custom** for exact numbers; the frame rate as 30, 60, or **Custom**; and, under **Scene sizes**, each scene's own size (see [Published scenes](#published-scenes)). Saved in the scene file, and undone like any edit. |
| Stream | Set the relay, its token, the failover relay, the broadcast's name, and on demand (see [Stream settings](#stream-settings)). |
| Recording | Change or open the recordings folder, and pick the picture and the audio scene a recording holds, and whether each source records a file of its own too (see [Recording](#recording)). |
| Hotkeys | Bind keys, by what they are for: going live, ending the stream, the panic key, and recording; Studio mode, TAKE, and AUTO; each scene; and replay and clips (see [Hotkeys](#hotkeys)). |
Expand Down Expand Up @@ -719,6 +722,52 @@ appearance or removing their layer keeps them off when they reconnect, until
a new link. Since an appearance is your edit, it drops what you could redo.
Off, every guest waits under Add, as before.

### Both formats at once

A landscape show can go out in portrait too, for phones, beside the
landscape picture for desks and TVs: switch on **Both at once** in Settings,
Show. Each scene of the show's size then has two arrangements of the same
layers, and the studio draws, encodes, and sends both pictures. A scene of its
own size, such as a published portrait scene, stays one picture, in its own
shape, so it cannot go on program meanwhile. A scene given the show's size as
its own follows the show's size instead when you turn it on.

Turning it on is one edit you can undo, and starts each portrait arrangement
from the landscape one. The largest picture, covering a quarter of the canvas
or more, is the main one: it is cropped to the top 60% of the portrait canvas,
with the largest camera stacked under it, cropped to fill the rest, or fitted
there keeping its outline if it has a round or rounded shape. Without a
camera, the main picture covers the whole portrait canvas. A backdrop covering
the landscape canvas covers the portrait one, cropped. What sits on the main
picture, such as a lower third or a logo, keeps its place on it. A layer set
anywhere else keeps its place on the turned canvas, scaled to fit, until you
move it in portrait. A scene laid out by slots arranges them in a row in
portrait, or in a grid or inset if that is how it arranges them in landscape,
and picks its own portrait arrangement in BUILD's scene inspector. A layer
showing another scene shows that scene's landscape picture in portrait too,
placed and cropped like any other landscape source.

LIVE shows both pictures side by side, each in its tally frame, with its
format, size, and what its video sends under it. BUILD edits one picture at a
time: **Landscape** or **Portrait** under the picture picks which, and the other
shows small beside it, where a click edits it instead. A drag, Fit, Fill, the
inspector's X, Y, W, H, and crop all place the layer in the picture edited,
and leave its place in the other alone. A layer in a slot has one crop in both.
Each scene's pad shows the pair, and Studio mode's NEXT card shows both
pictures of the scene cued.

A layer is shown in both pictures, or only in one: **Format** in its
inspector, and the chip under the pictures naming the selected layer, picks
**Both**, **Landscape**, or **Portrait**. A layer shown only in landscape keeps
no portrait place. Switching **Both at once** off removes every layer shown
only in portrait and forgets every portrait place and arrangement, as one
edit.

Only a landscape canvas turns: picking Portrait in Settings, Show, switches
Both off. The portrait picture goes out as its own broadcast, at
`<broadcast>/portrait` (see [Output](#output)), so a published scene cannot
be named `portrait` meanwhile.

### Audio scenes

Audio from every source is mixed whether or not its scene is live, and cuts
Expand Down Expand Up @@ -1158,6 +1207,15 @@ items = [
{ source = "logo", rect = { x = 1600, y = 40, width = 280, height = 120 } },
]

[[scenes]]
name = "Game"
portrait = { arrangement = "row" } # its slots in portrait, while the show goes out in both
items = [
# Its place in the portrait picture, while the show goes out in both.
{ source = "game", rect = { x = 0, y = 0, width = 1920, height = 1080 }, portrait = { rect = { x = 0, y = 0, width = 1080, height = 1152 }, crop = { left = 0.2, right = 0.2 } } },
{ source = "logo", rect = { x = 1600, y = 40, width = 280, height = 120 }, format = "landscape" }, # shown only in landscape
]

[[scenes]]
name = "Portrait"
size = { width = 1080, height = 1920 } # its own size; the program's without one
Expand Down Expand Up @@ -1225,6 +1283,16 @@ default), and `delay`, in every audio scene holding it.
`published = ["camera", "mic"]`, a top-level key, names the sources
[published independently](#published-independently).

`portrait = true`, a top-level key, sends a portrait picture beside the
landscape one (see [Both formats at once](#both-formats-at-once)). Meanwhile,
in a scene of the show's size or `[[top]]`, a layer's `format`, `landscape`
or `portrait`, shows it only in that picture, both without one, and its
`portrait`, a `rect` and `crop`, places it in the portrait picture, where it
sits on the landscape canvas, fitted to the turned one, without one. A layer
in a slot, or shown only in landscape, takes no `portrait` place, and a
scene's `portrait = { arrangement = "..." }` arranges its slots there, `row`
without one.

Each entry of a scene's `items` is one of its layers, back to front, and each
`[[top]]` an always-on-top layer: the `source` it shows and its `rect` on the
canvas, in pixels, or, in a scene, its `slot`, `main` (one per scene) or
Expand Down Expand Up @@ -1319,7 +1387,12 @@ under the broadcast's path, while on air. A canvas under 360 lines has no
half size. Each published scene publishes the same way at
`<broadcast>/<scene name>`, at
its own size and half of it, its tracks sharing the connection's bandwidth
with the program's, and its own encode of the audio scene it carries.
with the program's, and its own encode of the audio scene it carries. A show
going out in both formats publishes its portrait picture the same way at
`<broadcast>/portrait`, at the canvas turned and half of it, with its own
encode of the audio scene on program. The sources published independently
are drawn into it, with no layer layout of their own. Each relay carries both broadcasts,
and GO LIVE's checks time the encoder and size the connection for both.
Timestamps come from the studio's own clock, so a source that stalls repeats
its last picture instead of holding up the stream.

Expand Down Expand Up @@ -1394,6 +1467,7 @@ encode engine for them all. The Health window counts the sessions open:
| The program's canvas size | 1 while on air |
| Its half size | 1 while someone watches it |
| Each published scene's own size and half size | as the program's |
| The portrait picture's, while the show goes out in both | as the program's |
| The recording's own size | 1 while recording |
| Each captured source or media file | 1 while used, as [above](#source-tracks) |

Expand Down Expand Up @@ -1771,8 +1845,8 @@ Below that it shows:
watches, since the relay pulls nothing otherwise. Source tracks never reach
the relay, so they are left out of the media.

Then, for the program's broadcast and each published scene's, each with its
own viewers:
Then, for the program's broadcast, the portrait picture's, and each published
scene's, each with its own viewers:

- **Watching**: the sessions subscribed to it, from the relays'
`.stats` broadcasts. A relay shares those only with a connection allowed to
Expand Down
8 changes: 8 additions & 0 deletions quest/m0/destinations.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,14 @@ the go-live sheet and live control the studio has, which land them for MoQ
relays. This quest adds its destinations to that sheet rather than
building a second way to start a stream.

Proposed 2026-11-17 by
[Landscape and portrait at once](https://github.com/moq-dev/studio/pull/200),
waiting on the maintainer there: a show going out in both formats has two
broadcasts, the program's and `<broadcast>/portrait`. MoQ relays carry both,
and each destination here names the one it takes, landscape by default, so a
phone-first platform takes the portrait picture. That picture is another
encoder-sharing key: two destinations taking it share its encoder.

## Required

- [moq's egress dial changes are pinned](/quest/m0/pin-egress-dial.md) - RTMPS that keeps `tcUrl`, and a dial policy the caller supplies
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/ux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ shows those pictures beside the frames they implement.
from capture, dockable into spare room but never docked by default
(rejected: a chat column in the main window; leaving chat to the browser).
- **Formats:** ✅ one format per show in m0, landscape or portrait, and the
layout follows it. [Both at once](/quest/m1/dual-format.md) is m1 (rejected:
layout follows it. [Both at once](https://github.com/moq-dev/studio/pull/200) is m1 (rejected:
designing every m0 page for two pictures).
- **Layout follows the picture:** ✅ the picture is larger than any other
part of the window but the title bar, as large as the window allows, and
Expand Down
9 changes: 9 additions & 0 deletions quest/m1/auto-place.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,15 @@ ready; local first with a hosted model as an opt-in tie-breaker).
reflow, eases any list of items from one placement to the next on the
program clock, keyed by what changed.

- **In each format:** while the show goes out in landscape and portrait at
once ([moq-dev/studio#200](https://github.com/moq-dev/studio/pull/200)),
the layer keeps out of the way in each picture on its own. The analysis
reads each picture, the slots are each canvas's own, and the slot chosen
is the layer's place in that picture: its `rect` in landscape, its
`portrait` place in portrait, which moq-scene's `Item::placed` sets. A
layer shown in one format only is placed in that one alone. Off in
portrait, it keeps the place turning **Both at once** on gave it.

Measure, per AGENTS.md: the frame time and GPU copies the analysis adds, and
that the encoder's load does not move. Should the signals prove too weak, a
small on-device model is a later quest, not a hosted one.
Expand Down
26 changes: 26 additions & 0 deletions quest/m1/dual-format-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# [XS] Watch one show on a phone and a desk

## Goal

The maintainer has watched one show going out in both formats on a phone and
on a desktop at once, and each reads as made for its screen: the portrait
picture's main picture is cropped around what matters, the camera stacked
under it is not cut off, and overlays sit where a phone viewer sees them. If
it does not, how turning **Both at once** on places the portrait arrangement
changes until it does.

## Plan

[Landscape and portrait at once](https://github.com/moq-dev/studio/pull/200)
draws, encodes, and publishes the portrait picture at `<broadcast>/portrait`,
and its GPU and engine tests check the frames and the broadcast. Whether the
derived arrangement reads well on a real phone, and whether a viewer's player
picks the broadcast for its screen, are judgments no test makes.

## Waiting

- Maintainer: switch on **Both at once** in Settings, Show, on a show with a
game or a screen as its main picture and a camera, go live to a relay,
and watch `<broadcast>` on a desktop and `<broadcast>/portrait` on a phone.
Done when both read as intended, or the change the derivation needs is
written into this Plan.
32 changes: 32 additions & 0 deletions quest/m1/dual-format-encodes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# [S] Time two program encodes for real

## Goal

Setup and the go-live sheet say whether this machine keeps up with a show
going out in landscape and portrait at once from two encodes timed together,
not from one encode's time doubled. The numbers come from a real GPU encoder
with its session cap, such as NVENC on `nixos`'s RTX 3070 Ti, beside the
software encoders.

## Plan

Decided 2026-11-17 (maintainer, on
[moq-dev/studio#200](https://github.com/moq-dev/studio/pull/200)): ✅
estimate two encodes as twice one until this lands (rejected: timing two
encodes in that PR).

- `setup::Encoding::keeps_up_all` and `total_of` double one encode's frame
time and bitrate. Time the landscape and the turned canvas encoding at once
instead, as `setup::time` times one, and keep the doubled estimate only
where two sessions cannot open.
- A check that opens two hardware sessions holds both while it runs, so it
must finish before going live opens the program's, as the one-encode check
already does.
- Measure on `nixos` (NVENC, session cap 8, with CARLA and a transcode job
holding some) and on a software encoder: frame time and encoder load for
one encode and for two, per AGENTS.md. Record whether doubling over- or
under-estimates, and by how much.

## Hosts

- linux - the RTX 3070 Ti on `nixos`
39 changes: 0 additions & 39 deletions quest/m1/dual-format.md

This file was deleted.

40 changes: 40 additions & 0 deletions rs/moq-scene/src/clip.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,13 @@ impl Clip {
})
.chain(drawn)
.collect(),
// Copied from a show going out in both formats, its items
// may say where each picture shows them, which pasting
// keeps or drops as the destination does.
portrait: items
.items
.iter()
.any(|item| item.format.is_some() || item.portrait.is_some()),
..Collection::new()
}
.validate()?
Expand Down Expand Up @@ -262,6 +269,11 @@ impl Draft {
slot => (slot, item.rect),
};
let filters = rename(&item.filters);
// Where it shows in portrait means nothing to a layer drawn in one
// format, and a slot places it there too.
let dual = self.collection().dual(layer);
let format = item.format.filter(|_| dual);
let portrait = item.portrait.filter(|_| dual && slot.is_none());
self.insert_item(
layer,
index,
Expand All @@ -270,6 +282,8 @@ impl Draft {
filters,
slot,
rect,
format,
portrait,
..item.clone()
},
)?;
Expand Down Expand Up @@ -364,6 +378,32 @@ mod tests {
assert_eq!(pasted.top, collection.scenes[0].items);
}

/// A layer copied from a show going out in both formats reads back with
/// its portrait place, which a paste into a show going out in landscape
/// alone drops.
#[test]
fn a_layer_with_a_portrait_place_copies() {
let mut collection = collection();
collection.portrait = true;
let item = &mut collection.scenes[0].items[0];
item.portrait = Some(crate::Placement {
rect: Rect::fill(&collection.canvas.turned()),
crop: Default::default(),
});
item.format = Some(crate::Format::Portrait);
collection.validate().unwrap();
let clip = Clip::items(&collection, collection.scenes[0].items.clone(), Path::new(DIR));
let text = clip.to_toml().unwrap();
let pasted = paste(&collection, &Layer::scene("Scene"), &text, DIR);
assert_eq!(pasted.scenes[0].items[1], collection.scenes[0].items[0]);
let mut landscape = collection.clone();
landscape.portrait = false;
landscape.scenes[0].items.clear();
let pasted = paste(&landscape, &Layer::scene("Scene"), &text, DIR);
assert_eq!(pasted.scenes[0].items[0].portrait, None);
assert_eq!(pasted.scenes[0].items[0].format, None);
}

/// A layer the studio placed on its own is the host's once copied.
#[test]
fn a_copy_is_the_hosts() {
Expand Down
Loading