Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 41 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ The **strip** holds:
| --- | --- |
| LIVE | Watch the program while on air, and TAKE or AUTO a cued scene. |
| BUILD | Edit the scene on program, or the one open off air, shown in the middle, with its layers and what can be added on the left and the inspector on the right (see [BUILD](#build)). |
| MIX | Add, rename, reorder, and remove audio scenes, pick the audio scene on program and the one to listen to and on which output, set each sound's gain, pan, mute, and delay, and send each sound and audio scene into audio scenes, with a meter on every one. The program shows small in the strip's place for the sound. |
| MIX | Ride each sound's fader and mute it, open its strip for its pan and delay, and send each sound and audio scene into audio scenes, from dots under the strips or as a matrix under **Routes**. Add, rename, reorder, and remove audio scenes, put one on the stream, and listen to one on the output you pick (see [Audio scenes](#audio-scenes)). The program shows small in the strip's place for the sound. |
| REVIEW | Pick a stretch of the last 30 seconds of the program or a source from a strip of its pictures, and replay it in a replay source, slowed down if you like, or save it as a clip (see [Replay](#replay)). Beside it, this show's recordings and clips as cards with their length and size, one repaired after a crash marked so, and **Open the folder**. The program's card is the live picture. |

The layout follows the program's shape and the window's size, giving the
Expand Down Expand Up @@ -495,27 +495,46 @@ audio scene holding every microphone, which both a "Stream" audio scene (Mics,
without music you may not broadcast) and a "Headphones" one (Mics and the
music) hold. A sound an audio scene reaches more than one way plays in it
once, and an audio scene never holds itself, even through others. Add,
rename, reorder, and remove audio scenes at the top of the MIX page.

The MIX page is a mixer. Each sound has a strip, with its gain, pan, mute,
delay, and meter, which apply in every audio scene holding it, and a send to
each audio scene; each audio scene has a strip with its meter and a send to
each other audio scene. A source added from BUILD's Add, or
pasted, is sent to the audio scene on program, so it is heard at once.
**On program** picks the audio scene the broadcast carries, and switching
it, or editing an audio scene, never drops or repeats audio. A recording holds
the audio scene on program unless Settings' Recording tab picks another, such as one
keeping music the stream leaves out, and each published scene carries the one
on program unless BUILD's inspector picks another beside its **Publish** switch;
switching either while it runs never drops or repeats audio either. The audio
scene on program cannot be removed; removing one a recording or published
scene picks returns it to the one on program.

**Listen** plays an audio scene to an audio output, from the `capture`
feature's build, as the strip's **Listen** plays the audio scene on program: the system default, or the one picked under **Listen on**,
which the studio remembers for you rather than in the scene file. Use
headphones: a microphone that hears the speakers feeds its audio scene back
into itself.
rename, reorder, and remove audio scenes on the MIX page.

The MIX page is a mixing desk. Each sound has a strip: its name, a meter, a
fader for its gain in decibels (unity is marked, and a double click returns
to it), its gain to read or type, and its mute, which turns amber while it
is on. These apply in every audio scene holding the sound. Click a sound's
name to open its strip to its gain, pan (a double click centers it), and
delay, and the audio scenes it is heard in; the other strips narrow beside it
rather than hide. Each audio scene has a strip after the sounds with its
meter, on the same scale, and **Listen**. A meter's top is full scale, where
its last segment turns red: the sound clips.

Under each strip is a dot for each audio scene, marked with its initial:
filled where the strip is sent into it, the accent's color for the one the
stream carries; ringed where it is heard there only through another audio
scene; and faint where it is not heard. A click sends it in or takes it out.
An audio scene shows a dash under its own strip and under any it holds, which
it cannot be sent back into. **Routes** shows the same sends as a matrix, a
row for each sound and audio scene and a column for each audio scene, with
what each one holds at its foot.

A source added from BUILD's Add, or pasted, is sent to the
audio scene on program, so it is heard at once. The audio scene on program is
tagged **Stream hears this**; **Put on stream** on another's strip puts that
one on program instead, and switching it, or editing an audio scene, never
drops or repeats audio. **+** after the audio scenes adds one, holding
nothing. Right-click an audio scene's strip, or its column under **Routes**,
to rename it, move it left or right, or remove it. A recording holds the
audio scene on program unless Settings' Recording tab picks another, such as one keeping music the stream leaves out, which is tagged
**In the recording**, and each published scene carries the one on program
unless BUILD's inspector picks another beside its **Publish** switch; switching
either while it runs never drops or repeats audio either. The audio scene on
program cannot be removed; removing one a recording or published scene picks
returns it to the one on program.

**Listen** on an audio scene's strip plays it to an audio output, from the
`capture` feature's build: the system default, or the one picked from the tag
on the strip while it plays, which the studio remembers for you rather than
in the scene file. Use headphones: a microphone that hears the speakers feeds
its audio scene back into itself.

A source's **delay** holds it back in every audio scene, up to 2 s, so a microphone
lines up with a camera that lags it. A broadcast's picture already follows its
Expand Down
26 changes: 26 additions & 0 deletions quest/m0/pin-output-kind.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# [XS] moq-audio's output kind is pinned

## Goal

The `moq` submodule includes an output's kind on
`moq_audio::playback::Device`: whether the system reports it as headphones
or a headset, as speakers, or does not say.

## Plan

cpal 0.18, which moq-audio already uses, reports it as
`DeviceDescription::device_type` on PipeWire and WASAPI, and as `Unknown`
elsewhere. `playback::list` reads the description for the name already, so
the change upstream is a field beside `name`, mapped from cpal's type so
cpal stays out of moq-audio's API, with a test of the mapping.

Once the wait clears, advance it with `just upgrade-moq` in its own PR,
fixing whatever the new API breaks here.

## Waiting

- moq-dev/moq: `moq_audio::playback::Device` on moq's `main` reports an
output's kind. Check `rs/moq-audio/src/playback/device.rs` there. The
change has no quest in moq's tree yet; it was found by
[moq-dev/studio#178](https://github.com/moq-dev/studio/pull/178) and is
listed in its Follow-ups for `/quest-plan` there.
4 changes: 3 additions & 1 deletion quest/m0/ux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,9 @@ quest links its own.

An agent can prove a page matches its mock-up's layout, that nothing clips at
any window size, and what a frame costs. It cannot judge these, so each child's
PR hands them to the maintainer as `## Waiting` with what to look at:
PR adds a follow-up quest for the maintainer to try the page, such as
[Try the LIVE page](/quest/m0/ux/try-live.md), with a `Maintainer:` item saying
what to look at, rather than waiting on it:

- **Taste:** whether the page reads as bold and calm at once. The PR shows its
screenshots beside the mock-up frames.
Expand Down
25 changes: 25 additions & 0 deletions quest/m0/ux/listen-speakers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# [S] Listen warns on speakers

## Goal

**Listen** on the MIX page warns, on the listened audio scene's strip and
before it plays, when the output it plays on is not headphones and the audio
scene carries a microphone: the one feedback loop the studio cannot prevent.
Where the system cannot tell what an output is, it says nothing.

## Plan

Split from the MIX page, which landed Listen's output picker on the listened
audio scene's strip (`Panels::listen_tag` in `rs/moq-studio/src/ui/mixer.rs`)
but cannot tell headphones from speakers yet.

- `mix::Speaker` carries the output's kind from
`moq_audio::playback::Device`, and the system default's kind is the
default device's.
- The warning is amber, as everything that needs attention but is not an
error, and does not block Listen: a speaker in another room is fine.
- A headset counts as headphones.

## Required

- [moq-audio's output kind is pinned](/quest/m0/pin-output-kind.md) - whether an output is headphones, where the system can tell
55 changes: 0 additions & 55 deletions quest/m0/ux/mix.md

This file was deleted.

Binary file modified quest/m0/ux/mock/mix-mix.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified quest/m0/ux/mock/mix-strip.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 8 additions & 5 deletions quest/m0/ux/mock/mix.html
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,13 @@
/* masters: the audio scenes */
.strip.ms{width:auto;flex:1;background:linear-gradient(#1d1d27,#15151b);border-color:#3b3b4a}
.strip.ms .mt i{width:14px}
.strip.ms .hd{gap:10px;justify-content:center}
.strip.ms .hd{gap:6px;justify-content:center}
.strip.ms .nm2{font-size:18px;letter-spacing:.08em}
.strip.ms.pg{border-color:var(--accent);box-shadow:0 0 28px #4f8cff33}
.mtag{display:flex;align-items:center;gap:5px;height:18px;padding:0 7px;border-radius:4px;background:var(--p3);color:var(--dim);font:700 11px/1 var(--disp);letter-spacing:.1em;text-transform:uppercase;white-space:nowrap}
.mtag.air{background:var(--accent);color:#fff}
.mtag.sel{background:#4f8cff2b;color:#8db4ff}
.mtag.ghost{background:var(--p2);box-shadow:inset 0 0 0 1px var(--line)}
.mtag svg.i{width:12px;height:12px}
.addms{width:40px;flex:none;border:2px dashed var(--line);border-radius:12px;display:grid;place-items:center;color:var(--dim)}

Expand Down Expand Up @@ -180,13 +181,15 @@
const dot=(c,i,l=1)=>`<i class="sd ${{d:'d',v:'v','-':'',x:'x'}[c]} ${i==0?'pg':''}">${c=='x'||!l?'':SC[i][0]}</i>`;
const sends=s=>`<div class="sends">${[...s].map((c,i)=>dot(c,i)).join('')}</div>`;
const scale=`<div class="sc">${[['+6',0],['0',14,'u'],['6',28],['12',42],['24',58],['36',72],['48',86],['60',100]].map(([t,y,c])=>`<span class="${c||''}" style="--y:${y}%">${t}</span>`).join('')}</div>`;
const fz=s=>`<div class="fz"><div class="mt"><i style="--n:${seg(s.l)};--p:${seg(s.pk)}"></i><i style="--n:${seg(s.r)};--p:${seg(s.pk-3)}"></i></div>${scale}<div class="fdr" style="--t:${s.t}%"><i></i></div></div><span class="db">${s.db} dB</span>`;
const fz=s=>`<div class="fz"><div class="mt"><i style="--n:${seg(s.l)};--p:${seg(s.pk)}"></i></div>${scale}<div class="fdr" style="--t:${s.t}%"><i></i></div></div><span class="db">${s.db} dB</span>`;
/* an audio scene has no gain or mute of its own: its meter on the same scale, and Listen */
const mfz=s=>`<div class="fz"><div class="mt"><i style="--n:${seg(s.l)};--p:${seg(s.pk)}"></i></div>${scale}</div><span class="db" style="visibility:hidden"></span>`;
const strip=s=>`<div class="strip ${s.m?'muted':''}" data-id="${s.id}">
<div class="hd"><div class="ic" style="background:${s.bg}">${s.ic}</div><span class="nm2">${s.n}</span></div>${fz(s)}
<div class="bts"><div class="bt ${s.m?'off':''}">${I(s.mi||'vol','sm')}</div></div>${sends(s.s)}</div>`;
const master=s=>`<div class="strip ms ${s.pg?'pg':''}" data-id="${s.id}">
<div class="hd"><div class="row" style="gap:7px"><i class="sd d ${s.pg?'pg':''}">${s.id}</i><span class="nm2">${s.n}</span></div>${s.tag}</div>${fz(s)}
<div class="bts"><div class="bt">${I('vol','sm')}</div><div class="bt ${s.on?'on':''}">${I('phones','sm')}</div></div>${sends(s.s)}</div>`;
<div class="hd"><div class="row" style="gap:7px"><i class="sd d ${s.pg?'pg':''}">${s.id}</i><span class="nm2">${s.n}</span></div>${s.pg?'':'<span class="mtag ghost">Put on stream</span>'}${s.tag}</div>${mfz(s)}
<div class="bts"><div class="bt ${s.on?'on':''}">${I('phones','sm')}</div></div>${sends(s.s)}</div>`;
const legend=`<div class="lg"><span>Where each sound is heard</span><i class="sd d"></i>sent<i class="sd v" style="margin-left:8px"></i>through another audio scene<i class="sd" style="margin-left:8px;box-shadow:inset 0 0 0 1px var(--line)"></i>not heard</div>`;
const sub=v=>`<div class="sub"><div class="seg"><span class="${v=='s'?'on':''}">STRIPS</span><span class="${v=='r'?'on':''}">ROUTES</span></div><div class="grow"></div>${legend}</div>`;
const desk=(open)=>`<div class="desk ${open?'c':''}">
Expand Down Expand Up @@ -246,7 +249,7 @@
</div></div>`;

const V={
'mix':{label:'MIX',cls:'M',note:'<b>MIX.</b> One strip per sound with a tall meter, a fader in decibels, and a mute; the audio scenes stand to the right as heavier master strips. The dots under a strip say where it is heard, one per audio scene: filled is sent, a ring arrives through another audio scene, and the accent is the one the stream hears. Any audio scene can be listened to with its headphone button. The dock stays underneath, so the scenes, the panic key, and the live picture stay on screen; the picture takes the slot where the sound strip sits on the other pages, since this page already is the sound.',
'mix':{label:'MIX',cls:'M',note:'<b>MIX.</b> One strip per sound with a tall meter, a fader in decibels, and a mute; the audio scenes stand to the right as heavier master strips, with their meters on the same scale but no fader or mute, since an audio scene has no gain of its own. The dots under a strip say where it is heard, one per audio scene: filled is sent, a ring arrives through another audio scene, and the accent is the one the stream hears. Any audio scene can be listened to with its headphone button. The dock stays underneath, so the scenes, the panic key, and the live picture stay on screen; the picture takes the slot where the sound strip sits on the other pages, since this page already is the sound.',
html:()=>topbar('MIX')+sub('s')+desk()+mixdock(),
after:f=>{air(f,'game');const d=f.querySelector('.strip[data-id=mic] .sends .sd.v');d.classList.add('hov');
f.querySelector('.strip[data-id=mic]').insertAdjacentHTML('beforeend','<div class="tip" style="left:40px;bottom:-22px">Desk mic reaches Headphones through Stream</div>')}},
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/ux/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ bar.
## Plan

- The mock-up lists a Sound pane. Its one setting, the output Listen plays
on, is on the [MIX page](/quest/m0/ux/mix.md) instead, so there is none.
on, is on the MIX page instead, so there is none.
The Guests pane has no frame; it follows the Stream pane's shape.
- Each setting keeps its home on disk: the scene file for what belongs to
the show (canvas, scene sizes, published scenes), per-user settings for
Expand Down
38 changes: 38 additions & 0 deletions quest/m0/ux/try-mix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# [XS] Try the MIX page

## Goal

The maintainer tries the MIX page on their own machines and either signs it
off or files what they find as quests: what the epic's
[What a person must check](/quest/m0/ux/README.md) asks of it, which no
headless run exercises.

## Plan

The MIX page (its strips, the audio scenes as master strips, the opened
strip, and Routes) is on `main` (`cargo run -p moq-studio --features
capture`; Listen and its output picker are disabled without `capture`).
What to try, beside the [mock-up](/quest/m0/ux/mock/mix-mix.webp):

- **Real devices:** a real microphone and headphones. **Listen** on an audio
scene, then pick the output from its **On …** tag; the meters moving with
real sound, the peak held, red only at a clip.
- **Feel:** riding a fader with a mouse and a trackpad, whether the taper is
fine enough near unity, a double click returning to it; mute and unmute
during a show; opening a strip while the others narrow.
- **Readability:** whether the send dots (filled, ringed when heard only
through another audio scene, faint when not heard) read at a glance to
someone who has never seen an audio scene. If they do not, **Routes** is
the fallback to point people at, and the legend's wording is the cheapest
thing to change.

An agent then turns each finding into a quest, or an edit to the quest that
owns that page, and deletes this one.

## Waiting

- Maintainer: try the MIX page as the Plan lists and report what to change,
or that nothing needs to. An agent cannot judge feel or taste, and the
agent hosts have no microphone or headphones; a kittest run already covers
the clicks (`shots::mix_sends_and_mutes`), and `just shots` the layout.
Check: a comment or quest records the result.
Loading