diff --git a/api/fishjam-server b/api/fishjam-server index 62073a78..80b73b6b 160000 --- a/api/fishjam-server +++ b/api/fishjam-server @@ -1 +1 @@ -Subproject commit 62073a78d3b6e176a37f52c32cacfc38681bc7f2 +Subproject commit 80b73b6bf84fc37e874ff120f0d37045ad0d6ed9 diff --git a/api/protos b/api/protos index 1a8a029d..244345c4 160000 --- a/api/protos +++ b/api/protos @@ -1 +1 @@ -Subproject commit 1a8a029d4ee99664ba5990c77df688e7bfe81ef5 +Subproject commit 244345c478e8cec97b5603268e16eb831a54e0df diff --git a/api/room-manager b/api/room-manager index 31836f4e..94c2b0b3 160000 --- a/api/room-manager +++ b/api/room-manager @@ -1 +1 @@ -Subproject commit 31836f4ed6c8551b3892cea50d2872cdf2292e71 +Subproject commit 94c2b0b3b21624d251d666b2000d514eccaad08b diff --git a/docs/explanation/moq-streaming.mdx b/docs/explanation/moq-streaming.mdx new file mode 100644 index 00000000..c582b967 --- /dev/null +++ b/docs/explanation/moq-streaming.mdx @@ -0,0 +1,139 @@ +--- +type: explanation +sidebar_position: 6 +title: MoQ Streaming +description: Understand how Media over QUIC (MoQ) works in Fishjam — the relay model, publish/subscribe architecture, paths, and token-based access control. +--- + +# MoQ Streaming with Fishjam + +_How Media over QUIC (MoQ) works in Fishjam_ + +## What is MoQ? + +[Media over QUIC (MoQ)](https://datatracker.ietf.org/wg/moq/about/) is a new internet standard for live media delivery, designed from the ground up for **scalable, low-latency streaming to large audiences**. + +Unlike WebRTC — which was primarily built for interactive, peer-to-peer conferencing — MoQ is optimized for the one-to-many broadcast model: one publisher, potentially thousands of simultaneous subscribers, all receiving the stream with minimal delay. + +A few properties make MoQ stand out: + +- **Built on QUIC.** QUIC is a modern transport protocol that eliminates head-of-line blocking, recovers from packet loss more gracefully than TCP, and establishes connections faster. For live video, this means more resilient delivery at low latency. +- **Standardized negotiation.** Because MoQ defines a common signaling and subscription protocol, any MoQ-compliant client can connect to any MoQ-compliant relay — you are not locked into a proprietary stack. +- **Relay-based architecture.** The relay is a first-class part of the MoQ protocol, not an add-on. Because relaying is built into the protocol's design, scaling delivery to large audiences is a native capability. + +:::info +Fishjam also supports WebRTC-based livestreaming (WHIP/WHEP). See [Livestreams](./livestreams) for that approach. +::: + +## How MoQ Works in Fishjam + +Fishjam provides a **MoQ relay** that publishers push media to and subscribers pull media from. + +``` +Publisher → Fishjam MoQ Relay → Subscriber(s) +``` + +The relay is responsible for distributing the stream: it receives media from the publisher once and fans it out to every subscriber. + +### The publish/subscribe model + +MoQ uses a **publish/subscribe** model: + +- A **publisher** connects to the relay, announces a stream under a specific path, and starts sending media. +- **Subscribers** connect to the relay, discover announced paths, and receive the media. + +The relay manages the flow between them. Neither side needs a direct connection to the other. + +### Paths + +Every stream is identified by a **path** — a slash-separated string like `my-room/alice-camera`. Paths are used in two distinct ways: + +1. **Addressing** — a subscriber consumes an exact path to receive that specific stream (e.g. `my-room/alice-camera`). +2. **Discovery** — a subscriber watches a prefix (e.g. `my-room`) to learn which streams are currently live under it. This returns a live feed of announced paths — each of which must then be consumed individually. This is how you can display all participants in a room without knowing their paths in advance. + +Note that consuming an exact path and discovering a prefix are separate operations. Consuming `my-room` directly would fail unless a publisher is broadcasting at that exact path. + +### Path Scoping + +Every connection goes to `relay.fishjam.io/`. Your Fishjam ID is automatically used as the token's root namespace by the Fishjam Server — you never include it in `publishPath` or `subscribePath`; it is set for you. All paths you specify are **relative to that root**. + +Path matching is **prefix-based**: a path of `"stream-name"` permits any broadcast whose full path starts with `stream-name/`, not just the exact string `"stream-name"`. + +#### Publisher paths + +The `publishPath` you set determines how much freedom the broadcaster has when naming their broadcast: + +- **Broad path** (`publishPath: "stream-name"`) — the client can publish as any sub-path under `stream-name`, such as `stream-name/alice` or `stream-name/bob-camera`. The client chooses its own identity; the relay only enforces the prefix. +- **Specific path** (`publishPath: "stream-name/alice"`) — the client can **only** publish as `stream-name/alice`. If the broadcaster tries to use `stream-name/bob`, the relay rejects the announcement. This is how you enforce a broadcaster's identity from the server side. + +Use the broad form when clients self-identify (e.g., users pick their own stream name). Use the specific form when your backend assigns identities (e.g., you issue a per-user token for a managed conference). + +#### Subscriber paths + +The `subscribePath` works the same way: it is a prefix that limits which broadcasts the subscriber can consume and discover. + +- **Broad path** (`subscribePath: "stream-name"`) — the subscriber can consume any broadcast under `stream-name/` and will surface all publishers in that namespace as they come and go. +- **Specific path** (`subscribePath: "stream-name/alice"`) — the subscriber can only receive from `stream-name/alice`. Broadcasts at `stream-name/bob` are invisible to this client. + +#### Example: a multi-publisher room + +A typical room setup uses a combination of both patterns: + +1. The backend issues each broadcaster a **specific** publisher token — `publishPath: "my-room/"` — so each user can only occupy their own slot. +2. The backend issues viewers a **broad** subscriber token — `subscribePath: "my-room"` — so they discover and consume every broadcast in the room. +3. When a new publisher joins or leaves, the viewer is informed by the relay + +## Access Control: MoQ Tokens + +Access to the relay is controlled by **MoQ tokens** — short-lived JWTs that are path-scoped: + +| Token type | Grants | Typical recipient | +| ---------------- | ---------------------------------- | ----------------- | +| Publisher token | Write access to a specific path | Streamer | +| Subscriber token | Read access to a path or namespace | Viewer | + +A token is attached to the relay URL as a query parameter (`?jwt=`). The relay validates the token and enforces its scope before allowing any media to flow. + +Keeping publisher and subscriber tokens separate ensures that a viewer can never accidentally publish to the stream, and a publisher cannot subscribe to paths it does not own. + +## Getting Tokens + +There are two ways to obtain MoQ tokens, depending on where you are in the development lifecycle. + +### Sandbox API (prototyping) + +The **Sandbox API** is a ready-made backend provided by Fishjam for development and prototyping. It issues tokens without requiring you to build your own server, so you can start streaming immediately. + +To get a publisher token, call: + +``` +GET https://fishjam.io/api/v1/connect/{FISHJAM_ID}/room-manager/moq/{PUBLISHER-PATH}/publisher +``` + +To get a subscriber token, call: + +``` +GET https://fishjam.io/api/v1/connect/{FISHJAM_ID}/room-manager/moq/{SUBSCRIBER-PATH}/subscriber +``` + +The Sandbox API is **not intended for production** — it has no authentication and is only available in the Sandbox environment. See [What is the Sandbox API?](./sandbox-api-concept) for more context. + +### Fishjam Server SDK (production) + +In production, your backend generates tokens using the **Fishjam Server SDK**. This gives you full control over who can publish and who can subscribe. + +The SDK's `createMoqToken` method accepts either a `publishPath` or a `subscribePath`: + +- `publishPath` — issues a publisher token scoped to that path. +- `subscribePath` — issues a subscriber token scoped to that path or namespace prefix. + +Your backend then delivers each token to the appropriate client (publisher or viewer), which uses it to connect to the relay. + +See the [MoQ Streaming tutorial](../tutorials/moq) for working code examples of both approaches. + +## See also + +- [MoQ Streaming tutorial](../tutorials/moq) — step-by-step guide to publishing and subscribing +- [What is the Sandbox API?](./sandbox-api-concept) — when and why to use the Sandbox API +- [Security & Token Model](./security-tokens) — broader overview of Fishjam's token system +- [Livestreams](./livestreams) — WebRTC-based livestreaming with WHIP/WHEP diff --git a/docs/tutorials/moq.mdx b/docs/tutorials/moq.mdx new file mode 100644 index 00000000..3d1d0ee3 --- /dev/null +++ b/docs/tutorials/moq.mdx @@ -0,0 +1,375 @@ +--- +type: tutorial +sidebar_position: 5 +title: MoQ Streaming +description: Stream live video and audio over Media over QUIC (MoQ) with Fishjam, from sandbox prototyping to production deployment. +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; + +# MoQ Streaming + +This section explains how to publish and subscribe to live streams using [Media over QUIC (MoQ)](https://datatracker.ietf.org/wg/moq/about/) with Fishjam. +MoQ uses QUIC as its transport layer (with a WebSocket fallback), delivering ultra-low latency at scale — making it ideal for interactive broadcasts, live events, and large-audience streams. + +:::info +If you're new to MoQ, then we recommend getting familiar with the [MoQ Streaming with Fishjam](../explanation/moq-streaming) explanation. +::: + +We show how to quickly prototype in [Quickstart with the Sandbox API](#quickstart-with-the-sandbox-api) and how to get ready for production in [Production MoQ with Server SDKs](#production-moq-with-server-sdks). + +:::info +Fishjam supports both WebRTC-based livestreaming (WHIP/WHEP) and MoQ streaming. \ +**MoQ** is a new protocol designed from the ground up for scalable, low-latency delivery to large audiences.\ +**WebRTC** is a mature, battle-tested technology built for low-latency peer-to-peer conferencing and interactive streaming. + +See [Livestreaming](./livestreaming) for the WebRTC approach. +::: + +## Quickstart with the Sandbox API + +If you don't have a backend server set up, you can prototype a MoQ streaming scenario using the Sandbox API. + +:::tip +MoQ is a protocol with a well-defined negotiation, so in theory any compliant MoQ client should work. That said, we recommend using the [`@moq`](https://github.com/moq-dev/moq) client libraries — the reference TypeScript implementation maintained alongside the protocol. For more details, see the [documentation](https://doc.moq.dev/js/). +::: + +### Publisher setup + +To start publishing a MoQ stream, you need two things: a _publisher token_ and the relay URL. + +#### Obtaining a publisher token + +Fetch a sandbox publisher token from the Room Manager: + +```ts +const FISHJAM_ID = "YOUR_FISHJAM_ID"; +const PUBLISHER_PATH = "stream/alice"; + +const response = await fetch( + `https://fishjam.io/api/v1/connect/${FISHJAM_ID}/room-manager/moq/${PUBLISHER_PATH}/publisher`, +); +const { token: publishToken } = await response.json(); +``` + +#### Connecting and publishing + +Install the MoQ packages: + +```bash npm2yarn +npm install @moq/lite @moq/publish +``` + +Use the token to connect to the Fishjam MoQ relay and start broadcasting: + +```ts +// @noErrors +import * as Moq from "@moq/lite"; +import * as Publish from "@moq/publish"; + +const FISHJAM_ID = "YOUR_FISHJAM_ID"; +const PUBLISHER_PATH = "stream/alice"; +const publishToken = ""; + +// ---cut--- +// Build the relay URL using the publisher token +const relayUrl = new URL( + `https://relay.fishjam.io/${FISHJAM_ID}?jwt=${publishToken}`, +); + +// Connect to the Fishjam MoQ relay +const connection = await Moq.Connection.connect(relayUrl); + +// Get camera and microphone access +const camera = await navigator.mediaDevices.getUserMedia({ + video: true, + audio: true, +}); + +// Set up a broadcast with video and audio tracks +const broadcast = new Publish.Broadcast({ + connection, + name: Moq.Path.from(PUBLISHER_PATH), + enabled: true, + video: { + source: camera.getVideoTracks()[0] as Publish.Video.Source, + hd: { enabled: true }, + }, + audio: { + enabled: true, + source: camera.getAudioTracks()[0] as Publish.Audio.Source, + }, +}); +``` + +The stream is now live on the MoQ relay! Viewers can now start watching the stream by following the steps in [Subscriber setup](#subscriber-setup) + +### Subscriber setup + +To receive a MoQ stream you need one thing: a _subscriber token_. + +#### Obtaining a subscriber token + +Fetch a sandbox subscriber token using the Sandbox API: + +```ts +const FISHJAM_ID = "YOUR_FISHJAM_ID"; +const SUBSCRIBER_PATH = "stream/alice"; + +const response = await fetch( + `https://fishjam.io/api/v1/connect/${FISHJAM_ID}/room-manager/moq/${SUBSCRIBER_PATH}/subscriber`, +); +const { token: subscribeToken } = await response.json(); +``` + +#### Connecting and subscribing + +Install the MoQ packages: + +```bash npm2yarn +npm install @moq/lite @moq/watch +``` + +Use the token to connect and receive the stream: + +```ts +// @noErrors +import * as Moq from "@moq/lite"; +import * as Watch from "@moq/watch"; + +const FISHJAM_ID = "YOUR_FISHJAM_ID"; +const SUBSCRIBER_PATH = "stream/alice"; +const subscribeToken = ""; + +// ---cut--- +// Build the relay URL using the subscriber token +const relayUrl = new URL( + `https://relay.fishjam.io/${FISHJAM_ID}?jwt=${subscribeToken}`, +); + +// Connect to the Fishjam MoQ relay +const connection = await Moq.Connection.connect(relayUrl); + +// Subscribe to the broadcast +const broadcast = new Watch.Broadcast({ + connection, + name: Moq.Path.from(SUBSCRIBER_PATH), + enabled: true, +}); + +// Attach to a canvas element — MultiBackend handles decoding and rendering +const canvas = document.querySelector("canvas")!; +const backend = new Watch.MultiBackend({ + element: canvas, + broadcast, + paused: false, +}); +``` + +That's it! The stream will appear in the canvas once the publisher starts broadcasting. + +## Production MoQ with Server SDKs + +The [Quickstart with the Sandbox API](#quickstart-with-the-sandbox-api) shows how to get MoQ streaming up and running quickly. +In a production scenario, your backend generates tokens with proper authorization, allowing you to control who can publish and who can subscribe. + +MoQ tokens are path-scoped: a publisher token grants write access to a specific path, and a subscriber token grants read access. Your backend needs to: + +1. Generate a publisher token for the path your broadcaster will use. +2. Generate a subscriber token for the path your viewers will use. +3. Deliver each token to the appropriate client. + + + + + ```ts + const fishjamId = ''; + const managementToken = ''; + + // ---cut--- + import { FishjamClient } from '@fishjam-cloud/js-server-sdk'; + + const fishjamClient = new FishjamClient({ + fishjamId, + managementToken, + }); + + const streamPath = 'stream/alice'; + + // Generate a token that allows publishing to 'stream/alice' + const { token: publishToken } = await fishjamClient.createMoqToken({ + publishPath: streamPath, + }); + + // Generate a token that allows subscribing to 'stream/alice' + const { token: subscribeToken } = await fishjamClient.createMoqToken({ + subscribePath: streamPath, + }); + ``` + + + + + + ```python + from fishjam import FishjamClient + + fishjam_client = FishjamClient( + fishjam_id=fishjam_id, + management_token=management_token, + ) + + stream_path = 'stream/alice' + + # Generate a token that allows publishing to 'stream/alice' + publish_token = fishjam_client.create_moq_token(publish_path=stream_path) + + # Generate a token that allows subscribing to 'stream/alice' + subscribe_token = fishjam_client.create_moq_token(subscribe_path=stream_path) + ``` + + + + +Deliver these tokens to your clients, then use them to connect as described in [Connecting and publishing](#connecting-and-publishing) and [Connecting and subscribing](#connecting-and-subscribing). + +:::tip +A single token should only grant either `publishPath` _or_ `subscribePath` — not both. +Streamers receive a publisher token; viewers receive a subscriber token. +This separation ensures a viewer can never accidentally publish to the stream. +::: + +### Subscribe to a namespace + +When multiple publishers join a room, you won't know their exact paths in advance. +Instead of consuming a single path, you can **discover** all broadcasts published under a namespace prefix and subscribe to each one as they appear. + +To do this, generate a subscriber token scoped to the room namespace instead of a single stream path. + + + + + ```ts + const fishjamId = ''; + const managementToken = ''; + + // ---cut--- + import { FishjamClient } from '@fishjam-cloud/js-server-sdk'; + + const fishjamClient = new FishjamClient({ fishjamId, managementToken }); + + const roomName = 'my-room'; + + const { token: alicePublisherToken } = await fishjamClient.createMoqToken({ + publishPath: roomName + "/alice", + }); + + const { token: bobPublisherToken } = await fishjamClient.createMoqToken({ + publishPath: roomName + "/bob", + }); + + + const { token: namespaceToken } = await fishjamClient.createMoqToken({ + subscribePath: roomName, + }); + ``` + + + + + + ```python + from fishjam import FishjamClient + + fishjam_client = FishjamClient( + fishjam_id=fishjam_id, + management_token=management_token, + ) + + room_name = 'my-room' + # Publishers under this namespace might use paths like: + # 'my-room/alice-camera', 'my-room/bob-screen', etc. + + namespace_token = fishjam_client.create_moq_token(subscribe_path=room_name) + ``` + + + + +Then on the client, call `connection.announced()` to receive a live feed of announce events. Each event tells you whether a path is becoming **active** (a publisher just joined) or going inactive (a publisher disconnected). Loop over them to subscribe and clean up as publishers come and go: + +```ts +// @noErrors +import * as Moq from "@moq/lite"; +import * as Watch from "@moq/watch"; + +const FISHJAM_ID = "YOUR_FISHJAM_ID"; +const ROOM_NAME = "my-room"; +const namespaceToken = ""; + +// ---cut--- +const relayUrl = new URL( + `https://relay.fishjam.io/${FISHJAM_ID}/${ROOM_NAME}?jwt=${namespaceToken}`, +); + +const connection = await Moq.Connection.connect(relayUrl); + +// Keep references so we can clean up when a publisher goes offline +const slots = new Map< + string, + { + broadcast: Watch.Broadcast; + backend: Watch.MultiBackend; + canvas: HTMLCanvasElement; + } +>(); + +const announced = connection.announced(); + +while (true) { + const entry = await announced.next(); + if (!entry) break; // connection closed + + const key = entry.path.toString(); + + if (entry.active) { + // New publisher — subscribe and render + const broadcast = new Watch.Broadcast({ + connection, + name: entry.path, + enabled: true, + }); + + const canvas = document.createElement("canvas"); + document.body.appendChild(canvas); + + const backend = new Watch.MultiBackend({ + element: canvas, + broadcast, + paused: false, + }); + + slots.set(key, { broadcast, backend, canvas }); + } else { + // Publisher disconnected — tear down its slot + const slot = slots.get(key); + if (!slot) continue; + slot.backend.close(); + slot.broadcast.close(); + slot.canvas.remove(); + slots.delete(key); + } +} +``` + +## See also + +If you want a better understanding of how MoQ works, see: + +- [MoQ Streaming with Fishjam](../explanation/moq-streaming) + +If you want to learn about streaming using WebRTC instead of MoQ, see: + +- [Livestreaming](./livestreaming) +- [WHIP/WHEP with Fishjam](../how-to/backend/whip-whep) diff --git a/packages/js-server-sdk b/packages/js-server-sdk index 0007dea8..ad25200e 160000 --- a/packages/js-server-sdk +++ b/packages/js-server-sdk @@ -1 +1 @@ -Subproject commit 0007dea898b99b6923059b364584054cf1580b02 +Subproject commit ad25200e98571af994b8d46ff284063493d800cd diff --git a/packages/python-server-sdk b/packages/python-server-sdk index 2f18378a..0889d548 160000 --- a/packages/python-server-sdk +++ b/packages/python-server-sdk @@ -1 +1 @@ -Subproject commit 2f18378a2339b3bf5870939864e0fff94f2e5a76 +Subproject commit 0889d54851167442d6a9318847a269f9c7466854 diff --git a/packages/web-client-sdk b/packages/web-client-sdk index ae228527..f9c524fd 160000 --- a/packages/web-client-sdk +++ b/packages/web-client-sdk @@ -1 +1 @@ -Subproject commit ae228527ec61ba2db2a61a590b4c999f065dcfeb +Subproject commit f9c524fd1ea91e5bcd57ee915282fefeaed45b65