Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import "./content-media-workbench.css";
type Provider = {status: string; enabled: boolean | null; next_action: string | null};
type Composition = {
composition_id: string; revision: number; status: string; title: string; duration_seconds: number;
shots: {title: string; seconds: number; kind: string; boundary: string; source_ref: string}[];
shots: {title: string; seconds: number; kind: string; boundary: string; source_ref: string; framing?: "full_frame"}[];
result: {files: Record<string, {sha256: string; bytes: number}>; qa: {frames: number; full_decode: string; narration: {mode: string}; blender_scenes?: {project_reopened: boolean}[]}; cost: {render_wall_seconds: number}} | null;
provider: Provider;
};
Expand Down Expand Up @@ -84,9 +84,9 @@ export function ContentMediaWorkbench({goalId}: {goalId: string}) {
{selected.result.files["blender-project.zip"] ? <a href={file("blender-project.zip")} download="blender-project.zip">{copy("Blender 工程与画面", "Blender project and passes")}</a> : null}</div>
<p>{copy("完整解码通过", "Full decode passed")} · {selected.result.qa.frames} {copy("帧", "frames")} · {copy("渲染耗时", "Render time")} {selected.result.cost.render_wall_seconds}s</p>
{selected.result.qa.blender_scenes?.length ? <p>{copy("三维工程已重新打开并检查,颜色与深度帧完整。", "The 3D project was reopened and checked; color and depth frames are complete.")}</p> : null}
</> : <ol className="content-media-timeline">{selected.shots.map((shot, i) => <li key={i}><span>{shot.title}</span><small>{shot.seconds}s · {shot.boundary}</small></li>)}</ol>}
</> : <ol className="content-media-timeline">{selected.shots.map((shot, i) => <li key={i}><span>{shot.title}</span><small>{shot.seconds}s · {shot.framing === "full_frame" ? copy("大画面 · ", "Full frame · ") : ""}{shot.boundary}</small></li>)}</ol>}
{["failed", "damaged"].includes(selected.status) ? <p>{hasBlender ? copy("请让 Agent 检查 Blender 依赖、场景范围与制作时限。修复本机依赖后可重试;修改场景需保存新修订。", "Ask your Agent to check the Blender extra, scene bounds and render time. Retry after dependency repair; save a new revision for scene changes.") : copy("请让 Agent 检查素材、字体和配音时长;修改脚本后保存新修订,或修复本机依赖后重试。", "Ask your Agent to check sources, fonts and narration timing. Save a new revision for story changes, or repair the local dependencies and retry.")}</p> : null}
<details><summary>{copy("来源与检查", "Sources and checks")}</summary><ul>{selected.shots.map((shot, i) => <li key={i}>{shot.title} — {shot.boundary}<br /><small>{shot.source_ref}</small></li>)}</ul>
<details><summary>{copy("来源与检查", "Sources and checks")}</summary><ul>{selected.shots.map((shot, i) => <li key={i}>{shot.title} — {shot.framing === "full_frame" ? copy("大画面 · ", "Full frame · ") : ""}{shot.boundary}<br /><small>{shot.source_ref}</small></li>)}</ul>
<p>{copy("本机私有作品。编码通过不代表创意质量、理解效果或发布许可;请观看并试听。", "Private local work. Encoding does not establish creative quality, audience understanding or publication approval. Watch and listen before sharing.")}</p></details>
</div></div> : null}
</section>;
Expand Down
41 changes: 41 additions & 0 deletions docs/reference/protocols/content-ops-media-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,3 +169,44 @@ missing bpy does not disable ordinary 2D work. A failed 3D attempt retains its
prepared input and supports retry after dependency repair. No fallback replaces
3D with a code projection. Mechanical readiness still does not establish
creative quality, audience understanding or publication approval.

## Per-shot picture framing

An `excerpt` or `blender` shot can opt into `framing: "full_frame"` with
`loopx-media` 0.3.0. The source fits the available width/height without cropping
or changing its aspect ratio; a small image may be enlarged. Its complete
`boundary` is wrapped in a separate measured strip below the picture. Neither
that strip nor the optional player captions are baked over source pixels.
The shot title remains timeline/source metadata; put a title that the viewer
must see in the source artwork, a title shot or timed captions. The fixed studio
header, drawn shot title and progress bar are omitted for this shot. The
selected layout's background/text tokens still apply. This is a rendering
choice, not an audience-understanding or creative-quality judgment.

Omitted framing and explicit `"editorial"` normalize to the original plan
structure and retain the existing digest/layout. Other values and framing on
non-spatial shot kinds are refused. Nonempty spatial `body` remains refused.
Full-frame framing changes the source-bound digest and needs a new revision;
it cannot inherit earlier content approval. The result additionally requires
`qa.full_frame_shots` to match the admitted ordered shot IDs. An older provider
that ignores the field cannot produce a ready result; explicitly upgrade the
optional package/manifest and retry after doctor. Default framing remains
compatible with the older provider. No core setting or global installation is
changed by preparing a full-frame plan.

The Agent prepares this choice in the same brief; the packaged workbench shows
“Full frame” beside its timeline/source entry and renders, cancels, retries and
reads back the same composition. Users do not enter source paths or JSON again.
Disabling the existing extension blocks new renders and keeps verified movies
readable. To return to editorial framing, prepare a revision without the field
and render it; an older LoopX core will refuse a plan containing the new field.

中文:`excerpt` 与 `blender` 镜头可选 `framing: "full_frame"`。素材按原比例
完整放入较大的画面区,必要时放大;来源说明在独立底栏完整换行,工作室页眉、
片内标题和进度条不再绘制。标题仍在分镜与来源信息中,需要观众看到的标题由素材、
标题镜头或字幕承载。省略该字段或显式选择 `editorial` 保留原计划摘要与构图。
此选项需要显式升级 `loopx-media` 0.3.0、注册新 manifest 并运行 doctor;旧渲染器
若忽略新字段,结果会被拒绝就绪。修改构图需新修订,仍会撤销旧内容批准。
工作台沿现有入口显示“大画面”并制作、取消、重试和查看;停用扩展阻止新制作但
保留已校验影片。回到旧构图时,移除字段后保存新修订再制作;旧版核心不会接受
含新字段的计划。该选项不改变来源可见性、发布权限或全局配置。
4 changes: 3 additions & 1 deletion loopx/capabilities/content_ops/media.py
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ def inspect(self, composition_id: str) -> dict:
status = "damaged"; break
return {"ok": True, "composition_id": composition_id, "revision": record["revision"], "status": status,
"title": record["plan"]["title"], "duration_seconds": record["duration_seconds"],
"plan_digest": record["plan_digest"], "shots": [{k: shot[k] for k in ("title", "seconds", "kind", "boundary", "source_ref")} for shot in record["plan"]["shots"]],
"plan_digest": record["plan_digest"], "shots": [{k: shot[k] for k in ("title", "seconds", "kind", "boundary", "source_ref", "framing") if k in shot} for shot in record["plan"]["shots"]],
"updated_at": record["updated_at"], "result": result, "item_state": record["item"]["state"],
"error_code": record.get("error_code"), "provider": self.binding().public_readiness(),
"visibility": "local_private", "publish_authorized": False}
Expand Down Expand Up @@ -219,9 +219,11 @@ def _execute(self, composition_id: str, attempt: str) -> None:
"sources": record["sources"], "attempt_directory": str(_safe_path(bundle, attempt)), "cancel_file": str(cancel)})
spatial = [{"id": shot["id"], "frames": round(shot["seconds"] * record["plan"]["fps"])}
for shot in record["plan"]["shots"] if shot["kind"] == "blender"]
full_frame = [shot["id"] for shot in record["plan"]["shots"] if shot.get("framing") == "full_frame"]
result = _owner("result", {"result": result, "plan_digest": record["plan_digest"],
"frame_count": record["frame_count"], "narration_mode": record["plan"]["narration"]["mode"],
**({"blender_shots": spatial} if spatial else {}),
**({"full_frame_shots": full_frame} if full_frame else {}),
**{k: record["plan"][k] for k in ("width", "height", "fps")}})
required = {"video.mp4", "poster.png", "captions.vtt", "captions.srt", "transcript.txt"}
if not required <= set(result["files"]) or set(result["files"]) - RESULT_FILES:
Expand Down
15 changes: 12 additions & 3 deletions loopx/control_plane/capabilities/content_media.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,15 +112,20 @@ export function validateMediaPlan(input: unknown) {
if (new Set(assets.map(asset => asset.id)).size !== assets.length) fail("Duplicate source reference");
const shots = array(raw.shots, 30).map(value => {
const shot = object(value);
fields(shot, ["id", "kind", "seconds", "title", "body", "nodes", "asset_id", "start_seconds", "source_ref", "boundary", "scene"]);
fields(shot, ["id", "kind", "seconds", "title", "body", "nodes", "asset_id", "start_seconds", "source_ref", "boundary", "scene", "framing"]);
const kind = shot.kind;
if (typeof kind !== "string" || !["title", "diagram", "excerpt", "blender"].includes(kind)) fail("Unsupported shot kind");
// Composition-local presentation vocabulary; omitted/editorial retains the
// original canonical bytes. It changes neither source nor publication scope.
if (shot.framing !== undefined && shot.framing !== "editorial" && shot.framing !== "full_frame") fail("Unsupported shot framing");
if (shot.framing !== undefined && kind !== "excerpt" && kind !== "blender") fail("Framing requires an excerpt or Blender shot");
const framing = shot.framing === "full_frame" ? {framing: "full_frame" as const} : {};
if (kind !== "blender" && shot.scene !== undefined) fail("Scene geometry requires a Blender shot");
const base = {id: token(shot.id), kind, seconds: number(shot.seconds, 1, 30), title: text(shot.title, 100),
body: shot.body == null || shot.body === "" ? "" : text(shot.body, 300), source_ref: text(shot.source_ref, 240), boundary: text(shot.boundary, 180)};
body: shot.body == null || shot.body === "" ? "" : text(shot.body, 300), source_ref: text(shot.source_ref, 240), boundary: text(shot.boundary, 180), ...framing};
if ((kind === "excerpt" || kind === "blender") && base.body) fail("Excerpt and Blender shots do not support body text; use a title, boundary or timed captions");
if (kind === "blender") {
fields(shot, ["id", "kind", "seconds", "title", "body", "source_ref", "boundary", "scene"]);
fields(shot, ["id", "kind", "seconds", "title", "body", "source_ref", "boundary", "scene", "framing"]);
const frames = number(base.seconds * fps, 12, 120, true);
return {...base, scene: blenderScene(shot.scene, frames)};
}
Expand Down Expand Up @@ -200,6 +205,9 @@ export function validateMediaResult(input: unknown) {
}) : [];
if (spatial.length !== expectedScenes.length || Boolean(files["blender-project.zip"]) !== Boolean(expectedScenes.length)) fail("Blender project output is missing or out of scope");
const narration = object(qa.narration);
const expectedFullFrame = array(request.full_frame_shots ?? [], 30).map(token);
const fullFrame = array(qa.full_frame_shots ?? [], 30).map(token);
if (new Set(fullFrame).size !== fullFrame.length || stable(fullFrame) !== stable(expectedFullFrame)) fail("Renderer did not verify the requested full-frame shots; upgrade the local media provider");
if (!["silent", "supplied", "system_speech"].includes(String(narration.mode)) || narration.mode !== request.narration_mode) fail("Narration result differs from the admitted plan");
if (narration.mode !== "silent" && !files["narration.wav"]) fail("Narration output is missing");
const audioMetrics = narration.mode === "silent" ? {} : {
Expand All @@ -209,6 +217,7 @@ export function validateMediaResult(input: unknown) {
return {ok: true, protocol: MEDIA_PROTOCOL, plan_digest: request.plan_digest, files,
qa: {full_decode: "passed", frames: qa.frames, width: qa.width, height: qa.height, fps: qa.fps,
source_hashes_verified: true, narration: {mode: narration.mode, ...audioMetrics, subjective_listening: "not_tested"},
...(fullFrame.length ? {full_frame_shots: fullFrame} : {}),
...(spatial.length ? {blender_scenes: spatial} : {}),
audience_understanding: "not_tested", creative_review: "not_tested"},
cost: {render_wall_seconds: number(cost.render_wall_seconds, 0, 120)}, publish_authorized: false};
Expand Down
33 changes: 32 additions & 1 deletion packages/loopx-media/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,43 @@ RTL shaping and arbitrary font discovery are not qualified. macOS speech is
optional; other platforms require supplied audio or explicit silence.
No cloud voice, generative image/video or game engine is bundled here.

Excerpt and Blender layouts display the title and source boundary; nonempty
The default excerpt and Blender layouts display the title and source boundary; nonempty
`body` text is explicitly refused. Preserve a needed qualifier in those visible
fields, a separate title scene or timed captions before preparing a new revision.
The renderer also refuses a stale/direct request with unsupported body text;
it cannot return ready while silently dropping that text.

## Picture framing

For a source-led composition, an Agent can set `framing: "full_frame"` on an
image/video excerpt or Blender shot. Version 0.3.0 fits the complete picture
without cropping and keeps its source boundary in a separate strip below it.
The studio header, drawn shot title and progress bar are omitted; the title
remains in the timeline/source readback. Include needed titles in the artwork,
a title shot or timed captions. Each shot can choose independently. Omitted
framing or `"editorial"` keeps the original canonical plan and layout.

Use the explicit package upgrade/reinstall and extension upgrade/doctor route
below for an existing provider. The core refuses full-frame readiness when an
older renderer omits its framing readback. Retry the unchanged plan after the
upgrade. The workbench displays the choice and retains its existing render,
cancel, retry and verified-download journey. To roll back the composition,
prepare a new revision without framing and render it. No global setting,
source classification or publication authority changes. See the
[bilingual contract](../../docs/reference/protocols/content-ops-media-v0.md#per-shot-picture-framing).

中文:0.3.0 在原 content-ops 镜头中加入可选“大画面”构图。素材保持完整比例,
来源说明另置底栏;需要观众看到的标题由素材、标题镜头或字幕承载。Agent 准备
分镜,工作台沿原入口制作和查看。已有安装应显式升级可选包及 manifest 并运行
doctor;移除字段、保存新修订并重制即可恢复原构图。默认计划和发布边界不变。

Public synthetic packaged-workbench captures (macOS, source qualification;
not an installed-App or comprehension claim):

| Editorial | Full frame | Narrow, scrolled to movie |
| --- | --- | --- |
| ![Editorial](examples/framing/editorial-desktop.png) | ![Full frame](examples/framing/full-frame-desktop.png) | ![Narrow](examples/framing/full-frame-narrow.png) |

## Optional Blender scenes

For real 3D geometry, install the same provider with its optional Blender extra
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion packages/loopx-media/extension.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
schema_version = "loopx_extension_manifest_v0"
id = "loopx-media"
version = "0.2.0"
version = "0.3.0"
requires_loopx_api = ">=1,<2"
permissions = ["content_media_write"]

Expand Down
2 changes: 1 addition & 1 deletion packages/loopx-media/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "loopx-media"
version = "0.2.0"
version = "0.3.0"
description = "Optional local composition renderer for LoopX content-ops"
requires-python = ">=3.11"
dependencies = ["Pillow>=11,<13", "imageio-ffmpeg>=0.6,<0.7", "fonttools>=4,<5"]
Expand Down
Loading
Loading