From dbd1619f91130bf1bf6861f6ca7c0022e9b196a2 Mon Sep 17 00:00:00 2001 From: Your Name Date: Wed, 22 Jul 2026 16:51:03 -0700 Subject: [PATCH 1/4] feat(video): require five cinematic shot dimensions Add a storyboard gate for action, camera, depth, transformation, and surprise. Enforce a 50% cinematic/hybrid floor so poster-only sequences fail before production spend. --- docs/video/STORYBOARD.cinematic.example.md | 72 ++++++++++ docs/video/cinematic-shot-system.md | 117 ++++++++++++++++ package.json | 4 +- tools/cinematic-storyboard.mjs | 156 +++++++++++++++++++++ tools/cinematic-storyboard.test.mjs | 65 +++++++++ 5 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 docs/video/STORYBOARD.cinematic.example.md create mode 100644 docs/video/cinematic-shot-system.md create mode 100644 tools/cinematic-storyboard.mjs create mode 100644 tools/cinematic-storyboard.test.mjs diff --git a/docs/video/STORYBOARD.cinematic.example.md b/docs/video/STORYBOARD.cinematic.example.md new file mode 100644 index 000000000..1ae9911ee --- /dev/null +++ b/docs/video/STORYBOARD.cinematic.example.md @@ -0,0 +1,72 @@ +--- +format: 1920x1080 +message: "The machine does not make the decision; it carries the decision into the world." +arc: Signal → Pursuit → Refusal → Transformation → Proof → Quiet +audience: kernel.chat readers and creative-tool builders +mode: autonomous +rhythm: impact → pursuit → stillness → transformation → proof → hold +--- + +## Frame 1 — The signal escapes + +- visual_mode: cinematic +- scene: A red signal tears out of a printed sentence and escapes into a nocturnal paper city. +- duration: 5s +- transition_in: cut +- action: The final word tears free, folds into a red paper animal, and runs off the page. +- camera: A macro push becomes a low tracking chase alongside the animal. +- depth: Foreground paper fibers whip past; the midground animal runs; background rooftops wake in sequence. +- transformation: A static printed claim becomes a living signal loose in the city. +- surprise: The punctuation mark stays behind and turns its head to watch. +- start_state: An extreme macro of an intact typeset sentence. +- end_state: A wide street with the red animal disappearing around a corner. + +The typography begins as evidence and becomes matter. The action—not a title entrance—creates the cut into the city. + +## Frame 2 — The ledger wall + +- visual_mode: hybrid +- scene: The camera travels beside an architectural ledger whose exact measurements open physical routes through the city. +- duration: 6s +- transition_in: velocity-matched whip +- action: Each measured figure punches a doorway through the ledger and the signal chooses the middle route. +- camera: A lateral dolly tracks the figures, then rack-focuses from the foreground 53 to the midground 459 doorway. +- depth: Foreground numerals cross the lens; the midground signal enters 459; background 1,407 recedes into haze. +- transformation: A flat comparison becomes three traversable routes with different physical costs. +- surprise: The largest number opens the smallest, most obstructed door. +- start_state: A ruled wall carrying three exact measurements. +- end_state: The middle doorway remains open with warm light beyond it. + +The numbers remain exact and typeset in the publication system. Their spatial consequence makes the comparison felt. + +## Frame 3 — Refusal + +- visual_mode: quiet +- scene: A hand rests beside an unpressed approval lever while the workshop waits without punishment. +- duration: 4s +- transition_in: hard cut +- action: Dust settles and the machine deliberately powers down when the hand withdraws. +- camera: A locked macro shot performs one slow rack focus from the lever to the resting machine. +- depth: Foreground hand exits; midground lever remains untouched; background machine light dims to black. +- transformation: A waiting system becomes a safely refused system. +- surprise: The room grows warmer after the machine turns off. +- start_state: Lever, hand, and machine suspended in expectation. +- end_state: Empty lever in a calm, warm room. + +Stillness is the event here. It is earned by the pursuit before it. + +## Frame 4 — The proof returns + +- visual_mode: cinematic +- scene: The red signal returns carrying a film canister that unfolds into the original printed page. +- duration: 6s +- transition_in: light leak +- action: The animal leaps onto the workbench, unfolds into a ribbon, and binds the canister into a finished spread. +- camera: A crane drops with the leap, orbits the binding action, then pulls back to reveal the whole room. +- depth: Foreground tools briefly occlude the binding; the midground ribbon transforms; background workers turn toward the result. +- transformation: A moving signal becomes a durable artifact owned in the room. +- surprise: The finished spread contains the opening sentence, now changed by one word. +- start_state: A dark workbench awaiting the returning signal. +- end_state: A completed spread under warm light, with the city visible beyond it. + +The closing image pays off the opening without returning to the same state. diff --git a/docs/video/cinematic-shot-system.md b/docs/video/cinematic-shot-system.md new file mode 100644 index 000000000..3e2ab717a --- /dev/null +++ b/docs/video/cinematic-shot-system.md @@ -0,0 +1,117 @@ +# The Shot Is the Unit + +Status: production law for generated and HyperFrames-authored films. + +The existing system is excellent at making resolved editorial frames. This layer prevents those frames from becoming a slideshow. A frame is now evidence of a shot; the shot is the authored unit. + +## The contract + +Every storyboard frame must declare these fields: + +```md +- visual_mode: cinematic | hybrid | graphic | quiet +- action: The physical event and its consequence. +- camera: The camera's observable path or lens change. +- depth: Foreground, midground, and background behavior. +- transformation: What is irreversibly different by the end. +- surprise: The turn the opening image does not predict. +- start_state: Optional but recommended description of the opening image. +- end_state: Optional but recommended description of the closing image. +``` + +`action` is not an entrance animation. “The title fades in” describes software. “The title tears through the paper and exposes the workshop behind it” describes an event. + +`camera` is not a transition. Crossfade, wipe, and push-slide describe the edit between shots. Orbit, crane, dive, track, rack focus, whip, and pull-back describe perception inside the shot. + +`depth` names all three planes and how they interact. Foreground material should occasionally occlude the subject. Midground owns the action. Background establishes scale, atmosphere, or consequence. + +`transformation` must change the visual proposition. A larger number is usually not a transformation. A ledger flooding until it becomes a city is. + +`surprise` must be visible, not merely narrated. It may be small and quiet, but the opening image must not fully predict it. + +## Mode budget + +Across a film: + +- At least 50% of frames are `cinematic` or `hybrid`. +- `graphic` frames carry measurements, proof, typography, and diagrams. +- `quiet` frames are earned rests and should remain at or below 20%. +- Brand typography and the tomato/ink/ivory palette connect the worlds; they do not have to occupy every pixel. + +This is a floor, not a recipe. A generated narrative film may be almost entirely cinematic. A data feature may sit near the minimum and use hybrid mechanisms to carry evidence. + +## Shot construction order + +Author each beat in this order: + +1. Emotional beat — what changes in the viewer. +2. Physical event — what happens in the world. +3. Camera path — how the viewer discovers it. +4. Depth stack — what passes before, around, and behind the subject. +5. Transformation — the ending image and its consequence. +6. Surprise — the turn. +7. Graphic layer — typography, labels, readings, and publication marks. + +Layout comes last. The still poster is selected from the shot, not used as the shot's source of truth. + +## Rhythm law + +Use contrast, not constant frenzy. Name the film's rhythm before building it, for example: + +`impact → pursuit → stillness → transformation → proof → eruption → hold` + +At least one adjacent pair must change two or more of these dimensions: + +- scale: macro / human / architectural +- velocity: still / drifting / fast +- camera: locked / travelling / unstable +- material: paper / light / liquid / physical object / typography +- density: sparse / layered / crowded + +Dynamic does not mean everything moves. It means movement changes meaning. + +## Hybrid shot recipe + +Hybrid is the native kernel.chat mode. It lets factual graphics live inside a physical world: + +- A measured number is printed on a ticket moving through a real chute. +- A comparison grid is a wall the camera travels along. +- A waveform becomes a thread that pulls the next scene open. +- A ruled frame is a window into a generated environment. +- A typographic word becomes matter: paper, stencil, shadow, signage, or an object the subject handles. + +The evidence stays exact. The presentation gains consequence. + +## Generation direction + +For generated shots, prompts describe one subject, one action, and one camera instruction. Keep identity and geometry constraints separate from motion. Prefer image-to-video when subject continuity matters. A motion prompt should say what moves, what remains fixed, how the camera moves, and what the final state is. + +Avoid asking one short clip to perform several unrelated transformations. Use the edit for discontinuity and the shot for one legible event. + +## HyperFrames direction + +HyperFrames remains responsible for exact typography, data, diagrams, compositing, and deterministic finishing. Generated video supplies physical worlds and hard-to-simulate events. Three.js or shaders may supply fully deterministic spatial shots when appropriate. + +Do not fake cinematography by applying the same 3% scale push to every flat frame. Camera motion needs parallax, occlusion, focus change, or a meaningful change of viewpoint. + +## Audio direction + +Every hero event gets an audio consequence. Design three layers: + +- environment: the space exists before the action; +- material: paper, metal, breath, glass, motor, cloth; +- punctuation: the single hit, cut, silence, or tonal change that marks transformation. + +Sound should sometimes lead the picture. A cue arriving 2–6 frames early creates anticipation; silence immediately after impact gives the event weight. + +## Gate + +Run: + +```bash +npm run lint:cinematic -- path/to/STORYBOARD.md +``` + +The gate rejects missing shot fields, cinematic frames made only from layout reveals, and films without a cinematic/hybrid majority. Warnings identify weak camera verbs and incomplete depth plans. + +The lint is intentionally semantic and conservative. Passing it does not make a shot good. Failing it means the plan has not yet described a shot. diff --git a/package.json b/package.json index 620ab67be..e83f250a1 100644 --- a/package.json +++ b/package.json @@ -11,6 +11,7 @@ "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0", "lint:adherence": "node scripts/check-adherence.mjs", "lint:editorial": "node scripts/check-editorial.mjs", + "lint:cinematic": "node tools/cinematic-storyboard.mjs", "artifacts:index": "node scripts/build-artifact-index.mjs", "sitemap": "node scripts/build-sitemap.mjs", "preview": "vite preview", @@ -43,7 +44,8 @@ "video:palmier:bed": "node tools/palmier/build-room-bed.mjs", "video:palmier:plan": "node tools/palmier/content-plan.mjs", "video:palmier:suite": "node tools/palmier/suite.mjs", - "test:palmier": "npx vitest run tools/palmier/suite.test.mjs" + "test:palmier": "npx vitest run tools/palmier/suite.test.mjs", + "test:cinematic": "npx vitest run tools/cinematic-storyboard.test.mjs" }, "homepage": "https://kernel.chat", "dependencies": { diff --git a/tools/cinematic-storyboard.mjs b/tools/cinematic-storyboard.mjs new file mode 100644 index 000000000..13d1ac7ee --- /dev/null +++ b/tools/cinematic-storyboard.mjs @@ -0,0 +1,156 @@ +import fs from 'node:fs' +import path from 'node:path' + +const REQUIRED_FIELDS = ['action', 'camera', 'depth', 'transformation', 'surprise'] +const VISUAL_MODES = new Set(['cinematic', 'hybrid', 'graphic', 'quiet']) +const EMPTY_VALUES = new Set(['none', 'n/a', 'na', 'static', 'tbd', 'todo']) +const POSTER_ACTIONS = /\b(appear|assemble|count(?:s)? up|draw(?:s)?|fade(?:s)?|reveal(?:s)?|slide(?:s)? in|type(?:s)? on)\b/i +const PHYSICAL_ACTIONS = /\b(bend|bloom|break|burn|chase|collapse|collide|crack|crash|drip|explode|fall|flood|flow|fold|fracture|grow|melt|open|peel|pour|race|rip|roll|shatter|spill|split|tear|transform|unfurl)\b/i +const CAMERA_ACTIONS = /\b(arc|crane|dive|dolly|drift|fly|orbit|pan|pull|push|rack focus|roll|track|truck|whip|zoom)\b/i +const DEPTH_LAYERS = /\b(bg|background|foreground|fg|midground|mg|near|middle|far|occlud|parallax)\b/gi + +function clean(value = '') { + return value.trim().replace(/^['"]|['"]$/g, '') +} + +export function parseCinematicStoryboard(markdown) { + const frames = [] + const heading = /^#{2,3}\s+(?:Frame|Beat|Scene)\s+([^\n]+)$/gim + const matches = [...markdown.matchAll(heading)] + + for (let index = 0; index < matches.length; index += 1) { + const start = matches[index].index + matches[index][0].length + const end = matches[index + 1]?.index ?? markdown.length + const body = markdown.slice(start, end) + const fields = {} + for (const match of body.matchAll(/^\s*-\s+([a-z][a-z0-9_-]*):\s*(.+?)\s*$/gim)) { + fields[match[1].toLowerCase()] = clean(match[2]) + } + frames.push({ + index, + heading: clean(matches[index][1]), + fields, + }) + } + + return frames +} + +function isEmpty(value) { + return !value || EMPTY_VALUES.has(value.toLowerCase()) +} + +function uniqueDepthLayers(value = '') { + return new Set([...value.matchAll(DEPTH_LAYERS)].map((match) => { + const token = match[0].toLowerCase() + if (token === 'bg' || token === 'background' || token === 'far') return 'background' + if (token === 'mg' || token === 'midground' || token === 'middle') return 'midground' + return 'foreground' + })) +} + +export function lintCinematicStoryboard(markdown, options = {}) { + const frames = parseCinematicStoryboard(markdown) + const findings = [] + + if (!frames.length) { + return { + ok: false, + frames, + findings: [{ level: 'error', code: 'no-frames', message: 'No storyboard frames found.' }], + summary: { frames: 0, cinematic: 0, hybrid: 0, graphic: 0, quiet: 0 }, + } + } + + for (const frame of frames) { + const label = `Frame ${frame.heading}` + for (const field of REQUIRED_FIELDS) { + if (isEmpty(frame.fields[field])) { + findings.push({ level: 'error', code: `missing-${field}`, frame: frame.index + 1, message: `${label} needs a concrete ${field}.` }) + } + } + + const mode = frame.fields.visual_mode?.toLowerCase() + if (!VISUAL_MODES.has(mode)) { + findings.push({ level: 'error', code: 'invalid-visual-mode', frame: frame.index + 1, message: `${label} needs visual_mode: cinematic, hybrid, graphic, or quiet.` }) + } + + if (!isEmpty(frame.fields.camera) && !CAMERA_ACTIONS.test(frame.fields.camera)) { + findings.push({ level: 'warning', code: 'weak-camera-verb', frame: frame.index + 1, message: `${label} camera direction does not name an observable camera action.` }) + } + + if (!isEmpty(frame.fields.depth) && uniqueDepthLayers(frame.fields.depth).size < 3) { + findings.push({ level: 'warning', code: 'shallow-depth-plan', frame: frame.index + 1, message: `${label} should name foreground, midground, and background behavior.` }) + } + + if ((mode === 'cinematic' || mode === 'hybrid') && !isEmpty(frame.fields.action)) { + if (POSTER_ACTIONS.test(frame.fields.action) && !PHYSICAL_ACTIONS.test(frame.fields.action)) { + findings.push({ level: 'error', code: 'poster-action', frame: frame.index + 1, message: `${label} uses only layout/reveal motion; add a physical event with consequences.` }) + } + } + + if (frame.fields.end_state && frame.fields.end_state === frame.fields.start_state) { + findings.push({ level: 'error', code: 'unchanged-state', frame: frame.index + 1, message: `${label} starts and ends in the same state.` }) + } + } + + const summary = { frames: frames.length, cinematic: 0, hybrid: 0, graphic: 0, quiet: 0 } + for (const frame of frames) { + const mode = frame.fields.visual_mode?.toLowerCase() + if (mode in summary) summary[mode] += 1 + } + + const dynamicCount = summary.cinematic + summary.hybrid + const minimumDynamicRatio = options.minimumDynamicRatio ?? 0.5 + if (dynamicCount / frames.length < minimumDynamicRatio) { + findings.push({ + level: 'error', + code: 'poster-sequence-ratio', + message: `Only ${dynamicCount}/${frames.length} frames are cinematic or hybrid; require at least ${Math.ceil(frames.length * minimumDynamicRatio)}.`, + }) + } + + const quietLimit = Math.max(1, Math.floor(frames.length * 0.2)) + if (summary.quiet > quietLimit) { + findings.push({ level: 'warning', code: 'too-many-quiet-holds', message: `${summary.quiet}/${frames.length} frames are quiet holds; the target ceiling is ${quietLimit}.` }) + } + + return { + ok: !findings.some((finding) => finding.level === 'error'), + frames, + findings, + summary, + } +} + +export function formatReport(result, file = 'STORYBOARD.md') { + const lines = [`Cinematic storyboard: ${file}`, `Frames: ${result.summary.frames} · cinematic ${result.summary.cinematic} · hybrid ${result.summary.hybrid} · graphic ${result.summary.graphic} · quiet ${result.summary.quiet}`] + for (const finding of result.findings) { + lines.push(`${finding.level.toUpperCase()} ${finding.code}${finding.frame ? ` [frame ${finding.frame}]` : ''}: ${finding.message}`) + } + lines.push(result.ok ? 'PASS: the storyboard clears the cinematic gate.' : 'FAIL: the storyboard is still behaving like an animated poster sequence.') + return lines.join('\n') +} + +function runCli() { + const args = process.argv.slice(2) + const json = args.includes('--json') + const files = args.filter((arg) => !arg.startsWith('--')) + if (!files.length) { + console.error('Usage: node tools/cinematic-storyboard.mjs [more files] [--json]') + process.exitCode = 2 + return + } + + const reports = files.map((file) => { + const markdown = fs.readFileSync(file, 'utf8') + return { file: path.resolve(file), ...lintCinematicStoryboard(markdown) } + }) + if (json) console.log(JSON.stringify(reports, null, 2)) + else console.log(reports.map((report) => formatReport(report, report.file)).join('\n\n')) + if (reports.some((report) => !report.ok)) process.exitCode = 1 +} + +if (process.argv[1] && path.resolve(process.argv[1]) === path.resolve(new URL(import.meta.url).pathname)) { + runCli() +} diff --git a/tools/cinematic-storyboard.test.mjs b/tools/cinematic-storyboard.test.mjs new file mode 100644 index 000000000..150777dfd --- /dev/null +++ b/tools/cinematic-storyboard.test.mjs @@ -0,0 +1,65 @@ +import { describe, expect, it } from 'vitest' +import { lintCinematicStoryboard, parseCinematicStoryboard } from './cinematic-storyboard.mjs' + +const dynamicFrame = ` +## Frame 1 — The fold + +- visual_mode: cinematic +- action: The paper city folds inward and traps the red signal. +- camera: A slow crane dives through the collapsing rooftops. +- depth: Foreground fibers occlude the midground city while the background sky compresses. +- transformation: A wide city becomes a single sealed envelope. +- surprise: The envelope opens from inside. +- start_state: An intact city at night. +- end_state: One sealed envelope in daylight. +` + +describe('cinematic storyboard parser', () => { + it('preserves custom cinematic fields on each frame', () => { + const [frame] = parseCinematicStoryboard(dynamicFrame) + expect(frame.heading).toBe('1 — The fold') + expect(frame.fields.visual_mode).toBe('cinematic') + expect(frame.fields.camera).toContain('crane') + }) +}) + +describe('cinematic storyboard lint', () => { + it('passes a physical, spatial, transforming shot', () => { + const result = lintCinematicStoryboard(dynamicFrame) + expect(result.ok).toBe(true) + expect(result.findings).toEqual([]) + }) + + it('rejects poster-only motion in a cinematic frame', () => { + const result = lintCinematicStoryboard(` +## Frame 1 — Metric +- visual_mode: cinematic +- action: A number counts up and labels fade in. +- camera: Static. +- depth: Background grid and foreground number. +- transformation: The number becomes larger. +- surprise: A rule appears. +`) + expect(result.ok).toBe(false) + expect(result.findings.map((finding) => finding.code)).toEqual(expect.arrayContaining([ + 'poster-action', + 'weak-camera-verb', + 'shallow-depth-plan', + ])) + }) + + it('enforces a dynamic majority across the film', () => { + const graphic = (number) => ` +## Frame ${number} — Poster +- visual_mode: graphic +- action: Type stamps onto the sheet. +- camera: Camera pushes toward the type. +- depth: Foreground type, midground rule, background paper. +- transformation: Blank paper becomes a printed claim. +- surprise: The final word is physically misregistered. +` + const result = lintCinematicStoryboard(`${dynamicFrame}${graphic(2)}${graphic(3)}`) + expect(result.ok).toBe(false) + expect(result.findings.some((finding) => finding.code === 'poster-sequence-ratio')).toBe(true) + }) +}) From b973259e3ad39f899d99f8d2c273792077361e4a Mon Sep 17 00:00:00 2001 From: Your Name Date: Wed, 22 Jul 2026 16:59:15 -0700 Subject: [PATCH 2/4] feat(video): compile directed candidate batches Route each shot by motion needs, carry five continuity locks across the film, and score four intentional candidates with a 100-point rubric before any paid generation. --- docs/video/STORYBOARD.cinematic.example.md | 7 + docs/video/cinematic-shot-system.md | 20 ++ package.json | 3 +- tools/shot-compiler.mjs | 238 +++++++++++++++++++++ tools/shot-compiler.test.mjs | 41 ++++ 5 files changed, 308 insertions(+), 1 deletion(-) create mode 100644 tools/shot-compiler.mjs create mode 100644 tools/shot-compiler.test.mjs diff --git a/docs/video/STORYBOARD.cinematic.example.md b/docs/video/STORYBOARD.cinematic.example.md index 1ae9911ee..de00dcac2 100644 --- a/docs/video/STORYBOARD.cinematic.example.md +++ b/docs/video/STORYBOARD.cinematic.example.md @@ -5,6 +5,13 @@ arc: Signal → Pursuit → Refusal → Transformation → Proof → Quiet audience: kernel.chat readers and creative-tool builders mode: autonomous rhythm: impact → pursuit → stillness → transformation → proof → hold +visual_world: "Cinematic macro paper-craft noir with tangible fibers, warm practical light, deep ink shadow, and one tomato-red signal" +continuity_subject: "The same small tomato-red folded-paper animal, sharp triangular ears, one white paper eye, no anatomy changes" +continuity_location: "One coherent nocturnal paper city connected to the same warm workshop" +palette: "Ivory paper, brown-black ink, tomato red, and warm practical amber only" +material: "Hand-cut paper with visible fibers, crisp folds, miniature practical construction" +lens_family: "Anamorphic macro family with shallow depth of field and consistent oval bokeh" +candidates: 4 --- ## Frame 1 — The signal escapes diff --git a/docs/video/cinematic-shot-system.md b/docs/video/cinematic-shot-system.md index 3e2ab717a..de1f57e3b 100644 --- a/docs/video/cinematic-shot-system.md +++ b/docs/video/cinematic-shot-system.md @@ -115,3 +115,23 @@ npm run lint:cinematic -- path/to/STORYBOARD.md The gate rejects missing shot fields, cinematic frames made only from layout reveals, and films without a cinematic/hybrid majority. Warnings identify weak camera verbs and incomplete depth plans. The lint is intentionally semantic and conservative. Passing it does not make a shot good. Failing it means the plan has not yet described a shot. + +## Compile + +Once the gate passes, compile the storyboard into an executable production plan: + +```bash +npm run video:compile-shots -- path/to/STORYBOARD.md --output=shot-plan.json +``` + +The compiler adds the layer that a preset picker cannot: + +- film-wide subject, location, palette, material, and lens locks; +- per-shot routing between generated video and deterministic HyperFrames work; +- separate keyframe and motion prompts; +- four deliberately different candidates instead of four accidental retries; +- hard-reject criteria followed by a 100-point selection rubric; +- estimated batch cost with both keyframe and paid-generation approval gates intact; +- a finishing handoff for the selected shot. + +The output is a local JSON plan. Compilation never submits a paid request. A generation runner may consume the plan later, but it must preserve the existing explicit cost-confirm contract. diff --git a/package.json b/package.json index e83f250a1..4417067c6 100644 --- a/package.json +++ b/package.json @@ -43,9 +43,10 @@ "video:palmier:cuts": "node tools/palmier/cut-sheet.mjs", "video:palmier:bed": "node tools/palmier/build-room-bed.mjs", "video:palmier:plan": "node tools/palmier/content-plan.mjs", + "video:compile-shots": "node tools/shot-compiler.mjs", "video:palmier:suite": "node tools/palmier/suite.mjs", "test:palmier": "npx vitest run tools/palmier/suite.test.mjs", - "test:cinematic": "npx vitest run tools/cinematic-storyboard.test.mjs" + "test:cinematic": "npx vitest run tools/cinematic-storyboard.test.mjs tools/shot-compiler.test.mjs" }, "homepage": "https://kernel.chat", "dependencies": { diff --git a/tools/shot-compiler.mjs b/tools/shot-compiler.mjs new file mode 100644 index 000000000..fb5794b55 --- /dev/null +++ b/tools/shot-compiler.mjs @@ -0,0 +1,238 @@ +import fs from 'node:fs' +import path from 'node:path' +import { fileURLToPath } from 'node:url' +import { estimateUsd, getModel } from './video-models.mjs' +import { lintCinematicStoryboard, parseCinematicStoryboard } from './cinematic-storyboard.mjs' + +const DEFAULT_NEGATIVE = [ + 'unmotivated camera shake', + 'rubber geometry', + 'identity drift', + 'extra limbs or objects', + 'morphing typography', + 'illegible text', + 'generic AI glow', + 'unrequested scene cuts', +].join(', ') + +const CAMERA_COMPLEXITY = /\b(arc(?:s|ing)?|crane(?:s|d|ing)?|div(?:e|es|ed|ing)|doll(?:y|ies|ied|ying)|orbit(?:s|ed|ing)?|pull(?:s|ed|ing)?|push(?:es|ed|ing)?|rack(?:s|ed|ing)? focus|track(?:s|ed|ing)?|truck(?:s|ed|ing)?|whip(?:s|ped|ping)?)\b/gi +const DIALOGUE = /\b(say|says|speak|speaks|dialogue|voice|lip[- ]?sync)\b/i +const HIGH_ACTION = /\b(chase|collide|crash|explode|fall|flood|fracture|race|rip|shatter|tear)\b/i + +export const REVIEW_RUBRIC = Object.freeze([ + { id: 'identity', weight: 20, question: 'Are subject identity, wardrobe, geometry, and material stable?' }, + { id: 'action', weight: 20, question: 'Is the physical event legible and completed?' }, + { id: 'camera', weight: 15, question: 'Does the camera path feel intentional and spatially coherent?' }, + { id: 'transformation', weight: 15, question: 'Does the ending image materially differ from the opening image?' }, + { id: 'composition', weight: 10, question: 'Is there one clear focal hierarchy at every sampled moment?' }, + { id: 'continuity', weight: 10, question: 'Does the shot cut cleanly from its neighbors?' }, + { id: 'integrity', weight: 10, question: 'Is the image free of warping, duplicate objects, and text corruption?' }, +]) + +export const HARD_REJECTS = Object.freeze([ + 'subject identity changes', + 'the action never completes', + 'the camera contradicts the requested direction', + 'new unrequested subjects or objects appear', + 'load-bearing text is malformed', + 'the final frame cannot connect to the next shot', +]) + +function clean(value = '') { + return value.trim().replace(/^['"]|['"]$/g, '') +} + +export function parseStoryboardGlobals(markdown) { + const match = markdown.match(/^---\s*\n([\s\S]*?)\n---/) + if (!match) return {} + const globals = {} + for (const line of match[1].split('\n')) { + const field = line.match(/^([a-z][a-z0-9_-]*):\s*(.*?)\s*$/i) + if (field) globals[field[1].toLowerCase()] = clean(field[2]) + } + return globals +} + +function durationSeconds(value) { + const match = String(value || '').match(/\d+(?:\.\d+)?/) + return match ? Number(match[0]) : 5 +} + +function cameraComplexity(camera = '') { + return new Set([...camera.matchAll(CAMERA_COMPLEXITY)].map((match) => match[0].toLowerCase())).size +} + +export function routeShot(frame) { + const mode = frame.fields.visual_mode?.toLowerCase() + const combined = `${frame.fields.action || ''} ${frame.fields.camera || ''} ${frame.fields.scene || ''}` + + if (mode === 'graphic' || mode === 'quiet') { + return { renderer: 'hyperframes', modelId: null, reason: `${mode} shots need exact, deterministic editorial control.` } + } + if (DIALOGUE.test(combined)) { + return { renderer: 'generated-video', modelId: 'veo-3-fast', reason: 'Dialogue or voice action benefits from a model with native audiovisual generation.' } + } + if (HIGH_ACTION.test(combined) || cameraComplexity(frame.fields.camera) >= 2) { + return { renderer: 'generated-video', modelId: 'kling-pro', reason: 'Complex physical action or compound camera choreography needs the motion-control route.' } + } + return { renderer: 'generated-video', modelId: 'seedance-lite', reason: 'Controlled image-to-video is the economical continuity-first default.' } +} + +function continuityLocks(globals, frame) { + return { + subject: frame.fields.subject_lock || globals.continuity_subject || 'Preserve the approved keyframe subject exactly.', + location: frame.fields.location_lock || globals.continuity_location || 'Preserve the established world and spatial logic.', + palette: frame.fields.palette_lock || globals.palette || 'Preserve the approved production palette.', + material: frame.fields.material_lock || globals.material || 'Preserve material texture and surface behavior.', + optics: frame.fields.lens || globals.lens_family || 'Use one coherent lens family across adjacent shots.', + } +} + +function keyframePrompt(frame, globals, locks) { + const world = globals.visual_world ? `${globals.visual_world}. ` : '' + return [ + world, + `Opening composition: ${frame.fields.start_state || frame.fields.scene}.`, + `Depth staging: ${frame.fields.depth}.`, + `Continuity: ${locks.subject} ${locks.location} ${locks.palette} ${locks.material}`, + 'Create a production keyframe, not a poster. No captions or interface chrome.', + ].join(' ').replace(/\s+/g, ' ').trim() +} + +function motionPrompt(frame, locks) { + return [ + `Subject action: ${frame.fields.action}.`, + `Camera: ${frame.fields.camera}.`, + `Transformation: ${frame.fields.transformation}.`, + `Visible turn: ${frame.fields.surprise}.`, + `Final composition: ${frame.fields.end_state || frame.fields.transformation}.`, + `Keep fixed: ${locks.subject} ${locks.location} ${locks.material}`, + 'One continuous event. Preserve crisp geometry. Finish in a stable, editable final pose.', + ].join(' ').replace(/\s+/g, ' ').trim() +} + +function candidateVariants(frame, basePrompt, count) { + const variants = [ + { id: 'faithful', direction: 'Prioritize exact action completion and continuity. Use restrained motion.' }, + { id: 'spatial', direction: `Prioritize foreground occlusion, parallax, and the requested camera path: ${frame.fields.camera}.` }, + { id: 'transform', direction: `Prioritize the irreversible visual transformation and hold the final state: ${frame.fields.transformation}.` }, + { id: 'wildcard', direction: `Preserve every continuity lock, but make this visible surprise unusually bold: ${frame.fields.surprise}.` }, + ] + return variants.slice(0, count).map((variant, index) => ({ + ...variant, + index, + prompt: `${basePrompt} Candidate direction: ${variant.direction}`, + })) +} + +export function compileStoryboard(markdown, options = {}) { + const lint = lintCinematicStoryboard(markdown) + if (!lint.ok && !options.allowInvalid) { + const error = new Error('Storyboard failed the cinematic gate.') + error.findings = lint.findings + throw error + } + + const globals = parseStoryboardGlobals(markdown) + const frames = parseCinematicStoryboard(markdown) + const candidateCount = Math.max(1, Math.min(Number(options.candidates || globals.candidates || 4), 4)) + const shots = frames.map((frame) => { + const route = routeShot(frame) + const locks = continuityLocks(globals, frame) + const seconds = durationSeconds(frame.fields.duration) + const prompt = route.renderer === 'generated-video' ? motionPrompt(frame, locks) : null + const model = route.modelId ? getModel(route.modelId) : null + const candidates = route.renderer === 'generated-video' ? candidateVariants(frame, prompt, candidateCount) : [] + const costPerCandidate = model ? estimateUsd(model.id, seconds) : 0 + + return { + shotId: `shot-${String(frame.index + 1).padStart(2, '0')}`, + title: frame.heading, + mode: frame.fields.visual_mode, + durationSeconds: seconds, + route: { ...route, modelLabel: model?.label || 'HyperFrames' }, + continuityLocks: locks, + inputs: { + startFrameRequired: route.renderer === 'generated-video', + endFrameRecommended: route.renderer === 'generated-video' && Boolean(frame.fields.end_state), + keyframePrompt: route.renderer === 'generated-video' ? keyframePrompt(frame, globals, locks) : null, + negativePrompt: route.renderer === 'generated-video' ? DEFAULT_NEGATIVE : null, + }, + candidates, + review: { rubric: REVIEW_RUBRIC, hardRejects: HARD_REJECTS, passingScore: 82 }, + estimatedUsd: { + perCandidate: costPerCandidate, + batch: Math.round(costPerCandidate * candidates.length * 100) / 100, + }, + handoff: route.renderer === 'hyperframes' + ? 'Author as a deterministic HyperFrames composition.' + : 'Approve the keyframe, generate candidates, reject hard failures, then hand the winner to HyperFrames for type and finishing.', + } + }) + + const generatedShots = shots.filter((shot) => shot.route.renderer === 'generated-video') + return { + version: 1, + source: globals.message || 'Untitled cinematic storyboard', + globals: { + format: globals.format || '1920x1080', + visualWorld: globals.visual_world || null, + rhythm: globals.rhythm || null, + continuity: { + subject: globals.continuity_subject || null, + location: globals.continuity_location || null, + palette: globals.palette || null, + material: globals.material || null, + lensFamily: globals.lens_family || null, + }, + }, + policy: { + candidateCount, + paidGenerationRequiresApproval: true, + keyframeApprovalRequired: true, + selectionMethod: 'hard rejects first, then weighted rubric; highest score above 82 wins', + }, + shots, + totals: { + shots: shots.length, + generatedShots: generatedShots.length, + hyperframesShots: shots.length - generatedShots.length, + candidates: generatedShots.reduce((total, shot) => total + shot.candidates.length, 0), + estimatedUsd: Math.round(generatedShots.reduce((total, shot) => total + shot.estimatedUsd.batch, 0) * 100) / 100, + }, + } +} + +function runCli() { + const args = process.argv.slice(2) + const input = args.find((arg) => !arg.startsWith('--')) + const outputArg = args.find((arg) => arg.startsWith('--output=')) + const candidatesArg = args.find((arg) => arg.startsWith('--candidates=')) + if (!input) { + console.error('Usage: node tools/shot-compiler.mjs [--output=shot-plan.json] [--candidates=1..4]') + process.exitCode = 2 + return + } + + try { + const plan = compileStoryboard(fs.readFileSync(input, 'utf8'), { + candidates: candidatesArg?.split('=')[1], + }) + const json = `${JSON.stringify(plan, null, 2)}\n` + if (outputArg) { + const output = path.resolve(outputArg.slice('--output='.length)) + fs.writeFileSync(output, json) + console.log(`Compiled ${plan.totals.shots} shots and ${plan.totals.candidates} candidates to ${output}`) + console.log(`Estimated generation batch: $${plan.totals.estimatedUsd.toFixed(2)} · approval still required`) + } else { + process.stdout.write(json) + } + } catch (error) { + console.error(error.message) + for (const finding of error.findings || []) console.error(`${finding.level.toUpperCase()} ${finding.code}: ${finding.message}`) + process.exitCode = 1 + } +} + +const cliPath = process.argv[1] ? path.resolve(process.argv[1]) : '' +if (cliPath === fileURLToPath(import.meta.url)) runCli() diff --git a/tools/shot-compiler.test.mjs b/tools/shot-compiler.test.mjs new file mode 100644 index 000000000..05df024d8 --- /dev/null +++ b/tools/shot-compiler.test.mjs @@ -0,0 +1,41 @@ +import { describe, expect, it } from 'vitest' +import { compileStoryboard, parseStoryboardGlobals, routeShot } from './shot-compiler.mjs' +import { parseCinematicStoryboard } from './cinematic-storyboard.mjs' +import fs from 'node:fs' +import path from 'node:path' + +const example = fs.readFileSync(path.resolve('docs/video/STORYBOARD.cinematic.example.md'), 'utf8') + +describe('shot compiler', () => { + it('parses film-wide continuity memory', () => { + const globals = parseStoryboardGlobals(`---\nvisual_world: Paper noir city\ncontinuity_subject: Red animal\n---`) + expect(globals.visual_world).toBe('Paper noir city') + expect(globals.continuity_subject).toBe('Red animal') + }) + + it('routes quiet and graphic work to deterministic finishing', () => { + const frames = parseCinematicStoryboard(example) + expect(routeShot(frames[2]).renderer).toBe('hyperframes') + }) + + it('routes compound cinematic motion to the motion-control model', () => { + const frames = parseCinematicStoryboard(example) + expect(routeShot(frames[3]).modelId).toBe('kling-pro') + }) + + it('compiles prompts, locks, candidates, costs, and review policy', () => { + const plan = compileStoryboard(example, { candidates: 3 }) + expect(plan.totals.shots).toBe(4) + expect(plan.totals.generatedShots).toBe(3) + expect(plan.totals.candidates).toBe(9) + expect(plan.policy.paidGenerationRequiresApproval).toBe(true) + expect(plan.shots[0].inputs.keyframePrompt).toContain('Depth staging') + expect(plan.shots[0].candidates).toHaveLength(3) + expect(plan.shots[0].review.rubric.reduce((sum, item) => sum + item.weight, 0)).toBe(100) + expect(plan.totals.estimatedUsd).toBeGreaterThan(0) + }) + + it('refuses to compile a storyboard that fails the cinematic gate', () => { + expect(() => compileStoryboard('## Frame 1 — Empty\n- visual_mode: graphic')).toThrow('Storyboard failed the cinematic gate.') + }) +}) From 6afcf899b89f742834f25c1ace2ffc09b0d9f898 Mon Sep 17 00:00:00 2001 From: Your Name Date: Wed, 22 Jul 2026 17:02:39 -0700 Subject: [PATCH 3/4] feat(canvas): preserve creative lineage across 26-node films Compile shot plans into keyframe, batch, blind-critic, selection, sound, and master stages. Audit all 68 links for cycles, provenance, approvals, and receipt completeness. --- docs/video/cinematic-shot-system.md | 20 ++ package.json | 3 +- src/pages/CreativeCanvasPage.tsx | 18 ++ tools/creative-graph-compiler.mjs | 258 +++++++++++++++++++++++++ tools/creative-graph-compiler.test.mjs | 44 +++++ 5 files changed, 342 insertions(+), 1 deletion(-) create mode 100644 tools/creative-graph-compiler.mjs create mode 100644 tools/creative-graph-compiler.test.mjs diff --git a/docs/video/cinematic-shot-system.md b/docs/video/cinematic-shot-system.md index de1f57e3b..e90c12b73 100644 --- a/docs/video/cinematic-shot-system.md +++ b/docs/video/cinematic-shot-system.md @@ -135,3 +135,23 @@ The compiler adds the layer that a preset picker cannot: - a finishing handoff for the selected shot. The output is a local JSON plan. Compilation never submits a paid request. A generation runner may consume the plan later, but it must preserve the existing explicit cost-confirm contract. + +## Creative graph + +The shot plan can become a reusable GALLEY canvas workflow: + +```bash +npm run video:compile-graph -- path/to/STORYBOARD.md --output=creative-graph.json +``` + +This incorporates the strongest idea from node-based creative environments—models as inspectable steps—while adding production guarantees that a general canvas does not provide automatically: + +- every node carries typed lineage back to its brief, shot, continuity source, batch, cost, and approval state; +- generated shots expand into approved keyframe → candidate batch → blind critic → selected take; +- the critic sees the work and rubric, not the model brand, limiting reputation bias; +- rejected candidates remain attached to the decision instead of disappearing from history; +- deterministic and generated shots coexist in the same directed acyclic graph; +- sound direction joins the master as a first-class dependency; +- the final master requires a receipt containing sources, prompts, routes, costs, rejections, scores, and approvals. + +The graph uses the existing Creative Canvas node contract and can be loaded through its external state bridge. Its audit rejects cycles, dangling links, missing lineage, missing batch members, and masters without a receipt policy. diff --git a/package.json b/package.json index 4417067c6..2bd759f4b 100644 --- a/package.json +++ b/package.json @@ -44,9 +44,10 @@ "video:palmier:bed": "node tools/palmier/build-room-bed.mjs", "video:palmier:plan": "node tools/palmier/content-plan.mjs", "video:compile-shots": "node tools/shot-compiler.mjs", + "video:compile-graph": "node tools/creative-graph-compiler.mjs", "video:palmier:suite": "node tools/palmier/suite.mjs", "test:palmier": "npx vitest run tools/palmier/suite.test.mjs", - "test:cinematic": "npx vitest run tools/cinematic-storyboard.test.mjs tools/shot-compiler.test.mjs" + "test:cinematic": "npx vitest run tools/cinematic-storyboard.test.mjs tools/shot-compiler.test.mjs tools/creative-graph-compiler.test.mjs" }, "homepage": "https://kernel.chat", "dependencies": { diff --git a/src/pages/CreativeCanvasPage.tsx b/src/pages/CreativeCanvasPage.tsx index fb829a82b..61106817a 100644 --- a/src/pages/CreativeCanvasPage.tsx +++ b/src/pages/CreativeCanvasPage.tsx @@ -136,6 +136,22 @@ interface StudioNode { videoUrl?: string result?: string status?: 'idle' | 'running' | 'done' | 'error' + lineage?: { + role: string + origin?: string + shotId?: string + continuitySource?: string + batchId?: string + candidateId?: string + candidateIndex?: number + estimatedUsd?: number + approval?: string + immutable?: boolean + blindToModel?: boolean + threshold?: number + receiptRequired?: boolean + [key: string]: unknown + } } interface StudioEdge { @@ -277,6 +293,7 @@ interface ExternalCanvasState { imageUrl?: string result?: string status?: StudioNode['status'] + lineage?: StudioNode['lineage'] data?: { value?: string; agentId?: string; modelId?: string; output?: string } }> edges?: Array<{ id: string; from?: string; to?: string; fromNode?: string; toNode?: string }> @@ -316,6 +333,7 @@ function normalizeExternalState(state: ExternalCanvasState): { nodes: StudioNode imageUrl: node.imageUrl, result: node.result ?? node.data?.output, status: node.status ?? 'idle', + lineage: node.lineage, } }) const edges = state.edges diff --git a/tools/creative-graph-compiler.mjs b/tools/creative-graph-compiler.mjs new file mode 100644 index 000000000..6877c195a --- /dev/null +++ b/tools/creative-graph-compiler.mjs @@ -0,0 +1,258 @@ +import fs from 'node:fs' +import path from 'node:path' +import { fileURLToPath } from 'node:url' +import { compileStoryboard } from './shot-compiler.mjs' + +function stableId(...parts) { + return parts.join('-').toLowerCase().replace(/[^a-z0-9-]+/g, '-').replace(/-+/g, '-').replace(/^-|-$/g, '') +} + +function node({ id, kind, x, y, title, content, model, lineage }) { + return { id, kind, x, y, title, content, ...(model ? { model } : {}), status: 'idle', lineage } +} + +function edge(from, to, role) { + return { id: stableId('edge', from, to, role), from, to, role } +} + +export function compileCreativeGraph(plan) { + const nodes = [] + const edges = [] + const batches = [] + const decisions = [] + + const briefId = 'film-brief' + const continuityId = 'continuity-constitution' + const soundId = 'sound-direction' + nodes.push(node({ + id: briefId, + kind: 'prompt', + x: 80, + y: 100, + title: 'Film brief', + content: plan.source, + model: 'Claude 4.5', + lineage: { role: 'brief', origin: 'shot-compiler', immutable: true }, + })) + nodes.push(node({ + id: continuityId, + kind: 'note', + x: 80, + y: 390, + title: 'Continuity constitution', + content: JSON.stringify(plan.globals.continuity, null, 2), + model: 'Manual note', + lineage: { role: 'continuity', origin: briefId, immutable: true }, + })) + nodes.push(node({ + id: soundId, + kind: 'agent', + x: 1540, + y: 80, + title: 'Sound director', + content: `Design environment, material, and punctuation layers for the film rhythm: ${plan.globals.rhythm || 'derive from the approved edit'}. Sound may lead hero events by 2–6 frames.`, + model: 'Director', + lineage: { role: 'sound-direction', origin: briefId, approval: 'required' }, + })) + edges.push(edge(briefId, soundId, 'brief')) + + const selectedShotIds = [] + for (const [shotIndex, shot] of plan.shots.entries()) { + const y = 80 + shotIndex * 360 + const baseLineage = { + shotId: shot.shotId, + mode: shot.mode, + origin: briefId, + continuitySource: continuityId, + estimatedUsd: shot.estimatedUsd.batch, + } + + if (shot.route.renderer === 'hyperframes') { + const compId = stableId(shot.shotId, 'hyperframes') + nodes.push(node({ + id: compId, + kind: 'agent', + x: 480, + y, + title: `${shot.shotId} · deterministic composition`, + content: `${shot.handoff}\n\nShot direction:\n${shot.title}`, + model: 'Coder Agent', + lineage: { ...baseLineage, role: 'composition', approval: 'review' }, + })) + edges.push(edge(briefId, compId, 'brief'), edge(continuityId, compId, 'continuity')) + selectedShotIds.push(compId) + decisions.push({ shotId: shot.shotId, kind: 'deterministic', nodeId: compId, rule: shot.route.reason }) + continue + } + + const keyframeId = stableId(shot.shotId, 'keyframe') + nodes.push(node({ + id: keyframeId, + kind: 'image', + x: 440, + y, + title: `${shot.shotId} · approved keyframe`, + content: `${shot.inputs.keyframePrompt}\n\nAvoid: ${shot.inputs.negativePrompt}`, + model: 'GPT Image', + lineage: { ...baseLineage, role: 'keyframe', approval: 'required-before-motion' }, + })) + edges.push(edge(briefId, keyframeId, 'brief'), edge(continuityId, keyframeId, 'continuity')) + + const batchId = stableId(shot.shotId, 'motion-batch') + const candidateIds = [] + for (const candidate of shot.candidates) { + const candidateId = stableId(shot.shotId, 'candidate', String(candidate.index + 1)) + candidateIds.push(candidateId) + nodes.push(node({ + id: candidateId, + kind: 'video', + x: 800, + y: y + candidate.index * 72, + title: `${shot.shotId} · ${candidate.id}`, + content: candidate.prompt, + model: shot.route.modelLabel, + lineage: { + ...baseLineage, + role: 'candidate', + batchId, + candidateId: candidate.id, + candidateIndex: candidate.index, + approval: 'paid-generation', + estimatedUsd: shot.estimatedUsd.perCandidate, + }, + })) + edges.push(edge(keyframeId, candidateId, 'start-frame'), edge(continuityId, candidateId, 'continuity')) + } + batches.push({ + id: batchId, + shotId: shot.shotId, + strategy: 'parallel creative directions', + nodeIds: candidateIds, + estimatedUsd: shot.estimatedUsd.batch, + approval: 'human-required', + }) + + const criticId = stableId(shot.shotId, 'critic') + const winnerId = stableId(shot.shotId, 'winner') + nodes.push(node({ + id: criticId, + kind: 'agent', + x: 1160, + y, + title: `${shot.shotId} · blind critic`, + content: `Reject any hard failure first. Then score surviving candidates with this rubric:\n${JSON.stringify(shot.review, null, 2)}\nDo not reward spectacle that violates the brief or continuity constitution. Return scores, evidence, and one winner.`, + model: 'VFX Supervisor', + lineage: { ...baseLineage, role: 'critic', blindToModel: true, approval: 'automatic-review' }, + })) + for (const candidateId of candidateIds) edges.push(edge(candidateId, criticId, 'candidate-for-review')) + edges.push(edge(continuityId, criticId, 'continuity-standard')) + + nodes.push(node({ + id: winnerId, + kind: 'output', + x: 1500, + y: y + 180, + title: `${shot.shotId} · selected take`, + content: `Promote only a candidate scoring at least ${shot.review.passingScore}/100 with no hard reject. Preserve the critic's evidence and rejected alternatives in the receipt.`, + model: 'Compiled result', + lineage: { ...baseLineage, role: 'selection', approval: 'human-final', threshold: shot.review.passingScore }, + })) + edges.push(edge(criticId, winnerId, 'decision')) + for (const candidateId of candidateIds) edges.push(edge(candidateId, winnerId, 'candidate-lineage')) + selectedShotIds.push(winnerId) + decisions.push({ shotId: shot.shotId, kind: 'competitive-selection', criticNodeId: criticId, winnerNodeId: winnerId, candidates: candidateIds }) + } + + const masterId = 'film-master' + nodes.push(node({ + id: masterId, + kind: 'output', + x: 1880, + y: 320, + title: 'Film master and receipt', + content: 'Assemble approved shots and sound in story order. Emit the master plus a receipt containing every source, prompt, model route, cost, rejection, score, approval, and finishing decision.', + model: 'Compiled result', + lineage: { role: 'master', origin: briefId, approval: 'human-final', receiptRequired: true }, + })) + for (const selectedId of selectedShotIds) edges.push(edge(selectedId, masterId, 'approved-shot')) + edges.push(edge(soundId, masterId, 'sound-direction')) + + return { + version: 1, + projectName: plan.source, + nodes, + edges, + view: { x: 0, y: 0, zoom: 0.7 }, + batches, + decisions, + policy: { + modelsAreStepsNotDestinations: true, + preserveRejectedAlternatives: true, + paidGenerationRequiresApproval: true, + keyframesBeforeMotion: true, + blindCriticToModelIdentity: true, + receiptRequired: true, + }, + } +} + +export function auditCreativeGraph(graph) { + const findings = [] + const nodeIds = new Set(graph.nodes.map((item) => item.id)) + const indegree = new Map(graph.nodes.map((item) => [item.id, 0])) + const children = new Map(graph.nodes.map((item) => [item.id, []])) + for (const item of graph.edges) { + if (!nodeIds.has(item.from) || !nodeIds.has(item.to)) findings.push({ level: 'error', code: 'dangling-edge', edgeId: item.id }) + else { + indegree.set(item.to, (indegree.get(item.to) || 0) + 1) + children.get(item.from).push(item.to) + } + } + const queue = graph.nodes.filter((item) => (indegree.get(item.id) || 0) === 0).map((item) => item.id) + let visited = 0 + while (queue.length) { + const id = queue.shift() + visited += 1 + for (const child of children.get(id) || []) { + indegree.set(child, indegree.get(child) - 1) + if (indegree.get(child) === 0) queue.push(child) + } + } + if (visited !== graph.nodes.length) findings.push({ level: 'error', code: 'cycle' }) + if (!graph.nodes.every((item) => item.lineage?.role)) findings.push({ level: 'error', code: 'missing-lineage' }) + if (!graph.policy?.receiptRequired) findings.push({ level: 'error', code: 'missing-receipt-policy' }) + for (const batch of graph.batches || []) { + if (batch.nodeIds.length < 2) findings.push({ level: 'warning', code: 'single-candidate-batch', batchId: batch.id }) + if (batch.nodeIds.some((id) => !nodeIds.has(id))) findings.push({ level: 'error', code: 'missing-batch-node', batchId: batch.id }) + } + return { ok: !findings.some((item) => item.level === 'error'), findings } +} + +function runCli() { + const args = process.argv.slice(2) + const input = args.find((arg) => !arg.startsWith('--')) + const outputArg = args.find((arg) => arg.startsWith('--output=')) + if (!input) { + console.error('Usage: node tools/creative-graph-compiler.mjs [--output=creative-graph.json]') + process.exitCode = 2 + return + } + try { + const markdown = fs.readFileSync(input, 'utf8') + const graph = compileCreativeGraph(compileStoryboard(markdown)) + const audit = auditCreativeGraph(graph) + if (!audit.ok) throw new Error(`Creative graph audit failed: ${JSON.stringify(audit.findings)}`) + const output = outputArg ? path.resolve(outputArg.slice('--output='.length)) : null + const json = `${JSON.stringify(graph, null, 2)}\n` + if (output) { + fs.writeFileSync(output, json) + console.log(`Compiled ${graph.nodes.length} nodes, ${graph.edges.length} links, and ${graph.batches.length} candidate batches to ${output}`) + } else process.stdout.write(json) + } catch (error) { + console.error(error.message) + process.exitCode = 1 + } +} + +const cliPath = process.argv[1] ? path.resolve(process.argv[1]) : '' +if (cliPath === fileURLToPath(import.meta.url)) runCli() diff --git a/tools/creative-graph-compiler.test.mjs b/tools/creative-graph-compiler.test.mjs new file mode 100644 index 000000000..b27c057e7 --- /dev/null +++ b/tools/creative-graph-compiler.test.mjs @@ -0,0 +1,44 @@ +import fs from 'node:fs' +import path from 'node:path' +import { describe, expect, it } from 'vitest' +import { compileStoryboard } from './shot-compiler.mjs' +import { auditCreativeGraph, compileCreativeGraph } from './creative-graph-compiler.mjs' + +const storyboard = fs.readFileSync(path.resolve('docs/video/STORYBOARD.cinematic.example.md'), 'utf8') + +describe('creative graph compiler', () => { + const graph = compileCreativeGraph(compileStoryboard(storyboard, { candidates: 3 })) + + it('turns each generated shot into keyframe, candidate, critic, and selection stages', () => { + const shotOne = graph.nodes.filter((item) => item.lineage?.shotId === 'shot-01') + expect(shotOne.filter((item) => item.lineage.role === 'keyframe')).toHaveLength(1) + expect(shotOne.filter((item) => item.lineage.role === 'candidate')).toHaveLength(3) + expect(shotOne.filter((item) => item.lineage.role === 'critic')).toHaveLength(1) + expect(shotOne.filter((item) => item.lineage.role === 'selection')).toHaveLength(1) + }) + + it('preserves rejected alternatives and requires a receipt', () => { + expect(graph.policy.preserveRejectedAlternatives).toBe(true) + expect(graph.policy.receiptRequired).toBe(true) + expect(graph.nodes.find((item) => item.id === 'film-master').lineage.receiptRequired).toBe(true) + }) + + it('creates auditable candidate batches with human spend approval', () => { + expect(graph.batches).toHaveLength(3) + expect(graph.batches.every((batch) => batch.nodeIds.length === 3)).toBe(true) + expect(graph.batches.every((batch) => batch.approval === 'human-required')).toBe(true) + }) + + it('produces an acyclic graph with complete lineage', () => { + expect(auditCreativeGraph(graph)).toEqual({ ok: true, findings: [] }) + }) + + it('detects a broken provenance graph', () => { + const broken = structuredClone(graph) + delete broken.nodes[0].lineage + broken.edges.push({ id: 'bad', from: 'missing', to: 'film-master', role: 'bad' }) + const audit = auditCreativeGraph(broken) + expect(audit.ok).toBe(false) + expect(audit.findings.map((item) => item.code)).toEqual(expect.arrayContaining(['missing-lineage', 'dangling-edge'])) + }) +}) From bc4cb5bba285619ab4af3ce97993eb5de295842a Mon Sep 17 00:00:00 2001 From: Your Name Date: Wed, 22 Jul 2026 17:09:24 -0700 Subject: [PATCH 4/4] docs(video): park the 3D asset tournament Record provider roles, six implementation phases, objective Blender checks, and activation conditions while explicitly authorizing no integration or spend. --- docs/video/3d-asset-system-plan.md | 124 +++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 docs/video/3d-asset-system-plan.md diff --git a/docs/video/3d-asset-system-plan.md b/docs/video/3d-asset-system-plan.md new file mode 100644 index 000000000..5e6d139a3 --- /dev/null +++ b/docs/video/3d-asset-system-plan.md @@ -0,0 +1,124 @@ +# 3D Asset Tournament + +Status: PARKED — reference plan only. Do not implement, subscribe, generate, or spend without a new explicit request. + +## Purpose + +Add persistent physical assets to the cinematic system when a film needs real camera parallax, repeatable subjects, stable props, or reusable environments. The system should own provider selection, comparison, cleanup, provenance, and delivery instead of depending on one generation company. + +## Proposed provider roles + +| Role | Provider | +|---|---| +| Default hosted generation | Tripo | +| Premium hero-asset challenger | Hyper3D Rodin | +| Rigging and animation specialist | Meshy | +| Local research route | TRELLIS.2 and Hunyuan3D | +| Cleanup and final authority | Blender | +| Optional real-time delivery | Unity | + +No provider is the permanent winner. Models remain replaceable stages. + +## Pipeline + +```text +approved references + ↓ +provider candidates + ↓ +local download and provenance receipt + ↓ +Blender normalization and inspection + ↓ +standardized turntable renders + ↓ +geometry, material, and continuity scoring + ↓ +human selection + ↓ +cleanup, rigging, and animation + ↓ +cinematic render or Unity handoff + ↓ +HyperFrames typography and finishing +``` + +## Evaluation rubric + +Every candidate should be inspected under identical lighting and camera conditions for: + +- silhouette fidelity; +- multiview consistency; +- manifold geometry and hidden surfaces; +- topology and deformation readiness; +- polygon budget; +- UV integrity and texture seams; +- PBR map completeness; +- scale, orientation, origin, and pivot correctness; +- rig deformation where applicable; +- Blender and Unity import health; +- continuity with the approved references and neighboring shots. + +Hard failures are non-manifold geometry that prevents intended use, missing load-bearing surfaces, severe identity drift, unusable UVs, corrupted exports, or license/provenance uncertainty. + +## Recommended pilot + +Budget ceiling: $50. + +1. Generate ten reference assets through Tripo. +2. Send the strongest five references through Meshy. +3. Compare no more than two hero assets with Rodin. +4. Import every candidate into Blender. +5. Normalize scale, orientation, pivot, camera, and lighting. +6. Render identical turntables and inspection passes. +7. Score results without showing the evaluator the provider name. +8. Record quality, failure rate, latency, and actual cost per accepted asset. +9. Decide whether a multi-provider system beats a Tripo-only pipeline by enough to justify its complexity. + +## Expected operating cost + +Indicative only; re-verify provider pricing before activation. + +- Lean stack without Rodin: approximately $30–$80 per month. +- Tripo + Meshy + Rodin pilot stack: approximately $150–$250 per month. +- Three-provider comparison: approximately $0.90–$1.40 per asset before retakes. +- Adding a cloud-hosted open model may raise a comparison to approximately $1.40–$4.40. +- Broad candidate tournaments may cost approximately $4–$13 per accepted asset. + +## Implementation phases + +### Phase 1 — provider-neutral contract + +Define job submission, status, result download, estimated cost, supported inputs, output formats, and immutable receipts. Reuse GALLEY's existing estimate-before-spend and local-ownership rules. + +### Phase 2 — Tripo adapter + +Support text, image, and multiview generation; PBR options; topology controls; async polling; and local GLB download. This is the first integration because it offers the best balance of breadth, cost, and accessibility. + +### Phase 3 — Blender inspection + +Automate import, transforms, manifold checks, polygon counts, material inventory, missing texture detection, UV checks, turntable setup, and inspection renders. Blender becomes the authoritative asset record. + +### Phase 4 — provider tournament + +Add Meshy and Rodin adapters behind the same contract. Generate controlled candidates, blind the evaluator to provider identity, retain rejected alternatives, and promote only candidates clearing the rubric. + +### Phase 5 — local research models + +Evaluate TRELLIS.2 and Hunyuan3D when suitable compute is available. Use them for private drafts, cost control, and scientific independence rather than assuming they automatically replace hosted production services. + +### Phase 6 — delivery + +Add optional rigging, animation, cinematic Blender rendering, Unity validation, and HyperFrames compositing. Ship the asset, source references, license context, prompts, costs, checks, and transformation history together. + +## Activation conditions + +Do not begin implementation until a real film or interactive artifact needs at least one of: + +- the same object from several camera angles; +- persistent character or prop identity across shots; +- physically correct camera parallax or occlusion; +- a reusable environment; +- a deliverable GLB, FBX, OBJ, USDZ, or Unity asset. + +At activation time, re-check provider models, API availability, prices, retention, training policy, commercial terms, and rate limits. Never handle provider secrets in the repository, and never submit a paid task without the existing explicit approval gate.