Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions docs/video/3d-asset-system-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# 3D Asset Tournament

Status: PARKED — reference plan only. Do not implement, subscribe, generate, or spend without a new explicit request.

## Purpose

Add persistent physical assets to the cinematic system when a film needs real camera parallax, repeatable subjects, stable props, or reusable environments. The system should own provider selection, comparison, cleanup, provenance, and delivery instead of depending on one generation company.

## Proposed provider roles

| Role | Provider |
|---|---|
| Default hosted generation | Tripo |
| Premium hero-asset challenger | Hyper3D Rodin |
| Rigging and animation specialist | Meshy |
| Local research route | TRELLIS.2 and Hunyuan3D |
| Cleanup and final authority | Blender |
| Optional real-time delivery | Unity |

No provider is the permanent winner. Models remain replaceable stages.

## Pipeline

```text
approved references
provider candidates
local download and provenance receipt
Blender normalization and inspection
standardized turntable renders
geometry, material, and continuity scoring
human selection
cleanup, rigging, and animation
cinematic render or Unity handoff
HyperFrames typography and finishing
```

## Evaluation rubric

Every candidate should be inspected under identical lighting and camera conditions for:

- silhouette fidelity;
- multiview consistency;
- manifold geometry and hidden surfaces;
- topology and deformation readiness;
- polygon budget;
- UV integrity and texture seams;
- PBR map completeness;
- scale, orientation, origin, and pivot correctness;
- rig deformation where applicable;
- Blender and Unity import health;
- continuity with the approved references and neighboring shots.

Hard failures are non-manifold geometry that prevents intended use, missing load-bearing surfaces, severe identity drift, unusable UVs, corrupted exports, or license/provenance uncertainty.

## Recommended pilot

Budget ceiling: $50.

1. Generate ten reference assets through Tripo.
2. Send the strongest five references through Meshy.
3. Compare no more than two hero assets with Rodin.
4. Import every candidate into Blender.
5. Normalize scale, orientation, pivot, camera, and lighting.
6. Render identical turntables and inspection passes.
7. Score results without showing the evaluator the provider name.
8. Record quality, failure rate, latency, and actual cost per accepted asset.
9. Decide whether a multi-provider system beats a Tripo-only pipeline by enough to justify its complexity.

## Expected operating cost

Indicative only; re-verify provider pricing before activation.

- Lean stack without Rodin: approximately $30–$80 per month.
- Tripo + Meshy + Rodin pilot stack: approximately $150–$250 per month.
- Three-provider comparison: approximately $0.90–$1.40 per asset before retakes.
- Adding a cloud-hosted open model may raise a comparison to approximately $1.40–$4.40.
- Broad candidate tournaments may cost approximately $4–$13 per accepted asset.

## Implementation phases

### Phase 1 — provider-neutral contract

Define job submission, status, result download, estimated cost, supported inputs, output formats, and immutable receipts. Reuse GALLEY's existing estimate-before-spend and local-ownership rules.

### Phase 2 — Tripo adapter

Support text, image, and multiview generation; PBR options; topology controls; async polling; and local GLB download. This is the first integration because it offers the best balance of breadth, cost, and accessibility.

### Phase 3 — Blender inspection

Automate import, transforms, manifold checks, polygon counts, material inventory, missing texture detection, UV checks, turntable setup, and inspection renders. Blender becomes the authoritative asset record.

### Phase 4 — provider tournament

Add Meshy and Rodin adapters behind the same contract. Generate controlled candidates, blind the evaluator to provider identity, retain rejected alternatives, and promote only candidates clearing the rubric.

### Phase 5 — local research models

Evaluate TRELLIS.2 and Hunyuan3D when suitable compute is available. Use them for private drafts, cost control, and scientific independence rather than assuming they automatically replace hosted production services.

### Phase 6 — delivery

Add optional rigging, animation, cinematic Blender rendering, Unity validation, and HyperFrames compositing. Ship the asset, source references, license context, prompts, costs, checks, and transformation history together.

## Activation conditions

Do not begin implementation until a real film or interactive artifact needs at least one of:

- the same object from several camera angles;
- persistent character or prop identity across shots;
- physically correct camera parallax or occlusion;
- a reusable environment;
- a deliverable GLB, FBX, OBJ, USDZ, or Unity asset.

At activation time, re-check provider models, API availability, prices, retention, training policy, commercial terms, and rate limits. Never handle provider secrets in the repository, and never submit a paid task without the existing explicit approval gate.
79 changes: 79 additions & 0 deletions docs/video/STORYBOARD.cinematic.example.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
format: 1920x1080
message: "The machine does not make the decision; it carries the decision into the world."
arc: Signal → Pursuit → Refusal → Transformation → Proof → Quiet
audience: kernel.chat readers and creative-tool builders
mode: autonomous
rhythm: impact → pursuit → stillness → transformation → proof → hold
visual_world: "Cinematic macro paper-craft noir with tangible fibers, warm practical light, deep ink shadow, and one tomato-red signal"
continuity_subject: "The same small tomato-red folded-paper animal, sharp triangular ears, one white paper eye, no anatomy changes"
continuity_location: "One coherent nocturnal paper city connected to the same warm workshop"
palette: "Ivory paper, brown-black ink, tomato red, and warm practical amber only"
material: "Hand-cut paper with visible fibers, crisp folds, miniature practical construction"
lens_family: "Anamorphic macro family with shallow depth of field and consistent oval bokeh"
candidates: 4
---

## Frame 1 — The signal escapes

- visual_mode: cinematic
- scene: A red signal tears out of a printed sentence and escapes into a nocturnal paper city.
- duration: 5s
- transition_in: cut
- action: The final word tears free, folds into a red paper animal, and runs off the page.
- camera: A macro push becomes a low tracking chase alongside the animal.
- depth: Foreground paper fibers whip past; the midground animal runs; background rooftops wake in sequence.
- transformation: A static printed claim becomes a living signal loose in the city.
- surprise: The punctuation mark stays behind and turns its head to watch.
- start_state: An extreme macro of an intact typeset sentence.
- end_state: A wide street with the red animal disappearing around a corner.

The typography begins as evidence and becomes matter. The action—not a title entrance—creates the cut into the city.

## Frame 2 — The ledger wall

- visual_mode: hybrid
- scene: The camera travels beside an architectural ledger whose exact measurements open physical routes through the city.
- duration: 6s
- transition_in: velocity-matched whip
- action: Each measured figure punches a doorway through the ledger and the signal chooses the middle route.
- camera: A lateral dolly tracks the figures, then rack-focuses from the foreground 53 to the midground 459 doorway.
- depth: Foreground numerals cross the lens; the midground signal enters 459; background 1,407 recedes into haze.
- transformation: A flat comparison becomes three traversable routes with different physical costs.
- surprise: The largest number opens the smallest, most obstructed door.
- start_state: A ruled wall carrying three exact measurements.
- end_state: The middle doorway remains open with warm light beyond it.

The numbers remain exact and typeset in the publication system. Their spatial consequence makes the comparison felt.

## Frame 3 — Refusal

- visual_mode: quiet
- scene: A hand rests beside an unpressed approval lever while the workshop waits without punishment.
- duration: 4s
- transition_in: hard cut
- action: Dust settles and the machine deliberately powers down when the hand withdraws.
- camera: A locked macro shot performs one slow rack focus from the lever to the resting machine.
- depth: Foreground hand exits; midground lever remains untouched; background machine light dims to black.
- transformation: A waiting system becomes a safely refused system.
- surprise: The room grows warmer after the machine turns off.
- start_state: Lever, hand, and machine suspended in expectation.
- end_state: Empty lever in a calm, warm room.

Stillness is the event here. It is earned by the pursuit before it.

## Frame 4 — The proof returns

- visual_mode: cinematic
- scene: The red signal returns carrying a film canister that unfolds into the original printed page.
- duration: 6s
- transition_in: light leak
- action: The animal leaps onto the workbench, unfolds into a ribbon, and binds the canister into a finished spread.
- camera: A crane drops with the leap, orbits the binding action, then pulls back to reveal the whole room.
- depth: Foreground tools briefly occlude the binding; the midground ribbon transforms; background workers turn toward the result.
- transformation: A moving signal becomes a durable artifact owned in the room.
- surprise: The finished spread contains the opening sentence, now changed by one word.
- start_state: A dark workbench awaiting the returning signal.
- end_state: A completed spread under warm light, with the city visible beyond it.

The closing image pays off the opening without returning to the same state.
157 changes: 157 additions & 0 deletions docs/video/cinematic-shot-system.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# The Shot Is the Unit

Status: production law for generated and HyperFrames-authored films.

The existing system is excellent at making resolved editorial frames. This layer prevents those frames from becoming a slideshow. A frame is now evidence of a shot; the shot is the authored unit.

## The contract

Every storyboard frame must declare these fields:

```md
- visual_mode: cinematic | hybrid | graphic | quiet
- action: The physical event and its consequence.
- camera: The camera's observable path or lens change.
- depth: Foreground, midground, and background behavior.
- transformation: What is irreversibly different by the end.
- surprise: The turn the opening image does not predict.
- start_state: Optional but recommended description of the opening image.
- end_state: Optional but recommended description of the closing image.
```

`action` is not an entrance animation. “The title fades in” describes software. “The title tears through the paper and exposes the workshop behind it” describes an event.

`camera` is not a transition. Crossfade, wipe, and push-slide describe the edit between shots. Orbit, crane, dive, track, rack focus, whip, and pull-back describe perception inside the shot.

`depth` names all three planes and how they interact. Foreground material should occasionally occlude the subject. Midground owns the action. Background establishes scale, atmosphere, or consequence.

`transformation` must change the visual proposition. A larger number is usually not a transformation. A ledger flooding until it becomes a city is.

`surprise` must be visible, not merely narrated. It may be small and quiet, but the opening image must not fully predict it.

## Mode budget

Across a film:

- At least 50% of frames are `cinematic` or `hybrid`.
- `graphic` frames carry measurements, proof, typography, and diagrams.
- `quiet` frames are earned rests and should remain at or below 20%.
- Brand typography and the tomato/ink/ivory palette connect the worlds; they do not have to occupy every pixel.

This is a floor, not a recipe. A generated narrative film may be almost entirely cinematic. A data feature may sit near the minimum and use hybrid mechanisms to carry evidence.

## Shot construction order

Author each beat in this order:

1. Emotional beat — what changes in the viewer.
2. Physical event — what happens in the world.
3. Camera path — how the viewer discovers it.
4. Depth stack — what passes before, around, and behind the subject.
5. Transformation — the ending image and its consequence.
6. Surprise — the turn.
7. Graphic layer — typography, labels, readings, and publication marks.

Layout comes last. The still poster is selected from the shot, not used as the shot's source of truth.

## Rhythm law

Use contrast, not constant frenzy. Name the film's rhythm before building it, for example:

`impact → pursuit → stillness → transformation → proof → eruption → hold`

At least one adjacent pair must change two or more of these dimensions:

- scale: macro / human / architectural
- velocity: still / drifting / fast
- camera: locked / travelling / unstable
- material: paper / light / liquid / physical object / typography
- density: sparse / layered / crowded

Dynamic does not mean everything moves. It means movement changes meaning.

## Hybrid shot recipe

Hybrid is the native kernel.chat mode. It lets factual graphics live inside a physical world:

- A measured number is printed on a ticket moving through a real chute.
- A comparison grid is a wall the camera travels along.
- A waveform becomes a thread that pulls the next scene open.
- A ruled frame is a window into a generated environment.
- A typographic word becomes matter: paper, stencil, shadow, signage, or an object the subject handles.

The evidence stays exact. The presentation gains consequence.

## Generation direction

For generated shots, prompts describe one subject, one action, and one camera instruction. Keep identity and geometry constraints separate from motion. Prefer image-to-video when subject continuity matters. A motion prompt should say what moves, what remains fixed, how the camera moves, and what the final state is.

Avoid asking one short clip to perform several unrelated transformations. Use the edit for discontinuity and the shot for one legible event.

## HyperFrames direction

HyperFrames remains responsible for exact typography, data, diagrams, compositing, and deterministic finishing. Generated video supplies physical worlds and hard-to-simulate events. Three.js or shaders may supply fully deterministic spatial shots when appropriate.

Do not fake cinematography by applying the same 3% scale push to every flat frame. Camera motion needs parallax, occlusion, focus change, or a meaningful change of viewpoint.

## Audio direction

Every hero event gets an audio consequence. Design three layers:

- environment: the space exists before the action;
- material: paper, metal, breath, glass, motor, cloth;
- punctuation: the single hit, cut, silence, or tonal change that marks transformation.

Sound should sometimes lead the picture. A cue arriving 2–6 frames early creates anticipation; silence immediately after impact gives the event weight.

## Gate

Run:

```bash
npm run lint:cinematic -- path/to/STORYBOARD.md
```

The gate rejects missing shot fields, cinematic frames made only from layout reveals, and films without a cinematic/hybrid majority. Warnings identify weak camera verbs and incomplete depth plans.

The lint is intentionally semantic and conservative. Passing it does not make a shot good. Failing it means the plan has not yet described a shot.

## Compile

Once the gate passes, compile the storyboard into an executable production plan:

```bash
npm run video:compile-shots -- path/to/STORYBOARD.md --output=shot-plan.json
```

The compiler adds the layer that a preset picker cannot:

- film-wide subject, location, palette, material, and lens locks;
- per-shot routing between generated video and deterministic HyperFrames work;
- separate keyframe and motion prompts;
- four deliberately different candidates instead of four accidental retries;
- hard-reject criteria followed by a 100-point selection rubric;
- estimated batch cost with both keyframe and paid-generation approval gates intact;
- a finishing handoff for the selected shot.

The output is a local JSON plan. Compilation never submits a paid request. A generation runner may consume the plan later, but it must preserve the existing explicit cost-confirm contract.

## Creative graph

The shot plan can become a reusable GALLEY canvas workflow:

```bash
npm run video:compile-graph -- path/to/STORYBOARD.md --output=creative-graph.json
```

This incorporates the strongest idea from node-based creative environments—models as inspectable steps—while adding production guarantees that a general canvas does not provide automatically:

- every node carries typed lineage back to its brief, shot, continuity source, batch, cost, and approval state;
- generated shots expand into approved keyframe → candidate batch → blind critic → selected take;
- the critic sees the work and rubric, not the model brand, limiting reputation bias;
- rejected candidates remain attached to the decision instead of disappearing from history;
- deterministic and generated shots coexist in the same directed acyclic graph;
- sound direction joins the master as a first-class dependency;
- the final master requires a receipt containing sources, prompts, routes, costs, rejections, scores, and approvals.

The graph uses the existing Creative Canvas node contract and can be loaded through its external state bridge. Its audit rejects cycles, dangling links, missing lineage, missing batch members, and masters without a receipt policy.
6 changes: 5 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
"lint:adherence": "node scripts/check-adherence.mjs",
"lint:editorial": "node scripts/check-editorial.mjs",
"lint:cinematic": "node tools/cinematic-storyboard.mjs",
"artifacts:index": "node scripts/build-artifact-index.mjs",
"sitemap": "node scripts/build-sitemap.mjs",
"preview": "vite preview",
Expand Down Expand Up @@ -42,8 +43,11 @@
"video:palmier:cuts": "node tools/palmier/cut-sheet.mjs",
"video:palmier:bed": "node tools/palmier/build-room-bed.mjs",
"video:palmier:plan": "node tools/palmier/content-plan.mjs",
"video:compile-shots": "node tools/shot-compiler.mjs",
"video:compile-graph": "node tools/creative-graph-compiler.mjs",
"video:palmier:suite": "node tools/palmier/suite.mjs",
"test:palmier": "npx vitest run tools/palmier/suite.test.mjs"
"test:palmier": "npx vitest run tools/palmier/suite.test.mjs",
"test:cinematic": "npx vitest run tools/cinematic-storyboard.test.mjs tools/shot-compiler.test.mjs tools/creative-graph-compiler.test.mjs"
},
"homepage": "https://kernel.chat",
"dependencies": {
Expand Down
Loading
Loading