Agentic Edition Studio

by RodGrc · built while making Haiku

A starter for teaching an AI coding agent to edit video. Picture and sound become timed evidence. The edit is React. The judgment lives in files the agent can open one page at a time.

This repository is the empty studio. It does not contain finished videos, personal notes, or a sound library.

Why it exists

Editing is not only cutting. Someone looks, listens, and decides what is worth telling. An agent has four gaps for that job:

Gap	What goes wrong	What this repo does about it
Video	The model sees sampled stills, not motion. A 60 s clip at 30 fps is 1,800 frames; sending 1 per second skips 29 of every 30.	sureshot picks frames by events (cuts, motion, text) and keeps a timeline.
Audio	The model gets, at best, a transcript.	sureshot adds speech with timestamps, silences and signal events.
Editing environment	Editors are made for hands and eyes.	The edit is a Remotion (React) project: every cut is a line you can review.
Context	Your brand and rules do not fit in one prompt.	AGENTS.md, PROJECT.md and a wiki/ with an index, read page by page.

Pieces

Piece	Path	Role
Editor	this folder	Remotion project, one folder per video
Perception	.agents/skills/sureshot/	Speech, on-screen text, cuts, motion, silence in one timeline
Remotion rules	.agents/skills/remotion-best-practices/	How to write a composition
Judgment	wiki/	Index plus the pages that apply to the job
Sounds	assets/audio/	Shelves and a catalog shape. No audio files yet
Manual	AGENTS.md	How the agent should work in any project

Workflow

ingest -> understand -> decide -> produce -> verify
ffmpeg    sureshot      agent     Remotion    sureshot + you
          (+ Gemini)    + wiki    + FFmpeg

Whatever fails in verify goes back to decide. A render with no error can still have bad cuts, drifting audio or empty stretches.

Requirements

• Node.js 18 or newer
• Windows, with Python 3.10 to 3.13 for sureshot and the sound scripts

sureshot installs the rest (ffmpeg, speech, OCR, Gemini key) with a guided command that asks before each step:

python .agents/skills/sureshot/scripts/sureshot.py setup --plan
python .agents/skills/sureshot/scripts/sureshot.py setup
python .agents/skills/sureshot/scripts/sureshot.py doctor

doctor confirms everything works. Details: .agents/skills/sureshot/INSTALL.md. OCR uses Tesseract by default; PaddleOCR is optional (setup --ocr paddle). A free Gemini key from Google AI Studio is only needed for the agent and digest commands.

The sound scripts need numpy:

python -m pip install -r scripts/requirements.txt

Media you add later (mp3, wav, mp4, png) is marked for Git LFS in .gitattributes. Install Git LFS before committing those files.

Quick start

npm install
npm run dev

Studio opens on Example: 1080x1920, 30 fps, 150 frames.

Use it with an agent

1. Open this folder in your agent (Codex, Claude Code, Cursor). It reads AGENTS.md and the skills in .agents/skills/.

2. Put the source video in a new project:

   npm run new -- my-reel --width 1080 --height 1920 --fps 30 --frames 150

   Register the snippet it prints in src/Root.tsx and drop files in that project's assets/.

3. Ask for the job. Example prompt:

   Edit a 30 s reel from projects/<folder>/assets/source.mp4. Probe it, analyze it with sureshot, read wiki/INDEX.md and open only the pages that apply, write the beat table and the decision in PROJECT.md, then build it in Remotion and verify sync.

4. Review the render. Corrections go back to the agent as frame numbers.

Using sureshot by itself

python .agents/skills/sureshot/scripts/sureshot.py analyze video.mp4 --stt --ocr
python .agents/skills/sureshot/scripts/sureshot.py view video.mp4 --map
python .agents/skills/sureshot/scripts/sureshot.py inspect video.mp4 00:03-00:05
python .agents/skills/sureshot/scripts/sureshot.py report video.mp4

Command	What you get
analyze	Five layers on one clock: structure (cuts), text (OCR), motion, audio (silence, energy), speech
view --map	Where there is activity, without rereading everything
inspect	Frames only for the range you ask about
report	Evidence, metrics and provenance as YAML
agent / digest	Optional Gemini summary of the stored timeline

sureshot does not interpret. It leaves evidence so the agent can decide and cite timestamps. Full manual: .agents/skills/sureshot/SKILL.md.

Project layout

projects/<YYMMDD-name>/
  PROJECT.md    brief, beat table, decisions, what the edit taught
  Video.tsx     main composition
  assets/       inputs worth keeping
  _scratch/     temp and analysis frames (gitignored)
  renders/      exported mp4 (gitignored)

Wiki and sounds

• wiki/INDEX.md lists the pages. The agent reads the index first and opens only what applies. Principles and cases are empty on purpose: add pages from the section templates as you learn.
• assets/audio/LIBRARY.md describes the sound shelves and catalog shape. scripts/prep_sfx.py, snap_cues.py, gen_placement.py and verify_sync.py land a sound on a frame and check the hit.

Links

• Portfolio: rod.run
• Haiku: haiku.ac

License

MIT. See LICENSE.