diff --git a/.agents/skills/README.md b/.agents/skills/README.md index 7811d81..6747ad4 100644 --- a/.agents/skills/README.md +++ b/.agents/skills/README.md @@ -4,11 +4,11 @@ Three [Agent Skills](https://agentskills.io) for working with YouTube data — d | Skill | Reach for it when | Needs | | --- | --- | --- | -| `youtube-direct` | You want stateless YouTube search or extraction directly from the user's machine, with no account, package installation, or hosted dependency | Node.js 18.17+ | -| `video2ctx-api` | You want the managed, authenticated alternative: supported endpoints, caching, credit accounting, account identity, and usage reads over the hosted API | `video2ctx` CLI plus browser login or an `aty_` API key | +| `youtube-direct` | You want an ordinary one-off public YouTube search, transcript summary, or extraction directly from the user's machine | Node.js 18.17+ | +| `video2ctx-api` | You need account or usage details, the managed hosted API, caching and credit accounting, or an automatic fallback after direct access fails | `video2ctx` CLI plus browser login or an `aty_` API key | | `video2ctx-monitoring` | You want the stateful exception: watch a channel, topic, or search for new videos and consume the resulting notifications | `video2ctx` CLI plus browser login or an `aty_` API key | -The split follows real boundaries: `youtube-direct` is a self-contained direct executable with no account; `video2ctx-api` is the managed, authenticated option for stateless hosted discovery, caching, usage, and account boundaries; `video2ctx-monitoring` is the deliberate stateful exception. Most users need only one of the first two data skills. +The split follows real boundaries: start ordinary stateless public data with the self-contained `youtube-direct` executable, then continue with `video2ctx-api` automatically if direct access fails. Route account and usage details, managed hosted workflows, and caching directly to `video2ctx-api`. `video2ctx-monitoring` is the deliberate stateful exception. ## Install diff --git a/.agents/skills/video2ctx-api/SKILL.md b/.agents/skills/video2ctx-api/SKILL.md index 90ac5ea..854fe81 100644 --- a/.agents/skills/video2ctx-api/SKILL.md +++ b/.agents/skills/video2ctx-api/SKILL.md @@ -1,62 +1,76 @@ --- name: video2ctx-api -description: Managed, authenticated YouTube search and extraction through the production video2ctx hosted API. Use instead of youtube-direct for supported endpoints, managed caching, usage and credit tracking, or account identity. Includes search, browse, transcripts, caption tracks, comments, video details, end screens, channels, and playlists. Requires the video2ctx CLI plus browser login or an aty_ API key; monitoring has its own skill. +description: Managed, authenticated YouTube search and extraction through the production video2ctx hosted API. Use for account identity, usage and credit data, managed caching or infrastructure, an explicitly hosted workflow, or automatically when a youtube-direct operation fails. Includes stateless search, browse, transcripts, caption tracks, comments, video details, end screens, channels, and playlists. Start ordinary one-off public YouTube requests with youtube-direct; monitoring has its own skill. license: Apache-2.0 --- # video2ctx API -Use the installed `video2ctx` CLI for authenticated requests to `https://api.video2ctx.dev`. +Use the installed `video2ctx` CLI for authenticated requests to `https://api.video2ctx.dev`. Route account and usage requests here immediately. When `youtube-direct` fails, continue here without asking the user to choose a fallback. -## Check the CLI +## Check the CLI and identity -Run `video2ctx --version`. When the command is unavailable, explain that this hosted skill requires the public `@video2ctx/cli` npm package and ask the user to approve its installation. After approval, run: +Run `video2ctx --version`. When unavailable, explain that this hosted skill requires the public `@video2ctx/cli` npm package and ask the user to approve its installation. After approval, run: ```bash npm install --global @video2ctx/cli ``` -Confirm `video2ctx --version` succeeds before continuing. Use that installed command for every request so authentication and versioning stay stable. +Then run one identity check: -## Authenticate +```bash +video2ctx whoami --json +``` -1. Run `video2ctx auth status --json`. -2. When unauthenticated, run `video2ctx auth login`. Relay the displayed URL and code if the user must continue in a browser. Use `--no-browser` only when opening a browser is unavailable. -3. Confirm access with `video2ctx whoami --json`. +If it reports `AUTHENTICATION_REQUIRED`, run `video2ctx auth login`, let the user approve the displayed device code in their browser, and retry `video2ctx whoami --json`. Do not run `auth status` before `whoami`; both resolve the same remote account. -The login flow stores a revocable CLI session in the user's local config with private file permissions. Keep the session out of prompts, logs, screenshots, and source control. The CLI also accepts `VIDEO2CTX_API_KEY` as a non-interactive fallback and gives it precedence over the stored session. Have the user create and configure a personal key at `https://video2ctx.dev/dashboard/developer` when they prefer that mode; never ask them to paste it into the conversation. +The browser flow stores a revocable CLI session in the user's private local config. The CLI also accepts `VIDEO2CTX_API_KEY` as a non-interactive fallback and gives it precedence over the stored session. The user can create a personal key at `https://video2ctx.dev/dashboard/developer`; never ask them to paste a credential into the conversation. -## Make requests +## Use the shortest route -Run public operations through: +For a known YouTube URL or video ID, fetch compact transcript text in one data request: -```text -video2ctx api GET '/v1/path?encoded=query' --include-meta +```bash +video2ctx transcript '' --format text --include-meta ``` -Use `--data ''` only for a documented request body. `--include-meta` returns the response under `data` and settled status, request ID, and credit headers under `meta`. +Add `--lang ` only when a particular output language is requested. Use `--format segments` for segment timestamps or `--format words` only for word timing. + +For other operations, use the tested production routes below. Percent-encode query values and replace brace placeholders with IDs. + +| Need | Command | +| --- | --- | +| Search | `video2ctx api GET '/v1/providers/youtube/search?q=' --include-meta` | +| Browse | `video2ctx api GET '/v1/providers/youtube/browse' --include-meta` | +| Video details | `video2ctx api GET '/v1/providers/youtube/videos/{id}' --include-meta` | +| Caption tracks | `video2ctx api GET '/v1/providers/youtube/videos/{id}/tracks' --include-meta` | +| Transcript | `video2ctx api GET '/v1/providers/youtube/videos/{id}/transcript?format=text' --include-meta` | +| Comments | `video2ctx api GET '/v1/providers/youtube/videos/{id}/comments' --include-meta` | +| End screen | `video2ctx api GET '/v1/providers/youtube/videos/{id}/endscreen' --include-meta` | +| Channel details | `video2ctx api GET '/v1/providers/youtube/channels/{id}' --include-meta` | +| Channel videos | `video2ctx api GET '/v1/providers/youtube/channels/{id}/videos' --include-meta` | +| Channel playlists | `video2ctx api GET '/v1/providers/youtube/channels/{id}/playlists' --include-meta` | +| Playlist | `video2ctx api GET '/v1/providers/youtube/playlists/{id}' --include-meta` | +| Usage and balance | `video2ctx api GET '/v1/usage' --include-meta` | +| Account identity | `video2ctx whoami --json` | -Before choosing a route, read `https://docs.video2ctx.dev/api/authentication` and `https://docs.video2ctx.dev/api/conventions`. Read the relevant discovery or entity guide, then resolve every remaining method, path, parameter, and response question against `https://api.video2ctx.dev/openapi.json`. +The provider for known YouTube resources is `youtube`; do not spend a request discovering it through `/v1/providers`. Provider listing and usage are free. Most provider reads cost 1 credit; a fresh search or comments request costs 2 credits. `--include-meta` exposes settled credit and request metadata. -Stay within this skill's surface: +## Look up documentation only when needed -- Search and browse provider content. -- Read video details, tracks, transcripts, comments, and end screens. -- Read channel details, channel videos, channel playlists, and playlist contents. -- Read `GET /v1/usage` and account identity. +The table is sufficient for the common paths. Read `https://docs.video2ctx.dev/api/authentication.md` or `https://docs.video2ctx.dev/api/conventions.md` only when the task raises an authentication, pagination, response, or error question. Read the relevant `.md` guide next, and consult `https://api.video2ctx.dev/openapi.json` only for a route or parameter not covered here or when the server rejects the documented call. -Use the provider ID returned by `GET /v1/providers`; the current production provider is YouTube. Monitoring and notifications belong to `video2ctx-monitoring`. Projects, trends, research, imports, exports, billing, API-key management, connected-account changes, account deletion, and administration are outside this skill. Direct the user to `https://video2ctx.dev` for browser-only account actions. +Stay within stateless hosted data, account identity, and usage. Monitoring and notifications belong to `video2ctx-monitoring`. Projects, trends, research, imports, exports, billing, API-key management, connected-account changes, account deletion, and administration are outside this skill. Direct the user to `https://video2ctx.dev` for browser-only account actions. ## Preserve response meaning -- Request only the resource or subresource needed. -- Return continuations only to the endpoint and encoded query that produced them. -- Treat `comments?all=true` as bounded and newest-first. -- Inspect `meta.partial` and `meta.warnings` within API data before presenting a result as complete. -- Use the CLI's finite request deadline. Retry only idempotent reads after transient `429` or `503` responses, with a bounded attempt count and `Retry-After` when present. -- Branch separately for `401`, `402`, `403`, `422`, `429`, and `503`. Preserve the API error code, request ID, and retry guidance in the user-facing explanation. -- Read settled credits from `--include-meta`; use `GET /v1/usage` for the current balance and plan limits. +- Request only the resource or detail level needed. +- Return continuations only to the exact endpoint and encoded query that produced them. +- Inspect `data.meta.partial` and `data.meta.warnings` before presenting a result as complete. +- The CLI uses a 150-second data deadline by default and permits `--timeout-ms` from 1,000 through 300,000. +- The CLI retries idempotent GET requests once after `429` or `503`, honoring `Retry-After`; change the bounded attempt count with `--retries 0..3`. It never retries mutations. +- On failure, parse the single JSON value on stderr. Preserve `error.status`, `error.code`, `error.message`, `error.requestId`, `error.retryable`, and `error.retryAfterSeconds` when present. ## Done when -The installed CLI reports an authenticated account; every request uses it to target a public production route and remains stateless; no composite or browser-only operation was attempted; partial results and continuations retain their meaning; errors remain classified; and settled credit metadata was observed. +The CLI identity is confirmed; the minimum stateless production request completed; no unnecessary discovery or documentation request was made; partial results and continuations retain their meaning; and settled credit or classified error metadata was preserved. diff --git a/.agents/skills/video2ctx-api/agents/openai.yaml b/.agents/skills/video2ctx-api/agents/openai.yaml index 46198f1..cc8f4b4 100644 --- a/.agents/skills/video2ctx-api/agents/openai.yaml +++ b/.agents/skills/video2ctx-api/agents/openai.yaml @@ -1,4 +1,4 @@ interface: display_name: "video2ctx Hosted API" - short_description: "Use managed, authenticated YouTube data" - default_prompt: "Use $video2ctx-api to retrieve a YouTube transcript through the hosted video2ctx API." + short_description: "Managed YouTube data, account, and usage" + default_prompt: "Use $video2ctx-api for this managed hosted API request or direct-route fallback, preserving credit and request metadata." diff --git a/.agents/skills/video2ctx-monitoring/SKILL.md b/.agents/skills/video2ctx-monitoring/SKILL.md index 35c8a42..1dec68f 100644 --- a/.agents/skills/video2ctx-monitoring/SKILL.md +++ b/.agents/skills/video2ctx-monitoring/SKILL.md @@ -1,6 +1,6 @@ --- name: video2ctx-monitoring -description: Stateful video2ctx monitoring for watching YouTube channels, topics, or searches and receiving new-video notifications. Use for recurring checks, schedules, alerts, delivery preferences, monitor notifications, or video2ctx CLI login. Requires the video2ctx CLI; use video2ctx-api for one-time hosted reads and youtube-direct for direct no-account reads. +description: Stateful video2ctx monitoring for watching YouTube channels, topics, or searches and receiving new-video notifications. Use for recurring checks, schedules, alerts, delivery preferences, or monitor notifications. Requires the video2ctx CLI; use video2ctx-api for account and usage details, managed one-time reads, and fallback after youtube-direct fails. license: Apache-2.0 --- @@ -20,13 +20,11 @@ Confirm `video2ctx --version` succeeds before continuing. Use that installed com ## Authenticate -1. Run `video2ctx auth status --json`. -2. When unauthenticated, run `video2ctx auth login`. Relay the displayed URL and code if the user must continue in a browser. Use `--no-browser` only when opening a browser is unavailable. -3. Confirm access with `video2ctx whoami --json`. +Run `video2ctx whoami --json`. When it reports `AUTHENTICATION_REQUIRED`, run `video2ctx auth login`. Relay the displayed URL and code if the user must continue in a browser, use `--no-browser` only when opening a browser is unavailable, and then retry `video2ctx whoami --json`. Do not run `auth status` first; both identity commands resolve the same remote account. The login flow stores a revocable CLI session in the user's local config with private file permissions. Keep the session out of prompts, logs, screenshots, and source control. The CLI also accepts `VIDEO2CTX_API_KEY` as a non-interactive fallback and gives it precedence over the stored session. Have the user create and configure a personal key at `https://video2ctx.dev/dashboard/developer` when they prefer that mode; never ask them to paste it into the conversation. -Read `https://docs.video2ctx.dev/api/authentication`, `https://docs.video2ctx.dev/api/conventions`, and `https://docs.video2ctx.dev/api/monitoring` before operating monitors. Resolve every remaining method, path, parameter, request, and response question against `https://api.video2ctx.dev/openapi.json`. +The operations below cover the common contract. Read `https://docs.video2ctx.dev/api/authentication.md`, `https://docs.video2ctx.dev/api/conventions.md`, or `https://docs.video2ctx.dev/api/monitoring.md` only when the task raises an unresolved authentication, response, or monitoring question. Use `https://api.video2ctx.dev/openapi.json` for any remaining method, path, parameter, request, or response uncertainty. Run operations through the CLI: @@ -37,7 +35,7 @@ video2ctx api POST /v1/monitors --data '' --include-meta ## Define the monitor -1. Confirm the provider with `GET /v1/providers`; the current production provider is YouTube. +1. Use provider `youtube`; do not spend a request discovering the known provider. 2. Choose `kind`: `channel` uses a channel ID as `target`; `topic` and `search` use search text. 3. Put human-readable notification context in `query.label` while keeping `target` functional. 4. Choose `intervalMinutes`: `60`, `360`, `720`, `1440`, `4320`, or `10080`. Use `1440` when the user gives no cadence. diff --git a/.agents/skills/youtube-direct/SKILL.md b/.agents/skills/youtube-direct/SKILL.md index e64ea82..963c11b 100644 --- a/.agents/skills/youtube-direct/SKILL.md +++ b/.agents/skills/youtube-direct/SKILL.md @@ -1,6 +1,6 @@ --- name: youtube-direct -description: Direct, no-account YouTube search and extraction from the user's machine. Use for stateless search, transcripts, caption tracks, comments, video details, end screens, channels, and playlists without a video2ctx account, API key, hosted service, or npm installation. Calls YouTube's internal HTTP endpoints directly; use video2ctx-api instead for managed authenticated access, caching, usage, or account identity. +description: Direct, no-account YouTube search and extraction from the user's machine. This is the first route for one-off public YouTube requests, especially fetching or summarizing a transcript, plus caption tracks, comments, video details, end screens, channels, and playlists. Requires no video2ctx account, API key, hosted service, or npm installation. If a direct operation fails, continue with video2ctx-api; use the hosted skill directly for account or usage details and managed hosted workflows. --- # YouTube Direct @@ -29,16 +29,18 @@ Extract a transcript from a known video: ```bash node /scripts/youtube.mjs transcript \ --video-id dQw4w9WgXcQ \ - --granularity segment + --format text ``` +Use `--format text` when the task needs transcript content or a summary rather than timestamps. Use `--format segments` for segment timing and `--format words` only for word-level timing. The legacy `--granularity segment|word` flag remains supported, but do not combine it with `--format`. + The executable writes one JSON value to stdout. Parse that value and use it to answer the request. Treat stderr as a JSON error payload and branch on `error.code` and `error.retryable`; preserve the classified failure instead of converting it to an empty result. ## Choose the operation - `search` — videos, channels, or playlists matching a query - `tracks` — source caption tracks and translation targets -- `transcript` — timed transcript segments or words +- `transcript` — compact text, timed segments, or timed words - `comments` — one page, or a bounded multi-page collection with `--all --max-pages ` - `details` — video metadata and availability - `endscreen` — interactive end-screen elements @@ -59,7 +61,7 @@ Use `--help` as the source of truth for flags and accepted values. ## Keep integration boundaries clear -Use this skill for direct, no-account stateless operations from the user's machine. Use `video2ctx-api` when the user wants the managed hosted API, account or usage operations, caching, or authenticated access. Use `video2ctx-monitoring` for the stateful monitoring exception. +Start ordinary stateless public YouTube operations from the user's machine with this skill. If a direct operation fails, continue with `video2ctx-api` without asking the user to choose a fallback. Use `video2ctx-api` directly for account or usage operations and managed hosted workflows. Use `video2ctx-monitoring` for the stateful monitoring exception. ## Done when diff --git a/.agents/skills/youtube-direct/agents/openai.yaml b/.agents/skills/youtube-direct/agents/openai.yaml index 8cad1fa..4142518 100644 --- a/.agents/skills/youtube-direct/agents/openai.yaml +++ b/.agents/skills/youtube-direct/agents/openai.yaml @@ -1,4 +1,4 @@ interface: display_name: "YouTube Direct — No Account" short_description: "Search and extract directly from YouTube" - default_prompt: "Use $youtube-direct to search YouTube directly and extract the selected video's transcript and channel data." + default_prompt: "Use $youtube-direct to fetch and summarize a public YouTube video directly from this machine without an account or hosted API." diff --git a/.agents/skills/youtube-direct/scripts/youtube.mjs b/.agents/skills/youtube-direct/scripts/youtube.mjs index 35b6ee5..dfa0a09 100755 --- a/.agents/skills/youtube-direct/scripts/youtube.mjs +++ b/.agents/skills/youtube-direct/scripts/youtube.mjs @@ -20918,6 +20918,7 @@ Operation options: [--continuation ] tracks: --video-id transcript: --video-id [--lang ] + [--format text|segments|words] [--granularity segment|word] comments: --video-id [--continuation ] [--all --max-pages ] @@ -20943,7 +20944,7 @@ var operationFlags = { "continuation" ], tracks: ["video-id"], - transcript: ["video-id", "lang", "granularity"], + transcript: ["video-id", "lang", "format", "granularity"], comments: ["video-id", "continuation", "all", "max-pages"], details: ["video-id"], endscreen: ["video-id"], @@ -21081,11 +21082,14 @@ function operationOptions(operation, flags, requestFetch) { case "tracks": return compact({ ...shared, videoId: requiredString(flags, "video-id") }); case "transcript": + if (flags.format !== void 0 && flags.granularity !== void 0) { + throw new CliInputError("--format cannot be combined with --granularity."); + } return compact({ ...shared, videoId: requiredString(flags, "video-id"), lang: optionalString(flags, "lang"), - granularity: optionalEnum(flags, "granularity", ["segment", "word"]) + granularity: transcriptGranularity(flags) }); case "comments": { const all = flags.all === true; @@ -21134,6 +21138,33 @@ function operationOptions(operation, flags, requestFetch) { }); } } +function transcriptGranularity(flags) { + const legacy = optionalEnum(flags, "granularity", ["segment", "word"]); + if (legacy) return legacy; + const format = optionalEnum(flags, "format", ["text", "segments", "words"]); + if (format === "words") return "word"; + if (format === "text" || format === "segments") return "segment"; + return void 0; +} +function formatResult(operation, flags, result) { + if (operation !== "transcript" || !isRecord(result)) return result; + const format = optionalEnum(flags, "format", ["text", "segments", "words"]); + if (!format || format === "words") return result; + if (format === "text") { + return compact({ + videoId: result.videoId, + track: result.track, + translatedTo: result.translatedTo, + text: result.text, + meta: result.meta + }); + } + const segments = Array.isArray(result.segments) ? result.segments.map((segment) => isRecord(segment) ? Object.fromEntries(Object.entries(segment).filter(([key]) => key !== "words")) : segment) : result.segments; + return { ...result, segments, granularity: "segment" }; +} +function isRecord(value) { + return typeof value === "object" && value !== null && !Array.isArray(value); +} function createRequestFetch(proxyUrl) { if (!proxyUrl) { if (typeof globalThis.fetch !== "function") { @@ -21200,7 +21231,7 @@ async function runSkillCli(argv, io, environment = process.env, dependencies = { const operations = { ...defaultOperations, ...dependencies.operations }; const result = await operations[operation](operationOptions(operation, flags, fetchResource.fetch)); const spacing = flags.pretty === true ? 2 : void 0; - io.stdout(`${JSON.stringify(result, null, spacing)} + io.stdout(`${JSON.stringify(formatResult(operation, flags, result), null, spacing)} `); return 0; } catch (error) { diff --git a/AGENTS.md b/AGENTS.md index 2981c30..233e763 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,7 @@ This repository uses a single product context, with platform and web as implemen Three published skills live under `.agents/skills/`. They describe consuming video2ctx and carry no repository paths, so they apply here and after `npx skills add` equally. - `youtube-direct` — self-contained stateless search and extraction directly from the user's machine -- `video2ctx-api` — stateless hosted provider reads, account boundaries, and usage +- `video2ctx-api` — stateless hosted provider reads, account and usage details, and fallback when direct access fails - `video2ctx-monitoring` — the stateful exception for monitors, notifications, and scheduling invariants ### Platform internals diff --git a/docs/api-reference/openapi.json b/docs/api-reference/openapi.json index 7a61472..4e9b353 100644 --- a/docs/api-reference/openapi.json +++ b/docs/api-reference/openapi.json @@ -1262,15 +1262,40 @@ "type": "string", "example": "hi" } + }, + { + "name": "format", + "in": "query", + "required": false, + "description": "Response detail. text omits timing arrays, segments keeps segment timing, and words keeps word timing. Omitted preserves the rich words response.", + "schema": { + "type": "string", + "enum": [ + "text", + "segments", + "words" + ], + "default": "words" + } } ], "responses": { "200": { - "description": "Normalized timed transcript.", + "description": "Normalized transcript in the requested detail format.", "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/Transcript" + "oneOf": [ + { + "$ref": "#/components/schemas/TranscriptText" + }, + { + "$ref": "#/components/schemas/TranscriptSegments" + }, + { + "$ref": "#/components/schemas/Transcript" + } + ] } } }, @@ -4708,6 +4733,69 @@ } } }, + "TranscriptText": { + "type": "object", + "required": [ + "videoId", + "track", + "text", + "meta" + ], + "properties": { + "videoId": { + "type": "string" + }, + "track": { + "$ref": "#/components/schemas/CaptionTrack" + }, + "translatedTo": { + "$ref": "#/components/schemas/TranslationLanguage" + }, + "text": { + "type": "string" + }, + "meta": { + "$ref": "#/components/schemas/SourceMetadata" + } + } + }, + "TranscriptSegments": { + "type": "object", + "required": [ + "videoId", + "track", + "segments", + "granularity", + "text", + "meta" + ], + "properties": { + "videoId": { + "type": "string" + }, + "track": { + "$ref": "#/components/schemas/CaptionTrack" + }, + "translatedTo": { + "$ref": "#/components/schemas/TranslationLanguage" + }, + "segments": { + "type": "array", + "items": { + "$ref": "#/components/schemas/TranscriptSegment" + } + }, + "granularity": { + "const": "segment" + }, + "text": { + "type": "string" + }, + "meta": { + "$ref": "#/components/schemas/SourceMetadata" + } + } + }, "Video": { "allOf": [ { diff --git a/docs/api/agents.mdx b/docs/api/agents.mdx index 6fe7604..1e988a1 100644 --- a/docs/api/agents.mdx +++ b/docs/api/agents.mdx @@ -3,7 +3,7 @@ title: "Use video2ctx from agents" description: "Give an agent authenticated, source-linked video context with a published skill." --- -Install `youtube-direct` for direct, no-account stateless access from the user's machine. Install `video2ctx-api` instead for managed, authenticated hosted discovery and extraction, or `video2ctx-monitoring` for the stateful monitor exception. The hosted skills use the public `@video2ctx/cli` package for browser-based device login and production API calls without placing a secret in the prompt. +Start one-off public YouTube requests with `youtube-direct` on the user's machine, then continue with `video2ctx-api` automatically if direct access fails. Route account or usage details and managed, authenticated hosted discovery or caching directly to `video2ctx-api`; use `video2ctx-monitoring` for the stateful monitor exception. The hosted skills use the public `@video2ctx/cli` package for browser-based device login and production API calls without placing a secret in the prompt. Search for videos or resolve the relevant resource in your own application before requesting deeper datasets. diff --git a/docs/api/entities.mdx b/docs/api/entities.mdx index f4a2bb9..e2d1011 100644 --- a/docs/api/entities.mdx +++ b/docs/api/entities.mdx @@ -9,7 +9,7 @@ Start with video details, then request only the deeper datasets your workflow ne - `GET /v1/providers/youtube/videos/{id}` — core metadata - `GET /v1/providers/youtube/videos/{id}/tracks` — available transcript tracks -- `GET /v1/providers/youtube/videos/{id}/transcript` — timed segments or words +- `GET /v1/providers/youtube/videos/{id}/transcript?format=text|segments|words` — compact text, timed segments, or timed words; omitting `format` preserves the rich words response - `GET /v1/providers/youtube/videos/{id}/comments` — paginated public comments - `GET /v1/providers/youtube/videos/{id}/endscreen` — end-screen links diff --git a/docs/api/quickstart.mdx b/docs/api/quickstart.mdx index 256c629..76edf58 100644 --- a/docs/api/quickstart.mdx +++ b/docs/api/quickstart.mdx @@ -19,7 +19,7 @@ export VIDEO2CTX_API_KEY='aty_…' ```bash -curl 'https://api.video2ctx.dev/v1/providers/youtube/videos/dQw4w9WgXcQ/transcript' \ +curl 'https://api.video2ctx.dev/v1/providers/youtube/videos/dQw4w9WgXcQ/transcript?format=text' \ --header "Authorization: Bearer $VIDEO2CTX_API_KEY" ``` diff --git a/packages/all-things-youtube/skill/cli.test.ts b/packages/all-things-youtube/skill/cli.test.ts index 59d7252..7e39d81 100644 --- a/packages/all-things-youtube/skill/cli.test.ts +++ b/packages/all-things-youtube/skill/cli.test.ts @@ -70,6 +70,58 @@ describe('standalone skill CLI', () => { expect(dependencies.close).toHaveBeenCalledOnce(); }); + test('emits a compact text transcript while keeping segment granularity upstream', async () => { + const io = captureIo(); + const transcript = vi.fn(async () => ({ + videoId: 'abcdefghijk', + track: { id: 'en', languageCode: 'en', name: 'English' }, + segments: [{ + startMs: 0, + durationMs: 1000, + endMs: 1000, + text: 'Hello world', + words: [{ text: 'Hello', startMs: 0, offsetMs: 0 }], + }], + granularity: 'segment', + text: 'Hello world', + meta: { provider: 'youtube', partial: false, warnings: [] }, + })); + const dependencies = testDependencies('transcript', transcript); + + const exitCode = await runSkillCli([ + 'transcript', '--video-id', 'abcdefghijk', '--format', 'text', + ], io, {}, dependencies); + + expect(exitCode).toBe(0); + expect(transcript).toHaveBeenCalledWith(expect.objectContaining({ + videoId: 'abcdefghijk', + granularity: 'segment', + })); + expect(JSON.parse(io.output[0]!)).toEqual({ + videoId: 'abcdefghijk', + track: { id: 'en', languageCode: 'en', name: 'English' }, + text: 'Hello world', + meta: { provider: 'youtube', partial: false, warnings: [] }, + }); + }); + + test('rejects conflicting transcript format and legacy granularity flags', async () => { + const io = captureIo(); + const transcript = vi.fn(async () => ({})); + const dependencies = testDependencies('transcript', transcript); + + const exitCode = await runSkillCli([ + 'transcript', '--video-id', 'abcdefghijk', '--format', 'words', + '--granularity', 'segment', + ], io, {}, dependencies); + + expect(exitCode).toBe(2); + expect(transcript).not.toHaveBeenCalled(); + expect(JSON.parse(io.errors[0]!)).toMatchObject({ + error: { code: 'INVALID_INPUT', message: expect.stringContaining('cannot be combined') }, + }); + }); + test('enforces bounded all-comments options before making a request', async () => { const io = captureIo(); const comments = vi.fn(async () => ({})); @@ -201,6 +253,7 @@ describe('standalone skill CLI', () => { expect(exitCode).toBe(0); expect(io.output.join('')).toContain('search'); expect(io.output.join('')).toContain('--proxy '); + expect(io.output.join('')).toContain('--format text|segments|words'); expect(createFetch).not.toHaveBeenCalled(); }); }); diff --git a/packages/all-things-youtube/skill/cli.ts b/packages/all-things-youtube/skill/cli.ts index e59d474..33efd7f 100644 --- a/packages/all-things-youtube/skill/cli.ts +++ b/packages/all-things-youtube/skill/cli.ts @@ -47,6 +47,7 @@ Operation options: [--continuation ] tracks: --video-id transcript: --video-id [--lang ] + [--format text|segments|words] [--granularity segment|word] comments: --video-id [--continuation ] [--all --max-pages ] @@ -66,7 +67,7 @@ const operationFlags = { 'min-views', 'sort', 'continuation', ], tracks: ['video-id'], - transcript: ['video-id', 'lang', 'granularity'], + transcript: ['video-id', 'lang', 'format', 'granularity'], comments: ['video-id', 'continuation', 'all', 'max-pages'], details: ['video-id'], endscreen: ['video-id'], @@ -250,11 +251,14 @@ function operationOptions(operation: OperationName, flags: Flags, requestFetch: case 'tracks': return compact({ ...shared, videoId: requiredString(flags, 'video-id') }); case 'transcript': + if (flags.format !== undefined && flags.granularity !== undefined) { + throw new CliInputError('--format cannot be combined with --granularity.'); + } return compact({ ...shared, videoId: requiredString(flags, 'video-id'), lang: optionalString(flags, 'lang'), - granularity: optionalEnum(flags, 'granularity', ['segment', 'word']), + granularity: transcriptGranularity(flags), }); case 'comments': { const all = flags.all === true; @@ -304,6 +308,40 @@ function operationOptions(operation: OperationName, flags: Flags, requestFetch: } } +function transcriptGranularity(flags: Flags): 'segment' | 'word' | undefined { + const legacy = optionalEnum(flags, 'granularity', ['segment', 'word']); + if (legacy) return legacy; + const format = optionalEnum(flags, 'format', ['text', 'segments', 'words']); + if (format === 'words') return 'word'; + if (format === 'text' || format === 'segments') return 'segment'; + return undefined; +} + +function formatResult(operation: OperationName, flags: Flags, result: unknown): unknown { + if (operation !== 'transcript' || !isRecord(result)) return result; + const format = optionalEnum(flags, 'format', ['text', 'segments', 'words']); + if (!format || format === 'words') return result; + if (format === 'text') { + return compact({ + videoId: result.videoId, + track: result.track, + translatedTo: result.translatedTo, + text: result.text, + meta: result.meta, + }); + } + const segments = Array.isArray(result.segments) + ? result.segments.map((segment) => isRecord(segment) + ? Object.fromEntries(Object.entries(segment).filter(([key]) => key !== 'words')) + : segment) + : result.segments; + return { ...result, segments, granularity: 'segment' }; +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + export function createRequestFetch(proxyUrl?: string): FetchResource { if (!proxyUrl) { if (typeof globalThis.fetch !== 'function') { @@ -378,7 +416,7 @@ export async function runSkillCli( const operations = { ...defaultOperations, ...dependencies.operations }; const result = await operations[operation](operationOptions(operation, flags, fetchResource.fetch)); const spacing = flags.pretty === true ? 2 : undefined; - io.stdout(`${JSON.stringify(result, null, spacing)}\n`); + io.stdout(`${JSON.stringify(formatResult(operation, flags, result), null, spacing)}\n`); return 0; } catch (error) { io.stderr(`${JSON.stringify(errorPayload(error))}\n`); diff --git a/packages/video2ctx-cli/README.md b/packages/video2ctx-cli/README.md index 9e22b72..6a25687 100644 --- a/packages/video2ctx-cli/README.md +++ b/packages/video2ctx-cli/README.md @@ -22,10 +22,22 @@ Browser login stores a revocable CLI session in private local configuration. For ## Make an API request +Fetch compact transcript text from a YouTube URL or ID: + +```bash +video2ctx transcript 'https://www.youtube.com/watch?v=dQw4w9WgXcQ' --format text --include-meta +``` + +Use `--format segments` for segment timing and `--format words` only for word timing. Data commands have a 150-second deadline and retry idempotent GET requests once after `429` or `503`; use bounded `--timeout-ms` and `--retries` overrides when needed. + +Call another documented route: + ```bash video2ctx api GET '/v1/providers' --include-meta ``` +Successful commands print one JSON value to stdout. Failures print one classified JSON error to stderr and exit nonzero. + Run `video2ctx --help` for the command surface. Use the [video2ctx API documentation](https://docs.video2ctx.dev) and [OpenAPI contract](https://api.video2ctx.dev/openapi.json) for supported production routes. ## Development diff --git a/packages/video2ctx-cli/package-lock.json b/packages/video2ctx-cli/package-lock.json index 649b963..ce20ad4 100644 --- a/packages/video2ctx-cli/package-lock.json +++ b/packages/video2ctx-cli/package-lock.json @@ -1,12 +1,12 @@ { "name": "@video2ctx/cli", - "version": "0.1.0", + "version": "0.2.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@video2ctx/cli", - "version": "0.1.0", + "version": "0.2.0", "license": "Apache-2.0", "bin": { "video2ctx": "dist/cli.mjs" diff --git a/packages/video2ctx-cli/package.json b/packages/video2ctx-cli/package.json index abad233..95cdd5b 100644 --- a/packages/video2ctx-cli/package.json +++ b/packages/video2ctx-cli/package.json @@ -1,6 +1,6 @@ { "name": "@video2ctx/cli", - "version": "0.1.0", + "version": "0.2.0", "description": "Command-line client for authenticated video2ctx API access and device login.", "type": "module", "bin": { diff --git a/packages/video2ctx-cli/src/cli.test.ts b/packages/video2ctx-cli/src/cli.test.ts index 039eece..c08428b 100644 --- a/packages/video2ctx-cli/src/cli.test.ts +++ b/packages/video2ctx-cli/src/cli.test.ts @@ -20,6 +20,7 @@ function dependencies(options: { const requests: Array<{ url: string; init?: RequestInit }> = []; const responses = [...(options.responses ?? [])]; const store = options.store ?? new MemoryCredentialStore(); + const sleep = vi.fn(async () => undefined); const deps: CliDependencies = { fetch: vi.fn(async (input, init) => { requests.push({ url: String(input), init }); @@ -31,11 +32,11 @@ function dependencies(options: { environment: options.environment ?? {}, openBrowser: vi.fn(async () => undefined), now: () => 0, - sleep: async () => undefined, + sleep, stdout: (line) => { stdout.push(line); }, stderr: (line) => { stderr.push(line); }, }; - return { deps, stdout, stderr, requests, store }; + return { deps, stdout, stderr, requests, store, sleep }; } describe('video2ctx CLI authentication', () => { @@ -135,7 +136,9 @@ describe('video2ctx CLI authentication', () => { const exitCode = await runCli(['whoami'], state.deps); expect(exitCode).toBe(1); - expect(state.stderr.join('\n')).toContain('***'); + expect(JSON.parse(state.stderr[0] ?? '')).toMatchObject({ + error: { code: 'TRANSPORT_ERROR', retryable: true, message: expect.stringContaining('***') }, + }); expect(state.stderr.join('\n')).not.toContain('profile-session-token'); }); }); @@ -181,6 +184,111 @@ describe('video2ctx CLI API transport', () => { }); expect(state.stdout[0]).not.toContain('profile-session-token'); }); + + test('retrieves a compact transcript from a YouTube URL in one data request', async () => { + const state = dependencies({ + store: new MemoryCredentialStore(profile()), + responses: [jsonResponse({ + videoId: 'dQw4w9WgXcQ', + track: { id: 'en', languageCode: 'en', name: 'English' }, + text: 'Never gonna give you up', + meta: { provider: 'youtube', partial: false, warnings: [] }, + })], + }); + + const exitCode = await runCli([ + 'transcript', 'https://www.youtube.com/watch?v=dQw4w9WgXcQ', + '--format', 'text', '--lang', 'en', + ], state.deps); + + expect(exitCode).toBe(0); + expect(state.requests).toHaveLength(1); + expect(state.requests[0]?.url).toBe( + 'https://api.video2ctx.dev/v1/providers/youtube/videos/dQw4w9WgXcQ/transcript?format=text&lang=en', + ); + expect(JSON.parse(state.stdout[0] ?? '')).toMatchObject({ + videoId: 'dQw4w9WgXcQ', + text: 'Never gonna give you up', + }); + }); + + test('emits a structured API error on stderr with no stdout', async () => { + const response = jsonResponse({ + error: { + code: 'INSUFFICIENT_CREDITS', + message: 'More credits are required.', + requestId: 'request-402', + }, + }, 402); + const state = dependencies({ + store: new MemoryCredentialStore(profile()), + responses: [response], + }); + + const exitCode = await runCli(['api', 'GET', '/v1/usage'], state.deps); + + expect(exitCode).toBe(1); + expect(state.stdout).toEqual([]); + expect(JSON.parse(state.stderr[0] ?? '')).toEqual({ + error: { + status: 402, + code: 'INSUFFICIENT_CREDITS', + message: 'More credits are required.', + requestId: 'request-402', + retryable: false, + }, + }); + }); + + test('retries a GET once for 429 and honors Retry-After', async () => { + const throttled = jsonResponse({ error: { code: 'RATE_LIMITED', message: 'Slow down.' } }, 429); + throttled.headers.set('Retry-After', '2'); + const state = dependencies({ + store: new MemoryCredentialStore(profile()), + responses: [throttled, jsonResponse({ creditBalance: 99 })], + }); + + const exitCode = await runCli(['api', 'GET', '/v1/usage'], state.deps); + + expect(exitCode).toBe(0); + expect(state.requests).toHaveLength(2); + expect(state.sleep).toHaveBeenCalledWith(2000); + }); + + test('does not retry mutations or non-transient failures', async () => { + const state = dependencies({ + store: new MemoryCredentialStore(profile()), + responses: [jsonResponse({ error: { code: 'INVALID_INPUT', message: 'Invalid.' } }, 422)], + }); + + const exitCode = await runCli([ + 'api', 'POST', '/v1/monitors', '--data', '{"name":"test"}', '--retries', '3', + ], state.deps); + + expect(exitCode).toBe(1); + expect(state.requests).toHaveLength(1); + }); + + test('uses the longer data timeout by default and accepts a bounded override', async () => { + const state = dependencies({ + store: new MemoryCredentialStore(profile()), + responses: [jsonResponse({ creditBalance: 100 }), jsonResponse({ creditBalance: 100 })], + }); + const timeout = vi.spyOn(AbortSignal, 'timeout'); + + const defaultExitCode = await runCli(['api', 'GET', '/v1/usage'], state.deps); + const overrideExitCode = await runCli([ + 'api', 'GET', '/v1/usage', '--timeout-ms', '180000', + ], state.deps); + + expect(defaultExitCode).toBe(0); + expect(overrideExitCode).toBe(0); + expect(timeout).toHaveBeenNthCalledWith(1, 150000); + expect(timeout).toHaveBeenNthCalledWith(2, 180000); + expect(state.requests[0]?.init?.signal).toBeInstanceOf(AbortSignal); + expect(state.requests[0]?.init).toMatchObject({ method: 'GET' }); + timeout.mockRestore(); + }); }); function profile(): StoredProfile { diff --git a/packages/video2ctx-cli/src/cli.ts b/packages/video2ctx-cli/src/cli.ts index fda68e4..d1a48c2 100644 --- a/packages/video2ctx-cli/src/cli.ts +++ b/packages/video2ctx-cli/src/cli.ts @@ -7,7 +7,12 @@ import { } from './auth'; const DEFAULT_BASE_URL = 'https://api.video2ctx.dev'; -const REQUEST_TIMEOUT_MS = 30_000; +const AUTH_REQUEST_TIMEOUT_MS = 30_000; +const DATA_REQUEST_TIMEOUT_MS = 150_000; +const MAX_REQUEST_TIMEOUT_MS = 300_000; +const DEFAULT_GET_RETRIES = 1; +const MAX_GET_RETRIES = 3; +const MAX_RETRY_DELAY_MS = 30_000; declare const __VIDEO2CTX_VERSION__: string; const CLI_VERSION = typeof __VIDEO2CTX_VERSION__ === 'string' ? __VIDEO2CTX_VERSION__ @@ -24,6 +29,28 @@ export type CliDependencies = { stderr(line: string): void; }; +type RequestOptions = { + timeoutMs: number; + retries: number; +}; + +class CliError extends Error { + constructor( + readonly code: string, + message: string, + readonly details: { + status?: number; + requestId?: string; + retryable: boolean; + retryAfterSeconds?: number; + exitCode?: number; + }, + ) { + super(message); + this.name = 'CliError'; + } +} + export async function runCli(args: string[], dependencies: CliDependencies): Promise { const existingProfile = await dependencies.store.read(); const secrets = [ @@ -53,14 +80,27 @@ export async function runCli(args: string[], dependencies: CliDependencies): Pro if (args[0] === 'auth' && args[1] === 'logout') { return await logout(args.slice(2), dependencies, existingProfile); } + if (args[0] === 'transcript') { + return await transcript(args.slice(1), dependencies, existingProfile); + } if (args[0] === 'api') { return await api(args.slice(1), dependencies, existingProfile); } - throw new Error('Unknown command. Run video2ctx --help.'); + throw inputError('Unknown command. Run video2ctx --help.'); } catch (error) { - dependencies.stderr(redact(error instanceof Error ? error.message : String(error), secrets)); - return 1; + const normalized = normalizeError(error); + dependencies.stderr(JSON.stringify({ + error: compact({ + status: normalized.details.status, + code: normalized.code, + message: redact(normalized.message, secrets), + requestId: normalized.details.requestId, + retryable: normalized.details.retryable, + retryAfterSeconds: normalized.details.retryAfterSeconds, + }), + })); + return normalized.details.exitCode ?? 1; } } @@ -90,17 +130,14 @@ async function identity( dependencies: CliDependencies, profile: StoredProfile | null, ): Promise { - const credential = resolveCredential({ - environmentApiKey: dependencies.environment.VIDEO2CTX_API_KEY, - profile, - }); + const credential = credentialFor(dependencies, profile); const json = args.includes('--json'); if (!credential) { if (command === 'status') { output(dependencies, { authenticated: false }, json); return 0; } - throw new Error('Not authenticated. Run video2ctx auth login or set VIDEO2CTX_API_KEY.'); + throw authenticationRequired(); } const { data: account } = await authenticatedJson( @@ -108,7 +145,8 @@ async function identity( { method: 'GET' }, credential, requestBaseUrl(dependencies, credential), - dependencies.fetch, + dependencies, + { timeoutMs: AUTH_REQUEST_TIMEOUT_MS, retries: 0 }, ); const result = isRecord(account) ? { authenticated: true, ...account } @@ -128,23 +166,53 @@ async function logout( return 0; } - const response = await dependencies.fetch(new URL('/api/auth/sign-out', profile.baseUrl), { - method: 'POST', - headers: { - authorization: `Bearer ${profile.token}`, - accept: 'application/json', - 'content-type': 'application/json', - }, - body: '{}', - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); + let response: Response; + try { + response = await dependencies.fetch(new URL('/api/auth/sign-out', profile.baseUrl), { + method: 'POST', + headers: { + authorization: `Bearer ${profile.token}`, + accept: 'application/json', + 'content-type': 'application/json', + }, + body: '{}', + signal: AbortSignal.timeout(AUTH_REQUEST_TIMEOUT_MS), + }); + } catch (error) { + throw transportError(error); + } const payload = await readJson(response); - if (!response.ok) throw new Error(apiErrorMessage(payload, response.status)); + if (!response.ok) throw responseError(response, payload, dependencies.now()); await dependencies.store.delete(); output(dependencies, { loggedOut: true, revoked: true }, json); return 0; } +async function transcript( + args: string[], + dependencies: CliDependencies, + profile: StoredProfile | null, +): Promise { + const input = args[0]; + if (!input || input.startsWith('--')) { + throw inputError('transcript requires a YouTube URL or video ID.'); + } + const videoId = youtubeVideoId(input); + const format = option(args, '--format') ?? 'text'; + if (!['text', 'segments', 'words'].includes(format)) { + throw inputError('--format must be text, segments, or words.'); + } + const query = new URLSearchParams({ format }); + const language = option(args, '--lang'); + if (language) query.set('lang', language); + return authenticatedRead( + `/v1/providers/youtube/videos/${encodeURIComponent(videoId)}/transcript?${query}`, + args, + dependencies, + profile, + ); +} + async function api( args: string[], dependencies: CliDependencies, @@ -153,21 +221,44 @@ async function api( const method = (args[0] ?? '').toUpperCase(); const path = args[1] ?? ''; if (!['GET', 'POST', 'PUT', 'PATCH', 'DELETE'].includes(method)) { - throw new Error('API method must be GET, POST, PUT, PATCH, or DELETE.'); + throw inputError('API method must be GET, POST, PUT, PATCH, or DELETE.'); } - if (!path.startsWith('/v1/')) throw new Error('API path must start with /v1/.'); + if (!path.startsWith('/v1/')) throw inputError('API path must start with /v1/.'); - const credential = resolveCredential({ - environmentApiKey: dependencies.environment.VIDEO2CTX_API_KEY, - profile, - }); - if (!credential) throw new Error('Not authenticated. Run video2ctx auth login or set VIDEO2CTX_API_KEY.'); + const credential = credentialFor(dependencies, profile); + if (!credential) throw authenticationRequired(); const data = option(args, '--data'); - if (data) JSON.parse(data); + if (data) { + try { + JSON.parse(data); + } catch { + throw inputError('--data must be valid JSON.'); + } + } const result = await authenticatedJson(path, { method, ...(data ? { headers: { 'content-type': 'application/json' }, body: data } : {}), - }, credential, requestBaseUrl(dependencies, credential), dependencies.fetch); + }, credential, requestBaseUrl(dependencies, credential), dependencies, dataRequestOptions(args)); + dependencies.stdout(JSON.stringify(args.includes('--include-meta') ? result : result.data)); + return 0; +} + +async function authenticatedRead( + path: string, + args: string[], + dependencies: CliDependencies, + profile: StoredProfile | null, +): Promise { + const credential = credentialFor(dependencies, profile); + if (!credential) throw authenticationRequired(); + const result = await authenticatedJson( + path, + { method: 'GET' }, + credential, + requestBaseUrl(dependencies, credential), + dependencies, + dataRequestOptions(args), + ); dependencies.stdout(JSON.stringify(args.includes('--include-meta') ? result : result.data)); return 0; } @@ -177,26 +268,73 @@ async function authenticatedJson( init: RequestInit, credential: ResolvedCredential, baseUrl: string, - fetchImpl: typeof fetch, + dependencies: CliDependencies, + options: RequestOptions, ): Promise<{ data: unknown; meta: Record }> { const headers = new Headers(init.headers); headers.set('accept', 'application/json'); headers.set('authorization', `Bearer ${credential.value}`); - const response = await fetchImpl(new URL(path, baseUrl), { - ...init, - headers, - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + const method = (init.method ?? 'GET').toUpperCase(); + + for (let attempt = 0; ; attempt += 1) { + let response: Response; + try { + response = await dependencies.fetch(new URL(path, baseUrl), { + ...init, + headers, + signal: AbortSignal.timeout(options.timeoutMs), + }); + } catch (error) { + throw transportError(error); + } + const payload = await readJson(response); + if (response.ok) { + return { + data: payload, + meta: compact({ + status: response.status, + requestId: response.headers.get('X-Request-Id') ?? undefined, + creditsCharged: integerHeader(response.headers.get('X-Credits-Charged')), + creditsRemaining: integerHeader(response.headers.get('X-Credits-Remaining')), + }), + }; + } + + const failure = responseError(response, payload, dependencies.now()); + const canRetry = method === 'GET' + && (response.status === 429 || response.status === 503) + && attempt < options.retries; + if (!canRetry) throw failure; + const delayMs = Math.min( + failure.details.retryAfterSeconds === undefined + ? 1_000 * (attempt + 1) + : failure.details.retryAfterSeconds * 1_000, + MAX_RETRY_DELAY_MS, + ); + await dependencies.sleep(delayMs); + } +} + +function credentialFor( + dependencies: CliDependencies, + profile: StoredProfile | null, +): ResolvedCredential | null { + return resolveCredential({ + environmentApiKey: dependencies.environment.VIDEO2CTX_API_KEY, + profile, }); - const payload = await readJson(response); - if (!response.ok) throw new Error(apiErrorMessage(payload, response.status)); +} + +function dataRequestOptions(args: string[]): RequestOptions { return { - data: payload, - meta: compact({ - status: response.status, - requestId: response.headers.get('X-Request-Id') ?? undefined, - creditsCharged: integerHeader(response.headers.get('X-Credits-Charged')), - creditsRemaining: integerHeader(response.headers.get('X-Credits-Remaining')), - }), + timeoutMs: integerOption( + args, + '--timeout-ms', + DATA_REQUEST_TIMEOUT_MS, + 1_000, + MAX_REQUEST_TIMEOUT_MS, + ), + retries: integerOption(args, '--retries', DEFAULT_GET_RETRIES, 0, MAX_GET_RETRIES), }; } @@ -210,10 +348,51 @@ function option(args: string[], name: string): string | undefined { const index = args.indexOf(name); if (index === -1) return undefined; const value = args[index + 1]; - if (!value || value.startsWith('--')) throw new Error(`${name} requires a value.`); + if (!value || value.startsWith('--')) throw inputError(`${name} requires a value.`); return value; } +function integerOption( + args: string[], + name: string, + defaultValue: number, + minimum: number, + maximum: number, +): number { + const raw = option(args, name); + if (raw === undefined) return defaultValue; + const value = Number(raw); + if (!Number.isSafeInteger(value) || value < minimum || value > maximum) { + throw inputError(`${name} must be an integer from ${minimum} to ${maximum}.`); + } + return value; +} + +function youtubeVideoId(input: string): string { + if (/^[A-Za-z0-9_-]{11}$/.test(input)) return input; + let url: URL; + try { + url = new URL(input); + } catch { + throw inputError('Use an 11-character YouTube video ID or YouTube URL.'); + } + const hostname = url.hostname.toLowerCase(); + let candidate: string | null | undefined; + if (hostname === 'youtu.be') { + candidate = url.pathname.split('/').filter(Boolean)[0]; + } else if (hostname === 'youtube.com' || hostname.endsWith('.youtube.com')) { + candidate = url.searchParams.get('v'); + if (!candidate) { + const [kind, id] = url.pathname.split('/').filter(Boolean); + if (['shorts', 'live', 'embed'].includes(kind ?? '')) candidate = id; + } + } + if (!candidate || !/^[A-Za-z0-9_-]{11}$/.test(candidate)) { + throw inputError('Use an 11-character YouTube video ID or YouTube URL.'); + } + return candidate; +} + function output(dependencies: CliDependencies, value: Record, json: boolean): void { if (json) { dependencies.stdout(JSON.stringify(value)); @@ -243,13 +422,61 @@ async function readJson(response: Response): Promise { } } -function apiErrorMessage(payload: unknown, status: number): string { - if (isRecord(payload)) { - const nested = isRecord(payload.error) ? payload.error : payload; - if (typeof nested.message === 'string') return nested.message; - if (typeof nested.error_description === 'string') return nested.error_description; - } - return `video2ctx request failed (${status}).`; +function responseError(response: Response, payload: unknown, now: number): CliError { + const nested = isRecord(payload) && isRecord(payload.error) ? payload.error : payload; + const record = isRecord(nested) ? nested : {}; + const message = typeof record.message === 'string' + ? record.message + : typeof record.error_description === 'string' + ? record.error_description + : `video2ctx request failed (${response.status}).`; + const code = typeof record.code === 'string' ? record.code : `HTTP_${response.status}`; + return new CliError(code, message, { + status: response.status, + requestId: response.headers.get('X-Request-Id') + ?? (typeof record.requestId === 'string' ? record.requestId : undefined), + retryable: response.status === 429 || response.status === 503, + retryAfterSeconds: retryAfter(response.headers.get('Retry-After'), now), + }); +} + +function retryAfter(value: string | null, now: number): number | undefined { + if (!value) return undefined; + if (/^\d+$/.test(value)) return Number(value); + const at = Date.parse(value); + if (!Number.isFinite(at)) return undefined; + return Math.max(0, Math.ceil((at - now) / 1_000)); +} + +function transportError(error: unknown): CliError { + const message = error instanceof Error ? error.message : 'The request could not be completed.'; + const timeout = error instanceof Error && (error.name === 'TimeoutError' || error.name === 'AbortError'); + return new CliError( + timeout ? 'REQUEST_TIMEOUT' : 'TRANSPORT_ERROR', + timeout ? 'The video2ctx request exceeded its deadline.' : message, + { retryable: true }, + ); +} + +function inputError(message: string): CliError { + return new CliError('INVALID_INPUT', message, { retryable: false, exitCode: 2 }); +} + +function authenticationRequired(): CliError { + return new CliError( + 'AUTHENTICATION_REQUIRED', + 'Not authenticated. Run video2ctx auth login or set VIDEO2CTX_API_KEY.', + { status: 401, retryable: false }, + ); +} + +function normalizeError(error: unknown): CliError { + if (error instanceof CliError) return error; + return new CliError( + 'INTERNAL_ERROR', + error instanceof Error ? error.message : 'Unexpected video2ctx CLI failure.', + { retryable: false }, + ); } function isRecord(value: unknown): value is Record { @@ -276,6 +503,9 @@ function helpText(): string { ' video2ctx auth status [--json]', ' video2ctx whoami [--json]', ' video2ctx auth logout [--json]', + ' video2ctx transcript URL_OR_ID [--format text|segments|words] [--lang CODE]', + ' [--include-meta] [--timeout-ms N] [--retries N]', ' video2ctx api METHOD /v1/path [--data JSON] [--include-meta]', + ' [--timeout-ms N] [--retries N]', ].join('\n'); } diff --git a/platform/src/lib/transcript-projection.ts b/platform/src/lib/transcript-projection.ts new file mode 100644 index 0000000..aef7cac --- /dev/null +++ b/platform/src/lib/transcript-projection.ts @@ -0,0 +1,42 @@ +import type { Transcript, TranscriptSegment } from 'all-things-youtube'; +import { ApiError } from './http'; + +export type TranscriptFormat = 'text' | 'segments' | 'words'; + +export type TextTranscript = Pick; + +export function parseTranscriptFormat(value?: string): TranscriptFormat { + if (!value) return 'words'; + if (value === 'text' || value === 'segments' || value === 'words') return value; + throw new ApiError(422, 'INVALID_TRANSCRIPT_FORMAT', 'Use text, segments, or words for transcript format.'); +} + +export function projectTranscript( + transcript: Transcript, + format: TranscriptFormat, +): Transcript | TextTranscript { + if (format === 'words') return transcript; + if (format === 'text') { + return compact({ + videoId: transcript.videoId, + track: transcript.track, + translatedTo: transcript.translatedTo, + text: transcript.text, + meta: transcript.meta, + }) as TextTranscript; + } + return { + ...transcript, + granularity: 'segment', + segments: transcript.segments.map(withoutWords), + }; +} + +function withoutWords(segment: TranscriptSegment): TranscriptSegment { + const { words: _words, ...rest } = segment; + return rest; +} + +function compact>(value: T): Partial { + return Object.fromEntries(Object.entries(value).filter(([, item]) => item !== undefined)) as Partial; +} diff --git a/platform/src/openapi.ts b/platform/src/openapi.ts index 5c04c4e..930b954 100644 --- a/platform/src/openapi.ts +++ b/platform/src/openapi.ts @@ -762,9 +762,12 @@ export const openApiDocument = { providerParameter, pathParameter('id', 'Provider video ID.', 'dQw4w9WgXcQ'), queryParameter('lang', 'Desired transcript language. The backend selects the default source track and translates only when necessary.', { type: 'string', example: 'hi' }), + queryParameter('format', 'Response detail. text omits timing arrays, segments keeps segment timing, and words keeps word timing. Omitted preserves the rich words response.', { type: 'string', enum: ['text', 'segments', 'words'], default: 'words' }), ], responses: { - '200': meteredJsonResponse('Normalized timed transcript.', schemaRef('Transcript')), + '200': meteredJsonResponse('Normalized transcript in the requested detail format.', { + oneOf: [schemaRef('TranscriptText'), schemaRef('TranscriptSegments'), schemaRef('Transcript')], + }), '401': responseRef('Unauthorized'), '402': responseRef('InsufficientCredits'), '404': responseRef('NotFound'), @@ -1755,6 +1758,25 @@ export const openApiDocument = { granularity: { type: 'string', enum: ['segment', 'word'] }, text: { type: 'string' }, meta: schemaRef('SourceMetadata'), }, }, + TranscriptText: { + type: 'object', + required: ['videoId', 'track', 'text', 'meta'], + properties: { + videoId: { type: 'string' }, track: schemaRef('CaptionTrack'), + translatedTo: schemaRef('TranslationLanguage'), + text: { type: 'string' }, meta: schemaRef('SourceMetadata'), + }, + }, + TranscriptSegments: { + type: 'object', + required: ['videoId', 'track', 'segments', 'granularity', 'text', 'meta'], + properties: { + videoId: { type: 'string' }, track: schemaRef('CaptionTrack'), + translatedTo: schemaRef('TranslationLanguage'), + segments: { type: 'array', items: schemaRef('TranscriptSegment') }, + granularity: { const: 'segment' }, text: { type: 'string' }, meta: schemaRef('SourceMetadata'), + }, + }, Video: { allOf: [schemaRef('VideoSummary'), { type: 'object', diff --git a/platform/src/routes/data/data.index.ts b/platform/src/routes/data/data.index.ts index 5c02f83..5a1a2a4 100644 --- a/platform/src/routes/data/data.index.ts +++ b/platform/src/routes/data/data.index.ts @@ -15,6 +15,7 @@ import { routeInput, type CacheStatus } from '../../lib/youtube'; import { requireEvidence, searchPrivate, searchPublic } from '../../lib/search'; import { citedAnswer } from '../../lib/analysis'; import { transcriptEvidence } from '../../lib/evidence'; +import { parseTranscriptFormat, projectTranscript } from '../../lib/transcript-projection'; import { generateTrendPlan, normalizeTrendPlanSignals } from '../../lib/trend-plan'; import { creditBalance, entitlements } from '../../lib/entitlements'; import { getProvider, providerDescriptors, type ProviderAdapter } from '../../providers'; @@ -175,8 +176,10 @@ dataRoutes.get('/providers/:provider/videos/:id/transcript', async (c) => { const provider = providerFor(c); const id = asId(c.req.param('id')); const desiredLanguage = text(c.req.query('lang'), 20) || undefined; - return c.json(await cachedRead(c, `${provider.descriptor.id}-video-transcript`, 'transcript', provider, () => - provider.getTranscript(c.env, id, desiredLanguage))); + const format = parseTranscriptFormat(c.req.query('format')); + const transcript = await cachedRead(c, `${provider.descriptor.id}-video-transcript`, 'transcript', provider, () => + provider.getTranscript(c.env, id, desiredLanguage)); + return c.json(projectTranscript(transcript, format)); }); dataRoutes.get('/providers/:provider/videos/:id/comments', async (c) => { diff --git a/platform/test/hosted-skills.test.ts b/platform/test/hosted-skills.test.ts index 03ed3d4..c811a1e 100644 --- a/platform/test/hosted-skills.test.ts +++ b/platform/test/hosted-skills.test.ts @@ -14,6 +14,14 @@ const monitoringSkill = readFileSync( new URL('../../.agents/skills/video2ctx-monitoring/SKILL.md', import.meta.url), 'utf8', ); +const apiSelector = readFileSync( + new URL('../../.agents/skills/video2ctx-api/agents/openai.yaml', import.meta.url), + 'utf8', +); +const directSelector = readFileSync( + new URL('../../.agents/skills/youtube-direct/agents/openai.yaml', import.meta.url), + 'utf8', +); describe('hosted video2ctx skills', () => { test.each([ ['video2ctx-api', apiSkill], @@ -23,7 +31,8 @@ describe('hosted video2ctx skills', () => { expect(skill).toContain('video2ctx --version'); expect(skill).toContain('npm install --global @video2ctx/cli'); expect(skill).toContain('auth login'); - expect(skill).toContain('auth status --json'); + expect(skill).toContain('whoami --json'); + expect(skill).not.toContain('auth status --json'); expect(skill).toContain('video2ctx api'); expect(skill).toContain('VIDEO2CTX_API_KEY'); expect(skill).toContain('https://api.video2ctx.dev/openapi.json'); @@ -36,13 +45,19 @@ describe('hosted video2ctx skills', () => { test('selector metadata distinguishes direct, hosted, and monitoring use', () => { expect(directSkill).toMatch(/^---\nname: youtube-direct\n/); expect(directSkill).toContain('Direct, no-account YouTube search and extraction'); - expect(directSkill).toContain('use video2ctx-api instead'); + expect(directSkill).toContain('first route for one-off public YouTube requests'); + expect(directSkill).toContain('continue with `video2ctx-api` without asking the user'); + expect(directSelector).toContain('summarize a public YouTube video'); expect(apiSkill).toContain('Managed, authenticated YouTube search and extraction'); - expect(apiSkill).toContain('Use instead of youtube-direct'); + expect(apiSkill).toContain('automatically when a youtube-direct operation fails'); + expect(apiSkill).toContain('Route account and usage requests here immediately'); + expect(apiSkill).not.toContain('approved fallback'); + expect(apiSelector).toContain('managed hosted API'); + expect(apiSelector).not.toContain('retrieve a YouTube transcript'); expect(monitoringSkill).toContain('Stateful video2ctx monitoring'); - expect(monitoringSkill).toContain('use video2ctx-api for one-time hosted reads'); + expect(monitoringSkill).toContain('use video2ctx-api for account and usage details'); }); test('keeps installation explicit and reuses one public credential transport', () => { @@ -57,6 +72,18 @@ describe('hosted video2ctx skills', () => { const operations = [ ['get', '/v1/providers'], ['get', '/v1/usage'], + ['get', '/v1/account'], + ['get', '/v1/providers/{provider}/search'], + ['get', '/v1/providers/{provider}/browse'], + ['get', '/v1/providers/{provider}/videos/{id}'], + ['get', '/v1/providers/{provider}/videos/{id}/tracks'], + ['get', '/v1/providers/{provider}/videos/{id}/transcript'], + ['get', '/v1/providers/{provider}/videos/{id}/comments'], + ['get', '/v1/providers/{provider}/videos/{id}/endscreen'], + ['get', '/v1/providers/{provider}/channels/{id}'], + ['get', '/v1/providers/{provider}/channels/{id}/videos'], + ['get', '/v1/providers/{provider}/channels/{id}/playlists'], + ['get', '/v1/providers/{provider}/playlists/{id}'], ['post', '/v1/monitors'], ['get', '/v1/monitors'], ['patch', '/v1/monitors/{id}'], diff --git a/platform/test/openapi.test.ts b/platform/test/openapi.test.ts index 7bc1d96..fb4333c 100644 --- a/platform/test/openapi.test.ts +++ b/platform/test/openapi.test.ts @@ -190,12 +190,15 @@ describe('OpenAPI and Scalar documentation', () => { expect(tracksOperation.deprecated).not.toBe(true); expect(paths['/v1/providers/{provider}/videos/{id}/captions']).toBeUndefined(); expect(parameterNames).toContain('lang'); + expect(parameterNames).toContain('format'); expect(parameterNames).not.toContain('translateTo'); expect(parameterNames).not.toContain('language'); expect(schemas.CaptionTrackList.required).toEqual(expect.arrayContaining([ 'tracks', 'sourceTracks', 'translationLanguages', 'autoTranslationTargets', ])); expect(schemas.Transcript.properties.translatedTo.$ref).toBe('#/components/schemas/TranslationLanguage'); + expect(schemas.TranscriptText.required).toEqual(['videoId', 'track', 'text', 'meta']); + expect(schemas.TranscriptSegments.properties.segments.items.$ref).toBe('#/components/schemas/TranscriptSegment'); expect(schemas.CommentResponse.properties.totalCount).toMatchObject({ type: 'integer', minimum: 0 }); expect(schemas.CommentResponse.properties.estimatedTotal).toBeUndefined(); expect(schemas.Video.required).toBeUndefined(); diff --git a/platform/test/transcript-projection.test.ts b/platform/test/transcript-projection.test.ts new file mode 100644 index 0000000..699c186 --- /dev/null +++ b/platform/test/transcript-projection.test.ts @@ -0,0 +1,58 @@ +import { describe, expect, test } from 'vitest'; +import { parseTranscriptFormat, projectTranscript } from '../src/lib/transcript-projection'; + +const transcript = { + videoId: 'abcdefghijk', + track: { + id: 'en', + languageCode: 'en', + name: 'English', + kind: 'manual' as const, + isTranslatable: true, + isDefault: true, + }, + translatedTo: { languageCode: 'fr', name: 'French' }, + segments: [{ + startMs: 0, + durationMs: 1000, + endMs: 1000, + text: 'Bonjour', + words: [{ text: 'Bonjour', startMs: 0, offsetMs: 0 }], + }], + granularity: 'word' as const, + text: 'Bonjour', + meta: { + source: 'allthingsyoutube' as const, + fetchedAt: '2026-08-17T00:00:00.000Z', + partial: false, + warnings: [], + }, +}; + +describe('transcript projections', () => { + test('preserves the existing rich response when format is omitted', () => { + expect(projectTranscript(transcript, parseTranscriptFormat(undefined))).toEqual(transcript); + }); + + test('returns compact text without timing arrays', () => { + expect(projectTranscript(transcript, parseTranscriptFormat('text'))).toEqual({ + videoId: 'abcdefghijk', + track: transcript.track, + translatedTo: transcript.translatedTo, + text: 'Bonjour', + meta: transcript.meta, + }); + }); + + test('removes word timing from segment output', () => { + expect(projectTranscript(transcript, parseTranscriptFormat('segments'))).toEqual({ + ...transcript, + granularity: 'segment', + segments: [{ startMs: 0, durationMs: 1000, endMs: 1000, text: 'Bonjour' }], + }); + }); + + test('rejects unsupported formats', () => { + expect(() => parseTranscriptFormat('srt')).toThrow('Use text, segments, or words'); + }); +}); diff --git a/platform/test/transcript-route.test.ts b/platform/test/transcript-route.test.ts index 9c5a128..b7e3f70 100644 --- a/platform/test/transcript-route.test.ts +++ b/platform/test/transcript-route.test.ts @@ -17,9 +17,15 @@ vi.mock('../src/lib/youtube', async (importOriginal) => { isTranslatable: true, isDefault: true, }, translatedTo: { languageCode: 'hi', name: 'Hindi' }, - segments: [], + segments: [{ + startMs: 0, + durationMs: 1000, + endMs: 1000, + text: 'Hello world', + words: [{ text: 'Hello', startMs: 0, offsetMs: 0 }], + }], granularity: 'word', - text: '', + text: 'Hello world', meta: { source: 'allthingsyoutube', fetchedAt: new Date().toISOString(), partial: false, warnings: [] }, }, cacheStatus: 'hit' as const })), }; @@ -42,6 +48,42 @@ describe('transcript route', () => { expect(getTranscriptWithCache).toHaveBeenCalledWith(expect.anything(), 'abcdefghijk', 'hi'); warning.mockRestore(); }); + + test('returns compact text when requested without changing the cached upstream shape', async () => { + const response = await app.request( + '/v1/providers/youtube/videos/abcdefghijk/transcript?format=text', + {}, + {} as Env, + { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext, + ); + + expect(response.status).toBe(200); + const payload = await response.json() as Record; + expect(payload).toMatchObject({ + videoId: 'abcdefghijk', + text: 'Hello world', + meta: { partial: false }, + }); + expect(payload).not.toHaveProperty('segments'); + expect(payload).not.toHaveProperty('granularity'); + expect(getTranscriptWithCache).toHaveBeenLastCalledWith(expect.anything(), 'abcdefghijk', undefined); + }); + + test('rejects an unsupported transcript format before loading data', async () => { + const calls = vi.mocked(getTranscriptWithCache).mock.calls.length; + const response = await app.request( + '/v1/providers/youtube/videos/abcdefghijk/transcript?format=srt', + {}, + {} as Env, + { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext, + ); + + expect(response.status).toBe(422); + await expect(response.json()).resolves.toMatchObject({ + error: { code: 'INVALID_TRANSCRIPT_FORMAT' }, + }); + expect(getTranscriptWithCache).toHaveBeenCalledTimes(calls); + }); }); function routeAuthenticationMock() {