Repository navigation
feat(web): internal route to backfill transcripts for existing videos - #26
Merged
Merged
Conversation
Imported Loom videos only get a transcript when someone opens them, and the public API only lets a video's owner request one. This adds POST /api/internal/transcripts/backfill, authenticated with the media server secret like the Loom import routes, so an operator can start transcription for a list of videos. Each id goes through transcribeVideo as the video's own owner, so the storage paths and every existing rule still apply (media processing suspension, transcription disabled, live claims, uploads in progress). AI generation is not requested. Videos that already have a transcription status are reported with that status and left alone, which also makes the route safe to call again to read progress. The response includes how many transcriptions started in the last hour are still processing, so a caller can hold back instead of flooding the in-process workflow runtime.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Imported Loom videos only get a transcript when someone opens them, and the public API only lets a video's owner request one. Transcribing the existing backlog needs a way to start transcription for videos owned by many people without signing in as each of them.
What
POST /api/internal/transcripts/backfillnext to the Loom import routes, authenticated the same way: thex-media-server-secretheader compared withMEDIA_SERVER_WEBHOOK_SECRETthroughtimingSafeEqualover SHA-256 digests. Up to 25 ids per request (same id pattern as the redrive route), deduplicated, five at a time.transcribeVideoas the video's own owner. The owner matters beyond permissions: the audio is read from and the VTT written to<ownerId>/<videoId>/. Every rule inside it still applies: media processing suspension, transcription disabled on the video or organization, live transcription claims, uploads in progress, and the atomicNULL → PROCESSINGclaim.aiGenerationEnabled = false), so the workflow ends after saving the transcript. Summaries still follow Cap's normal behavior: a later view of a transcribed video without a summary can start one, as for any other video.COMPLETE,PROCESSING,SKIPPED,NO_AUDIO,ERROR) are returned asskippedwith that status and never touched; missing videos and screenshots are skipped too. This makes calling the route again a safe way to read progress.started | skipped | failedwith a reason, the current transcription and AI generation status, and the duration. The response also carriesinFlight, the number of transcriptions claimed in the last hour that are still processing, so a caller can hold back instead of starting more work than the in-process workflow runtime can carry.New files only:
apps/web/app/api/internal/transcripts/backfill/route.ts,apps/web/lib/transcript-backfill.tsand its test. No schema change, no edit to upstream files.Verified
__tests__/unit/transcript-backfill.test.ts(16 tests: 401 without or with a wrong secret, 400 for invalid batches, owner id andfalsepassed totranscribeVideo, every existing status left alone, missing and screenshot videos, failures and throws mapped per id, repeated ids handled once, unknown in-flight count) andloom-importer-redrive.test.tspass.bun run typecheckclean; Biome clean on the touched files.Operating note
Cap Web's workflows run in process, so a deploy while a bulk backfill is running leaves the videos it was transcribing in
PROCESSING, which nothing retries for older videos. Turn off Cap Web's GitHub autodeploy in Railway for the duration of a bulk run and back on after it.