Manage: persistent ingest history — every run (UI, CLI, kubectl) with running/stale states (#15)

[gangtao]

Closes #15.

Problem

The Manage tab's "Ingest jobs" panel only reflected the in-memory JobManager: a kubectl exec … tpk ingest — how production was re-indexed for 0.0.6 — was invisible both while running and afterwards, and nothing survived a restart. The persistent record (kg_ingest_log) already existed but only fed the corpus table's "Last ingest" cell, and only recorded finished runs with no error text.

Change

• Ingest writes a started row when a run begins and its ok / failed twin (same run_id) when it ends. A new error column carries the failure reason (RuntimeError: graphify exploded: exit 137). Logging the start is best-effort — it can never stop an ingest.
• ingest.list_runs folds the two rows per run, newest first. A start with no end is running (< 2 h) or stale — the process died mid-run (OOM-kill, pod restart) and will never write the end row. ingest.latest_status applies the same rule to the corpus table, so "Last ingest" can't show "started" forever.
• GET /api/ingest-log?limit=50&entry= (corpus:view, like /api/repos; limit 1–200): {runs: [{run_id, entry_key, status, nodes, edges, git_sha, error, started_at, finished_at, duration_s}]}. Rows written before this change have no start row → started_at / duration_s are null.
• Manage tab: the panel is now Ingest history — every run, with status, counts, duration, relative time, commit in the tooltip, error text for failures, and "running… (started outside the UI)" for a CLI/kubectl run in progress. A job the UI itself started stays in the live block above (phase + progress) and is dropped from the history until its end row lands, so nothing shows twice.

Schema / upgrade

No migration and nothing breaking: the stream is append-only, the error column is added with _add_column_if_missing on startup (idempotent, backfilled "", same mechanism as daily_token_limit), and existing rows keep working. Rolling back leaves inert extra rows/column. GET /api/jobs unchanged; export/import unaffected (the column has a default).

Testing

• Ingest (real DB): started then ok with the same run_id, and the start row is visible during extraction; a failure writes failed with the error.
• API (real DB): 401/403/200 gate; folding start+end into one run with duration; a pre-existing end-only row; running vs stale; entry filter, limit and its bounds; the corpus table reports an abandoned start as stale.
• pytest: 343 passed, 10 skipped, 1 failed — tests/test_ingest.py::test_ingest_repo_passes_extraction_and_backend, environment-dependent, failing on main too. Web build clean.
• Browser check against a real tpk serve in a throwaway database seeded the way real runs write: all four states rendered (running / failed with error / ok with duration / stale), corpus table shows started for the live run and stale for the abandoned one.

Not in scope: cancelling a run, log retention, and per-phase progress for CLI runs (they write only start + end).

🤖 Generated with Claude Code