Refactor HTML generation to support SQLite storage and serving#83
Closed
ananis25 wants to merge 5 commits intosimonw:mainfrom
Closed
Refactor HTML generation to support SQLite storage and serving#83ananis25 wants to merge 5 commits intosimonw:mainfrom
ananis25 wants to merge 5 commits intosimonw:mainfrom
Conversation
Introduces db.py with schema creation, session import with dedup, page retrieval, and project/session listing queries. Uses stdlib sqlite3 with WAL mode for future Turso compatibility. https://claude.ai/code/session_019CmHK1C5R4AHwmLAsw2PEt
Refactors generate_html() and generate_html_from_session_data() to both use a shared render_session_pages() that returns pages in memory as a dict. Also adds write_pages_to_dir() helper. This enables the upcoming SQLite storage flow where HTML is generated in memory and stored in the database. https://claude.ai/code/session_019CmHK1C5R4AHwmLAsw2PEt
New CLI command: `claude-code-transcripts sync` scans ~/.claude/projects, renders HTML in memory, and stores sessions in SQLite with dedup. Skips sessions already present by session_id. https://claude.ai/code/session_019CmHK1C5R4AHwmLAsw2PEt
New serve.py module with HTTP handler that routes URLs to SQLite queries. Master index and project index pages are rendered dynamically from DB. Session pages are served directly from the session_pages table. URL routing: / -> master index, /<project>/ -> project index, /<project>/<session>/<page>.html -> session page content. https://claude.ai/code/session_019CmHK1C5R4AHwmLAsw2PEt
Commands now import sessions into SQLite instead of writing static HTML files. The -o/--output and --gist flags still produce file output as an escape hatch. Also adds import_session_file_to_db() and import_session_data_to_db() helpers for the common import flow. https://claude.ai/code/session_019CmHK1C5R4AHwmLAsw2PEt
Author
|
Sorry, I didn't notice this opened a PR against your repo. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR refactors the HTML generation pipeline to decouple rendering from file I/O, enabling storage of rendered transcripts in SQLite and serving them via HTTP. The changes introduce a new database layer, new CLI commands (
syncandserve), and update existing commands to support both legacy file-based output and new database-backed workflows.Key Changes
Refactored
generate_html()function: Split into three focused functions:render_session_pages(): Pure rendering function that returns HTML pages and metadata in memorywrite_pages_to_dir(): Writes rendered pages to filesystemgenerate_html(): Orchestrates the pipeline (maintains backward compatibility)New database module (
src/claude_code_transcripts/db.py):sessionsandsession_pagestablesget_db(),session_exists(),import_session(),get_session_page(),list_projects(),list_sessions()New HTTP server module (
src/claude_code_transcripts/serve.py):make_handler()factory creates request handlers bound to a database/), project index (/<project>/), session pages (/<project>/<session>/<page>.html)New import functions:
import_session_file_to_db(): Import from JSONL/JSON file pathimport_session_data_to_db(): Import from parsed session data dictUpdated CLI commands:
localandjsoncommands: Default to database import (unless-o,--output-auto, or--gistspecified)Comprehensive test coverage:
tests/test_db.py: Database operations (schema, CRUD, dedup)tests/test_sync.py: Sync command functionalitytests/test_serve.py: HTTP server routing and renderingImplementation Details
-oflag behavior unchangedhttps://claude.ai/code/session_019CmHK1C5R4AHwmLAsw2PEt