feat: lessons as a typed artifact with follow-through tracking (#428)#472
Open
Tet-9 wants to merge 1 commit into
Open
feat: lessons as a typed artifact with follow-through tracking (#428)#472Tet-9 wants to merge 1 commit into
Tet-9 wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
docs
Tet-9
force-pushed
the
feat/428-lessons-follow-through
branch
from
2d075f3 to
f1b51cc
Compare
Contributor
Author
|
@plind-junior , on for review |
…dev#428) Adds ClaimType.LESSON and kb.mark_lesson_followed -- an append-only observation of whether a surfaced lesson was actually followed in a given turn. Never edits the claim; appends a lesson.followed audit event with followed, optional context, and claim_type. Registered across all four surfaces: CLI (mark-lesson-followed), MCP (kb_mark_lesson_followed), JSONL (kb.mark_lesson_followed), and capabilities.METHODS. Added to the _STANDARD MCP tool profile alongside kb.confirm, its closest existing analogue. Lessons resurface through the normal retrieval path with no special-casing anywhere in context.py -- verified directly, not assumed. The propose-time repeat guard (vouchdev#147) comes for free: propose_claim already runs find_similar_on_propose for every claim type regardless, so no new code was needed for that acceptance criterion. Not restricted to ClaimType.LESSON: mark_lesson_followed accepts any claim id, matching the issue's own proposed surface ("either a new ClaimType.LESSON or a flag on workflow/warning claims"). Deliberately does not build the "effectiveness computation" system vouchdev#428 names as a downstream consumer (vouchdev#2) -- that reference resolves to an unrelated PR and no such system exists anywhere in this codebase under any name (verified via grep). These events are shaped for such a consumer to exist later; inventing an unspecified analytics system was out of scope. tests/test_lessons.py: 13 tests covering the claim type, retrieval resurfacing, the observe-not-edit invariant, multiple observations, missing- claim handling, the not-restricted-to-LESSON case, registration in capabilities/JSONL/MCP, and the repeat-guard warning.
Tet-9
force-pushed
the
feat/428-lessons-follow-through
branch
from
f1b51cc to
3094bcf
Compare
github-actions
Bot
added
retrieval
Contributor
Author
|
@plind-junior , if you may review this now |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #428
What's included
ClaimType.LESSON — resurfaces through the normal retrieval path like any claim, no special-casing
kb.mark_lesson_followed(claim_id, followed, context?) — append-only observation appended to audit.log.jsonl as a lesson.followed event; never edits the claim's text/status/any field
Registered across all four surfaces: CLI (vouch mark-lesson-followed), MCP tool, JSONL handler, capabilities.METHODS, and the _STANDARD tool profile
Repeat guard: no new code needed — propose_claim already runs find_similar_on_propose (#147) for every claim type, so proposing a near-duplicate lesson already surfaces a non-blocking similar_approved/similar_pending warning
Deliberately out of scope
The issue names an "effectiveness computation" system (cross-referenced as #2) as a downstream consumer of these events. That reference resolves to an unrelated PR, and no such system exists anywhere in this codebase under any name (verified via grep across src/vouch/). Rather than invent an unspecified analytics/scoring system, this PR ships events shaped for such a consumer (followed, context, claim_type fields) without building the consumer itself.
Design note
mark_lesson_followed isn't restricted to ClaimType.LESSON — it accepts any claim id, matching the issue's own proposed surface ("either a new ClaimType.LESSON or a flag on workflow/warning claims"). Covered by a dedicated test.
Testing
tests/test_lessons.py: 13 tests — claim type, retrieval resurfacing, observe-not-edit invariant, repeated observations, missing-claim error, not-restricted-to-LESSON, registration across capabilities/JSONL/MCP, and the repeat-guard warning
Full suite: ruff check / mypy clean, existing test_capabilities.py and similarity-warning tests (test_propose_similarity.py) unaffected
CHANGELOG.md updated under [Unreleased]