Implement severity-based drift classification and tiered documentation#7
Merged
Conversation
- Add Severity enum (GREEN/YELLOW/RED) and DriftEvent dataclass to drift_detect.py - Classify all drift changes with severity, human-readable explanation, impacted endpoints - New exit codes: 0=no drift, 10=GREEN, 11=YELLOW, 2=RED, 3=runtime mismatches - Pre-commit hook: auto-fix GREEN/YELLOW drift, only block RED (breaking) - CI pipeline: pass GREEN, fail YELLOW (require review), fail+escalate RED - Slack notifications: severity badges, action-required text, only for YELLOW/RED - Tier 1 docs (api_reference.md) regenerated deterministically - Tier 2/3 review section appended for YELLOW/RED changes - Runtime audit: surface repeated mismatches as Runtime Drift Signals - Backward compatible with existing baseline format Co-Authored-By: unknown <>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add severity-based drift classification and tiered documentation
Summary
Refines the Drift Guard system to classify all detected API drift by severity and adjust behavior accordingly:
Severityenum (GREEN/YELLOW/RED),DriftEventdataclass, and enhancedDriftReport. Schema comparison now tracks field types, required flags, and enum values to classify changes (e.g., new optional field = GREEN, type change = YELLOW, removed endpoint = RED).cmd_checkregenerates Tier 1 docs always, appends a "Proposed Narrative Updates" review section for YELLOW/RED, and only sends Slack for YELLOW/RED. New_build_runtime_signals()groups repeated audit mismatches for human review.severityparam; adds "Action required" block with severity-specific framing (RED = merge blocked, YELLOW = PR review required).Review & Testing Checklist for Human
detect_drift()function indrift_detect.pywas fully rewritten. Verify the classification rules match your expectations — especially edge cases likeallOfschemas,$refresolution, and fields with defaults being classified as optional vs required. No unit tests were added for the new classification logic.2for any drift. Now10=GREEN,11=YELLOW,2=RED only. Verify no external scripts or tooling depend on exit code2meaning "any drift detected."cmd_checkupdates baseline even for RED drift (guard.py~line 220): After detecting RED/breaking drift, the baseline is overwritten. This means runningchecktwice in a row yields "no drift" on the second run. Confirm this is intended — it means the RED warning is a one-shot signal.git add artifacts/openapi_baseline.json docs/api_reference.mdautomatically for GREEN/YELLOW changes. Verify you're comfortable with the hook modifying staged files silently._find_endpoints_using_schema()uses string matching (drift_detect.py~line 189): It converts the operation dict to a string and checks if the schema name appears. This could produce false positives (e.g., schema "User" matching a description mentioning "User"). Test with schemas that have common names.Test Plan
git commit. Verify the hook auto-updates baseline + docs and allows the commit.str→int), stage, commit. Verify the hook auto-updates but the CI fails with "PR review required."SLACK_WEBHOOK_URLand trigger YELLOW/RED drift. Verify the Slack message includes severity badge and action-required text. Verify GREEN drift does NOT send Slack.bash scripts/demo_bad_client.shmultiple times to generate repeated mismatches, thenpython -m drift_guard.guard audit. Verify the "Runtime Drift Signals" section appears with counts.Notes
added_endpoints,removed_endpoints,schema_changeslists are still populated for existing consumers.detect_drift()edge cases.