Integrate audit findings into docs/api_reference.md#8
Conversation
- cmd_audit now appends a Runtime Mismatches section to docs/api_reference.md with summary count, table (Method, Path, Issue Type, Details, Client IP), and audit timestamp - Section is inserted before the auto-generated footer line - Running audit again replaces the previous section (no stale accumulation) - Running baseline fully regenerates api_reference.md, clearing the audit section - artifacts/audit_report.md kept as-is for CI artifact uploads Co-Authored-By: unknown <>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
| _append_audit_section_to_api_ref(entries, total, now) | ||
| console.print( | ||
| f" Appended Runtime Mismatches section to [cyan]{API_REF_PATH}[/cyan]" | ||
| ) |
There was a problem hiding this comment.
🟡 Misleading success message printed when api_reference.md does not exist
When API_REF_PATH does not exist, _append_audit_section_to_api_ref silently returns at line 215-216 without writing anything. However, cmd_audit unconditionally prints "Appended Runtime Mismatches section to docs/api_reference.md" at lines 337-339, misleading the user into thinking the section was successfully written.
Root Cause
The function _append_audit_section_to_api_ref has an early return when the file doesn't exist (drift_guard/guard.py:215-216), but the caller at drift_guard/guard.py:336-339 does not check whether the operation actually succeeded:
# Line 215-216: silently returns
if not API_REF_PATH.exists():
return# Lines 336-339: always prints success
_append_audit_section_to_api_ref(entries, total, now)
console.print(
f" Appended Runtime Mismatches section to [cyan]{API_REF_PATH}[/cyan]"
)Impact: If a user runs audit before running baseline (which creates api_reference.md), they see a success message but the audit section is never written. This could cause confusion during CI or manual debugging.
| _append_audit_section_to_api_ref(entries, total, now) | |
| console.print( | |
| f" Appended Runtime Mismatches section to [cyan]{API_REF_PATH}[/cyan]" | |
| ) | |
| # Append Runtime Mismatches section to docs/api_reference.md | |
| if API_REF_PATH.exists(): | |
| _append_audit_section_to_api_ref(entries, total, now) | |
| console.print( | |
| f" Appended Runtime Mismatches section to [cyan]{API_REF_PATH}[/cyan]" | |
| ) | |
| else: | |
| console.print( | |
| f" [dim]Skipped updating {API_REF_PATH} (file not found)[/dim]" | |
| ) | |
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
Adds a "Runtime Mismatches" section to
docs/api_reference.mdso developers see audit findings alongside the API reference instead of in a separateartifacts/audit_report.mdfile.When
python -m drift_guard.guard auditruns:artifacts/audit_report.md(unchanged, for CI artifact uploads)## Runtime Mismatchessection intodocs/api_reference.md, inserted just before the auto-generated footer linebaselineorcheckfully regeneratesapi_reference.mdfrom the OpenAPI spec, which naturally clears the audit sectionNo changes to
openapi_to_md.py— the clearing behavior is implicit becauseopenapi_to_markdown()rewrites the entire file.Review & Testing Checklist for Human
_AUTOGEN_FOOTER_PREFIX = "_This file was auto-generated by"is hardcoded inguard.pybut the actual footer is generated inopenapi_to_md.py(line 188). If the footer wording ever changes in one place but not the other, the audit section silently falls through to appending at the end of the file instead of inserting before the footer. Verify these stay in sync — consider whether they should share a constant.str.index(): The logic to strip an existing audit section finds"## Runtime Mismatches"by string search, then looks for the footer after it. Ifapi_reference.mdcontains that exact heading in a different context (unlikely but possible), it would incorrectly strip content. Review the stripping logic in_append_audit_section_to_api_ref(lines 220–231 of the new code) for edge cases._append_audit_section_to_api_refhandlestotal == 0, butcmd_auditexits withsys.exit(0)before calling it when there are no mismatches. The zero-mismatch branch is unreachable.detailsfield escapes|characters for the markdown table, butmethod,path,issue_type, andclient_ipdo not. Probably fine in practice but worth noting.Test Plan
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reloadcurl -X DELETE http://localhost:8000/v1/auditcurl -X POST http://localhost:8000/v1/users -H "Content-Type: application/json" -d '{"name": "Test", "email": "t@t.com", "age": 30}'python -m drift_guard.guard auditdocs/api_reference.mdhas a## Runtime Mismatchessection with the table before the footerpython -m drift_guard.guard baselinedocs/api_reference.mdNotes