Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .agents/skills/python-structured-logging/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
name: python-structured-logging
description: Review or improve Python logging with structlog, stdlib logging, or existing wrappers. Use for logging changes, not unrelated Python refactoring.
---

# Python Structured Logging

Make logs useful operational events while preserving the user's scope and project contracts.

## Scope and contracts

- Review requests need findings with locations and concrete effects, not edits.
- Inspect the logger API, formatter/processors, event conventions, and failure ownership. Keep the stack and wrappers; migration requires authorization.
- Preserve business behavior, signatures, exception propagation, and retry decisions.
- Event names and fields may feed alerts, dashboards, and queries. Preserve existing contracts, including dotted names. Before an authorized rename, inspect available consumers and explain their updates; report external consumers you cannot verify.
- With no existing convention, use stable `snake_case` events such as `invoice_processed` and put variable values in fields. Follow existing field names; give new units explicit keys such as `duration_ms`.
- Replace diagnostic prints only in scope; preserve intentional CLI output.

## API and selective reading

Read the matching example pair for substantial refactoring, integration setup, or API uncertainty. Simple field additions and local reviews do not require examples.

- **structlog:** keyword fields and a locally bound logger; [good](examples/structlog/good.py), [bad](examples/structlog/bad.py).
- **stdlib:** `extra` or existing adapters; no arbitrary keyword fields or `.bind()`. Avoid reserved LogRecord keys. The formatter must emit added attributes; [good](examples/stdlib/good.py), [bad](examples/stdlib/bad.py).
- **Wrappers:** inspect their interface and output before adapting either pattern.

Read only the relevant reference when:

- Changing formatters/processors, diagnosing missing fields, serialization, or duplicate handlers: [output pipeline](references/output-pipeline.md).
- Working with cross-module or concurrent context or cleanup: [context lifecycle](references/context-lifecycle.md).
- Resolving failure ownership or sanitizing exception output: [exceptions and sensitive data](references/exceptions-and-sensitive-data.md).

## Context, exceptions, and noise

- Bind/inject shared context at request/job boundaries where supported; repeating `extra` is acceptable. A local `.bind()` does not enrich independent loggers. Clean up scoped context on success and failure without leaking between operations.
- Allowlist needed payload fields before binding. Never log credentials, tokens, authorization headers, or raw sensitive payloads. Identifiers and payment-derived fields still require the project's data policy; masking is not blanket permission.
- Log a failure once at the layer owning its operational outcome. Lower layers may propagate without logging. Use `logger.exception(...)` inside `except` when a traceback is appropriate; preserve control flow and derive retryability from the real policy.
- Exception messages and tracebacks can expose secrets despite safe fields. Inspect rendered output and use the existing sanitization path.
- Follow project levels: `INFO` for outcomes, `DEBUG` for optional diagnostics, `WARNING` for actionable degradation, `ERROR` for failed operations needing attention. Avoid per-item noise and redundant boundaries; retain useful progress for stalled or long-running work.

## Verify

Use focused tests/output capture proportional to the change, through the complete configured runtime output boundary (see [output pipeline](references/output-pipeline.md)):

- Contracts and application behavior preserved; fields survive formatting without API errors.
- One appropriate failure record, with traceback when needed; no sensitive values in fields, messages, or rendered exceptions.
- Request/job context isolated and cleaned up.

Report checks performed and unverified pipeline assumptions.
4 changes: 4 additions & 0 deletions .agents/skills/python-structured-logging/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Python Structured Logging"
short_description: "Practical guidance for safer, more useful Python logs"
default_prompt: "Use $python-structured-logging to review Python logging, report concrete findings, and preserve the existing stack and event contracts. Apply changes only when requested."
24 changes: 24 additions & 0 deletions .agents/skills/python-structured-logging/examples/stdlib/bad.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import logging

logger = logging.getLogger(__name__)


class PaymentGatewayError(RuntimeError):
pass


def process_invoice(
invoice_id: str,
customer_id: str,
request_id: str,
attempt: int,
amount_cents: int,
) -> None:
try:
if amount_cents < 0:
raise PaymentGatewayError("negative amount")

logger.info(f"invoice_{invoice_id}_processed for customer {customer_id} request={request_id} attempt={attempt}")
except PaymentGatewayError as exc:
logger.error(f"invoice processing failed for {invoice_id}: {exc}")
raise
31 changes: 31 additions & 0 deletions .agents/skills/python-structured-logging/examples/stdlib/good.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import logging

logger = logging.getLogger(__name__)


class PaymentGatewayError(RuntimeError):
pass


def process_invoice(
invoice_id: str,
customer_id: str,
request_id: str,
attempt: int,
amount_cents: int,
) -> None:
# This operation owns the failure record; callers propagate without logging.
context = {
"request_id": request_id,
"invoice_id": invoice_id,
"customer_id": customer_id,
"attempt": attempt,
}
try:
if amount_cents < 0:
raise PaymentGatewayError("negative amount")

logger.info("invoice_processed", extra={**context, "amount_cents": amount_cents})
except PaymentGatewayError:
logger.exception("invoice_processing_failed", extra={**context, "retryable": False})
raise
24 changes: 24 additions & 0 deletions .agents/skills/python-structured-logging/examples/structlog/bad.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import structlog

logger = structlog.get_logger(__name__)


class PaymentGatewayError(RuntimeError):
pass


def process_invoice(
invoice_id: str,
customer_id: str,
request_id: str,
attempt: int,
amount_cents: int,
) -> None:
try:
if amount_cents < 0:
raise PaymentGatewayError("negative amount")

logger.info(f"invoice_{invoice_id}_processed for customer {customer_id} request={request_id} attempt={attempt}")
except PaymentGatewayError as exc:
logger.error(f"invoice processing failed for {invoice_id}: {exc}")
raise
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import structlog

logger = structlog.get_logger(__name__)


class PaymentGatewayError(RuntimeError):
pass


def process_invoice(
invoice_id: str,
customer_id: str,
request_id: str,
attempt: int,
amount_cents: int,
) -> None:
# This operation owns the failure record; callers propagate without logging.
context = {
"request_id": request_id,
"invoice_id": invoice_id,
"customer_id": customer_id,
"attempt": attempt,
}
log = logger.bind(**context)
try:
if amount_cents < 0:
raise PaymentGatewayError("negative amount")

log.info("invoice_processed", amount_cents=amount_cents)
except PaymentGatewayError:
log.exception("invoice_processing_failed", retryable=False)
raise
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Python Logging Integration Notes

Shared scope, event contracts, and naming rules live in [SKILL.md](../SKILL.md).
Read only the reference needed for the current change:

- [Output pipeline](output-pipeline.md): formatter/processor changes, missing fields, serialization, or duplicate handlers.
- [Context lifecycle](context-lifecycle.md): cross-module context, concurrent requests/jobs, or cleanup.
- [Exceptions and sensitive data](exceptions-and-sensitive-data.md): failure ownership or sanitizing rendered exceptions.

## Sources

- [Python logging API](https://docs.python.org/3/library/logging.html)
- [Python logging cookbook](https://docs.python.org/3/howto/logging-cookbook.html)
- [structlog context variables](https://www.structlog.org/en/stable/contextvars.html)
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Context lifecycle

`logger.bind(...)` returns a logger with additional context; pass that logger to code that needs it. It does not automatically enrich independent loggers elsewhere.

For a structlog application already using contextvars, check that `merge_contextvars` is in the processor chain. Clear context at the start of a request/job, then bind its identifiers. Reset or clear context when the operation ends, including failure paths. Use scoped binding/token reset for nested operations that must restore parent context instead of clearing it.

Test two overlapping requests with distinct IDs and a subsequent operation with no ID. Their outputs must not share identifiers. Thread/task boundaries and hybrid sync/async frameworks may require explicit propagation; do not assume every execution context shares the same values.

For stdlib, retain the existing adapter, filter, or record factory. Check the supported Python version and adapter behavior before relying on per-call `extra` merging.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Exception ownership and sensitive output

Choose the owner based on the call chain: a request boundary, worker, or command may already record failures. Adding another exception log below it can duplicate alerts. If a lower layer owns the only failure record and propagates the exception, document that its caller must not log it again.

`logger.exception` normally renders the exception message too. Allowlisting structured fields alone does not sanitize credentials embedded in a URL, exception text, or captured locals. Exercise the configured sanitizer with synthetic sensitive values and inspect its final output. Never use real secrets as fixtures.

The paired examples use a non-retryable negative amount to illustrate preserved behavior. Both versions raise the same exception for the same input. The good version changes logging only; it does not add retry logic or invent payment metadata.
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Output pipeline

Stdlib `extra` adds attributes to a LogRecord; a default text formatter does not include arbitrary attributes. Inspect the project's formatter before claiming the output is structured. Preserve the existing handler configuration; reusable modules should not call `basicConfig()` or install root handlers.

Use `extra={"invoice_id": invoice_id}` with stdlib, and `invoice_id=invoice_id` with structlog. Do not use reserved LogRecord keys such as `name`, `message`, or `levelname` as `extra` fields. Formatter-required fields must also be handled for third-party records that lack them.

Capture the final rendered output as well as records. Check JSON decoding where JSON is the configured format, field types, exception rendering, and duplicate records from handler propagation. A test that only captures the pre-render event dictionary cannot establish downstream compatibility.

At the application's configuration boundary, include framework/server/access/error/lifecycle loggers and the actual stdout/stderr or collector path, not just the application logger. Exercise an unexpected failure through that runtime: a safe application formatter is insufficient if another output path can emit raw data. Keep reusable modules free of global logging configuration.

Formatter failure is part of this safety boundary. Probe non-finite numbers, unsupported objects and cyclic structured values through the public logging API, including stdlib `handleError`/diagnostic fallback. Use bounded safe normalization or a safe structured fallback: logging must not raise into application code, disclose the original exception/source through fallback, or emit invalid JSON when JSON is the output contract. Capture both streams and verify that a subsequent event still emits; silent record loss is not a successful fallback.
4 changes: 2 additions & 2 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Describe what this PR changes and why.
- [ ] `examples/stdlib/`
- [ ] plugin metadata
- [ ] `README.md`
- [ ] `_.github/`
- [ ] `.github/`

## Behavior impact
Explain any user-visible change in skill triggering, recommendations, examples, or installation flow.
Expand All @@ -37,7 +37,7 @@ Notes (include details if you could not run something):
- [ ] Updated examples to match the guidance

## Checklist
- [ ] I kept the repo aligned with a `structlog`-first but Python-wide logging stance
- [ ] I kept the repo aligned with the existing logging stack and user-requested scope
- [ ] I did not silently contradict the bundled reference guide
- [ ] I preserved the split between `structlog` and stdlib examples where applicable
- [ ] I kept installation and metadata paths consistent with the current repo layout
41 changes: 41 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Quality

on:
pull_request:
push:
branches: ['**']

permissions:
contents: read

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: pip
cache-dependency-path: requirements-dev.txt
- run: python -m pip install -r requirements-dev.txt
- run: python scripts/validate_repo.py
- run: python -m unittest discover -s tests -v
- run: python scripts/check_release_state.py

demo:
runs-on: ubuntu-latest
strategy:
matrix:
stack: [stdlib, structlog]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: pip
cache-dependency-path: evals/demo/requirements-${{ matrix.stack }}.txt
- run: python -m pip install -r evals/demo/requirements-${{ matrix.stack }}.txt
- run: python -m unittest discover -s tests/demo -v
env:
DEMO_TEST_STACK: ${{ matrix.stack }}
8 changes: 8 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ jobs:
with:
python-version: '3.12'

- name: Install validation dependencies
run: python -m pip install -r requirements-dev.txt

- name: Validate skill and examples
run: |
python scripts/validate_repo.py
python -m unittest discover -s tests -v

- name: Validate release state
run: |
python3 scripts/check_release_state.py --expected-version "${GITHUB_REF_NAME}"
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
venv
.uv*
.DS_Store
__pycache__/
*.py[cod]
66 changes: 42 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,27 +29,26 @@ npx skills add https://github.com/mrKazzila/python-structured-logging-skill

### Manual install

#### Claude Code
Copy the inner `plugins/python-structured-logging/skills/python-structured-logging` directory, including its examples and references. Run these commands from the repository root; choose the location for your client.

Copy this repository into the target project's `/.claude` folder.
| Client / scope | Destination |
| --- | --- |
| Claude Code / project | `<project>/.claude/skills/python-structured-logging` |
| Codex / user | `~/.agents/skills/python-structured-logging` |
| OpenCode / user | `~/.config/opencode/skills/python-structured-logging` |

See the [official Claude Skills documentation](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview).

#### Codex CLI

Copy [`plugins/python-structured-logging/skills/python-structured-logging`](plugins/python-structured-logging/skills/python-structured-logging) into your Codex skills path, typically `~/.codex/skills/python-structured-logging`.

See the [Agent Skills specification](https://agentskills.io/specification) for the standard skill format.

#### OpenCode

Clone the entire repo into the OpenCode skills directory:
For example, for Codex:

```sh
git clone https://github.com/mrKazzila/python-structured-logging-skill.git ~/.opencode/skills/python-structured-logging-skill
mkdir -p ~/.agents/skills
cp -R plugins/python-structured-logging/skills/python-structured-logging ~/.agents/skills/
```

Do not copy only the inner `skills/` folder. OpenCode auto-discovers `SKILL.md` files under `~/.opencode/skills/`.
For an existing installation, update its contents rather than nesting another copy. Do not copy the entire repository into a skill directory.

Verify discovery in a fresh client session: use `/python-structured-logging` for a manually installed Claude Code skill, `/skills` or `$python-structured-logging` in Codex, or ask OpenCode to list available skills. For Claude plugin installation, the command is namespaced as `/python-structured-logging:python-structured-logging`.

See the official [Claude Code](https://code.claude.com/docs/en/skills), [Codex](https://learn.chatgpt.com/docs/build-skills), and [OpenCode](https://opencode.ai/docs/skills/) documentation. Paths were checked against documentation on 2026-09-18; discovery still needs a smoke test in your client version.

The skill inspects the logging stack already in use before changing anything. If the project already uses `structlog`, it leans into bound context and structured fields. If the project intentionally uses stdlib `logging`, it improves event names, `extra` payloads, and traceback handling without forcing a migration.

Expand All @@ -71,24 +70,43 @@ The skill inspects the logging stack already in use before changing anything. If

## When Not to Use

- The user explicitly asked not to change logging behavior.
- The task is unrelated to logging. Review-only requests are supported and must not change files.
- The task is a one-off throwaway script where logging would add more noise than value.
- The project has strict logging conventions and the current task is unrelated to logging or observability.

## Verify / Contribute

- Edit the skill or reference guide.
- Update examples if the recommended behavior changes.
- Run checks before finishing:
Requires Python 3.12+ and either `uv` with `just`, or a Python virtual environment. No local Codex installation is needed.

```sh
just check
```

Equivalent commands without `just` or `uv`:

```sh
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements-dev.txt
.venv/bin/python scripts/validate_repo.py
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python scripts/check_release_state.py
```

Dependencies are pinned in `requirements-dev.txt`. The first installation requires access to a package index. `just validate` runs structural validation; `just test` runs executable example and validator regression checks. CI runs these checks on pull requests, branch pushes, and before a release.

Behavioral agent evaluation is separate: see [evals/README.md](evals/README.md) for fixtures, prompts, acceptance criteria, and skill/no-skill comparisons. Passing structural checks does not establish agent quality.

Keep changes scoped and update examples or eval cases when the recommended behavior changes.

## Generated demo project

Try the skill on a disposable FastAPI order API with stdlib or structlog logging. The application starts with intentional logging defects; an external checker measures HTTP behavior, rendered logs, exception ownership, secret handling, and request context before and after the agent's changes.

```sh
just validate
python scripts/check_release_state.py
python3 scripts/demo.py create --stack stdlib --dest /tmp/logging-demo-stdlib
```

- `just validate` uses `uv` plus `pyyaml`. If `pyyaml` is not already available, `uv` may need a writable cache and network access to fetch it.
- `uv run python scripts/check_release_state.py` is an equivalent release-state check if you prefer to keep the command under `uv`.
- Keep the skill concise, direct, and usable by agents during review and refactoring.
See the [demo guide](evals/demo/README.md) for environment setup, evaluation commands, and skill/no-skill trials. Run `just demo-test` to test the generator and evaluator; these tests are separate from `just check` and run in their own CI matrix.

## License

Expand Down
Loading
Loading