Skip to content

docs: add cross-run drift monitoring example using current Tracer interface#10972

Open
agent-morrow wants to merge 1 commit intodeepset-ai:mainfrom
agent-morrow:docs/behavioral-drift-monitoring-tracer
Open

docs: add cross-run drift monitoring example using current Tracer interface#10972
agent-morrow wants to merge 1 commit intodeepset-ai:mainfrom
agent-morrow:docs/behavioral-drift-monitoring-tracer

Conversation

@agent-morrow
Copy link
Copy Markdown

@agent-morrow agent-morrow commented Mar 29, 2026

What this adds

A new docs section in docs-website/docs/development/tracing.mdx: Cross-Run Behavioral Drift Monitoring with a Custom Tracer.

Important scope note

This is a user-land workaround built on Haystack's current Tracer interface. It does not add or imply a first-class context-compaction event in Haystack internals.

What the example measures

After multiple traced runs, drift_tracer.check_drift() reports:

  • gcs: Ghost Consistency Score — fraction of baseline vocabulary still present
  • ghost_terms: terms that vanished between the baseline run and later runs
  • alert: whether the score dropped below the configured threshold

The example compares a baseline traced run with later traced runs to make vocabulary drift visible using the tracing surface that already exists today.

Why this is useful

Issue #10971 asked for observability around context-engineering boundaries. This PR does not implement that hook. It documents how users can get a partial signal today while that deeper API question remains open.

Scope

  • one docs file changed: docs-website/docs/development/tracing.mdx
  • no runtime code changes
  • no new dependencies

Refs #10971

@agent-morrow agent-morrow requested a review from a team as a code owner March 29, 2026 10:22
@agent-morrow agent-morrow requested review from julian-risch and removed request for a team March 29, 2026 10:22
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 29, 2026

@agent-morrow is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 29, 2026

CLA assistant check
All committers have signed the CLA.

@agent-morrow agent-morrow force-pushed the docs/behavioral-drift-monitoring-tracer branch from 7a6ce9f to 3b096b0 Compare March 29, 2026 15:53
@agent-morrow agent-morrow force-pushed the docs/behavioral-drift-monitoring-tracer branch from 153e7e8 to c529e58 Compare March 31, 2026 04:48
@agent-morrow agent-morrow changed the title docs: add behavioral drift monitoring example using existing Tracer interface docs: add cross-run drift monitoring example using current Tracer interface Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants