Skip to content

Audit XMD tests against approved product behavior #768

Description

@taras

Story

As Product Owner, I want to review what XMD's tests show working from a product perspective, so future features can derive implementation evidence from approved behavior and bring only new verification decisions back to me.

Current gap

The repository has extensive specifications and tests, but much of their verification design was selected by Architects and Implementors without a Product Owner interview. Test volume and requirement coverage do not by themselves establish that the most important user journeys, observable behavior, failures, recovery and incomplete implementations are represented clearly.

#542 owns exact traceability between specification requirement IDs and tests. This audit decides the product behavior those requirements and tests should cover; it does not duplicate the traceability mechanism.

Audit

Inventory the test corpus by product feature. For each feature, the Architect prepares a first-pass product verification design and reviews one behavior at a time with the Product Owner:

  1. Describe the behavior the feature must provide.
  2. Show what a person does and observes.
  3. Identify an incomplete implementation that could appear to work.
  4. Propose verification that distinguishes the complete behavior.
  5. Record the Product Owner's feedback.

Compare the accepted product verification with the specifications and existing tests. Keep lower-level unit, boundary, fixture, runtime and command choices delegated to the Architect and Planner when they follow the approved behavior.

Propose reusable verification rules separately. Only explicit Product Owner approval adds them to .agents/product-verification.md. Missing or misleading coverage becomes a focused follow-up Story; do not rewrite the test corpus inside the audit merely to increase counts.

Acceptance

  • Every product feature represented in the test corpus has an explicit audit disposition and visible reviewed count.
  • Accepted product verification covers its user journey, observable behavior, important failures, recovery and a plausible incomplete implementation.
  • Existing specification requirements and tests map to that behavior or are recorded as debt.
  • Each missing, misleading or redundant product-level verification item has one owning follow-up Story.
  • Lower-level evidence remains delegated when it follows approved product behavior.
  • Reusable rules enter the rulebook only after explicit Product Owner approval.
  • The resulting work is reconciled with Quest: Ensure every specification requirement has exactly one test #542 rather than creating a competing requirement-to-test index.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions