Skip to content

Replace Chapter 8 with Microsoft Waza eval lab#4

Merged
queuedecay merged 8 commits into
mainfrom
copilot/replace-chapter-8-content
May 12, 2026
Merged

Replace Chapter 8 with Microsoft Waza eval lab#4
queuedecay merged 8 commits into
mainfrom
copilot/replace-chapter-8-content

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 12, 2026

Bug Fix

What was the bug?

Chapter 8 relied on third-party eval tooling with licensing assumptions customers may not have. The lab also lacked the Waza workspace artifacts required to evaluate chat customizations.

How did you fix it?

  • Chapter 8 content

    • Replaced the prior eval/observability lab with Microsoft Waza guidance.
    • Added GitHub Actions and VS Code Chat Customizations Evaluations workflows.
    • Clarified that customers should use customer-owned eval data.
  • Waza CI

    • Added .github/workflows/waza-eval-suite.yml.
    • Runs Waza against evals/token-optimization/eval.yaml.
    • Uploads JSON and JUnit result artifacts.
waza run evals/token-optimization/eval.yaml \
  --verbose \
  --output waza-results/results.json \
  --reporter junit:waza-results/results.xml
  • Required customization artifacts

    • Added a workspace skill: .github/skills/token-optimization.skill.md
    • Added a prompt: .github/prompts/context-triage.prompt.md
    • Added instructions: .github/instructions/token-optimization.instructions.md
    • Added an agent: .github/agents/token-eval-reviewer.agent.md
  • Waza eval scaffold

    • Added skills/token-optimization/SKILL.md.
    • Added a minimal eval suite under evals/token-optimization/.

Testing

Not included in this description; review should focus on the Chapter 8 replacement, Waza workflow, and required workspace customization files.

@queuedecay queuedecay marked this pull request as ready for review May 12, 2026 21:55
@queuedecay queuedecay merged commit 8f20dd1 into main May 12, 2026
1 check passed
@queuedecay queuedecay deleted the copilot/replace-chapter-8-content branch May 12, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants