Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 48 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# token-optimization
A customer-facing engagement for context management and token optimization

This repository contains source material for a practical developer workshop on context management, token optimization, agent customization, tool/MCP hygiene, model choice, AI evals, usage visibility, and sustainable team practices.
This repository contains source material for a practical developer workshop on token optimization, context engineering, agent customization, tool/MCP hygiene, model choice, AI evals, usage visibility, and sustainable team practices.

Start with [`labs/README.md`](labs/README.md) for 1-hour, 2-hour, and 4-hour delivery outlines, then use the chapter files in order or as modular source material.
Start with [`labs/README.md`](labs/README.md) for 1-hour, 2-hour, and 4-hour delivery outlines, then use the track labs, deck outline, surface matrix, templates, exercises, and worksheets as modular source material.

## Sample web app

This repository also includes a static JavaScript sample app for GitHub Copilot Usage-Based Budgeting.
This repository also includes a static JavaScript sample app for GitHub Copilot Usage-Based Budgeting. It includes budget-scope guidance and a per-surface estimator that uses user-supplied token rates instead of hardcoded future pricing.

To run it locally:

Expand Down Expand Up @@ -57,16 +57,50 @@ The repo uses small project skills for workflows that should only load when rele
- Give attendees repeatable habits they can apply in other projects.
- Introduce eval-driven improvement so teams can measure whether changes help.

## Curriculum tracks

| Track | Labs |
| --- | --- |
| VS Code/IDE users | [`00`](labs/00-foundations.md), [`01`](labs/01-ide-context-and-prompt-flow.md), [`02`](labs/02-ide-instructions-tools-and-mcp.md), [`07`](labs/07-measurement-billing-and-governance.md), [`08`](labs/08-applied-repo-review-and-adoption.md) |
| GitHub Copilot CLI users | [`00`](labs/00-foundations.md), [`03`](labs/03-cli-context-and-tool-output.md), [`04`](labs/04-cli-agents-tools-and-cost-control.md), [`07`](labs/07-measurement-billing-and-governance.md), [`08`](labs/08-applied-repo-review-and-adoption.md) |
| GitHub.com/code review users | [`00`](labs/00-foundations.md), [`05`](labs/05-github-web-context-and-coding-agent.md), [`06`](labs/06-github-code-review-and-pr-hygiene.md), [`07`](labs/07-measurement-billing-and-governance.md), [`08`](labs/08-applied-repo-review-and-adoption.md) |
| Full cross-surface practitioner | [`00`](labs/00-foundations.md) through [`08`](labs/08-applied-repo-review-and-adoption.md) |

## Contents

- [`labs/README.md`](labs/README.md) — overview, prerequisites, and timed agendas
- [`labs/01-context-management-basics.md`](labs/01-context-management-basics.md)
- [`labs/02-instructions-and-agent-customizations.md`](labs/02-instructions-and-agent-customizations.md)
- [`labs/03-mcp-and-tool-optimization.md`](labs/03-mcp-and-tool-optimization.md)
- [`labs/04-chat-session-management.md`](labs/04-chat-session-management.md)
- [`labs/05-model-choice.md`](labs/05-model-choice.md)
- [`labs/06-chat-history-and-memory.md`](labs/06-chat-history-and-memory.md)
- [`labs/07-usage-and-billing-visibility.md`](labs/07-usage-and-billing-visibility.md)
- [`labs/08-ai-evals-and-observability.md`](labs/08-ai-evals-and-observability.md)
- [`labs/09-ideal-workshop-repo.md`](labs/09-ideal-workshop-repo.md)
- [`labs/10-next-steps-and-extra-topics.md`](labs/10-next-steps-and-extra-topics.md)
- [`labs/README.md`](labs/README.md) - overview, prerequisites, and timed agendas
- [`decks/token-optimization-context-engineering.pptx`](decks/token-optimization-context-engineering.pptx) - primary workshop delivery deck with embedded speaker notes
- [`decks/token-optimization-context-engineering.executive.pptx`](decks/token-optimization-context-engineering.executive.pptx) - executive briefing visual variant
- [`decks/token-optimization-context-engineering.technical.pptx`](decks/token-optimization-context-engineering.technical.pptx) - technical deep dive visual variant
- [`decks/token-optimization-context-engineering.outline.md`](decks/token-optimization-context-engineering.outline.md) - editable delivery deck outline
- [`decks/token-optimization-context-engineering.speaker-notes.md`](decks/token-optimization-context-engineering.speaker-notes.md) - speaker notes for the delivery deck
- [`tools/generate_context_deck.py`](tools/generate_context_deck.py) - regenerates the styled PPTX variants from the Markdown sources
- [`resources/copilot-surface-matrix.md`](resources/copilot-surface-matrix.md) - living reference for Copilot surfaces and context controls
- [`resources/context-inventory-worksheet.md`](resources/context-inventory-worksheet.md)
- [`resources/instruction-diet-worksheet.md`](resources/instruction-diet-worksheet.md)
- [`resources/customer-preflight-checklist.md`](resources/customer-preflight-checklist.md)
- [`resources/monday-morning-checklist.md`](resources/monday-morning-checklist.md)
- [`templates/README.md`](templates/README.md) - copy/paste starter Copilot customization files
- [`exercises/README.md`](exercises/README.md) - track-specific hands-on exercises
- [`facilitator/delivery-guide.md`](facilitator/delivery-guide.md)
- [`labs/00-foundations.md`](labs/00-foundations.md)
- [`labs/01-ide-context-and-prompt-flow.md`](labs/01-ide-context-and-prompt-flow.md)
- [`labs/02-ide-instructions-tools-and-mcp.md`](labs/02-ide-instructions-tools-and-mcp.md)
- [`labs/03-cli-context-and-tool-output.md`](labs/03-cli-context-and-tool-output.md)
- [`labs/04-cli-agents-tools-and-cost-control.md`](labs/04-cli-agents-tools-and-cost-control.md)
- [`labs/05-github-web-context-and-coding-agent.md`](labs/05-github-web-context-and-coding-agent.md)
- [`labs/06-github-code-review-and-pr-hygiene.md`](labs/06-github-code-review-and-pr-hygiene.md)
- [`labs/07-measurement-billing-and-governance.md`](labs/07-measurement-billing-and-governance.md)
- [`labs/08-applied-repo-review-and-adoption.md`](labs/08-applied-repo-review-and-adoption.md)
- [`labs/MIGRATION.md`](labs/MIGRATION.md)

## Regenerating the deck

The PPTX files are generated from the Markdown outline and speaker notes so content remains easy to review.

```powershell
python -m pip install -r requirements-dev.txt
python tools\generate_context_deck.py
```

The generator produces workshop, executive briefing, and technical deep dive variants. All generated PPTX files include embedded PowerPoint speaker notes from `decks/token-optimization-context-engineering.speaker-notes.md`.
Binary file added decks/Token Optimization.pptx
Binary file not shown.
Binary file not shown.
172 changes: 172 additions & 0 deletions decks/token-optimization-context-engineering.outline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Token Optimization and Context Engineering across GitHub Copilot

Source outline for the delivery deck. Keep this file easy to diff; regenerate the `.pptx` delivery artifacts after major edits with `python tools\generate_context_deck.py`.

## Slide 1: Title

- Token Optimization and Context Engineering
- Getting more value from every Copilot interaction
- Across Copilot CLI, VS Code, GitHub.com, coding agent, and code review

## Slide 2: Why this matters now

- Usage models are shifting from simple request counting toward more granular accounting.
- Long conversations, broad tool output, and unnecessary context can affect cost, latency, and quality.
- The durable habit is not "use less Copilot"; it is "send better context."

## Slide 3: Context engineering

- Context engineering means bringing the right information, in the right format, to the model.
- It turns ad hoc prompting into repeatable workflows.
- Core primitives: custom instructions, prompt files, skills, agents, retrieval, and human review gates.

## Slide 4: What counts as context

- Product and system instructions
- Repository, organization, path-specific, personal, and agent guidance
- Conversation history and summaries
- Files, selections, issues, pull requests, and tool results
- Retrieved docs, web pages, MCP output, and generated plans

## Slide 5: Important billing nuance

- "Context is resent" is a useful mental model, not a universal invoice formula.
- Some products and models use caching or product-specific accounting.
- The practical takeaway remains: stale or irrelevant context increases cost, latency, and confusion.

## Slide 6: Where context gets wasted

- Whole directories or large files when one function matters
- Long-running mixed-topic sessions
- Overgrown custom instructions
- Raw logs, build output, generated files, and tool noise
- High-cost models used for routine work
- Auto-review or agent tasks with vague scope

## Slide 7: The five levers

- Context hygiene
- Prompt discipline
- Model and surface selection
- Scope and tool control
- Measurement

## Slide 8: Lever 1 - Context hygiene

- Start fresh when the task changes.
- Summarize before switching focus.
- Keep stable source-of-truth docs short and current.
- Avoid re-discovery loops by preserving useful handoffs.

## Slide 9: Lever 2 - Prompt discipline

- Use Markdown structure.
- State outcome, scope, constraints, and success criteria.
- Reference specific files, issues, PRs, or selections.
- Add validation gates before risky edits.

## Slide 10: Lever 3 - Model and surface selection

- Choose the cheapest model or surface that can reliably finish the job.
- Use VS Code Plan for design, Agent for implementation, Ask for exploration.
- Use CLI when you need visible context/tool control.
- Use code review when the unit of work is a pull request.

## Slide 11: Lever 4 - Scope and tool control

- Keep workspace, repository, and tool scope tight.
- Use targeted instructions instead of one giant instruction file.
- Enable MCP tools only when the task needs them.
- Use human-in-the-loop approval for high-risk tool actions.

## Slide 12: Lever 5 - Measurement

- Measure tokens where exposed.
- Measure premium requests, review counts, and billing views where tokens are hidden.
- Measure quality with retries, false positives, PR churn, and human rework.
- Optimize after you have a baseline.

## Slide 13: Surface matrix

- CLI: most visible token/context controls.
- VS Code: daily coding workflow with Ask, Plan, Agent, custom instructions, prompt files, and review.
- GitHub.com: repo, issue, PR, and discussion context.
- Coding/cloud agent: asynchronous implementation from scoped tasks.
- Code review: PR-focused feedback with product-specific constraints.

## Slide 14: VS Code pattern

- Curate project context with concise instructions and docs.
- Plan first for complex work.
- Implement from the plan in a fresh or focused session.
- Review changes against the plan.

## Slide 15: GitHub.com web pattern

- Ask from the page that already has the relevant context.
- Keep threads focused.
- Use repository, issue, and PR context deliberately.
- Treat generated files as drafts.

## Slide 16: Copilot CLI pattern

- Use sessions like branches: one task, one focused context.
- Filter tool output before it enters the conversation.
- Delegate noisy discovery when available.
- Use usage/context visibility to teach the token mental model.

## Slide 17: Coding agent pattern

- Write issues like implementation briefs.
- Include acceptance criteria, validation commands, and files to avoid.
- Keep tasks small enough to review.
- Review the generated PR like any other teammate's work.

## Slide 18: Code review pattern

- Keep PRs small.
- Tune repo and path-specific review instructions.
- Watch automatic review policy and quota implications.
- Validate Copilot findings; review comments are not approvals.

## Slide 19: Context inventory exercise

- List every context source.
- Mark required, useful, stale, redundant, sensitive, or unknown.
- Decide what stays, what moves, what gets summarized, and what gets removed.

## Slide 20: Instruction diet exercise

- Keep stable rules always-on.
- Move targeted rules to path-specific instructions.
- Move repeated workflows to prompts, skills, or agents.
- Link long docs instead of copying them into instructions.

## Slide 21: Governance

- Content exclusion
- Model access policies
- Code review settings
- Budgets and alerts
- Telemetry and dashboard ownership

## Slide 22: Delivery tracks

- 1 hour: mental model, surface view, demo, checklist
- 2 hours: practitioner lab and prompt/context refactor
- 4 hours: customer environment review and team operating model

## Slide 23: What improvement looks like

- Fewer irrelevant tokens
- Faster answers
- Fewer retries
- Higher-signal reviews
- Clearer ownership of policy and measurement

## Slide 24: Takeaways

- Context is a design input, not a dumping ground.
- Token optimization and context engineering improve both cost and quality.
- Surface controls differ; the habits transfer.
- Start with three changes this week and measure.
Binary file not shown.
99 changes: 99 additions & 0 deletions decks/token-optimization-context-engineering.speaker-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Speaker Notes: Token Optimization and Context Engineering

These notes are embedded into the generated PPTX variants by `python tools\generate_context_deck.py`.

## Slide 1

Open by saying this is not a "use less Copilot" session. It is a "get better answers with less waste" session.

## Slide 2

Anchor on the current shift: customers need practical habits before usage surprises become trust issues.

## Slide 3

Define context engineering as an evolution of prompt engineering. The goal is not clever phrasing; the goal is the right information in the right format.

## Slide 4

Walk left to right through the context inputs. Point out that the user prompt is often the smallest part.

## Slide 5

Use careful language. Product accounting can vary and caching may apply. The workshop teaches habits that remain useful even as billing details change.

## Slide 6

Emphasize that context waste hurts quality before it hurts the bill. Too much irrelevant context makes the model less focused.

## Slide 7

This is the map for the workshop. Do not create a second framework; map context engineering into these five levers.

## Slide 8

Use the branch analogy: one focused session per task. Clearing context is good between tasks, but clearing and re-reading the same files can be wasteful.

## Slide 9

Show one overloaded request and one structured request. Keep the demo short.

## Slide 10

Make model and surface selection practical. The right answer may be a different surface, not just a different model.

## Slide 11

Explain that every enabled tool is a capability and a possible source of noise. Tool descriptions and access boundaries are product UX.

## Slide 12

For surfaces without token visibility, measure proxies: retries, PR churn, review false positives, usage dashboards, and time-to-merge.

## Slide 13

Warn that the matrix is not parity. Some cells are intentionally "not available."

## Slide 14

VS Code is the daily practitioner path: Ask to understand, Plan to structure, Agent to implement, Review to validate.

## Slide 15

GitHub.com is strongest when the page context matters: repository, issue, pull request, or discussion.

## Slide 16

CLI remains the best live demo for token concepts because context and usage are visible.

## Slide 17

Coding agent tasks should look like good implementation tickets. Vague tasks create broad exploration and broad diffs.

## Slide 18

Code review is purpose-built. Do not promise model switching. Teach PR hygiene and instruction hygiene.

## Slide 19

Students should discover that some of their "helpful" context is stale, sensitive, or redundant.

## Slide 20

This is the most actionable demo. Trim one bloated instruction file and show where the pieces should move.

## Slide 21

Separate developer habits from admin controls. Admin controls compound because they apply every turn for every user.

## Slide 22

Match the track to the audience. Do not run the 4-hour customer review without preflight.

## Slide 23

Avoid unsupported savings promises. Talk about directional improvements and measurement.

## Slide 24

Close with a small commitment: pick three habits, measure for a week, then expand.
Binary file not shown.
26 changes: 26 additions & 0 deletions exercises/01-vscode-context-attachments/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Exercise 01: VS Code Context Attachments

## Goal

Practice replacing broad IDE context with deliberate file, selection, and test context.

## Task

Investigate a login bug in a sample project.

## Baseline

Read [`naive-transcript.md`](naive-transcript.md) and identify context that is stale, redundant, or too broad.

## Engineered flow

Read [`engineered-transcript.md`](engineered-transcript.md) and compare:

- Which files are attached?
- Which context is omitted?
- Which mode is used first?
- What validation gate is added?

## Output

Write one prompt for Ask mode, one for Plan mode, and one for Agent mode.
12 changes: 12 additions & 0 deletions exercises/01-vscode-context-attachments/engineered-transcript.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Engineered transcript

User:

> In `src/auth/session.ts`, explain why expired sessions are not rejected. Use the selected failing test and the stack trace below. Do not inspect unrelated directories. Return a diagnosis and patch plan before editing.

Likely result:

- Specific file and test context.
- Clear scope boundary.
- Plan before implementation.
- Validation is known before edits begin.
Loading