Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Contoso Travel Concierge Workshop — dev container image.
# Agent Optimization Workshop — dev container image.
#
# Base image is pinned here so we can customize freely later (system packages,
# CLIs, tools) without touching devcontainer.json. Post-create actions (env
Expand All @@ -12,7 +12,12 @@ ENV DEBIAN_FRONTEND=noninteractive \

# System tools we use across labs: curl (already present), jq (parsing outputs),
# git (already present), zip/unzip (bundling artifacts).
RUN apt-get update && apt-get install -y --no-install-recommends \
#
# The base image pre-configures the Yarn apt repo, whose signing key has since
# rotated and now breaks `apt-get update` (NO_PUBKEY 62D54FD4003F6525). We don't
# use Yarn, so remove that source before updating.
RUN rm -f /etc/apt/sources.list.d/yarn.list /usr/share/keyrings/yarn*.gpg \
&& apt-get update && apt-get install -y --no-install-recommends \
jq \
zip \
unzip \
Expand Down
28 changes: 0 additions & 28 deletions .devcontainer/README.md

This file was deleted.

26 changes: 12 additions & 14 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,25 +1,18 @@
{
"name": "Contoso Travel Concierge Workshop",
"name": "Agent Optimization Workshop",
"build": {
"dockerfile": "Dockerfile",
"context": "."
},
"remoteUser": "vscode",
"postCreateCommand": "bash .devcontainer/post-create.sh",

"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"ms-azuretools.vscode-docker",
"ms-azuretools.vscode-azurecontainerapps",
"ms-azuretools.vscode-bicep",
"github.copilot",
"github.copilot-chat",
"github.vscode-github-actions",
"davidanson.vscode-markdownlint",
"bierner.markdown-mermaid",
"redhat.vscode-yaml"
"ms-windows-ai-studio.windows-ai-studio"
],
"settings": {
"python.defaultInterpreterPath": "/usr/local/bin/python",
Expand All @@ -34,8 +27,13 @@
"remoteEnv": {
"PYTHONDONTWRITEBYTECODE": "1"
},
"mounts": [
// Persist coach progress between container rebuilds
"source=${localWorkspaceFolderBasename}-coach,target=/home/vscode/.contoso-coach,type=volume"
]
"features": {
"ghcr.io/devcontainers/features/azure-cli:1": {},
"ghcr.io/devcontainers/features/copilot-cli:1": {},
"ghcr.io/azure/azure-dev/azd:0": {},
"ghcr.io/devcontainers/features/github-cli:1": {}
}

// Persist coach progress between container rebuilds
// "mounts": [ "source=${localWorkspaceFolderBasename}-coach,target=/home/vscode/.contoso-coach,type=volume"]
}
31 changes: 21 additions & 10 deletions .devcontainer/post-create.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ set -euo pipefail

log() { printf "\n▸ %s\n" "$*"; }

# Install Dependencies
log "Installing workshop Python dependencies"
pip install --upgrade pip
if [[ -f requirements.txt ]]; then
Expand All @@ -17,27 +18,37 @@ if [[ -f requirements-dev.txt ]]; then
pip install -r requirements-dev.txt
fi

log "Ensuring azd extensions are up to date"
# Update Tools
log "Ensuring azd + the hosted-agents extension are ready"
if command -v azd >/dev/null 2>&1; then
azd version || true
# Hosted-agents extension used in Fundamentals lab 05.
# TODO(nitya): pin exact extension version once labs stabilize.
azd ext install azure.ai.agents || true
azd ext list || true
# Hosted-agent commands (`azd ai agent`, `azd deploy` for host: azure.ai.agent)
# come from the azure.ai.agents extension used in Fundamentals lab 05.
azd config set alpha.extensions on >/dev/null 2>&1 || true
if azd extension list --installed --output json 2>/dev/null | grep -q "azure.ai.agents"; then
echo "azure.ai.agents already installed — upgrading to the latest version"
azd extension upgrade azure.ai.agents || true
else
echo "Installing azure.ai.agents extension"
azd extension install azure.ai.agents || true
fi
azd extension list --installed || true
else
echo "⚠️ azd not installed — skipping extension setup"
fi

# Prepare Environment
log "Making workshop scripts executable"
chmod +x scripts/*.sh 2>/dev/null || true

log "Preparing coach progress directory"
mkdir -p /home/vscode/.contoso-coach
touch /home/vscode/.contoso-coach/.keep

log "Running spec tests as a smoke check"
if command -v pytest >/dev/null 2>&1; then
pytest -q || echo "⚠️ spec tests reported issues — see output above"
fi
# Python Dependencies
# log "Running spec tests as a smoke check"
# if command -v pytest >/dev/null 2>&1; then
# pytest -q || echo "⚠️ spec tests reported issues — see output above"
# fi

log "Devcontainer ready. Open .github/PLAN.md or README.md to get started."
log "Devcontainer ready. Open README.md to get started."
7 changes: 5 additions & 2 deletions .github/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,8 +248,11 @@ guardrail.

## Open questions

- Exact model list (agent model, evaluator model, embedding?) — decide during
Fundamentals authoring
- ~~Exact model list (agent model, evaluator model, embedding?)~~ **Decided:**
concierge `gpt-5.4-mini` (version `2026-03-17`) + judge `gpt-5.4` (version
`2026-03-05`), both GlobalStandard, auto-deployed by `azd up`
(`infra/main.bicep`). Model region/version/SKU reference:
<https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure?pivots=azure-direct-others>
- Whether to port the LAB540 CSVs as-is (rebranded to Contoso) or curate a
smaller eval-friendly subset

Expand Down
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,9 @@ cython_debug/
.abstra/

# Visual Studio Code
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# you could uncomment the following to ignore the entire vscode folder
# .vscode/
# Temporary file for partial code execution
Expand Down
20 changes: 20 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Contributing

> 🚧 This document is a work in progress. Please wait for final guidance to be released.

## Process

This is a **living resource** that is built and maintained with the help of coding agents. Follow these steps when you want to propose an addition or modification:

1. Update [`.github/PLAN.md`](./.github/PLAN.md) with the intent
2. Update the relevant spec in [`specs/`](./specs/) so tests know the new truth
3. Update or add tests in [`tests/`](./tests/)
4. Implement the change
5. `pytest -q` passes → open a PR

CI runs the spec suite on every PR
([`.github/workflows/verify-course.yml`](./.github/workflows/verify-course.yml)).

## Feedback

Issues and PRs welcome. See [`LICENSE`](./LICENSE) for terms.
117 changes: 74 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
<div align="center">
<table width="100%">
<tr>
<td<div style="background:#111111;color:#f2c200;padding:16px;margin:4px;border-radius:6px;font-family:sans-serif;">

🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛

### 🚧 Work in Progress 🚧

**This workshop is under active development and testing.**. <br/>
Please wait till this banner is removed before using it for self-guided or instructor-led delivery.

🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛🟨⬛

</div></td>
</tr>
</table>
</div>

<div align="center">

# Build, Evaluate & Optimize AI Agents with Microsoft Foundry & GitHub Copilot
Expand All @@ -6,24 +25,22 @@

</div>

> ⚠️ **Fictional scenario.** The Contoso Travel Concierge agent, its dataset
> (flights, hotels, car rentals), and all company names used in this workshop
> are entirely **fictitious** and created for educational purposes only. They
> do not represent — and should not be interpreted as representing — any real
> travel provider, product, price, or availability.

> 🤖 **Built with coding agents + human-in-the-loop.** This workshop was
> developed collaboratively using AI coding agents (GitHub Copilot and related
> tools) under continuous human oversight and review. All content, code,
> datasets, and pedagogy have been curated and validated by a human maintainer
> before publication. Contributions follow the same process.

> **Scenario.** Contoso is a fictitious enterprise retail company serving customers worldwide. Contoso Travel Concierge is their internal travel planning tool, used by employees to book flights, cars and hotels - and manage expenses - while ensuring they are in compliance with company travel policies.
>
> **Workshop.** This workshop was built with the help of coding agents grounded in our docs, with human oversight and review for correctness and consistency. If you find any discrepancies or have questions, [please file an appropriate issue](https://github.com/microsoft-foundry/agent-optimization-workshop/issues) for our attention.

<br/>

## Overview

A hands-on course that walks you through the full **Agent DevOps loop** on
Microsoft Foundry — from provisioning to deployment to continuous optimization
— using a single running scenario: the **Contoso Travel Concierge**, a
fictitious multi-agent travel assistant.
Building an AI agent on Microsoft Foundry can be fairly straightforward. Create a new project, deploy a relevant model, configure agent instructions and tools - then deploy it to get an endpoint you can send requests to (from a UI-based playground or a code-based client).

But agent behaviors can be non-deterministic. Building reliable agents - that meet required cost, latency and quality targets - requires end-to-end observability. Start with evaluations that assess the quality, performance and safety of your agent with built-in and custom metrics. Use tracing (logs) to debug issues and build your intuition for where your agent incurs cost and latency. And use application insights to understand your agent performance in production, at scale.

Real-world deployments can also help identify agent drift from the desired performance targets. For instance, models may evolve or new edge cases may be revealed from actual usage. Keeping your agent operating correctly while consistemtly meeting desired criteria - requires _continuous optimization_, reflected by the **Agent DevOps loop** shown below.


```mermaid
flowchart LR
Expand All @@ -33,6 +50,9 @@ flowchart LR
Monitor --> Protect([Protect]) --> Evaluate
```

This repository provides a series of hands-on labs that walks you through this loop on Microsoft Foundry — from provisioning to deployment to continuous optimization — usingthe **Contoso Travel Concierge** multi-agent scenario.


The course is organized in three phases:

| Phase | What you do | Time |
Expand All @@ -41,30 +61,50 @@ The course is organized in three phases:
| **Core Labs** | Observe → Evaluate → Optimize → Monitor on the Prompt Agent; capstone on the Hosted Agent | ~90 min |
| **More Labs** | Single-question deep dives against the deployed agents | 20–30 min each |

A repo-scoped [**workshop-coach**](./.github/agents/workshop-coach.agent.md)
GitHub Copilot agent supports self-guided learners — it tracks your progress,
guides you through the next step, and never does the task for you.

Full course design lives in [**`.github/PLAN.md`**](./.github/PLAN.md).
> **EXPERIMENTAL**: <br/>
> The repository is configured with a [**workshop-coach**](./.github/agents/workshop-coach.agent.md) agent (using GitHub Copilot) that self-guided learners can use to ask for explanations or debug issues, without losing track of their progress. The coach guides you through the next step but never does the task for you, so you learn by doing.

<br/>

## Quickstart

The fastest path from zero to running the first lab:
The repository is configured with a [`.devcontainer/`](./.devcontainer/) that defines the default Python environment and dependencies required to run the exercises. The fastest way to get started and running your first lab is:

1. **Fork the repo** to your profile to get a sandbox you can modify
1. **Launch the Dev Container** using GitHub Codespaces (browser) or Docker Desktop (device)

This should automatically invoke the scripts to install Python dependencies, update required tooling, and initialize environment for lab execution. Wait till you see an active VS Code terminal - and use these commands to verify installation status.

```bash
# Python 3.13+
python --version

1. **Open in a Codespace or Dev Container.** The [`.devcontainer/`](./.devcontainer/)
provisions Python 3.13, `az`, `azd`, `gh`, and coach state persistence.
2. **Install workshop deps** (post-create runs this automatically):
```bash
pip install -r requirements.txt
```
3. **Open [`labs/fundamentals/00-overview.md`](./labs/fundamentals/00-overview.md)**
and follow the trail.
# Azure CLI ("azure-cli": "2.89.0" or higher)
az version

# Azure Developer CLI (azd version 1.30.0 or higher)
azd version

# GitHub Copilot CLI (GitHub Copilot CLI 1.0.78. or higher)
copilot version

# GitHub CLI (gh version 2.97.0 or higher)
gh --version
```

Prefer local setup? You need Python 3.11+, `az`, `azd`, and `gh` on your PATH.
> Prefer local setup? Create a virtual environment and install the tools manually. You will need Python 3.13+, `az`, `azd`, `copilot` and `gh` on your PATH.

You are ready to get started on the first lab.
- **Open [`labs/fundamentals/00-overview.md`](./labs/fundamentals/00-overview.md)**
and follow the trail from there.


<br/>

## Core Labs

Every lab teaches **one node** of the Agent DevOps loop. Complete them in order.
The core labs track takes you through the steps of the Agent DevOps loop using our Contoso Travel Concierge scenario. Every lab teaches **one node** of the Agent DevOps loop. Complete them in order.

| # | Lab | Loop node |
|---|-----|-----------|
Expand All @@ -77,9 +117,11 @@ Every lab teaches **one node** of the Agent DevOps loop. Complete them in order.

Fundamentals prerequisites: [`labs/fundamentals/`](./labs/fundamentals/).

<br/>

## More Labs

Extensible library of one-question deep dives — take in any order after Core.
This section provides an evolving library of labs that offer "one-question deep dives". You should be able to explore them in any order once you complete the Core Labs. Watch for frequent updates to this section to learn new features or best practies.

| # | Lab | Loop node |
|---|-----|-----------|
Expand All @@ -89,19 +131,8 @@ Extensible library of one-question deep dives — take in any order after Core.
| 4 | [Datasets from real traces](./labs/more/trace-driven-datasets.md) | Evaluate |
| … | … see [`labs/more/README.md`](./labs/more/README.md) for the full index | |

## Contributing

This is a **living resource**. When you propose a change:

1. Update [`.github/PLAN.md`](./.github/PLAN.md) with the intent
2. Update the relevant spec in [`specs/`](./specs/) so tests know the new truth
3. Update or add tests in [`tests/`](./tests/)
4. Implement the change
5. `pytest -q` passes → open a PR

CI runs the spec suite on every PR
([`.github/workflows/verify-course.yml`](./.github/workflows/verify-course.yml)).
<br/>

## Feedback

Issues and PRs welcome. See [`LICENSE`](./LICENSE) for terms.
Have questions, contributions or feedback? Watch for updates to the [CONTRIBUTING](./CONTRIBUTING.md) guide.
Empty file.
6 changes: 3 additions & 3 deletions artifacts/prompts/reference/prompt-agent-baseline-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ Tokyo, Rome, and Cancún. Be warm, professional, and concise.

You have three attached datasets:

- **flights.csv** — flights (id, airline, route, cabin, price, seats)
- **hotels.csv** — hotels (id, name, city, stars, nightly price, amenities)
- **car_rentals.csv** — rental vehicles (id, company, city, type, daily price)
- **Flights** — flights (id, airline, route, cabin, price, seats)
- **Hotels** — hotels (id, name, city, stars, nightly price, amenities)
- **Car rentals** — rental vehicles (id, company, city, type, daily price)

When travelers ask travel questions, look up matching rows from the relevant
dataset before answering.
Expand Down
Loading
Loading