Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
with:
version: "0.12.3"
enable-cache: true
- run: uv sync --locked --dev --extra braintrust
- run: uv sync --locked --dev
- run: uv run ruff format --check .
- run: uv run ruff check .
- run: uv run ty check
Expand Down
36 changes: 0 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,42 +36,6 @@ The backend supports task environments defined by either:

CPU, memory, and storage values map to Hypeman vCPUs, base memory, and writable overlay size. Harbor `public` and `no-network` modes map to attached and detached Hypeman networking.

## Braintrust reporting

Install the official Braintrust integration with the backend:

```bash
uv tool install harbor \
--with harbor-hypeman \
--with 'braintrust>=0.33.0'
```

Then add the official Harbor plugin to a run:

```bash
harbor run \
--dataset ./tasks \
--agent codex \
--model openai/gpt-5.6 \
--env harbor_hypeman:HypemanEnvironment \
--plugin braintrust
```

Set the Braintrust credentials required by your project before running the command.

This repository includes two pinned `kernel-mcp-server` tasks for reproducible backend and reporting checks:

```bash
harbor run \
--dataset ./evals/kernel-mcp-server \
--agent codex \
--model openai/gpt-5.6 \
--env harbor_hypeman:HypemanEnvironment \
--plugin braintrust
```

The tasks start from fixed source commits and use focused hidden Bun tests for browser-region forwarding and MCP client-capability parsing.

## Behavior

- Dockerfile builds are cached by Harbor environment content hash and rebuilt with `--force-build`.
Expand Down
11 changes: 0 additions & 11 deletions evals/kernel-mcp-server/browser-region/environment/Dockerfile

This file was deleted.

12 changes: 0 additions & 12 deletions evals/kernel-mcp-server/browser-region/instruction.md

This file was deleted.

27 changes: 0 additions & 27 deletions evals/kernel-mcp-server/browser-region/task.toml

This file was deleted.

This file was deleted.

13 changes: 0 additions & 13 deletions evals/kernel-mcp-server/browser-region/tests/test.sh

This file was deleted.

11 changes: 0 additions & 11 deletions evals/kernel-mcp-server/client-capabilities/environment/Dockerfile

This file was deleted.

13 changes: 0 additions & 13 deletions evals/kernel-mcp-server/client-capabilities/instruction.md

This file was deleted.

27 changes: 0 additions & 27 deletions evals/kernel-mcp-server/client-capabilities/task.toml

This file was deleted.

This file was deleted.

13 changes: 0 additions & 13 deletions evals/kernel-mcp-server/client-capabilities/tests/test.sh

This file was deleted.

3 changes: 0 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,6 @@ dependencies = [
"pathspec>=1.0.3",
]

[project.optional-dependencies]
braintrust = ["braintrust>=0.33.0"]

[dependency-groups]
dev = [
"pytest>=8.4.2",
Expand Down
Loading