Skip to content

feat(sdk): add yaml-aligned eval authoring helpers#1430

Merged
christso merged 1 commit into
mainfrom
feat/sdk-yaml-helpers
Jun 19, 2026
Merged

feat(sdk): add yaml-aligned eval authoring helpers#1430
christso merged 1 commit into
mainfrom
feat/sdk-yaml-helpers

Conversation

@christso

Copy link
Copy Markdown
Collaborator

Summary

TypeScript .eval.ts suites can now use @agentv/sdk to author YAML-aligned eval definitions with camelCase ergonomics while still loading through AgentV's canonical YAML parser/runtime path. This keeps @agentv/core as the runner source of truth and preserves snake_case serialization with assertions as the durable field.

What Changed

  • Added defineEval(), evalSuite(), toEvalYamlObject(), and serializeEvalYaml() to @agentv/sdk, along with typed suite, test, execution, and workspace authoring interfaces.
  • Added a whitelist lowerer that converts known AgentV wire keys such as expectedOutput, inputFiles, lifecycle hook names, and execution aliases without rewriting opaque user payload keys.
  • Taught the TypeScript eval loader to detect SDK-authored suites and materialize them by lowering into the existing YAML parser path instead of widening the core runner API.
  • Added SDK unit coverage, loader fixtures/tests, and a runnable examples/features/sdk-eval-authoring/ dogfood example.
  • Updated SDK docs and README guidance to point new eval authoring toward @agentv/sdk instead of the deprecated @agentv/eval surface.

Testing

  • git diff --check
  • bun --filter @agentv/sdk test
  • bun test packages/core/test/evaluation/loaders/ts-eval-loader.test.ts
  • bun --filter @agentv/sdk build
  • bun --filter @agentv/core build
  • bun apps/cli/src/cli.ts eval examples/features/sdk-eval-authoring/evals/greeting.eval.ts --output <tmp>
  • bun run validate:examples
  • bun run typecheck
  • bun run lint

Compound Engineering
Codex

@christso christso force-pushed the feat/sdk-yaml-helpers branch from 87a6211 to 15ddee6 Compare June 19, 2026 06:29
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 19, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: dc7e0ae
Status: ✅  Deploy successful!
Preview URL: https://e750688a.agentv.pages.dev
Branch Preview URL: https://feat-sdk-yaml-helpers.agentv.pages.dev

View logs

@christso christso force-pushed the feat/sdk-yaml-helpers branch from 15ddee6 to dc7e0ae Compare June 19, 2026 06:31
@christso christso marked this pull request as ready for review June 19, 2026 06:33
@christso christso merged commit adb3c88 into main Jun 19, 2026
10 checks passed
@christso christso deleted the feat/sdk-yaml-helpers branch June 19, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant