Add run-evals orchestration skill and evals for published skills

[reidbaker-agent]

Human description

We need a way to unit test and regression test skills "evals" are how the industry seems to do it. The evals system authored by the devtools team is focused on user journeys and end to end flows along with token efficiency and model comparisons. This problem of unit and regression tests for contributor skills and even the few published skills for a package author is on the roadmap but too far out to wait. The pattern introduced in this pr is intended for local consumption to give me some way of evaluating regression in skills and the most valuable parts are the prompts and expectations. The rest of the "infra" can and should be replaced when something better comes along.

Finally these tests do not run automatically and they may not work for your harness and model. They require being run manually and were testing using an internal version of antigravity and gemini 3.1 pro and all pass.

Agent description

Description

Authored by /contributor-pr-description feedback welcome.
This PR introduces the run-evals orchestration skill, adds structural consistency testing, and creates evals.json tests for published skills in the dart_skills_lint package.

The run-evals skill executes evaluations for each skill in complete isolation (using a branched workspace).

Changes

• .agents/skills/run-evals/: Added the orchestration skill for executing skill evaluations in isolated branch workspaces.
• evals/: Added evals/README.md and code_quality_rubric.json to introduce a two-tiered evaluation architecture. This architecture separates skill-specific task assertions (defined in individual evals.json files) from universal quality standards (defined in shared rubric files). This modular approach allows us to enforce broad health and quality expectations across all agent runs without duplicating them into every skill's evaluation file.
• test/skills_evals_test.dart: Added a test suite enforcing that all published skills in skills/ contain an evals.json file. It also guarantees that every evals.json file across the repository shares the exact same JSON structure, while keeping that structure schema-agnostic and flexible. Re-architected JSON decoding and type promotion to use clean Dart 3 pattern matching switch expressions.
• evals.json Cleanup: Dropped the redundant "skill_name" key from all evaluation files (as it's inferred from the parent directory) and hoisted "repo_criteria" to the top of the files to prioritize readability.

Related PRs

• PR #193: Introduces the evals for the add-dart-lint-validation-rule skill, built on top of this PR's evals.json infrastructure.
• PR #194: Adds Meta-Evals (a dummy skill and static datasets of "good" and "bad" code) to test and validate this PR's code_quality_rubric.json, ensuring the rubric correctly catches anti-patterns while permitting clean code.

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive Skill Evaluations (Evals) Framework for AI agent skills, including documentation, universal code quality rubrics, and skill-specific evaluation configurations (evals.json) for several skills. It also adds a new run-evals skill and a Dart unit test (skills_evals_test.dart) to ensure structural consistency across all evaluation files. Feedback on the unit test suggests simplifying the type promotion logic to improve code readability.

[mariamhas]

Request to update readme to make it really clear the use case for these "evals" are essentially like unit tests for the skills within dart-lint-skills package and not for the agent client plugins or skills within the overall repo that this package currently lives in. Also what someone who comes across this "run evals" skills might use it for i.e. primarily used to evaluate contributions of skills related to the development of flutter framework, dart language, flutter/dart plugin/packages.

[reidbaker]

Lets change this to mean the expectations for the for what was done and the end state of the repo after. This will stay a list of requirements but will focus on changes to the repo and tracked files. Feel free to come up with a better name.

[reidbaker]

I think the evals structure might need the ability to have a starting configuration state and to specify what that state is.

Think skills that warn on dirty git starting state, evaluated skills with particular classes of errors. Different repo formats etc.