Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions sites/docs/src/content/ai/evals.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
title: AI Evaluations
sidenav: ai
description: >
Learn about Dart and Flutter's evaluation frameworks for measuring AI tooling
reliability.
---

:::experimental
Evaluation tooling and benchmarks are experimental and likely to change.
:::

To explore the evaluation strategy,
view the open-source dataset and scoring rubrics,
or get involved with community benchmark datasets,
visit the [Flutter Evals repository](https://github.com/flutter/evals).
Comment thread
lamek marked this conversation as resolved.

Evaluating the capabilities and reliability of AI agents requires testing
approaches that model actual developer tasks.
Because LLMs are non-deterministic,
standard unit testing is insufficient for verifying agentic behaviors like
codebase navigation, plan execution, and code synthesis.

To build developer confidence in AI tooling,
Dart and Flutter use an evaluation system ("evals")
to test critical user journeys (CUJs).
Evals measure both deterministic code correctness
(compilation, lints, automated tests) and qualitative performance
(reasoning, safety, and conciseness) using automated model judges
and expert human grading.
2 changes: 2 additions & 0 deletions sites/docs/src/data/sidenav/ai.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@
permalink: /ai/gemini-cli-extension
- title: Developer experience
permalink: /ai/best-practices/developer-experience
- title: "AI Evaluations (experimental)"
permalink: /ai/evals

- title: Build AI-powered apps
expanded: true
Expand Down
Loading