Summary
For many create nouns, the --set key=value flag rejects every field with unknown or read-only field "<field>", even though --help advertises those exact fields. --list-fields confirms the cause: these nouns return "No mutable fields defined for this command" — the mutable-field registry is empty, so no --set value is ever accepted. There is also no -f/--file fallback, leaving these nouns with no working create path via the CLI at all.
This appears to be the same root cause as #27 (connector) and #23 (PR), but the scope is much broader — it affects all AI-Evals nouns plus several core ones.
Version
harness version 0.1.10 (2026-06-18T04:38:52Z)
Reproduction
$ harness create eval_target --org default --project <proj> \
--set name=repro --set identifier=repro --set type=prompt
unknown or read-only field "type"; use --list-fields to see mutable fields
$ harness create eval_target --list-fields
No mutable fields defined for this command.
--help for the same command advertises the fields it then rejects:
$ harness create eval_target --help
Create an AI Evals target: harness create eval_target --set name=... identifier=... type=prompt
Scope — --list-fields audit
Working (registry populated):
| Noun |
--list-fields |
secret |
ID / Label / Type ✅ |
project |
ID / Label / Type ✅ |
organization |
ID / Label / Type ✅ |
Broken (empty registry → --set unusable, no -f fallback):
| Noun |
--list-fields |
eval_target |
No mutable fields ❌ |
eval_dataset |
No mutable fields ❌ |
eval_metric |
No mutable fields ❌ |
eval_model |
No mutable fields ❌ |
eval_metric_set |
No mutable fields ❌ |
evaluation |
No mutable fields ❌ |
pr |
No mutable fields ❌ (see #23) |
connector |
No mutable fields ❌ (see #27) |
environment |
No mutable fields ❌ |
service |
No mutable fields ❌ |
repository |
No mutable fields ❌ |
No workaround
The broken nouns also reject -f:
$ harness create eval_target -f /dev/null
unknown shorthand flag: 'f' in -f
So affected resources cannot be created through the CLI by any means; callers must drop to the REST API.
Expected
Either:
- Populate the mutable-field registry for these nouns so the
--set fields shown in --help actually work (and support nested keys like git_source.file_path for git-backed eval resources), or
- Add a
-f/--file <yaml|json> body flag as a general create path for complex/nested resources.
Impact
Standing up an AI-Evals stack (dataset → target → metric set → evaluation) or any git-backed eval is impossible via the CLI today — every component create fails. Same for code pr, connector, environment, service, and repository create.
Summary
For many
createnouns, the--set key=valueflag rejects every field withunknown or read-only field "<field>", even though--helpadvertises those exact fields.--list-fieldsconfirms the cause: these nouns return "No mutable fields defined for this command" — the mutable-field registry is empty, so no--setvalue is ever accepted. There is also no-f/--filefallback, leaving these nouns with no working create path via the CLI at all.This appears to be the same root cause as #27 (connector) and #23 (PR), but the scope is much broader — it affects all AI-Evals nouns plus several core ones.
Version
Reproduction
--helpfor the same command advertises the fields it then rejects:Scope —
--list-fieldsauditWorking (registry populated):
--list-fieldssecretprojectorganizationBroken (empty registry →
--setunusable, no-ffallback):--list-fieldseval_targeteval_dataseteval_metriceval_modeleval_metric_setevaluationprconnectorenvironmentservicerepositoryNo workaround
The broken nouns also reject
-f:So affected resources cannot be created through the CLI by any means; callers must drop to the REST API.
Expected
Either:
--setfields shown in--helpactually work (and support nested keys likegit_source.file_pathfor git-backed eval resources), or-f/--file <yaml|json>body flag as a general create path for complex/nested resources.Impact
Standing up an AI-Evals stack (dataset → target → metric set → evaluation) or any git-backed eval is impossible via the CLI today — every component create fails. Same for code
pr,connector,environment,service, andrepositorycreate.