Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 51 additions & 10 deletions docs/CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,11 +183,13 @@ Map-form entries accept three optional keys. String-form entries do not: the
shape selects the capabilities, so an entry needing an override is written as a
map.

| Key | Type | Effect |
| -------- | ------ | ----------------------------------------------------------------------------- |
| `env` | map | Layers over the group's `env:` for this command only. |
| `dir` | string | Runs this command in another directory, without needing a shell. |
| `silent` | bool | Suppresses live streaming; the output is still captured. |
| Key | Type | Effect |
| ------------------- | ------ | ----------------------------------------------------------------- |
| `env` | map | Layers over the group's `env:` for this command only. |
| `dir` | string | Runs this command in another directory, without needing a shell. |
| `silent` | bool | Suppresses live streaming; the output is still captured. |
| `continue-on-error` | bool | This command's failure is tolerated; the sequence continues. |
| `always` | bool | Runs even when an earlier command already failed. |

```yaml
groups:
Expand Down Expand Up @@ -223,11 +225,50 @@ commands:
SHA: '{{ output "build" }}'
```

`env` and `dir` fold into the cache fingerprint, so changing either re-runs the
group. `silent` does not: it changes nothing about what the command does or
produces, so toggling it keeps a valid cached result. A group that uses none of
these keys keeps the fingerprint it had before they existed — upgrading does not
invalidate existing caches.
### Failure policy

By default a sequence stops at the first non-zero exit, like `set -e`. Two
independent keys bend that, and they compose:

```yaml
groups:
- name: test-with-teardown
commands:
- ./setup.sh # starts a database
- { command: ./optional-lint.sh, continue-on-error: true }
- { command: go, params: [test, "./..."] }
- { command: ./teardown.sh, always: true } # runs even if tests fail
```

`continue-on-error` says this command's outcome does not matter: it is logged as
a warning, the sequence continues, and the group still succeeds with `exitCode`
0. Use it for best-effort steps.

`always` guarantees the command runs even after an earlier one failed. It does
**not** forgive that failure — the group still fails, reporting the original
error. This is how teardown works. When an `always` command fails after an
earlier failure, the first error is reported and the second is logged, so the
diagnostic keeps pointing at the real cause. Combine both keys when a cleanup
step must run and its own failure should also be ignored.

Entries between the failure and an `always` entry are skipped, not run.

`always` also survives cancellation: when the group hits its `timeout:` or you
interrupt the run, pending `always` commands still execute, detached from the
dead deadline and bounded by a 30-second grace period. A hung test that trips a
timeout still gets its teardown. Their own failures are logged rather than
reported, since the cancellation is the real cause.

One limit worth knowing: **`retries:` replays the whole sequence**, so an
`always` teardown runs once per attempt. Keep it idempotent.

### Caching interaction

`env`, `dir`, `continue-on-error`, and `always` all fold into the cache
fingerprint, so changing any of them re-runs the group. `silent` does not: it
changes nothing about what the command does or produces, so toggling it keeps a
valid cached result. A group that uses none of these keys keeps the fingerprint
it had before they existed — upgrading does not invalidate existing caches.

### Referencing other groups' output

Expand Down
12 changes: 10 additions & 2 deletions internal/cache/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ type Store interface {
}

// Compute returns a content fingerprint for the given cache spec. The
// fingerprint changes when the method, any command/param/form/env/dir in the
// group's command list, or any matched input file changes. For shell-form entries the
// fingerprint changes when the method, anything about a command in the group's
// list except silent, or any matched input file changes. For shell-form entries the
// fingerprint also changes when the shell program changes. A glob that matches
// nothing contributes nothing, so adding the first matching file naturally
// changes the fingerprint.
Expand Down Expand Up @@ -67,6 +67,14 @@ func Compute(spec *config.Cache, shell string, commands []config.CommandSpec) (s
for _, k := range slices.Sorted(maps.Keys(c.Env)) {
fmt.Fprintf(h, "%s\x04%s\x03", k, c.Env[k])
}
// Distinct markers: the two policies change whether the group
// succeeds, so they must never hash alike.
if c.ContinueOnError {
fmt.Fprintf(h, "coe\x03")
}
if c.Always {
fmt.Fprintf(h, "alw\x03")
}
fmt.Fprintf(h, "\x02")
}

Expand Down
23 changes: 23 additions & 0 deletions internal/cache/percommand_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,21 @@ func TestCompute_PerCommandKnobs(t *testing.T) {
spec: config.CommandSpec{Command: "go", Params: []string{"build"}, Silent: true},
wantBust: false,
},
{
name: "continue-on-error busts",
spec: config.CommandSpec{Command: "go", Params: []string{"build"}, ContinueOnError: true},
wantBust: true,
},
{
name: "always busts",
spec: config.CommandSpec{Command: "go", Params: []string{"build"}, Always: true},
wantBust: true,
},
{
name: "the two policy keys are distinguishable",
spec: config.CommandSpec{Command: "go", Params: []string{"build"}, Always: true, ContinueOnError: true},
wantBust: true,
},
{
name: "empty env map is the same as no env",
spec: config.CommandSpec{Command: "go", Params: []string{"build"}, Env: map[string]string{}},
Expand Down Expand Up @@ -82,6 +97,14 @@ func TestCompute_EnvOrderIsDeterministic(t *testing.T) {
}
}

func TestCompute_PolicyKeysAreDistinct(t *testing.T) {
dir := t.TempDir()
writeFile(t, filepath.Join(dir, "a.go"), "package main\n")
soft := config.CommandSpec{Command: "go", ContinueOnError: true}
always := config.CommandSpec{Command: "go", Always: true}
assert.NotEqual(t, fpFor(t, dir, soft), fpFor(t, dir, always))
}

// Two different env maps must not hash the same just because their
// concatenated bytes could line up.
func TestCompute_EnvPairsAreUnambiguous(t *testing.T) {
Expand Down
45 changes: 45 additions & 0 deletions internal/config/commands_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,31 @@ func TestCommandSpec_UnmarshalYAML_PerCommandKnobs(t *testing.T) {
Env: map[string]string{"A": "1", "B": "2"},
},
},
{
name: "per-command continue-on-error",
yaml: `{command: ./lint.sh, continue-on-error: true}`,
want: CommandSpec{Command: "./lint.sh", ContinueOnError: true},
},
{
name: "per-command always",
yaml: `{command: ./teardown.sh, always: true}`,
want: CommandSpec{Command: "./teardown.sh", Always: true},
},
{
name: "failure policy keys compose",
yaml: `{command: ./cleanup.sh, always: true, continue-on-error: true}`,
want: CommandSpec{Command: "./cleanup.sh", Always: true, ContinueOnError: true},
},
{
name: "continue-on-error must be a boolean",
yaml: `{command: go, continue-on-error: sometimes}`,
wantErr: `"continue-on-error" must be a boolean`,
},
{
name: "always must be a boolean",
yaml: `{command: go, always: yep}`,
wantErr: `"always" must be a boolean`,
},
{
name: "empty dir rejected",
yaml: `{command: go, dir: ""}`,
Expand Down Expand Up @@ -514,3 +539,23 @@ func TestLoadConfig_PerCommandKnobsFixture(t *testing.T) {
require.NoError(t, err)
assert.Contains(t, refs, "single", "an env template must register as a dependency")
}

func TestLoadConfig_FailurePolicyFixture(t *testing.T) {
cfg, err := LoadConfig("./test-resources/config-commands-valid.yml")
require.NoError(t, err)

policy := cfg.GroupByName("policy")
require.NotNil(t, policy)
list := policy.CommandList()
require.Len(t, list, 4)

assert.True(t, list[0].ContinueOnError)
assert.False(t, list[0].Always)
assert.True(t, list[2].Always)
assert.False(t, list[2].ContinueOnError)
assert.True(t, list[3].Always, "the two keys compose on one entry")
assert.True(t, list[3].ContinueOnError)

assert.False(t, list[1].Always, "an entry declaring neither stays strict")
assert.False(t, list[1].ContinueOnError)
}
19 changes: 16 additions & 3 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,10 @@ type CommandSpec struct {
Env map[string]string `yaml:"env,omitempty" json:"env,omitempty"`
Dir string `yaml:"dir,omitempty" json:"dir,omitempty"`
Silent bool `yaml:"silent,omitempty" json:"silent,omitempty"`
// Independent and composable: Always guarantees execution, ContinueOnError
// forgives the outcome.
ContinueOnError bool `yaml:"continue-on-error,omitempty" json:"continueOnError,omitempty"`
Always bool `yaml:"always,omitempty" json:"always,omitempty"`
// IsShell records that the entry was written in string form and therefore
// runs through the group's shell:. Argv-form entries always exec directly
// and ignore shell:, even when it is set.
Expand Down Expand Up @@ -288,15 +292,24 @@ func (cs *CommandSpec) unmarshalKey(key string, val *yaml.Node) error {
}
cs.Dir = val.Value
case "silent":
if err := val.Decode(&cs.Silent); err != nil {
return fmt.Errorf(`commands entry: "silent" must be a boolean: %w`, err)
}
return decodeBool(val, key, &cs.Silent)
case "continue-on-error":
return decodeBool(val, key, &cs.ContinueOnError)
case "always":
return decodeBool(val, key, &cs.Always)
default:
return fmt.Errorf("commands entry: unexpected key %q (use a string or a {command, params} map)", key)
}
return nil
}

func decodeBool(val *yaml.Node, key string, dst *bool) error {
if err := val.Decode(dst); err != nil {
return fmt.Errorf("commands entry: %q must be a boolean: %w", key, err)
}
return nil
}

// NewConfig parses YAML bytes into a Config and validates the schema.
func NewConfig(b []byte) (*Config, error) {
var cfg Config
Expand Down
10 changes: 10 additions & 0 deletions internal/config/test-resources/config-commands-valid.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,22 @@ groups:
env:
SHA: '{{ output "single" }}'

# failure policy — a tolerated failure, then a teardown that always runs
- name: policy
description: "Per-command failure policy"
commands:
- { command: "false", continue-on-error: true }
- { command: echo, params: [work] }
- { command: echo, params: [teardown], always: true }
- { command: "false", always: true, continue-on-error: true }

flows:
ci:
description: "Multi then single"
steps:
- run: [multi]
- run: [single]
- run: [knobs]
- run: [policy]

default: ci
8 changes: 8 additions & 0 deletions internal/engine/commands_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,14 @@ func TestEngine_CommandsFixture_EndToEnd(t *testing.T) {
single, ok := e.Outputs().Get("single")
require.True(t, ok)
assert.Equal(t, "single-step\n", single.Output)

// Two entries exit non-zero; both are tolerated, so the group succeeds and
// publishes only the surviving commands' output.
policy, ok := e.Outputs().Get("policy")
require.True(t, ok)
assert.Equal(t, "work\nteardown\n", policy.Output)
assert.Equal(t, 0, policy.ExitCode)
assert.Equal(t, result.StatusOK, policy.Status)
}

func TestEngine_ErrorDecoration_SingularVsMulti(t *testing.T) {
Expand Down
Loading