Skip to content

feat(orchestrator): make the build budget a per-queue setting - #602

Merged
behinddwalls merged 1 commit into
mainfrom
preetam/build-budget
Aug 18, 2026
Merged

feat(orchestrator): make the build budget a per-queue setting#602
behinddwalls merged 1 commit into
mainfrom
preetam/build-budget

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

How many builds a queue may have occupying CI at once was a constant in the wiring, with a TODO beside it saying so:

// TODO: move this onto entity.QueueConfig so operators can tune it per queue
// without a code change.
const defaultBuildBudget = 4

Four is a reasonable default and a poor universal answer. It is the only rationing lever the allocator has, and it decides how much speculation a queue does at all: a queue allowed one build never hedges an outcome, and a queue with a large CI pool behind it has no way to say so. A deployment running a busy trunk queue beside a quiet one has to pick a number that suits neither, and changing it means editing Go and shipping a binary.

It is also the setting a reader of the demo asks about first, because it is the one that visibly changes what a run does, and it was the only such knob with no way to set it.

What?

profiles.yaml gains a speculator block, per queue and in defaults, with one field:

defaults:
  speculator: {buildBudget: 4}

queues:
  - name: demo-queue
    speculator: {buildBudget: 12}

It inherits and overrides exactly as the other extension blocks do — a queue that says nothing takes the default, and the default itself falls back to 4 when unstated, so every existing configuration and the built-in topology behave as they did.

The block has no type. There is one speculator, composed from the queue's scorer, and what varies between queues is what it is allowed to spend — but the block is where an allocator choice would go if a second one ever exists, which a bare buildBudget: at queue level would not be.

The TODO proposed entity.QueueConfig instead. That is the gateway's record of which queues exist; the budget is speculation policy, which is what profiles already carry per queue, and it is resolved a few lines from the scorer it shares a speculator with. QueueConfig is left holding just the queue name.

A negative budget is rejected at startup rather than clamped. Sticky computes free slots as budget - funded, so a negative one yields no free slots ever: the queue would batch and then never build, which reads as a stuck queue rather than a misconfigured one. Absent or 0 takes the default — those are the same value in YAML and cannot be told apart, so the harmless reading wins.

The number is logged alongside the other resolved defaults, since a queue building less than expected is otherwise a silent condition.

Test Plan

  • ✅ a test that drives a real speculator per queue and counts what it proposes — eight dependency-free speculating batches against budgets of 5, 2 (inherited) and 2 (unlisted queue), asserting the proposals stop at the budget. Parsing a number proves nothing if it never reaches the allocator, so the assertion is on behaviour rather than on the parsed config
  • ✅ mutation-tested that assertion: reverting withSpeculator to the old constant fails all three cases, so it is not passing by construction
  • ✅ a negative budget fails loadProfilesConfig; an unstated one resolves to 4 while a stated one survives normalization
  • make test, make lint, make gazelle, make check-tidy
  • ✅ against a live stack, which is what proves the mounted file is read rather than just parsed in a test: buildBudget: -1 fails the orchestrator at boot with defaults: build budget -1 is negative, and buildBudget: 12 starts, logs default_build_budget: 12, and lands a six-change run whose deepest request records speculating [building ×12, built ×12] — the raised budget being spent

Sticky's own budget arithmetic is unchanged and already covered; what is new here is only where the number comes from.

@behinddwalls
behinddwalls marked this pull request as ready for review August 16, 2026 18:47
@behinddwalls
behinddwalls requested review from a team and sbalabanov as code owners August 16, 2026 18:47
Base automatically changed from preetam/watch-tui to main August 18, 2026 16:55
## Summary

### Why?

How many builds a queue may have occupying CI at once was a constant in the wiring, with a `TODO` beside it saying so:

```go
// TODO: move this onto entity.QueueConfig so operators can tune it per queue
// without a code change.
const defaultBuildBudget = 4
```

Four is a reasonable default and a poor universal answer. It is the only rationing lever the allocator has, and it decides how much speculation a queue does at all: a queue allowed one build never hedges an outcome, and a queue with a large CI pool behind it has no way to say so. A deployment running a busy trunk queue beside a quiet one has to pick a number that suits neither, and changing it means editing Go and shipping a binary.

It is also the setting a reader of the demo asks about first, because it is the one that visibly changes what a run does, and it was the only such knob with no way to set it.

### What?

`profiles.yaml` gains a `speculator` block, per queue and in `defaults`, with one field:

```yaml
defaults:
  speculator: {buildBudget: 4}

queues:
  - name: demo-queue
    speculator: {buildBudget: 12}
```

It inherits and overrides exactly as the other extension blocks do — a queue that says nothing takes the default, and the default itself falls back to 4 when unstated, so every existing configuration and the built-in topology behave as they did.

The block has no `type`. There is one speculator, composed from the queue's scorer, and what varies between queues is what it is allowed to spend — but the block is where an allocator choice would go if a second one ever exists, which a bare `buildBudget:` at queue level would not be.

The `TODO` proposed `entity.QueueConfig` instead. That is the gateway's record of which queues exist; the budget is speculation policy, which is what profiles already carry per queue, and it is resolved a few lines from the scorer it shares a speculator with. `QueueConfig` is left holding just the queue name.

**A negative budget is rejected at startup** rather than clamped. Sticky computes free slots as `budget - funded`, so a negative one yields no free slots ever: the queue would batch and then never build, which reads as a stuck queue rather than a misconfigured one. Absent or `0` takes the default — those are the same value in YAML and cannot be told apart, so the harmless reading wins.

The number is logged alongside the other resolved defaults, since a queue building less than expected is otherwise a silent condition.

## Test Plan

- ✅ a test that drives a real speculator per queue and counts what it proposes — eight dependency-free speculating batches against budgets of 5, 2 (inherited) and 2 (unlisted queue), asserting the proposals stop at the budget. Parsing a number proves nothing if it never reaches the allocator, so the assertion is on behaviour rather than on the parsed config
- ✅ mutation-tested that assertion: reverting `withSpeculator` to the old constant fails all three cases, so it is not passing by construction
- ✅ a negative budget fails `loadProfilesConfig`; an unstated one resolves to 4 while a stated one survives normalization
- ✅ `make test`, `make lint`, `make gazelle`, `make check-tidy`
- ✅ against a live stack, which is what proves the mounted file is read rather than just parsed in a test: `buildBudget: -1` fails the orchestrator at boot with `defaults: build budget -1 is negative`, and `buildBudget: 12` starts, logs `default_build_budget: 12`, and lands a six-change run whose deepest request records `speculating [building ×12, built ×12]` — the raised budget being spent

Sticky's own budget arithmetic is unchanged and already covered; what is new here is only where the number comes from.
@behinddwalls
behinddwalls added this pull request to the merge queue Aug 18, 2026
Merged via the queue into main with commit 86635af Aug 18, 2026
15 checks passed
@behinddwalls
behinddwalls deployed to stack-rebase August 18, 2026 17:15 — with GitHub Actions Active
@behinddwalls
behinddwalls deleted the preetam/build-budget branch August 18, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants