Skip to content

Deployment/RunbookRun FailureEncountered is always false in the API for completed executions #10183

Description

@NickJosevski

Severity

Low impact, but it's a public API field that is silently and permanently wrong. Not blocking — a reliable workaround exists (read the linked server task) — but every API consumer that trusts the field gets a wrong answer with no error to alert them.

Version

Present in 2025.1 (where it was reported) and still present on main today. It predates the 2025 execution-completion refactors — the equivalent ChangeState(null) on completion is visible in the codebase from at least August 2024, so treat it as longstanding rather than a regression.

Latest Version

Not applicable — none of the dropdown options fit, so to be precise about what was and wasn't done: confirmed present on main by code inspection, not by running a deployment on an instance. The code path is unambiguous (traced below, and every branch of it is unconditional), but nobody has re-run the customer's scenario against a current build. Worth someone doing that as the first triage step if there's any doubt.

What happened?

DeploymentResource.FailureEncountered is always false for any completed deployment, whether it succeeded or failed. Same for RunbookRunResource.FailureEncountered.

Reported against the Go client at OctopusDeploy/go-octopusdeploy#295, but it isn't a client bug — the API response itself carries false, so the .NET client, Terraform provider and raw HTTP consumers all see it.

The field is derived from in-flight execution plan state rather than stored as a fact about the outcome:

  • source/Octopus.Core/Features/Deployments/Mapping/DeploymentExtensionMethods.cs:51FailureEncountered = source.FailureEncountered
  • source/Octopus.Core/Model/Projects/Deployment.cs:247FailureEncountered => Json.FailureEncountered
  • source/Octopus.Core/Model/Projects/Deployment.cs:479Json.FailureEncountered => StateObject != null && StateObject.FailureEncountered

and on completion the state is deliberately discarded:

  • source/Octopus.Core/ExecutionsV2/Completion/ExecutionCompleter.cs:54executionDocument.ClearState()
  • source/Octopus.Core/Model/Projects/Deployment.cs:291ClearState() sets StateObject = null

Cleanup() runs for every completed execution regardless of outcome (ServiceBasedDeploymentUpdatingStrategy.cs:52-57), so StateObject is always null by the time anyone reads the deployment through the API, and the expression collapses to false.

The field can therefore only ever read true while a deployment is in flight or paused — for instance one sitting in a guided-failure interruption. That is a legitimate internal use, and it is how the field behaves today. It just isn't what an API consumer reads it as.

Expected: either the field reflects the deployment's outcome, or it doesn't exist on the public resource.

Reproduction

Steps to confirm it (see the note under Latest Version — these were derived from the code path and the original report, not executed by me):

  1. Run a deployment that fails.
  2. GET /api/{space}/deployments/{id} (or GET /api/{space}/releases/{id}/deployments).
  3. Observe "FailureEncountered": false in the response despite the deployment having failed.
  4. GET /api/tasks/{deployment.TaskId} returns "State": "Failed" — the correct outcome is available, just not on the deployment resource.

Both the single-deployment and release-deployments endpoints go through the same mapper, so both are affected.

More Information

The good news on scope. The document-level property has exactly two consumers in the whole codebase — the deployment and runbook run resource mappers. Nothing in Server reads it to make a decision, and the Portal doesn't use it at all (it reads task state, which is why the UI shows the correct red/green while the API says otherwise). The in-flight DeploymentState.FailureEncountered that drives execution is a different object and would not be touched. So the mechanical change is small: persist a FailureEncountered flag on the deployment JSON at completion — the value is already in hand at ExecutionCompleter.cs:39 — and have it survive ClearState(). No schema migration, since it lives in the existing JSON column.

The decisions that actually need making, which is why I'd rather this get triaged than picked up straight away:

  1. History can't be backfilled. The plan state for every already-completed deployment is gone. Post-fix, the field would be correct for new deployments and false for every old one, with no way for a caller to tell which regime a given deployment came from. Uniformly-wrong is at least predictable; intermittently-correct may be worse for anyone writing a version-agnostic client.

  2. What should it mean, exactly? RecordFailure is one-way — it's never reset (ExecutionPlan.cs:245-256, and BaseExecutionPlanner.cs:884 early-returns if already set). It's called from StepFailed, and it abandons all Success-condition steps, so in practice it means "this execution is now going to end badly". Two edges need confirming before persisting it as an outcome:

    • Cancellation — a cancelled deployment appears to route through the same path (KnownExecutionError.CreateCancelledError), which would give FailureEncountered: true alongside task State: Canceled, not Failed.
    • Guided failure — needs checking whether a failure the user resolves with Ignore/Retry reaches StepFailed. If it does, a deployment that ultimately succeeds would carry FailureEncountered: true, and the field would mean "something went wrong along the way" rather than "this failed".

    Either way the field would not be a synonym for "task failed", and shipping it as though it were would trade one wrong answer for a subtler one.

  3. Is the field worth keeping at all? TaskResource.State already answers this properly, distinguishes Failed / Canceled / TimedOut, and is what our own UI uses. A case can be made that the honest fix is to document FailureEncountered as in-flight-only and mark it obsolete, rather than give it a second meaning.

My own lean is (3) or (1)+(2) resolved deliberately — but that's a product/API call, not something to decide in the PR.

Runbook runs have the identical shape (RunbookRun.cs:285, :334) and would want whatever treatment deployments get.

What "done" looks like, depending on which way decision 3 goes:

  • If the field is made truthful: a failed deployment and a succeeded deployment, read back through GET /api/{space}/deployments/{id} after completion, return true and false respectively; a cancelled deployment and a guided-failure-then-succeeded deployment return whatever decision 2 settles on, with that choice written down somewhere a client author will find it; the same holds for runbook runs; and the docs state plainly that deployments completed before the fix always return false.
  • If the field is deprecated instead: it's marked obsolete on DeploymentResource and RunbookRunResource with a pointer to task state, the API docs say it is in-flight-only, and go-octopusdeploy#295 gets closed with that explanation.

Either outcome should close the loop on go-octopusdeploy#295 — the reporter is currently working around it with task state.

Workaround

Read the outcome from the server task the deployment points at: deployment.TaskIdGET /api/tasks/{id}, then use State (Queued, Executing, Failed, Canceled, TimedOut, Success, Cancelling) or FinishedSuccessfully. This is what the Octopus web portal does, it works on all existing versions, and it works for historical deployments — which no Server-side fix can.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugThis issue represents a verified problem we are committed to solving

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions