Skip to content

Improve logging when Hangfire operation canceled#874

Merged
Enkidu93 merged 1 commit intomainfrom
log_operation_canceled
Feb 17, 2026
Merged

Improve logging when Hangfire operation canceled#874
Enkidu93 merged 1 commit intomainfrom
log_operation_canceled

Conversation

@pmachapman
Copy link
Collaborator

@pmachapman pmachapman commented Feb 15, 2026

I had a really weird issue on ext-QA where my jobs were getting marked as Pending/Restarting:

{
  "_id": {
    "$oid": "69922109390b03b7aebfaa94"
  },
  "revision": 2,
  "engineRef": {
    "$oid": "67846017466a1867b5a21cd3"
  },
  "owner": "E4i07fEZraBMVQdCbDKR949GaMf4V8Jk@clients",
  "step": 0,
  "state": "Pending",
  "options": {
    "tags": "sf-dev"
  },
  "deploymentVersion": "1.15.0-rc.3",
  "executionData": {},
  "dateCreated": {
    "$date": "2026-02-15T19:39:53.463Z"
  },
  "message": "Restarting",
  "progress": 0
}

And it did not restart. I could not recreate it locally, nor diagnose the error on QA, apart from determine that it was coming from an OperationCanceled exception thrown in DoWork. Worse, it was only happening for some builds - a specific build from my local instance of Scripture Forge (changing translation engines did not help).

Redeploying the docker pod fixed the build, but I am still unable to determine the cause (it happened again later in the day).

To help diagnose the cause of any errors like this in future, this PR improves logging when this scenario occurs, so if it happens again, we will at least have a bit more info to go on.


This change is Reviewable

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.96%. Comparing base (ea081c6) to head (af27adc).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #874   +/-   ##
=======================================
  Coverage   66.96%   66.96%           
=======================================
  Files         384      384           
  Lines       20886    20889    +3     
  Branches     2700     2700           
=======================================
+ Hits        13986    13989    +3     
  Misses       5936     5936           
  Partials      964      964           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Collaborator

@Enkidu93 Enkidu93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@Enkidu93 reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ddaspit).

Copy link
Contributor

@ddaspit ddaspit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@ddaspit reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on pmachapman).

@Enkidu93 Enkidu93 merged commit 50ee6fa into main Feb 17, 2026
2 checks passed
@Enkidu93 Enkidu93 deleted the log_operation_canceled branch February 17, 2026 16:10
@Enkidu93
Copy link
Collaborator

We are seeing this same error in Mike's automatic testing. I'll go ahead and deploy this change. Hopefully, we'll then be able to track down the cause before a production release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments