Repository navigation
fix(queue): bound pending depth and make dedup constant time - #594
Open
walsykerem wants to merge 2 commits into
Open
walsykerem wants to merge 2 commits into
walsykerem wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 18af2f9. Configure here.
A full pending queue used to leave the image and push managers in a bad state. Refusing a pull deadlocked the image manager, a refused recovery leaked a push reservation, and a refused successor left the running pull without a handle to build with. - images: split releaseInflightPull and updateStatusByDigest into ...Locked bodies plus locking wrappers, and use the locked variants in the rejection path where createMu is already held. The record still ends failed with the queue error, credentials and timer cleared. - images: registerInflightPull now returns the handle it replaced, and a refused successor restores it with its credential expiry re-armed, so the pull already running keeps going and no expired handle stays current. - imagepush: a recovery refused by the full queue drops its inflight entry before failing the record, so a later push for the same digest and target can run and garbage collection stops treating it as live. Tests fill the queue through lib/queue directly and cover the refused create, the refused successor and the refused recovery.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

The in-memory job queue behind image pulls and pushes had no limit on waiting jobs. A caller could hand it 20,000 distinct keys; each one appended a pending entry, and duplicate detection walked the whole pending slice from the start. Memory and enqueue time therefore grew with the backlog: 20,001 pending entries, about 1.7 MB of heap, 20.6 s to fill the first batch and 56.3 s for a second batch at that depth, all under the race detector.
The queue now stops accepting work at 512 pending entries and reports a refused job with position -1. Keys live in a map, so duplicate detection stays constant time however deep the queue gets. The pull and push managers check that answer: a refused push drops its record and reservation and returns an error; a refused recovered job is marked failed with an explanation instead of waiting forever. No call blocks, and no job that was accepted is dropped.
After the change: 513 tracked jobs, and 0.36 s and 0.38 s for the two 20,000 call batches.
Note
Medium Risk
Touches core queueing, recovery, and borrowed-credential bookkeeping for image pulls and pushes; behavior changes when backlog hits 512, but refusals are explicit and terminal rather than silent growth.
Overview
Caps the shared in-memory job queue at 512 pending entries:
Enqueuenow returns -1 withqueue.ErrFull, and duplicate detection uses apendingIndexmap instead of scanning the pending slice.Image pull and push managers handle refusals instead of growing unbounded backlogs. Refused
CreatePushcalls drop the inflight reservation and persisted record so retries start clean; recovery paths mark jobs failed with an explanatory error when the queue cannot accept them. Image creates that are rejected before enqueue restore the previous inflight pull (so an already-running build keeps borrowed credentials and expiry), fail the new pending record, and returnErrFullwithout blocking other operations.Adds tests for the pending limit, full-queue create/recovery behavior, and successor refusal restoring prior inflight state.
Reviewed by Cursor Bugbot for commit a93ff27. Bugbot is set up for automated code reviews on this repo. Configure here.