Skip to content

[Bug]: Git status poller can fill the disk with orphaned tmp_pack_* files when the upstream fetch exceeds its 5s timeout #4296

Description

@exbaitman

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. Open, in the git panel, a repo whose current branch tracks an upstream that requires a non-trivial pack transfer to fetch (large repo, slow link, or a busy CPU — anything that pushes the upstream fetch past ~5s).
  2. Leave the panel open so GitStatusBroadcaster keeps polling.
  3. Watch .git/objects/pack/:
watch -n2 'ls .git/objects/pack/tmp_pack_* 2>/dev/null | wc -l; du -sh .git'
  1. Observe tmp_pack_* files accumulate (one per failed poll), each the full pack size, never cleaned up.

Expected behavior

Expected: a status refresh is bounded and idempotent — it should never leave orphaned temp files, and a slow/failed refresh must not cause repeated full re-downloads or unbounded disk growth.

Actual behavior

Actual: every poll that exceeds the timeout leaks a full-size tmp_pack_* and makes zero forward progress, so the leak repeats indefinitely until the disk is full.

Impact

Major degradation or frequent failure

Version or commit

| Commit | 28e481e (main, 2026-04-07) |

Environment

| OS | Ubuntu (Linux 6.8) | on a DO Droplet

Logs or stack traces

## Root cause

The per-repo poller (`GitStatusBroadcaster`, `GIT_STATUS_REFRESH_INTERVAL = 30s`) calls `refreshRemoteStatus`, which reaches `fetchUpstreamRefForStatus` in `apps/server/src/git/Layers/GitCore.ts`:


// apps/server/src/git/Layers/GitCore.ts:50-52
const STATUS_UPSTREAM_REFRESH_INTERVAL = Duration.seconds(15);
const STATUS_UPSTREAM_REFRESH_TIMEOUT  = Duration.seconds(5);   // <-- hard 5s cap
const STATUS_UPSTREAM_REFRESH_FAILURE_COOLDOWN = Duration.seconds(5);



// apps/server/src/git/Layers/GitCore.ts:893-909
const refspec = `+refs/heads/${upstream.upstreamBranch}:refs/remotes/${upstream.upstreamRef}`;
return executeGit(
  "GitCore.fetchUpstreamRefForStatus",
  fetchCwd,
  ["--git-dir", gitCommonDir, "fetch", "--quiet", "--no-tags", upstream.remoteName, refspec],
  { allowNonZeroExit: true, timeoutMs: Duration.toMillis(STATUS_UPSTREAM_REFRESH_TIMEOUT) },
);


`executeGit` enforces the timeout by interrupting a scoped effect, which terminates the spawned git child:


// apps/server/src/git/Layers/GitCore.ts:722-739
return yield* runGitCommand().pipe(
  Effect.scoped,
  Effect.timeoutOption(timeoutMs),           // interrupt -> scope closes -> git child killed
  Effect.flatMap((result) => Option.match(result, {
    onNone: () => Effect.fail(new GitCommandError({ /* "... timed out." */ })),
    onSome: Effect.succeed,
  })),
);


The failure chain:

1. The status fetch needs > 5s to transfer its pack (measured **4.35s wall / 4.4s user** on my repo — right on the edge; any CPU/network pressure tips it over).
2. On timeout, the git child is killed. A killed `git fetch` leaves its in-progress `.git/objects/pack/tmp_pack_*` behind and **does not update the ref**.
3. Because the ref never advances, the negotiation on the next poll is identical, so git transfers the **whole pack again** → killed again → another orphan.
4. Nothing ever removes `tmp_pack_*`. `git gc`/`git maintenance` don't run here and wouldn't be triggered on this path anyway.
5. Repeat every poll cycle → unbounded `.git` growth → disk full.

The core design problem: **the status path uses a real object-transferring `git fetch` on a timeout short enough that it can never complete, so it makes no progress while leaking a full pack each time.**

## Proposed fixes (roughly in priority order)

1. **Don't transfer objects just to compute status.** Ahead/behind can be detected with `git ls-remote <remote> <branch>` (remote tip OID vs. local) with no pack download at all. Only fetch objects when the tips actually differ, and do that as a separate, single-flighted, generously-timed background job — not on the hot poll path. This eliminates the failure mode entirely.
2. **Clean up on failure.** Whenever a status fetch times out or errors, remove any `tmp_pack_*` it created (or run it so git cleans up), so a slow refresh can never leak. This is the minimal safety fix even if #1 isn't taken.
3. **Make the timeout survivable / adaptive.** 5s is too low for a first fetch on a large repo. Raise it substantially (e.g. 30–60s), and/or back off on repeated failures instead of retrying at full cost every cycle. Ensure only one status fetch per repo runs at a time (single-flight) so retries can't pile up concurrently.
4. **Consider `--filter=blob:none`** for the status fetch to bound transfer size when objects genuinely must be fetched.

Any one of #1 or #2 stops the disk-fill; #1 is the durable architectural fix.

Screenshots, recordings, or supporting files

No response

Workaround

Delete the orphans and complete the fetch once so the poller becomes incremental:

find /path/to/repo/.git/objects/pack -name 'tmp_pack_*' -delete
git -C /path/to/repo fetch origin        # untimed; advances refs so future polls are cheap

As a host-side safety net I added a systemd timer that removes tmp_pack_* older than 30 minutes every 15 minutes — but that's papering over the leak; the fix belongs in the status-fetch path above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is broken or behaving incorrectly.needs-triageIssue needs maintainer review and initial categorization.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions