Skip to content

Codex adapter keeps dead sessions after the provider process exits — thread shows "Working" forever #10798

Description

@stephenjason89

Version: 0.0.34-nightly (reproduced against upstream/main @ 061543e9e) · macOS · provider codex

When a Codex App Server process dies mid-turn, the adapter forwards the session.exited event but keeps the dead runtime in its live-session map. Every liveness check downstream still reports the thread as running, so the UI shows "Working" indefinitely and no recovery path fires.

I hit this on a Gauntlet Loop that sat at "Working" for over 7 hours with no process behind it:

specialist_reports_sol_xhigh     8h 40m   assistant message                     1.5M tok
specialist_pipeline_sol_xhigh    8h 40m   /bin/zsh -lc 'vendor/bin/pint ...'    1.7M tok
critic_selected_fixes_sol_xhigh  8h 25m   reasoning                             83.0k tok

3 working  22 idle  1 settled                                     Σ 118.6M tok

The last provider event was ~7 hours before I looked. No codex process matching the thread remained. The worktree changes and test results were intact — only the session state was wrong.

Mechanism

CodexSessionRuntime watches the child's exit code and emits session/exited when the process dies without close() being called (CodexSessionRuntime.ts:2231). The adapter's event consumer maps that to a canonical session.exited and offers it to the runtime event queue — and then does nothing else (CodexAdapter.ts:2437).

Nothing removes the entry from the adapter's sessions map. Only stopSessionInternal deletes, and it runs solely from an explicit stopSession/stopAll. So after the process dies:

  • hasSession(threadId) stays true — the entry exists and stopped is still false (CodexAdapter.ts:2692)
  • listSessions() still returns it, because it filters only on stopped (CodexAdapter.ts:2685)
  • the session scope is never closed, so the runtime's client, queues, and stderr fibers leak

Startup reconciliation trusts listSessions() to decide which persisted bindings are still live, and ProviderSessionReaper permanently skips any session holding an activeTurnId. A dead runtime retained in the map therefore evades both, and the thread stays running with its activeTurnId set forever.

Every other adapter already deletes on this path — ClaudeAdapter.ts:4143, OpenCodeAdapter.ts:963 (deleteContextIfCurrent), AntigravityAdapter.ts:437, CursorAdapter.ts:478, GrokAdapter.ts:936 — each guarded on map identity so a replacement session for the same thread is not clobbered. CodexAdapter is the only one missing it.

Relationship to #4713

Related but distinct. #4713 is about the interrupt path, where a live provider never produces a terminal event and Stop becomes a no-op. This one needs no interrupt at all: the provider is already dead and did emit its terminal event, and the adapter drops it on the floor. repparw's comment on #4713 describes the same provider-death shape for opencode; this is the codex adapter's version of it, with a concrete one-line cause.

The open PRs I checked (#9391, #7854, #8859) all operate on ingestion or projection state downstream. None of them remove the dead runtime from the adapter map, so listSessions() keeps advertising it regardless.

Reproduction

A regression test against the existing fake runtime fails on main:

  • start a Codex adapter session
  • emit session/exited from the runtime
  • assert the event is forwarded, then assert hasSession is false and listSessions() excludes it

On main the event is forwarded, hasSession returns true, and the scope is never released.

Suggested fix

Delete the session when a terminal session.exited is forwarded, guarded on scope identity so a replacement session started for the same thread is untouched, and release its scope so the runtime's resources are freed.


Diagnosed with GPT-5.6 Sol and Claude Opus 5 in T3 Code.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions