Skip to content

fix: close the debug server instead of spending the shutdown budget on it (#64 item 6) - #87

Merged
jason-shen merged 1 commit into
streamcoreai:mainfrom
vsolano9:fix/64-debug-shutdown
Nov 7, 2026
Merged

jason-shen merged 1 commit into
streamcoreai:mainfrom
vsolano9:fix/64-debug-shutdown

Conversation

@vsolano9

@vsolano9 vsolano9 commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Addresses item 6 of #64: the debug server no longer shares main's shutdown context.

Cause

debugSrv.Shutdown(shutdownCtx) reused the 5s context that srv.Shutdown had just been given. Shutdown waits for in-flight requests, and a pprof CPU profile or trace holds its request open for as many seconds as the caller asked. So one ?seconds=30 profile made every shutdown run that context out. The force-exit watchdog is armed for the same five seconds, so the end of shutdown is a race between main returning and os.Exit(1).

Change

The debug server is stopped with Close(), first, and outside the graceful budget (stopDebugServer in main.go). The issue suggested a context of its own or shutting the debug server down first. Neither stops Shutdown from waiting for the profile to finish, so both would still cost seconds the sessions and plugins need. A profile cut short by a server that is going away is worth nothing. Closing the connection cancels the request context, which ends pprof's wait and releases the CPU profiler.

Item 6 only. Items 5 and 8–10 are untouched.

Tests

  • TestStopDebugServerDoesNotWaitForAnInFlightProfile starts the real debug server and opens a 30s CPU profile. It waits until the handler actually holds the CPU profiler (StartCPUProfile fails), with no sleeps. Then it requires stopDebugServer to return in under 1s, the profile request to end, and the profiler to be released.
  • TestStopDebugServerAcceptsNil covers the bind = "" case, which main now passes straight through.

Red proof: with stopDebugServer changed back to Shutdown with a 5s context, the first test fails with stopDebugServer took 5.000370844s with a profile in flight; want under 1s.

Stability: go test -race -count=40 -run '^TestStopDebugServer' . and the same with GOMAXPROCS=1 -count=20: all pass.

End to end

I built the server from main and from this branch. Each ran with only [server] port and [debug] bind = "127.0.0.1:16060", with a ?seconds=30 profile in flight, and then got SIGINT. Two runs, shutdown log lines only:

run 1, main:         exit 0, 5.09s
  Shutting down...
  debug HTTP shutdown error: context deadline exceeded
  Server stopped
  Shutdown timed out, forcing exit
run 2, main:         exit 1, 5.05s
  Shutting down...
  Shutdown timed out, forcing exit
this branch, both:   exit 0, 0.47s and 0.00s
  Shutting down...
  Server stopped

So on main the outcome is a coin flip at the 5s mark. In run 2 the watchdog won, so the process exited 1 before it logged anything about the debug server.

Validation

The four CI commands (Linux, Go 1.27.3):

  • gofmt -l .: clean
  • go build ./...: clean
  • go vet ./...: clean
  • go test -race ./...: ok in every package with tests (internal/turn has none)

Prepared with AI assistance (Claude Code). I ran the checks above myself and the outputs are quoted as observed.

…n it

debugSrv.Shutdown reused the 5s context main had just given the public
server. An in-flight pprof CPU profile or trace holds its request for as
many seconds as asked, so shutdown ran that context out and ended in a
race with the 5s force-exit watchdog (exit 0 or 1). Close the debug
server first instead: a profile cut short by a stopping server is worth
nothing, and closing cancels the request so pprof releases the profiler.

Addresses item 6 of streamcoreai#64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jason-shen
jason-shen merged commit f16e44f into streamcoreai:main Nov 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants