Skip to content

Conversation

@un-def
Copy link
Collaborator

@un-def un-def commented Nov 14, 2025

SSHTunnel.open() starts the ssh process with -f option, which causes it to become a daemon (it calls daemon(3) after authentication).

Then, SSHTunnel.close() uses ssh -O exit to ask that daemon process to exit (the processes communicate via the control socket, -S option).

The daemon exits, and then PID 1 should reap it.

Before #3165, PID 1 was bash, which handles SIGCHLD, properly wait(2)ing children, but in that pull request exec was added, making dstack server PID 1. dstack does nothing with SIGCHLD (neither handles nor explicitly ignores it), thus the default disposition (ignore the signal but do not discard children) leads to an evergrowing number of unreaped zombies.

Fixes: #3291

SSHTunnel.open() starts the ssh process with -f option, which
causes it to become a daemon (it calls daemon(3) after authentication).

Then, SSHTunnel.close() uses `ssh -O exit` to ask that daemon process
to exit (the processes communicate via the control socket, -S option).

The daemon exits, and then PID 1 should reap it.

Before #3165, PID 1 was bash,
which handles SIGCHLD, properly wait(2)ing children, but in that
pull request exec was added, making `dstack server` PID 1. dstack
does nothing with SIGCHLD (neither handles nor explicitly ignores
it), thus the default disposition (ignore the signal but do not discard
children) leads to an evergrowing number of unreaped zombies.

Fixes: #3291
@un-def un-def requested a review from r4victor November 14, 2025 15:07
@un-def un-def merged commit ead324c into master Nov 17, 2025
28 checks passed
@un-def un-def deleted the issue_3165_fix_docker_entrypoint_pid_1 branch November 17, 2025 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: BlockingIOError crashes dstack server due to SSHTunnel resource leaks

3 participants