Skip to content

fix(windows platform): report SERVICE_STOPPED to SCM when startup fails#25810

Open
klondikedragon wants to merge 2 commits into
vectordotdev:masterfrom
itlightning:itl/windows-service-stopped-on-failed-start
Open

fix(windows platform): report SERVICE_STOPPED to SCM when startup fails#25810
klondikedragon wants to merge 2 commits into
vectordotdev:masterfrom
itlightning:itl/windows-service-stopped-on-failed-start

Conversation

@klondikedragon

Copy link
Copy Markdown
Contributor

Summary

Fixes #25809.

When Vector runs as a Windows service and startup fails (for example, an invalid configuration), run_service returned without registering a service control handler or reporting any status to the Service Control Manager. The SCM left the service stuck in START_PENDING indefinitely: the vector.exe process stays alive, the service is NOT_STOPPABLE (so sc stop cannot clear it), no SCM error event is logged, and configured recovery actions never run. Clearing the state required disabling the service and terminating the process by hand.

This PR handles the Err arm of Application::prepare_start in run_service: register a minimal control handler and report SERVICE_STOPPED with the startup exit code as a service-specific exit code. The SCM then shows the failure (WIN32_EXIT_CODE 1066, SERVICE_EXIT_CODE = Vector's exit code, e.g. 78 for a config error), logs event 7024, and service recovery can act on it. The success path (report Running, run, report Stopped) is unchanged.

Vector configuration

The bug reproduces with any configuration that fails to load; tested by appending a syntactically invalid line to a previously working config:

# ...any valid config...
broken: [

How did you test this PR?

Windows 11 x64, Vector registered as a Windows service (vector), config made invalid as above.

Before (stock v0.56.0 release binary): stuck forever, process alive, no events.

> sc.exe start vector ; sc.exe query vector
SERVICE_NAME: vector
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        WAIT_HINT          : 0x7d0
# state identical minutes later; vector.exe still running; cleared only by
# disabling the service and terminating the process

After (this branch, cargo build --release): immediate honest stop.

> sc.exe start vector ; sc.exe query vector
SERVICE_NAME: vector
        STATE              : 1  STOPPED
        WIN32_EXIT_CODE    : 1066  (0x42a)
        SERVICE_EXIT_CODE  : 78  (0x4e)

System event log now records the failure:

Id 7024: The Vector Service service terminated with the following
         service-specific error: %%78

Service recovery now works. With a recovery ladder configured (sc.exe failure vector reset= 86400 actions= restart/10000/restart/30000/restart/60000 plus sc.exe failureflag vector 1, which Windows requires for non-crash failures), the SCM retries per the ladder:

Id 7031: The Vector Service service terminated unexpectedly. It has done this
         2 time(s). The following corrective action will be taken in 30000
         milliseconds: Restart the service.
Id 7024: ... service-specific error: %%78
Id 7031: ... 3 time(s) ... 60000 milliseconds: Restart the service.

After correcting the configuration, the SCM's scheduled restart brought the service back to RUNNING with no operator action (a subsequent manual sc.exe start returned FAILED 1056: An instance of the service is already running).

Regression check with a valid configuration: sc.exe start reaches RUNNING and sc.exe stop reaches STOPPED with exit code 0, repeatedly.

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Closes: #25809

When Application::prepare_start fails (for example an
invalid configuration), run_service returned without
registering a service control handler or reporting any
status to the SCM. The service then sits in START_PENDING
indefinitely and configured recovery (FailureActions)
never runs, because the SCM never observes a failure.

Register a handler and report SERVICE_STOPPED with the
startup exit code as a service-specific exit code, so the
failure is visible to the SCM and to service recovery.

(cherry picked from commit 0df4002)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows service stuck in START_PENDING forever when Vector fails to start

1 participant