Repository navigation
ENT-14927: Fixed cf-serverd crash when policy reload races with a new connection - #6464
Merged
Merged
Conversation
cf-serverd only checked ACTIVE_THREADS before reloading policy, but a connection is added to SERVER_ACCESS.connectionlist before its handler thread increments ACTIVE_THREADS and removed only after it decrements it. A reload in that window tripped the assertion in ClearAuthAndACLs(). Policy reload and shutdown cleanup now also wait for connections that are still in connectionlist. Ticket: ENT-14927 Changelog: Title Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Contributor
Author
|
@cf-bottom jenkins please |
|
Alright, I triggered a build: Jenkins: https://ci.cfengine.com/job/pr-pipeline/14996/ Packages: http://buildcache.cfengine.com/packages/testing-pr/jenkins-pr-pipeline-14996/ |
aleksandrychev
added
the
cherry-pick?
craigcomstock
approved these changes
Nov 6, 2026
Contributor
|
Backport to 3.24.x: #6492 |
Contributor
|
Backport to 3.27.x: #6493 |
github-actions
Bot
removed
the
cherry-pick?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: ENT-14927
cf-serverd decides it is safe to reload policy (and to free ACLs/TLS on shutdown) by checking
ACTIVE_THREADS == 0. But a connection is added toSERVER_ACCESS.connectionlistinServerEntryPoint()before its handler thread incrementsACTIVE_THREADS, and removed inDeleteConn()only after the thread has decremented it. A reload in either window hitsassert(SERVER_ACCESS.connectionlist == NULL)inClearAuthAndACLs().Seen in CI as a hub crash during bootstrap-tests (bootstrap-tests #1130):
Change: a new
ConnectionsInFlight()returnsMAX(ACTIVE_THREADS, ListLen(connectionlist)), reading the list undercft_count.PolicyUpdateIfSafe()andWaitOnThreads()use it instead ofACTIVE_THREADSalone. The lock order iscft_server_childrenthencft_count, which the reload path already does; nothing takes them in the reverse order.Testing (Debian 12,
--enable-debug): cf-serverd withallowallconnectsfor localhost, 16 threads opening/closing connections to 5308, and SIGHUP every 50 ms for 60 s:The same code is in 3.24.x and 3.27.x, so this should be backported.
🤖 Generated with Claude Code