Skip to content

feat: SentTaskHandle and ReceivedTaskHandle - #2636

Open
nicolas-grekas wants to merge 16 commits into
php:mainfrom
nicolas-grekas:bgworker-tasks
Open

nicolas-grekas wants to merge 16 commits into
php:mainfrom
nicolas-grekas:bgworker-tasks

Conversation

@nicolas-grekas

@nicolas-grekas nicolas-grekas commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #2617 and #2635: review the commits on top of them.

Tasks for background workers, the task half of #2319. A worker declares the namespaces it receives, and a sender names the namespace:

worker {
	file images.php
	background
	num 4
	tasks image.resize image.thumbnail
}
// anywhere
$task = new FrankenPHP\SentTaskHandle('image.resize', ['file' => 'photo.jpg']);
while (null !== $update = $task->read()) {
}

// the background worker, in its tick() loop
while ($task = $handle->receive()) {
    $task->update(['progress' => 50]);
    $task->complete(resize($task->getNamespace(), $task->getPayload()));
}
  1. Constructing a SentTaskHandle waits for a thread to pick the task up and throws on timeout, so a busy worker pushes back instead of queueing without bounds.
  2. A pool shares one queue for all its namespaces, and each task wakes one parked thread through its WorkerHandle, so the loop of feat: background workers and FrankenPHP\WorkerHandle #2617 doesn't change.
  3. Both task handles are Io\Poll\Handle with a getStream(), like the worker's. abandon(), or dropping the handle, gives up on the task, and a worker ending with a task open makes the sender's next read() throw.
  4. Each side waits on its own descriptor, eventfd on Linux, kqueue on macOS, a socket pair on Windows, pooled between tasks: 8 senders on 8 threads went from 2k to 32k tasks/s along the way.

The metrics follow: frankenphp_busy_workers and frankenphp_worker_queue_depth count tasks, plus frankenphp_worker_task_count{outcome} and frankenphp_worker_task_time for a Metrics implementation that also implements TaskMetrics.

Left out compared to #2319: a queue ahead of pickup, and cancelling a task before it.

@nicolas-grekas

Copy link
Copy Markdown
Contributor Author

Pushed 23d3094 after benchmarking the task path in the Docker builder image (harness kept out of the PR). Three changes: a send wakes one parked thread instead of every thread of the pool; the sender waits for the pickup in the kernel, on the task's socket, instead of in a Go select (waking a thread parked inside a cgo callback costs the scheduler a P hand-off); and the queue mutex is no longer held across a syscall nor taken twice per wake-up. One thread: 549 to 285µs per task. A pool of 8: 1745 to 320µs. 8 senders on 8 threads: 2k to 32k tasks/s. Absolute numbers are that VM's, the ratios are the point.

@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 5 times, most recently from 4c5af56 to 2b77a5b Compare September 9, 2026 17:53
@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 5 times, most recently from e0be4ac to fd177fc Compare September 14, 2026 06:16
@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 11 times, most recently from a8f1a7f to a9d72df Compare September 18, 2026 16:58
@nicolas-grekas nicolas-grekas changed the title feat: frankenphp_send_task(), frankenphp_receive_task() and friends feat: SentTaskHandle and ReceivedTaskHandle Sep 18, 2026
Names were scoped per server so a script could address a background worker
by name within its php_server. The namespaces of php#2635 and php#2636 do that
now and no PHP API takes a worker name, so naming goes back to what main
does: Caddy makes generated names unique, and the server label and the
ServerMetrics interface are gone. The extension worker fixes leave too,
main has the same bugs and they get a pull request of their own.
Since php#2660 worker threads come on top of num_threads and max_threads
bounds the whole process. Background threads were the one exception, which
took a reservation through calculateMaxThreads(), initPHPThreads() and the
automatic max_threads. They now follow the same rule as HTTP workers.
HTTP workers wait for their ready point without a bound too, on builds
without Zend max execution timers, so the bound belongs with both kinds
of worker. Background workers keep the bound of max_execution_time where
those timers exist, and the warning logged after 10 seconds elsewhere.
Init() waits for a background worker to reach its ready point, its first
WorkerHandle::tick(). On a build with Zend max execution timers,
max_execution_time ends a bootstrap that overstays; without them
FrankenPHP disables that limit, so a script that parks before ticking, or
one whose wake-up at start is lost, keeps the server start waiting for
ever with a warning as the only trace.

boot_timeout bounds that wait, 30 seconds by default, the same figure
PHP's own max_execution_time uses for the bootstrap it does bound. The
worker is then drained and stopped through the boot-failure path it
already has, and Init() returns the name of the worker that never ticked.
Zero waits for ever.
@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 2 times, most recently from 1792cb2 to f651960 Compare October 27, 2026 16:21
The timeout reported through a blocking send, so a second timeout could
wait for room in startupFailChan for good, Init() with it. It now reports
like the handlers do and moves the thread to ShuttingDown, so no run starts
after the one that overstayed, whether the timeout lands in a run or
between two failed boots.

The 30s default now only applies where Zend max execution timers are
missing, as the docs said: elsewhere max_execution_time bounds the
bootstrap, and a set_time_limit() raising it is no longer cut short.
Comments and docs that drifted are updated, and the thread budget and the
Caddy directive get the tests they lacked.
frankenphp_get_vars() copied the snapshot inside a Go callback with its
slot read-locked, so a bailout during the copy, past memory_limit or in an
autoloader an enum triggers, left the lock held: the publisher and the next
readers waited for good, Shutdown() with them. Snapshots are now counted
references, the copy runs in C with no lock held and lets go of the
snapshot on every path.

The class name of an enum went to the autoloaders as the persistent string
of the snapshot, so one keeping it pointed into memory the next
publication freed, and readers raced on its refcount. The lookup now uses a
request copy.

A worker waiting on another one at boot now gives up when the start fails,
so an HTTP worker reading vars in its bootstrap no longer keeps Init()
waiting on a publisher that never comes up, and a Caddy JSON config no
longer drops vars declared on an HTTP worker.
The sanitizer jobs run without the Zend allocator, which enforces no memory_limit. A fatal error in the autoloader an enum triggers bails out on every allocator.
Calling SentTaskHandle::__construct() again replaced the task of the handle,
so the first one was never closed on the sender's side: its descriptors and
payload leaked, and a worker past the update buffer waited for good. A
second construction now throws.

A task whose wait for a thread ended on a restart or the shutdown was
counted under no outcome; it is now "cancelled". A Caddy JSON config no
longer drops tasks declared on an HTTP worker either.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant