Utopia Queue

Important

This repository is a read-only mirror of the utopia-php monorepo. Development happens in packages/queue — please open issues and pull requests there.

Utopia Queue is a powerful Queue library. This library is aiming to be as simple and easy to learn and use. This library is maintained by the Appwrite team.

Although this library is part of the Utopia Framework project it is dependency free and can be used as standalone with any other PHP project or framework.

Getting started

Install using Composer:

composer require utopia-php/queue

Init in your application:

<?php

require_once __DIR__ . '/../../vendor/autoload.php';

// Create a worker using the Swoole adapter
use Utopia\Queue;
use Utopia\Queue\Message;

$connection = new Queue\Connection\Redis('redis');

if ($connection->ping()) {
    var_dump('Connection is ready.');
} else {
    var_dump('Connection is not ready.');
}

$adapter = new Queue\Adapter\Swoole($connection, 12, 'my-queue');
$server = new Queue\Server($adapter);

$server
    ->job()
    ->inject('message')
    ->action(function (Message $message) {
        var_dump($message);
    });

$server
    ->error()
    ->inject('error')
    ->action(function ($error) {
        echo $error->getMessage() . PHP_EOL;
    });

$server
    ->workerStart()
    ->action(function () {
        echo "Worker Started" . PHP_EOL;
    });

$server->start();


// Enqueue messages to the worker using the Redis adapter
$connection = new Redis('redis', 6379);
$client = new Client('my-queue', $connection);

$client->enqueue([
    'type' => 'test_number',
    'value' => 123
]);

NATS JetStream broker

Broker\Nats runs the queue on NATS JetStream instead of Redis, giving durable, server-persisted jobs and native at-least-once redelivery. It implements the same Publisher + Consumer interfaces as Broker\Redis, so it drops into the same Server and adapter setup.

use Utopia\NATS\Connection;
use Utopia\Queue\Broker\Nats;
use Utopia\Queue\Queue;

// Pass a Closure so each forked worker / pooled lease resolves its own connection —
// a NATS connection is single-owner and must not be shared across coroutines.
$broker = new Nats(
    fn (): Connection => Connection::connect('nats://127.0.0.1:4222'),
    ackWait: 30.0,   // redelivery window if a worker dies before commit()
    maxDeliver: 5,   // delivery attempts before a message is dead-lettered
);

$broker->enqueue(new Queue('my-queue'), ['type' => 'test_number', 'value' => 123]);

Each queue is a WorkQueue-retention stream (a message is removed once acknowledged) with a companion dead stream. commit() acknowledges a message, reject() schedules redelivery until maxDeliver and then dead-letters, retry() re-drives the dead stream onto the queue, and getQueueSize() reports pending (consumer num_pending) or failed (dead stream) counts. reap() is a no-op — redelivery after ackWait reclaims jobs stranded by a dead worker. Requires utopia-php/nats.

A NATS connection is single-owner. Run one message at a time per connection (the Swoole adapter with maxCoroutines: 1) or lease one connection per coroutine via Broker\Pool / Utopia\Pools.

System requirements

Utopia Framework requires PHP 8.0 or later. We recommend using the latest PHP version whenever possible.

Copyright and license

The MIT License (MIT) http://www.opensource.org/licenses/mit-license.php