# Laravel queues without a long-running worker

How Brewless runs Laravel queues on serverless containers. With Scaleway Queues a job reaches the worker the moment it is queued, with its tries and backoff.

## In short

- On a serverless container nothing stays running, so something has to call the queue worker instead of queue:work waiting for jobs.
- With Scaleway Queues, the queue itself calls the worker container as soon as a job is put on it, so a job runs right away and nothing wakes the worker in between.
- Every job the sqs and database connections queue is signed with a key derived from APP_KEY, and the worker only runs a job whose signature holds.
- A job delivered by the queue runs the way queue:work runs it, with its tries, its backoff, its failed() method and the failed jobs table.
- Give the worker a timeout in brewless.yml above your longest job; the queue keeps a job out of sight for that timeout plus half a minute, and at least three minutes.

On a server, Laravel's queue is a process that never ends: `php artisan queue:work` sits there and waits for jobs, and Supervisor restarts it when it dies. On serverless containers nothing stays running. There is no process to wait, and no Supervisor to keep one alive.

So on Brewless the worker does not wait for jobs. Jobs are brought to it. This post explains how, what Laravel features keep working, and the two settings in `brewless.yml` that matter.

## How does a Laravel queue work without a long-running worker?

Something outside the container calls the worker when there is work. With Scaleway Queues, the queue itself calls the worker container as soon as a job is put on it, and the job runs in that request. With jobs in the database, a trigger calls the worker every minute to look for them.

## Where the jobs wait

Where jobs wait is chosen per environment, with `queue` in `brewless.yml`:

```yaml brewless.yml
environments:
  production:
    region: fr-par
    database: shop-main
    queue: sqs
```

| `queue` | `QUEUE_CONNECTION` | How the worker is called |
| --- | --- | --- |
| `sqs` | `sqs` | By the queue at Scaleway Queues, as soon as a job is put on it |
| `database` | `database` | By a trigger, every minute |
| `false` | `sync` | Not at all: a job runs where it is dispatched |

With `sqs`, Brewless switches Scaleway Queues on for your project, makes a queue named after the application and the environment, and gives it a key of its own that may send and receive but not make or remove queues. That key goes into your own Secret Manager, as `SQS_ACCESS_KEY_ID` and `SQS_SECRET_ACCESS_KEY`, because Laravel's `sqs` connection otherwise reads the same `AWS_*` names as the `s3` disk. The `brewless/laravel` package hands them to the `sqs` connection.

Switching an environment from `database` to `sqs` in the file also moves the jobs that waited in the database to the new queue. The other way round is refused, because it would leave the queue behind with whatever is on it.

> [!NOTE]
> Scaleway Queues speaks the SQS API, but not all of it. Sending and receiving jobs work; asking a queue how many jobs it holds, as Laravel's `Queue::size()` does, is refused. Queues is also not available in Warsaw or Milan yet.

## Scaleway Queues: the job comes to the worker

Scaleway can attach a trigger to a queue: whenever a message arrives, it posts that message to a container. Brewless points that trigger at your worker container. So the moment your application dispatches a job, the queue posts it to the worker, and the worker runs it.

Nothing calls the worker between jobs. A quiet application has a worker that is scaled to zero, and, with a serverless database, a database that can sleep as well. When the next job arrives, the worker container starts first, so that job waits for a cold start. After that, it runs straight away.

This needs `brewless/laravel` 0.4 or later.

### Signed jobs

The queue's trigger posts the job to `/` on the worker container without any secret of Brewless's. A job is code that gets unserialised, so the worker must not run whatever is posted to it.

`brewless/laravel` therefore signs every job that the `sqs` and `database` connections queue. The key is derived from `APP_KEY`, which every container of the application has, including the web container that dispatches most jobs. The worker runs a job only when its signature holds:

- A job is good for an hour after it was signed (`BREWLESS_JOB_MAX_AGE`, 3600 seconds), and dropped after that.
- Within that hour each delivery runs once. A delivery that comes again is dropped; one that another request is still handling is answered with a 409, so the queue tries it later.
- After a rotation of `APP_KEY`, a job signed with a key in `APP_PREVIOUS_KEYS` still runs.

The marks that say a delivery ran are kept in the application's default cache. Give the containers a cache they share, the database or Redis, or a delivery that lands on another container is not recognised.

### Tries, backoff and failed jobs

A delivered job runs the way `queue:work` runs it. Its `$tries`, its `backoff()`, its `failed()` method and the `failed_jobs` table all apply, so a job class you wrote for a server needs no change:

```php app/Jobs/SendInvoice.php
final class SendInvoice implements ShouldQueue
{
    use Queueable;

    public int $tries = 5;

    /** @return list<int> */
    public function backoff(): array
    {
        return [10, 60, 300];
    }

    public function handle(): void
    {
        // ...
    }

    public function failed(Throwable $exception): void
    {
        // ...
    }
}
```

A job that is tried again goes back on its queue as a new message, signed again, and comes back to the worker after its backoff. A job that has used up its tries ends up in `failed_jobs`, where `queue:work` would put it.

The worker always answers the queue with a success, also for a job that failed or was dropped. Otherwise Scaleway would deliver the same message again every half minute, without end. Whether a job failed is your application's business, and it is in your own log and your own `failed_jobs` table.

Failed jobs can be put back on the queue from the **Queues** tab of the environment in the console. Why a job failed is in your application's `failed_jobs` table, and Brewless does not read it: an exception can quote the data it failed on.

## Worker timeout and visibility

Two clocks matter for a job on Scaleway Queues.

**The worker's timeout** is how long one call to the worker container may take. It is 120 seconds by default and set in `brewless.yml`:

```yaml brewless.yml
environments:
  production:
    queue: sqs
    worker:
      timeout: 600
      memory: 1024
```

**The visibility timeout** is how long the queue keeps a job out of sight once it handed it to the worker. If the worker has not finished by then, the queue delivers the job again. Brewless sets it from the worker's timeout: three minutes, or the timeout plus half a minute when that is longer.

| `worker.timeout` | Visibility timeout |
| --- | --- |
| 120 (default) | 180 seconds |
| 300 | 330 seconds |
| 600 | 630 seconds |

So the rule is simple: give a worker that runs long jobs a `timeout` above the longest job. The visibility follows on the next deploy, and a job that is still running is not handed out a second time.

> [!WARNING]
> A worker timeout below your longest job means that job is cut off when the container's time is up. Raise `worker.timeout` (up to 3600 seconds) or split the work into smaller jobs.

## The database queue

Without Scaleway Queues, jobs wait in your database's `jobs` table. Then nothing can tell the worker a job arrived, so a trigger calls it every minute. Each call runs one tick of the scheduler and then works the queue until it is empty, for at most 45 seconds. A tick that finds no job due does not start a worker at all.

With jobs in the database, `worker.queues` names the queues the worker works, the first first:

```yaml brewless.yml
environments:
  production:
    queue: database
    worker:
      queues: [default, mail]
```

Without it, the worker works the default queue of your application. Taking `queues` out of the file again keeps what the worker works; write `[default]` to go back. This needs `brewless/laravel` 0.3 or later.

The database queue has two costs. A job waits up to a minute before it runs. And the worker, with its database, is called every minute all day, so a serverless database never gets to sleep. For an application with a serverless database, Scaleway Queues is the cheaper choice.

## What stays the same

Your code dispatches jobs as it always did: `SendInvoice::dispatch($invoice)` and `->delay(...)` go through Laravel's own queue connection. What changes is only who starts the worker.

What you leave behind is Supervisor, `queue:restart` after every deploy, and a process to watch. The worker moves to a new release right after the web container has switched to it. If a worker gives no sign of life for more than three minutes, the console says so.

Laravel's scheduler on Brewless is the other half of the worker container; [Laravel's scheduler without a cron server](/blog/laravel-scheduler-without-cron) covers it. The full reference is [Queues and schedule](https://docs.brewless.eu/running/queues-and-schedule).

## Frequently asked questions

### Can I run php artisan queue:work on Brewless?

Not as a process that stays up. Brewless runs Laravel on serverless containers, and queue:work is a command that never ends, so it is refused as a release command and as a one-off command. The worker container is called instead, either by the queue or by a trigger every minute.

### How fast does a queued job run on Brewless?

With jobs on a queue at Scaleway Queues, the queue calls the worker as soon as a job is put on it. A worker that was scaled to zero starts first, so the first job after a quiet period waits for a cold start. With jobs in the database, a trigger calls the worker once a minute.

### Do Laravel job tries, backoff and failed jobs still work?

Yes. The worker runs a delivered job the way queue:work does, with its tries, its backoff, its failed() method and the failed_jobs table. A job that is tried again goes back on its queue as a new message, and failed jobs can be put back on the queue from the Brewless console.

### Should I use Scaleway Queues or the database queue?

Scaleway Queues runs a job right away and lets the worker and a serverless database sleep between jobs. The database queue needs no extra service, but a trigger then calls the worker every minute to look for jobs, which keeps the database awake all day. Scaleway Queues is not available in Warsaw or Milan yet.

### Why are queued jobs signed?

The queue posts each job to the worker container without a secret, and a job is code that gets unserialised. brewless/laravel signs every job with a key derived from APP_KEY and runs only a job whose signature holds, within an hour of signing and once per delivery.

Source: https://test.brewless.eu/blog/laravel-queues-without-a-worker