Laravel uploads on Scaleway Object Storage

On a server you own, storage/app is a folder that stays. On a serverless container it is not: the container starts when a request comes in, scales back to zero when it is quiet, and two copies of your app never share a disk. An upload written to local disk is gone with the container.

This post shows where uploads go instead when a Laravel app runs on Scaleway with Brewless: a private bucket on Scaleway Object Storage, through Laravel's own s3 disk, plus a second bucket for backups that the app can write to but never read.

Where do Laravel uploads go on a serverless container?#

They go to a bucket. Scaleway Object Storage speaks the S3 API, so Laravel's s3 disk stores and reads files there as it would on Amazon S3. With storage: in brewless.yml, Brewless makes a private bucket and sets the variables the s3 disk reads.

The local disk is still there for what lives within one request: a file you resize before you store it, an export you build and then upload. Anything that has to outlive the request goes to the bucket.

Ask for a bucket#

One key in brewless.yml asks for it:

environments:
  production:
    region: fr-par
    storage: shop-production-files

The next brewless deploy makes the bucket in the Scaleway project of the environment, in the region of the environment. You can also ask for one when you set an environment up with brewless init or brewless provision, or with Set up in the console.

Laravel needs the Flysystem adapter for S3:

composer require league/flysystem-aws-s3-v3 "^3.0"

Your config/filesystems.php needs no change. The s3 disk that ships with Laravel reads these variables, and Brewless puts them in a new revision of the environment's variables in your own Secret Manager:

With FILESYSTEM_DISK=s3, everything you store through Storage:: without naming a disk lands in the bucket:

$path = $request->file('invoice')->store('invoices');

// later
$contents = Storage::get($path);

Note

A variable change reaches the containers with the next deploy, not before. Adding storage: to an environment that runs adds its variables in the same deploy.

The identity behind the key#

Brewless does not hand your app its own Scaleway key. It makes an identity for the bucket that may read the buckets of the project and read, write and remove the objects in them. It may not make, change or remove a bucket. Scaleway grants these rights per project, not per bucket, so the identity reaches the objects of every bucket in that project. That is one reason to give each application a Scaleway project of its own.

Making identities needs the permission set IAMManager on the key you gave Brewless. Without it, the bucket is still made, and the result names AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for you to add yourself. See A bucket for files.

Private files and temporary URLs#

The bucket is private. Nothing in it has a public address, which is what you want for invoices, contracts, ID scans and exports. To let a visitor download one, make a temporary URL:

use Illuminate\Support\Facades\Storage;

public function __invoke(Invoice $invoice): RedirectResponse
{
    Gate::authorize('view', $invoice);

    return redirect()->away(
        Storage::temporaryUrl($invoice->path, now()->addMinutes(5)),
    );
}

The URL is signed with the app's key and stops working when it expires. The visitor downloads the file from Scaleway Object Storage directly, so a large file does not keep your container busy, and your authorisation check runs before the URL is made.

The other way is to stream the file through a route of your own with Storage::download() or Storage::response(). That keeps the file behind your app on every request, at the cost of a container serving the bytes.

Public assets and public media#

Two kinds of files look public, and they go to different places.

Your built frontend. With bunny.net connected, a deploy puts the hashed files of your frontend build in a storage zone in your own bunny.net account, and the pull zone serves those paths from there. A page of an earlier release still finds its files, and a request for one never wakes a container. This is only for what npm run build produces. See Built files behind bunny.net.

Uploads that everyone may see, such as avatars or product photos. Brewless does not put those on bunny.net Storage, and the bucket it makes is private. Serve them through a route of your own that reads from the bucket, or with temporary URLs that last longer. Behind bunny.net, a route that sends cache headers can be answered from the edge.

Tip

Large uploads pass through the web container on their way to the bucket. Its local disk (web.tmp-storage, 1000 MB by default) and its request timeout (web.timeout, 60 seconds by default) are set in brewless.yml. Raise them when users upload large files.

A bucket for backups#

Backups need the opposite of an uploads bucket: the app should be able to add one, and nothing else. If the app's key could remove backups, so could whoever got hold of that key. The backup: key asks for a second, private bucket, in a region of its own:

environments:
  production:
    region: fr-par
    storage: shop-production-files
    backup:
      name: shop-production-backups
      region: nl-ams
      expire_days: 35

region and expire_days are required. Without name, the bucket is named after the application, the environment and your project, with -backups. Brewless then:

• gives the bucket two lifecycle rules: Object Storage removes each object expire_days after it was put, and aborts an upload in parts that did not finish within a day;
• makes a write identity whose key the app gets. It may put a backup, also in parts, and nothing more: it cannot read, list or remove;
• makes a read identity for a restore. Its key is kept in your Secret Manager as <application>-<environment>-backup-read-key and never goes into the variables;
• puts a bucket policy on the bucket, so every other key of the project, the one of the files bucket included, can neither read nor remove a backup.

The app gets BACKUP_BUCKET, BACKUP_REGION, BACKUP_ENDPOINT, BACKUP_ACCESS_KEY_ID and BACKUP_SECRET_ACCESS_KEY. Your files disk is left as it is. A disk for them:

'backups' => [
    'driver' => 's3',
    'key' => env('BACKUP_ACCESS_KEY_ID'),
    'secret' => env('BACKUP_SECRET_ACCESS_KEY'),
    'region' => env('BACKUP_REGION'),
    'bucket' => env('BACKUP_BUCKET'),
    'endpoint' => env('BACKUP_ENDPOINT'),
    'use_path_style_endpoint' => false,
    'throw' => true,
],

Warning

The key may only put. A backup tool that lists the bucket, checks the newest backup or cleans up old ones is refused. Let the lifecycle rule do the cleaning and switch those steps off.

The bucket policy needs identities. When Brewless's key may not manage them, no policy is set, nothing is given to the app, and setting the environment up stops with what to do. To restore, read the read key from Secret Manager in the Scaleway console and use it on your own machine only. See A bucket for backups.

What stays when you change your mind#

A deploy never switches a bucket for another one. Another name under storage or backup, or another backup region, is refused before anything is made. storage: false or backup: false lets the bucket go from the environment, but the bucket and its files keep running at Scaleway, and your variables keep pointing at them until you change them. Removing data is yours to do, in your own Scaleway console.

Summary#

1. Add storage: to the environment and league/flysystem-aws-s3-v3 to the app.
2. Store through Storage::; the s3 disk and its variables are set for you.
3. Keep the bucket private and hand out files with temporary URLs or your own route.
4. Let bunny.net serve the built frontend, not uploads.
5. Add backup: for a write-only bucket in another region, and let expire_days remove old backups.

For the rest of a serverless Laravel setup, read How to host a Laravel app EU-sovereign.