# Move your S3 files from AWS to Brewless

Copy a Laravel app's uploads from an S3 bucket at AWS, or any S3 service, into its Scaleway bucket with brewless storage:import. Nothing is removed.

## In short

- `brewless storage:import` copies the files of an S3 bucket into the bucket of a Brewless environment, and never removes anything at either end.
- The source is described with the same AWS_* names Laravel's s3 disk uses, given as SOURCE_AWS_* variables, an AWS CLI profile or the .env of the old app.
- By default rclone copies in a job of your own Scaleway project, so the files go from AWS to Scaleway without passing through Brewless or your machine.
- The keys reach that job through one secret in your own Secret Manager, and that version is deleted when the copy ends.
- A file that is in the new bucket already is left alone, so you copy everything first and run it once more just before you move your domain.

A Laravel app that comes from Vapor, or from anywhere else on AWS, keeps its uploads in an S3 bucket. When it moves to Scaleway with [Brewless](/), the environment gets a new, private bucket on Scaleway Object Storage, and the files have to get there. This post shows how `brewless storage:import` copies them, which values go where, and what happens to your keys.

## How do you move an S3 bucket to Brewless?

Run `brewless storage:import` with the environment and the bucket the files are in now. It copies every file into the environment's bucket with rclone, in a serverless job of your own Scaleway project, and removes nothing. Run it again just before you move your domain and it only copies what changed.

```bash
export SOURCE_AWS_ACCESS_KEY_ID=AKIA…
export SOURCE_AWS_SECRET_ACCESS_KEY=…
brewless storage:import production s3://my-vapor-bucket --source-region=eu-west-1
```

The command says where it copies from and to, asks once, and follows the copy until it is done.

## Which value goes where

The names are the ones Laravel's `s3` disk reads from `config/filesystems.php`. For the source you give them; for the destination Brewless already set them in the environment's variables when it made the bucket.

| Laravel variable | Source (you give it) | Destination (Brewless set it) |
| --- | --- | --- |
| `AWS_BUCKET` | `s3://<bucket>[/<folder>]`, `SOURCE_AWS_BUCKET` or `--from-env` | `<application>-<environment>-<project>` |
| `AWS_DEFAULT_REGION` | `--source-region`, `SOURCE_AWS_DEFAULT_REGION`; at AWS it is looked up | the environment's region, such as `fr-par` |
| `AWS_ENDPOINT` | `--source-endpoint`; leave it out for AWS | `https://s3.fr-par.scw.cloud` |
| `AWS_USE_PATH_STYLE_ENDPOINT` | `--source-path-style` | `false` |
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | `SOURCE_AWS_*`, `--source-profile`, `--from-env`, or typed in | the environment's own key for its bucket |
| `AWS_URL` | not needed to copy | change or remove it after the move |

The `AWS_*` variables of your own shell are not read on purpose: they often belong to another AWS account than the one your files are in.

## Three ways to give the source

From the `.env` of the old application, which reads only its `AWS_*` lines:

```bash
brewless storage:import production --from-env=../old-app/.env
```

With variables, the same names as Laravel's with `SOURCE_` in front:

```bash
export SOURCE_AWS_ACCESS_KEY_ID=AKIA…
export SOURCE_AWS_SECRET_ACCESS_KEY=…
brewless storage:import production s3://my-vapor-bucket/uploads --source-region=eu-west-1
```

With a profile of the AWS CLI, from `~/.aws/credentials` and `~/.aws/config`:

```bash
brewless storage:import production s3://my-vapor-bucket --source-profile=migration
```

A key is never an option on the command line, where it would end up in your shell's history. Without a key in any of these places the command asks for it in the terminal, and run from a script it stops.

> [!NOTE]
> An app on Vapor runs with an IAM role, so its `.env` usually has no `AWS_ACCESS_KEY_ID`. Vapor sets `AWS_BUCKET` itself. Make a key at AWS that may only read the bucket, as below.

## A key at AWS that may only read

Do not hand out the key of your AWS account. Create an IAM user for the move, with only this inline policy, and an access key for an application outside AWS:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::my-vapor-bucket" },
        { "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::my-vapor-bucket/*" }
    ]
}
```

It can list and read the bucket and nothing else. Remove the user once you have moved.

## In your project or from your laptop

| | In your Scaleway project (default) | `--local` |
| --- | --- | --- |
| Where the files pass | from AWS straight to Object Storage | through your machine |
| Where your AWS key goes | into your own Secret Manager, deleted afterwards | nowhere: it stays on your machine |
| Terminal open | no, you can close your laptop | yes, until it is done |
| Files larger than 5 GB | copied | skipped and named |

In the default mode Brewless runs rclone in a serverless job in your project. Both keys, the one for AWS and the environment's own key for its bucket, reach the job in one secret of your Secret Manager. When the copy ends, however it ends, that version of the secret is deleted. The operation Brewless records only knows which version it was, never a key. That is the same rule as for your other [secrets](/blog/laravel-secrets-without-env-file): they stay in your account.

## Follow it, or come back later

While the copy runs, the command shows how many files of how many and how much of how much. `Ctrl+C` stops following, not the copy:

```bash
brewless storage:status production
```

shows the last copy into the environment's bucket, while it runs and after it ended. It is also on the environment's page in the console, where you can cancel it; what was copied stays.

`--dry-run` copies nothing and counts what the source holds first, which is handy to know how much AWS will bill for the data that leaves it.

## After the move

> [!WARNING]
> `brewless env:push` replaces the environment's variables as a whole. Do not push the `.env` of your old app with its own `AWS_BUCKET` and keys: that points the app back at the old bucket. Start from `brewless env:pull` instead.

Then change or remove `AWS_URL`, which pointed at the old bucket or CloudFront, run the import once more just before you move your domain, and remove the read key at AWS. Your app reads its files through the same `s3` disk as before, now from Scaleway; see [Laravel uploads on Scaleway Object Storage](/blog/laravel-uploads-scaleway-object-storage) for how that bucket is set up.

The full reference, with other S3 services and every option, is in the documentation: https://docs.brewless.eu/applications/moving-files

## Frequently asked questions

### Does storage:import delete anything?

No. It copies with rclone copy, never sync. Nothing at the source is changed or removed, and a file that already is in the environment's bucket with the same size and time is left as it is.

### Where does my AWS key go?

By default into a secret of your own Scaleway Secret Manager, next to the environment's own key for its bucket, as one rclone configuration for the job. When the copy ends, however it ends, that version of the secret is deleted. Brewless keeps no key. With --local the key never leaves your machine.

### What does it cost?

AWS bills the data that leaves it, per GB. On the Scaleway side the copy is a serverless job in your project, billed by Scaleway for the time it runs. A dry run counts how many files and how much data there is first.

### Can I run it again just before I switch my domain?

Yes. Files that are in the bucket already with the same size and time are skipped, so a second run only copies what was added or changed since the first.

### Does it work with something other than AWS?

Yes. Any S3 service works as the source. Give its endpoint and region, such as DigitalOcean Spaces, Cloudflare R2, Hetzner Object Storage, MinIO or another Scaleway project.

Source: https://test.brewless.eu/blog/move-s3-files-to-brewless