Encrypt uploads in the browser and send them in chunks
A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.
Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.
- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
ciphertext and tag per chunk. Each nonce holds the chunk index and a
last-chunk flag (the STREAM construction), so cut or reordered files
fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
wrapped with Argon2id (sodium, libsodium's interactive limits) in
shares.wrapped_key, which names its parameters. Password shares from
before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
pending share of its own, lists the files with their progress, retries
a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
abort. UploadChunkController only accepts chunks from the session that
started the share: a repeat is acknowledged, a skip gets 409 with the
count stored. Chunks go out as Blobs, which Chromium sends about eight
times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
needed and takes no files. The Docker image gains AUTO_HTTPS, which
serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
ZIP64) instead of decrypting whole files into memory and writing the
archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
endpoint's answers, completing a share, the streamed ZIP, cleanup,
and in Chromium a real chunked upload and the HTTPS warning; the
selected-files overflow test runs again. README, website, CHANGELOG
and .ai/rules follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
504971ad7f
commit
40e35bab0e
@@ -1,11 +1,10 @@
|
||||
<?php
|
||||
|
||||
use App\Models\Share;
|
||||
use App\Services\QrCodeService;
|
||||
use Illuminate\Http\UploadedFile;
|
||||
use App\Services\ShareService;
|
||||
use Illuminate\Support\Carbon;
|
||||
use Illuminate\Support\Facades\Storage;
|
||||
use Livewire\Features\SupportFileUploads\FileUploadConfiguration;
|
||||
use Livewire\Features\SupportFileUploads\TemporaryUploadedFile;
|
||||
use Tests\Screenshots\DemoData;
|
||||
use Tests\Screenshots\Publisher;
|
||||
|
||||
@@ -19,14 +18,9 @@ beforeEach(function () {
|
||||
// Sessions have to outlive a request: a sign-in, an unlocked share.
|
||||
config(['session.driver' => 'file']);
|
||||
Storage::fake('shares');
|
||||
Storage::fake('tmp-for-tests');
|
||||
|
||||
$this->travelTo(Carbon::parse('2026-10-01 09:30'));
|
||||
|
||||
// Livewire deletes temporary uploads a day older than now, by the files' real modification
|
||||
// times: under the frozen clock that is every file this run stores.
|
||||
config(['livewire.temporary_file_upload.cleanup' => false]);
|
||||
|
||||
DemoData::shares();
|
||||
});
|
||||
|
||||
@@ -71,27 +65,36 @@ function shoot(mixed $page, string $device, string $theme, string $name): void
|
||||
}
|
||||
|
||||
/**
|
||||
* Put files into the uploader the way a finished upload does: stored as Livewire temporary uploads
|
||||
* and handed to `_finishUpload` by their signed names. A browser test cannot select files through
|
||||
* the file input, because the in-process server does not store a multipart upload.
|
||||
* Put files into the uploader the way a finished upload does: registered through the page's own
|
||||
* `registerFiles`, their encrypted content stored here as the browser would have sent it (zeros of
|
||||
* the demo size), and the list refreshed to show them uploaded.
|
||||
*
|
||||
* @param array<string, int> $files relative path => size in kilobytes
|
||||
*/
|
||||
function selectFiles(mixed $page, array $files): void
|
||||
{
|
||||
$signed = [];
|
||||
$selection = collect($files)->map(fn (int $kilobytes, string $path): array => [
|
||||
'name' => basename($path),
|
||||
'size' => $kilobytes * 1024,
|
||||
'path' => str_contains($path, '/') ? $path : null,
|
||||
])->values();
|
||||
$wire = 'Livewire.find(document.querySelector("[data-test=drop-zone]").closest("[wire\\\\:id]").getAttribute("wire:id"))';
|
||||
|
||||
foreach ($files as $path => $kilobytes) {
|
||||
// Livewire's own storage for an arriving upload: the file and its name, type and size beside it.
|
||||
$stored = FileUploadConfiguration::storeTemporaryFile(UploadedFile::fake()->create(basename($path), $kilobytes), 'tmp-for-tests');
|
||||
$page->script('(async () => { await '.$wire.'.registerFiles('.json_encode($selection).') })()');
|
||||
$page->waitForText('Selected Files ('.count($files).')');
|
||||
|
||||
$signed[] = TemporaryUploadedFile::signPath(basename($stored));
|
||||
$shareService = app(ShareService::class);
|
||||
|
||||
foreach (Share::query()->whereNull('completed_at')->latest('id')->firstOrFail()->files as $file) {
|
||||
$header = $shareService->readHeader($file);
|
||||
|
||||
for ($index = 0; $index < $header['chunkCount']; $index++) {
|
||||
$length = min($header['chunkSize'], $file->file_size - $index * $header['chunkSize']);
|
||||
$shareService->storeChunk($file->refresh(), $index, encryptedChunk($file, str_repeat("\0", $length), $index, $index === $header['chunkCount'] - 1));
|
||||
}
|
||||
}
|
||||
|
||||
$relativePaths = array_map(fn (string $path): ?string => str_contains($path, '/') ? $path : null, array_keys($files));
|
||||
|
||||
$page->script('(() => { const wire = Livewire.find(document.querySelector("[data-test=drop-zone]").closest("[wire\\\\:id]").getAttribute("wire:id")); wire.$set("relativePaths", '.json_encode($relativePaths).', false); wire._finishUpload("files", '.json_encode($signed).', true) })()');
|
||||
|
||||
$page->script('(async () => { await '.$wire.'.$refresh() })()');
|
||||
$page->assertSee('Selected Files ('.count($files).')');
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user