A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.
Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.
- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
ciphertext and tag per chunk. Each nonce holds the chunk index and a
last-chunk flag (the STREAM construction), so cut or reordered files
fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
wrapped with Argon2id (sodium, libsodium's interactive limits) in
shares.wrapped_key, which names its parameters. Password shares from
before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
pending share of its own, lists the files with their progress, retries
a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
abort. UploadChunkController only accepts chunks from the session that
started the share: a repeat is acknowledged, a skip gets 409 with the
count stored. Chunks go out as Blobs, which Chromium sends about eight
times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
needed and takes no files. The Docker image gains AUTO_HTTPS, which
serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
ZIP64) instead of decrypting whole files into memory and writing the
archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
endpoint's answers, completing a share, the streamed ZIP, cleanup,
and in Chromium a real chunked upload and the HTTPS warning; the
selected-files overflow test runs again. README, website, CHANGELOG
and .ai/rules follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
318 lines
12 KiB
PHP
318 lines
12 KiB
PHP
<?php
|
|
|
|
use App\Livewire\FileUploader;
|
|
use App\Livewire\SystemPasswordPrompt;
|
|
use App\Models\Setting;
|
|
use App\Models\Share;
|
|
use App\Models\ShareFile;
|
|
use App\Services\ShareService;
|
|
use Illuminate\Support\Facades\Crypt;
|
|
use Illuminate\Support\Facades\Storage;
|
|
use Livewire\Features\SupportTesting\Testable;
|
|
use Livewire\Livewire;
|
|
|
|
/**
|
|
* Upload files through the page as the browser does: register them in one batch, then store each
|
|
* one's encrypted content (the chunk endpoint itself is covered by UploadChunkTest).
|
|
*
|
|
* @param array<string, string> $files name => content
|
|
* @return array<int, array<string, mixed>|null> what the page handed the browser
|
|
*/
|
|
function uploadThroughPage(Testable $component, array $files): array
|
|
{
|
|
$targets = [];
|
|
|
|
$component->call('registerFiles', collect($files)->map(fn (string $content, string $name): array => ['name' => $name, 'size' => strlen($content), 'path' => null])->values()->all())
|
|
->assertReturned(function (array $returned) use (&$targets): bool {
|
|
$targets = $returned;
|
|
|
|
return true;
|
|
});
|
|
|
|
foreach (array_values($files) as $position => $content) {
|
|
if ($targets[$position] !== null) {
|
|
$file = ShareFile::query()->findOrFail($targets[$position]['id']);
|
|
app(ShareService::class)->storeChunk($file, 0, encryptedChunk($file, $content, 0, true));
|
|
}
|
|
}
|
|
|
|
return $targets;
|
|
}
|
|
|
|
test('upload page can be rendered', function () {
|
|
$response = $this->get(route('upload'));
|
|
|
|
$response->assertOk();
|
|
});
|
|
|
|
test('upload page requires system password when configured', function () {
|
|
Setting::set('system_password', bcrypt('system-secret'));
|
|
|
|
$response = $this->get(route('upload'));
|
|
|
|
$response->assertRedirect(route('system-password'));
|
|
});
|
|
|
|
test('upload page accessible after system password verified', function () {
|
|
Setting::set('system_password', bcrypt('system-secret'));
|
|
|
|
$response = $this->withSession(['system_password_verified' => true])
|
|
->get(route('upload'));
|
|
|
|
$response->assertOk();
|
|
});
|
|
|
|
test('file upload creates share', function () {
|
|
Storage::fake('shares');
|
|
$component = Livewire::test(FileUploader::class);
|
|
uploadThroughPage($component, ['document.pdf' => 'the document']);
|
|
|
|
$component->call('createShare')
|
|
->assertRedirectContains('/share/');
|
|
|
|
$share = Share::query()->sole();
|
|
expect($share->isCompleted())->toBeTrue();
|
|
expect($share->files->pluck('original_name')->all())->toBe(['document.pdf']);
|
|
expect(session('pending_shares'))->not->toContain($share->token);
|
|
});
|
|
|
|
test('registering files hands the browser each file\'s chunk URL, the share key and the file\'s nonce prefix', function () {
|
|
Storage::fake('shares');
|
|
config(['uploads.chunk_size' => 4]);
|
|
$component = Livewire::test(FileUploader::class);
|
|
|
|
$targets = uploadThroughPage($component, ['a.txt' => 'abc']);
|
|
|
|
$file = ShareFile::query()->sole();
|
|
expect($targets[0])->toMatchArray([
|
|
'id' => $file->id,
|
|
'url' => url('upload/files/'.$file->id.'/chunks'),
|
|
'key' => $file->share->encryption_key,
|
|
'chunkSize' => 4,
|
|
'chunkCount' => 1,
|
|
]);
|
|
expect($targets[0]['noncePrefix'])->toBe(bin2hex(substr(file_get_contents(app(ShareService::class)->storedFilePath($file)), 12, 7)));
|
|
expect(session('pending_shares'))->toBe([$file->share->token]);
|
|
$component->assertSet('pendingToken', $file->share->token)
|
|
->assertSeeHtml('data-test="selected-file"');
|
|
});
|
|
|
|
test('each upload page gets its own pending share', function () {
|
|
Storage::fake('shares');
|
|
$first = Livewire::test(FileUploader::class);
|
|
$second = Livewire::test(FileUploader::class);
|
|
|
|
uploadThroughPage($first, ['one.txt' => 'one']);
|
|
uploadThroughPage($second, ['two.txt' => 'two']);
|
|
|
|
expect($first->get('pendingToken'))->not->toBe($second->get('pendingToken'));
|
|
expect(Share::query()->pluck('total_size')->all())->toBe([3, 3]);
|
|
});
|
|
|
|
test('a file an admin limit refuses gets no target and shows why, while the rest of the batch is registered', function () {
|
|
Storage::fake('shares');
|
|
Setting::set('max_file_size', 1024 * 1024);
|
|
$component = Livewire::test(FileUploader::class);
|
|
|
|
$component->call('registerFiles', [
|
|
['name' => 'small.txt', 'size' => 3, 'path' => null],
|
|
['name' => 'large.txt', 'size' => 2 * 1024 * 1024, 'path' => null],
|
|
]);
|
|
|
|
$component->assertReturned(fn (array $targets): bool => $targets[0] !== null && $targets[1] === null);
|
|
expect($component->errors()->first('files'))->toBe('"large.txt" is too large (2 MB). Maximum file size is 1 MB.');
|
|
expect(ShareFile::query()->pluck('original_name')->all())->toBe(['small.txt']);
|
|
});
|
|
|
|
test('removing files takes them out of the pending share', function () {
|
|
Storage::fake('shares');
|
|
$component = Livewire::test(FileUploader::class);
|
|
$targets = uploadThroughPage($component, ['keep.txt' => 'keep', 'remove.txt' => 'remove']);
|
|
|
|
$component->call('removeFiles', [$targets[1]['id']]);
|
|
|
|
expect(ShareFile::query()->pluck('original_name')->all())->toBe(['keep.txt']);
|
|
expect(Share::query()->sole()->total_size)->toBe(4);
|
|
});
|
|
|
|
test('a share cannot be created while a file is still uploading', function () {
|
|
Storage::fake('shares');
|
|
$component = Livewire::test(FileUploader::class)
|
|
->call('registerFiles', [['name' => 'unfinished.txt', 'size' => 10, 'path' => null]]);
|
|
|
|
$component->call('createShare');
|
|
|
|
expect($component->errors()->first('files'))->toBe('Wait until every file has finished uploading, or remove the ones that failed.');
|
|
expect(Share::query()->sole()->isCompleted())->toBeFalse();
|
|
});
|
|
|
|
test('the page offers a warning for browsers without a secure context', function () {
|
|
Livewire::test(FileUploader::class)
|
|
->assertSeeHtml('data-test="insecure-context"')
|
|
->assertSee('Uploads need a secure connection (HTTPS).');
|
|
});
|
|
|
|
test('file upload with password creates password-protected share', function () {
|
|
Storage::fake('shares');
|
|
$component = Livewire::test(FileUploader::class);
|
|
uploadThroughPage($component, ['secret.txt' => 'secret']);
|
|
|
|
$component
|
|
->set('usePassword', true)
|
|
->set('password', 'my-password')
|
|
->call('createShare')
|
|
->assertRedirectContains('/share/');
|
|
|
|
$share = Share::query()->first();
|
|
expect($share->isPasswordProtected())->toBeTrue();
|
|
});
|
|
|
|
test('switching password protection on leaves the field empty and offers a Generate button by default', function () {
|
|
$component = Livewire::test(FileUploader::class)
|
|
->set('usePassword', true);
|
|
|
|
$component->assertSet('password', '')
|
|
->assertSeeHtml('data-test="generate-password"')
|
|
->assertSeeHtml('data-test="copy-password"');
|
|
});
|
|
|
|
test('a generated password protects the share and is flashed, encrypted, for the page the upload leads to', function () {
|
|
Storage::fake('shares');
|
|
|
|
$component = Livewire::test(FileUploader::class);
|
|
uploadThroughPage($component, ['secret.txt' => 'secret']);
|
|
$component->set('usePassword', true)
|
|
->call('generatePassword');
|
|
$password = $component->get('password');
|
|
$component->call('createShare');
|
|
|
|
$share = Share::query()->first();
|
|
expect($password)->toMatch('/^[A-Za-z0-9]{20}$/');
|
|
expect(app(ShareService::class)->verifyPassword($share, $password))->toBeTrue();
|
|
expect(session('share_password.token'))->toBe($share->token);
|
|
expect(Crypt::decryptString(session('share_password.password')))->toBe($password);
|
|
});
|
|
|
|
test('a share without a password flashes no password', function () {
|
|
Storage::fake('shares');
|
|
|
|
$component = Livewire::test(FileUploader::class);
|
|
uploadThroughPage($component, ['document.pdf' => 'the document']);
|
|
|
|
$component->call('createShare')
|
|
->assertRedirectContains('/share/');
|
|
|
|
expect(session()->has('share_password'))->toBeFalse();
|
|
});
|
|
|
|
test('a prefilling generator fills in a password as protection is switched on', function () {
|
|
Setting::set('password_generator_mode', 'prefill');
|
|
|
|
$component = Livewire::test(FileUploader::class)
|
|
->set('usePassword', true);
|
|
|
|
expect($component->get('password'))->toMatch('/^[A-Za-z0-9]{20}$/');
|
|
});
|
|
|
|
test('a prefilling generator keeps a password the uploader already typed', function () {
|
|
Setting::set('password_generator_mode', 'prefill');
|
|
|
|
Livewire::test(FileUploader::class)
|
|
->set('password', 'my-own-password')
|
|
->set('usePassword', true)
|
|
->assertSet('password', 'my-own-password');
|
|
});
|
|
|
|
test('a generator switched off offers no Generate button and generates nothing', function () {
|
|
Setting::set('password_generator_mode', 'off');
|
|
|
|
Livewire::test(FileUploader::class)
|
|
->set('usePassword', true)
|
|
->assertDontSeeHtml('data-test="generate-password"')
|
|
->assertSeeHtml('data-test="copy-password"')
|
|
->call('generatePassword')
|
|
->assertSet('password', '');
|
|
});
|
|
|
|
test('file upload with expiration sets expires_at', function () {
|
|
Storage::fake('shares');
|
|
|
|
$component = Livewire::test(FileUploader::class);
|
|
uploadThroughPage($component, ['file.txt' => 'content']);
|
|
|
|
$component
|
|
->set('expiration', '24h')
|
|
->call('createShare')
|
|
->assertRedirectContains('/share/');
|
|
|
|
$share = Share::query()->first();
|
|
expect($share->expires_at)->not->toBeNull();
|
|
});
|
|
|
|
test('file upload with max downloads sets limit', function () {
|
|
Storage::fake('shares');
|
|
|
|
$component = Livewire::test(FileUploader::class);
|
|
uploadThroughPage($component, ['file.txt' => 'content']);
|
|
|
|
$component
|
|
->set('maxDownloads', 5)
|
|
->call('createShare')
|
|
->assertRedirectContains('/share/');
|
|
|
|
$share = Share::query()->first();
|
|
expect($share->max_downloads)->toBe(5);
|
|
});
|
|
|
|
test('a file of 6 GB is accepted when the admin limits allow it', function () {
|
|
Storage::fake('shares');
|
|
Setting::set('max_file_size', 15000 * 1024 * 1024);
|
|
Setting::set('max_size_per_share', 20 * 1024 * 1024 * 1024);
|
|
Setting::set('max_storage_quota', 50 * 1024 * 1024 * 1024);
|
|
|
|
$component = Livewire::test(FileUploader::class)
|
|
->call('registerFiles', [['name' => 'backup.dump', 'size' => 6 * 1024 * 1024 * 1024, 'path' => null]]);
|
|
|
|
$component->assertHasNoErrors('files');
|
|
expect(Share::query()->sole()->total_size)->toBe(6 * 1024 * 1024 * 1024);
|
|
expect(ShareFile::query()->sole()->file_size)->toBe(6 * 1024 * 1024 * 1024);
|
|
});
|
|
|
|
test('file upload requires at least one file', function () {
|
|
$component = Livewire::test(FileUploader::class)
|
|
->call('createShare');
|
|
|
|
expect($component->errors()->first('files'))->toBe('Please select at least one file to upload.');
|
|
expect(Share::query()->count())->toBe(0);
|
|
});
|
|
|
|
test('file upload blocks when storage is full', function () {
|
|
Storage::fake('shares');
|
|
Setting::set('max_storage_quota', 100);
|
|
Share::factory()->create(['total_size' => 100]);
|
|
|
|
$component = Livewire::test(FileUploader::class)
|
|
->call('registerFiles', [['name' => 'file.txt', 'size' => 1, 'path' => null]]);
|
|
|
|
expect($component->errors()->first('files'))->toBe('Storage is full. Please contact the administrator.');
|
|
expect(ShareFile::query()->count())->toBe(0);
|
|
});
|
|
|
|
test('system password prompt verifies correct password', function () {
|
|
Setting::set('system_password', bcrypt('system-secret'));
|
|
|
|
Livewire::test(SystemPasswordPrompt::class)
|
|
->set('password', 'system-secret')
|
|
->call('verify')
|
|
->assertRedirect(route('upload'));
|
|
});
|
|
|
|
test('system password prompt rejects incorrect password', function () {
|
|
Setting::set('system_password', bcrypt('system-secret'));
|
|
|
|
Livewire::test(SystemPasswordPrompt::class)
|
|
->set('password', 'wrong')
|
|
->call('verify')
|
|
->assertHasErrors(['password']);
|
|
});
|