noNameWEB/SealShare
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Encrypt uploads in the browser and send them in chunks

Browse Source 
A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.

Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.

- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
  ciphertext and tag per chunk. Each nonce holds the chunk index and a
  last-chunk flag (the STREAM construction), so cut or reordered files
  fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
  wrapped with Argon2id (sodium, libsodium's interactive limits) in
  shares.wrapped_key, which names its parameters. Password shares from
  before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
  pending share of its own, lists the files with their progress, retries
  a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
  abort. UploadChunkController only accepts chunks from the session that
  started the share: a repeat is acknowledged, a skip gets 409 with the
  count stored. Chunks go out as Blobs, which Chromium sends about eight
  times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
  needed and takes no files. The Docker image gains AUTO_HTTPS, which
  serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
  it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
  never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
  ZIP64) instead of decrypting whole files into memory and writing the
  archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
  dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
  for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
  default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
  UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
  endpoint's answers, completing a share, the streamed ZIP, cleanup,
  and in Chromium a real chunked upload and the HTTPS warning; the
  selected-files overflow test runs again. README, website, CHANGELOG
  and .ai/rules follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Reinhold / reini
2026-09-16 20:49:17 +02:00
co-authored by Claude Opus 5
parent 504971ad7f
commit 40e35bab0e
53 changed files with 2280 additions and 946 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Feature/ShareDownloadTest.php
+56 -81
View File
@@ -2,6 +2,7 @@
use App\Livewire\ShareDownload;
use App\Models\Share;
use App\Models\ShareFile;
use App\Services\FileEncryptionService;
use App\Services\ShareService;
use Illuminate\Http\UploadedFile;
@@ -81,102 +82,79 @@ test('non-password share shows files directly', function () {
test('download counter increments on zip download', function () {
Storage::fake('shares');
$share = createShareWithFile();
$share->load('files');
$encryptionService = app(FileEncryptionService::class);
$key = $share->encryption_key;
$content = 'test content';
foreach ($share->files as $file) {
$dir = Storage::disk('shares')->path($share->token);
if (! is_dir($dir)) {
mkdir($dir, 0755, true);
}
$encryptedPath = $dir.'/'.basename($file->stored_path);
$tempSource = tempnam(sys_get_temp_dir(), 'test');
file_put_contents($tempSource, $content);
$encryptionService->encryptFile($tempSource, $encryptedPath, $key);
unlink($tempSource);
$file->update(['file_size' => strlen($content)]);
}
$response = $this->withSession(['share_password_'.$share->token => null])
->get(route('share.download.all', $share));
$response->assertDownload();
$response = $this->get(route('share.download.all', $share));
$response->streamedContent();
$response->assertDownload('share-'.$share->token.'.zip');
expect($share->fresh()->download_count)->toBe(1);
});
test('zip download produces a valid archive', function () {
test('zip download streams a valid archive with every file\'s original content', function () {
Storage::fake('shares');
config(['uploads.chunk_size' => 1000]);
$binary = random_bytes(2500);
$share = app(ShareService::class)->createShare([
['file' => UploadedFile::fake()->createWithContent('notes.txt', 'hello zip content'), 'relativePath' => null],
['file' => UploadedFile::fake()->createWithContent('photo.bin', $binary), 'relativePath' => 'holiday/photo.bin'],
]);
$zipPath = tempnam(sys_get_temp_dir(), 'zip');
$share = createShareWithFile();
$share->load('files');
$encryptionService = app(FileEncryptionService::class);
$key = $share->encryption_key;
$content = 'hello zip content';
foreach ($share->files as $file) {
$dir = Storage::disk('shares')->path($share->token);
if (! is_dir($dir)) {
mkdir($dir, 0755, true);
}
$encryptedPath = $dir.'/'.basename($file->stored_path);
$tempSource = tempnam(sys_get_temp_dir(), 'test');
file_put_contents($tempSource, $content);
$encryptionService->encryptFile($tempSource, $encryptedPath, $key);
unlink($tempSource);
$file->update(['file_size' => strlen($content)]);
}
$response = $this->get(route('share.download.all', $share));
$response->assertDownload();
$zipPath = $response->getFile()->getPathname();
file_put_contents($zipPath, $this->get(route('share.download.all', $share))->streamedContent());
$zip = new ZipArchive;
$result = $zip->open($zipPath);
expect($result)->toBe(true);
expect($zip->numFiles)->toBe(1);
expect($zip->statIndex(0)['size'])->toBe(strlen($content));
expect($zip->open($zipPath))->toBeTrue();
expect($zip->numFiles)->toBe(2);
expect($zip->getFromName('notes.txt'))->toBe('hello zip content');
expect($zip->getFromName('holiday/photo.bin'))->toBe($binary);
$zip->close();
unlink($zipPath);
});
test('last download streams successfully before auto-delete', function () {
Storage::fake('shares');
$share = createShareWithFile();
$share->update(['max_downloads' => 1]);
$share->load('files');
$encryptionService = app(FileEncryptionService::class);
$key = $share->encryption_key;
$content = 'last download content';
$content = $this->get(route('share.download.all', $share))->streamedContent();
foreach ($share->files as $file) {
$dir = Storage::disk('shares')->path($share->token);
if (! is_dir($dir)) {
mkdir($dir, 0755, true);
}
$encryptedPath = $dir.'/'.basename($file->stored_path);
$tempSource = tempnam(sys_get_temp_dir(), 'test');
file_put_contents($tempSource, $content);
$encryptionService->encryptFile($tempSource, $encryptedPath, $key);
unlink($tempSource);
$file->update(['file_size' => strlen($content)]);
}
expect($content)->toStartWith("PK\x03\x04");
$this->assertModelMissing($share);
});
$response = $this->get(route('share.download.all', $share));
$response->assertDownload();
test('a share whose files are still uploading is not found anywhere a recipient or uploader could open it', function (string $route) {
Storage::fake('shares');
$share = Share::factory()->pending()->create();
$file = ShareFile::factory()->for($share)->uploading()->create();
// Share was deleted after download
expect(Share::query()->find($share->id))->toBeNull();
$response = $this->get(route($route, ['share' => $share, 'shareFile' => $file]));
$response->assertNotFound();
})->with([
'download page' => 'share.download',
'download all' => 'share.download.all',
'download one file' => 'share.download.file',
'share created page' => 'share.created',
]);
test('a password share created before key wrapping still unlocks and downloads', function () {
Storage::fake('shares');
$salt = str_repeat('cd', 32);
$share = Share::factory()->withPassword('old-password')->create(['encryption_salt' => $salt]);
$file = ShareFile::factory()->for($share)->create(['stored_path' => 'shares/'.$share->token.'/old.enc', 'file_size' => 11]);
$source = tempnam(sys_get_temp_dir(), 'old');
file_put_contents($source, 'old content');
Storage::disk('shares')->makeDirectory($share->token);
app(FileEncryptionService::class)->encryptFile($source, Storage::disk('shares')->path($share->token.'/old.enc'), bin2hex(hash_pbkdf2('sha256', 'old-password', hex2bin($salt), 100000, 32, true)), 1024);
unlink($source);
Livewire::test(ShareDownload::class, ['share' => $share])
->set('password', 'old-password')
->call('verifyPassword')
->assertSet('authenticated', true);
expect($this->get(route('share.download.file', [$share, $file]))->streamedContent())->toBe('old content');
});
test('share auto-deletes after reaching download limit', function () {
@@ -199,13 +177,10 @@ test('share auto-deletes after reaching download limit', function () {
/**
* Helper to create a share with an actual encrypted file.
*/
function createShareWithFile(?string $password = null): Share
function createShareWithFile(?string $password = null, string $content = 'test content'): Share
{
$service = app(ShareService::class);
$file = UploadedFile::fake()->create('testfile.txt', 100);
return $service->createShare([
['file' => $file, 'relativePath' => null],
return app(ShareService::class)->createShare([
['file' => UploadedFile::fake()->createWithContent('testfile.txt', $content), 'relativePath' => null],
], [
'password' => $password,
]);