noNameWEB/SealShare
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Encrypt uploads in the browser and send them in chunks

Browse Source 
A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.

Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.

- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
  ciphertext and tag per chunk. Each nonce holds the chunk index and a
  last-chunk flag (the STREAM construction), so cut or reordered files
  fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
  wrapped with Argon2id (sodium, libsodium's interactive limits) in
  shares.wrapped_key, which names its parameters. Password shares from
  before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
  pending share of its own, lists the files with their progress, retries
  a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
  abort. UploadChunkController only accepts chunks from the session that
  started the share: a repeat is acknowledged, a skip gets 409 with the
  count stored. Chunks go out as Blobs, which Chromium sends about eight
  times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
  needed and takes no files. The Docker image gains AUTO_HTTPS, which
  serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
  it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
  never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
  ZIP64) instead of decrypting whole files into memory and writing the
  archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
  dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
  for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
  default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
  UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
  endpoint's answers, completing a share, the streamed ZIP, cleanup,
  and in Chromium a real chunked upload and the HTTPS warning; the
  selected-files overflow test runs again. README, website, CHANGELOG
  and .ai/rules follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Reinhold / reini
2026-09-16 20:49:17 +02:00
co-authored by Claude Opus 5
parent 504971ad7f
commit 40e35bab0e
53 changed files with 2280 additions and 946 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Unit/FileEncryptionServiceTest.php
+133 -83
View File
@@ -1,7 +1,6 @@
<?php
use App\Services\FileEncryptionService;
use Symfony\Component\HttpFoundation\StreamedResponse;
beforeEach(function () {
$this->service = new FileEncryptionService;
@@ -16,6 +15,27 @@ afterEach(function () {
}
});
/**
* The whole decrypted content of an encrypted file.
*/
function decryptToString(FileEncryptionService $service, string $path, string $key): string
{
return implode('', iterator_to_array($service->decryptedChunks($path, $key), false));
}
/**
* A chunk encrypted the way the uploader's browser does, built here without the service: the
* nonce is prefix, index and last-chunk flag; the tag follows the ciphertext.
*/
function browserChunk(string $plaintext, string $keyHex, string $noncePrefix, int $index, bool $isLast): string
{
$tag = '';
$nonce = $noncePrefix.pack('N', $index).($isLast ? "\x01" : "\x00");
$ciphertext = openssl_encrypt($plaintext, 'aes-256-gcm', hex2bin($keyHex), OPENSSL_RAW_DATA, $nonce, $tag, '', 16);
return $ciphertext.$tag;
}
test('encrypt and decrypt round-trip works', function () {
$sourcePath = $this->tempDir.'/source.txt';
$encryptedPath = $this->tempDir.'/encrypted.enc';
@@ -25,14 +45,10 @@ test('encrypt and decrypt round-trip works', function () {
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$this->service->encryptFile($sourcePath, $encryptedPath, $key, 1024);
expect(file_exists($encryptedPath))->toBeTrue();
expect(file_get_contents($encryptedPath))->not->toBe($content);
$decrypted = $this->service->decryptFile($encryptedPath, $key);
expect($decrypted)->toBe($content);
expect(file_get_contents($encryptedPath))->not->toContain($content);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe($content);
});
test('decrypt with wrong key fails', function () {
@@ -41,12 +57,9 @@ test('decrypt with wrong key fails', function () {
file_put_contents($sourcePath, 'Secret data');
$correctKey = $this->service->generateRandomKey();
$wrongKey = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $this->service->generateRandomKey(), 1024);
$this->service->encryptFile($sourcePath, $encryptedPath, $correctKey);
$this->service->decryptFile($encryptedPath, $wrongKey);
decryptToString($this->service, $encryptedPath, $this->service->generateRandomKey());
})->throws(RuntimeException::class, 'Decryption failed');
test('derive key produces consistent results', function () {
@@ -98,78 +111,50 @@ test('password-derived key encrypt/decrypt round-trip works', function () {
file_put_contents($sourcePath, $content);
$password = 'user-password';
$salt = $this->service->generateSalt();
$key = bin2hex($this->service->deriveKey($password, $salt));
$key = bin2hex($this->service->deriveKey('user-password', $this->service->generateSalt()));
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$decrypted = $this->service->decryptFile($encryptedPath, $key);
$this->service->encryptFile($sourcePath, $encryptedPath, $key, 1024);
expect($decrypted)->toBe($content);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe($content);
});
test('decrypt file stream returns streamed response', function () {
$sourcePath = $this->tempDir.'/source.txt';
$encryptedPath = $this->tempDir.'/encrypted.enc';
$content = 'Streamed content';
file_put_contents($sourcePath, $content);
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$response = $this->service->decryptFileStream($encryptedPath, $key, 'test.txt', 'text/plain');
expect($response)->toBeInstanceOf(StreamedResponse::class);
expect($response->headers->get('Content-Type'))->toBe('text/plain');
expect($response->headers->get('Content-Disposition'))->toContain('test.txt');
});
test('chunked file has SEALCHK1 magic header', function () {
test('an encrypted file starts with the SEALCHK2 header and its chunk size', function () {
$sourcePath = $this->tempDir.'/source.txt';
$encryptedPath = $this->tempDir.'/encrypted.enc';
file_put_contents($sourcePath, 'test content');
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$this->service->encryptFile($sourcePath, $encryptedPath, $this->service->generateRandomKey(), 1024);
$header = file_get_contents($encryptedPath, false, null, 0, 8);
expect($header)->toBe('SEALCHK1');
expect(file_get_contents($encryptedPath, false, null, 0, 12))->toBe('SEALCHK2'.pack('N', 1024));
});
test('multi-chunk round-trip works', function () {
$sourcePath = $this->tempDir.'/large.bin';
$encryptedPath = $this->tempDir.'/large.enc';
// Create a file larger than one 4 MB chunk (5 MB)
$chunkSize = 4 * 1024 * 1024;
$content = random_bytes($chunkSize + (1024 * 1024));
$content = random_bytes(2500);
file_put_contents($sourcePath, $content);
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$decrypted = $this->service->decryptFile($encryptedPath, $key);
$this->service->encryptFile($sourcePath, $encryptedPath, $key, 1000);
expect($decrypted)->toBe($content);
expect(filesize($encryptedPath))->toBe(19 + 3 * 16 + 2500);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe($content);
});
test('exact chunk boundary round-trip works', function () {
$sourcePath = $this->tempDir.'/exact.bin';
$encryptedPath = $this->tempDir.'/exact.enc';
// Create a file exactly equal to one chunk (4 MB)
$content = random_bytes(4 * 1024 * 1024);
$content = random_bytes(2000);
file_put_contents($sourcePath, $content);
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$decrypted = $this->service->decryptFile($encryptedPath, $key);
$this->service->encryptFile($sourcePath, $encryptedPath, $key, 1000);
expect($decrypted)->toBe($content);
expect(filesize($encryptedPath))->toBe(19 + 2 * 16 + 2000);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe($content);
});
test('empty file round-trip works', function () {
@@ -179,58 +164,123 @@ test('empty file round-trip works', function () {
file_put_contents($sourcePath, '');
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$decrypted = $this->service->decryptFile($encryptedPath, $key);
$this->service->encryptFile($sourcePath, $encryptedPath, $key, 1000);
expect($decrypted)->toBe('');
expect(filesize($encryptedPath))->toBe(19 + 16);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe('');
});
test('chunks encrypted the way the browser does decrypt to the original file', function () {
$key = $this->service->generateRandomKey();
$header = $this->service->createHeader(4);
$noncePrefix = substr($header, 12, 7);
$encryptedPath = $this->tempDir.'/browser.enc';
file_put_contents($encryptedPath, $header
.browserChunk('abcd', $key, $noncePrefix, 0, false)
.browserChunk('ef', $key, $noncePrefix, 1, true));
expect(decryptToString($this->service, $encryptedPath, $key))->toBe('abcdef');
});
test('a chunk with the wrong last-chunk flag is rejected', function () {
$key = $this->service->generateRandomKey();
$noncePrefix = random_bytes(7);
$this->service->decryptChunk(browserChunk('abcd', $key, $noncePrefix, 0, false), $key, $noncePrefix, 0, true);
})->throws(RuntimeException::class, 'Decryption failed');
test('a file cut short at a chunk boundary fails to decrypt', function () {
$sourcePath = $this->tempDir.'/source.bin';
$encryptedPath = $this->tempDir.'/truncated.enc';
file_put_contents($sourcePath, random_bytes(3000));
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key, 1000);
$handle = fopen($encryptedPath, 'r+b');
ftruncate($handle, 19 + 2 * (1000 + 16));
fclose($handle);
decryptToString($this->service, $encryptedPath, $key);
})->throws(RuntimeException::class, 'Decryption failed');
test('a file with two chunks swapped fails to decrypt', function () {
$key = $this->service->generateRandomKey();
$header = $this->service->createHeader(4);
$noncePrefix = substr($header, 12, 7);
$encryptedPath = $this->tempDir.'/swapped.enc';
file_put_contents($encryptedPath, $header
.browserChunk('efgh', $key, $noncePrefix, 1, false)
.browserChunk('abcd', $key, $noncePrefix, 0, false)
.browserChunk('ij', $key, $noncePrefix, 2, true));
decryptToString($this->service, $encryptedPath, $key);
})->throws(RuntimeException::class, 'Decryption failed');
test('SEALCHK1 files from before still decrypt', function () {
$key = $this->service->generateRandomKey();
$encryptedPath = $this->tempDir.'/sealchk1.enc';
$baseNonce = random_bytes(12);
$file = 'SEALCHK1'.pack('N', 4).$baseNonce;
foreach (['abcd', 'ef'] as $index => $plaintext) {
$nonce = $baseNonce;
$indexBytes = pack('N', $index);
for ($i = 0; $i < 4; $i++) {
$nonce[8 + $i] = $nonce[8 + $i] ^ $indexBytes[$i];
}
$tag = '';
$ciphertext = openssl_encrypt($plaintext, 'aes-256-gcm', hex2bin($key), OPENSSL_RAW_DATA, $nonce, $tag, '', 16);
$file .= $tag.$ciphertext;
}
file_put_contents($encryptedPath, $file);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe('abcdef');
});
test('legacy format backward compatibility', function () {
$sourcePath = $this->tempDir.'/source.txt';
$encryptedPath = $this->tempDir.'/legacy.enc';
$content = 'Legacy encrypted content';
file_put_contents($sourcePath, $content);
$key = $this->service->generateRandomKey();
$binaryKey = hex2bin($key);
// Manually create a legacy format file: [nonce][tag][ciphertext]
$nonce = random_bytes(12);
$tag = '';
$ciphertext = openssl_encrypt($content, 'aes-256-gcm', $binaryKey, OPENSSL_RAW_DATA, $nonce, $tag, '', 16);
$ciphertext = openssl_encrypt($content, 'aes-256-gcm', hex2bin($key), OPENSSL_RAW_DATA, $nonce, $tag, '', 16);
file_put_contents($encryptedPath, $nonce.$tag.$ciphertext);
$decrypted = $this->service->decryptFile($encryptedPath, $key);
expect($decrypted)->toBe($content);
expect(decryptToString($this->service, $encryptedPath, $key))->toBe($content);
});
test('wrong key on chunked file throws exception', function () {
$sourcePath = $this->tempDir.'/source.txt';
$encryptedPath = $this->tempDir.'/encrypted.enc';
file_put_contents($sourcePath, 'Chunked secret data');
file_put_contents($sourcePath, random_bytes(2500));
$correctKey = $this->service->generateRandomKey();
$wrongKey = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $this->service->generateRandomKey(), 1000);
$this->service->encryptFile($sourcePath, $encryptedPath, $correctKey);
$this->service->decryptFile($encryptedPath, $wrongKey);
decryptToString($this->service, $encryptedPath, $this->service->generateRandomKey());
})->throws(RuntimeException::class, 'Decryption failed');
test('decrypt file stream with file size sets content-length header', function () {
$sourcePath = $this->tempDir.'/source.txt';
$encryptedPath = $this->tempDir.'/encrypted.enc';
$content = 'Content with known size';
test('a wrapped key unwraps with its password to the same data key', function () {
$dataKey = $this->service->generateRandomKey();
file_put_contents($sourcePath, $content);
$wrapped = $this->service->wrapKey($dataKey, 'correct horse battery');
$key = $this->service->generateRandomKey();
$this->service->encryptFile($sourcePath, $encryptedPath, $key);
$response = $this->service->decryptFileStream($encryptedPath, $key, 'test.txt', 'text/plain', strlen($content));
expect($response->headers->get('Content-Length'))->toBe((string) strlen($content));
expect($wrapped)->toStartWith('argon2id$')->not->toContain($dataKey);
expect($this->service->unwrapKey($wrapped, 'correct horse battery'))->toBe($dataKey);
});
test('a wrapped key does not unwrap with a wrong password', function () {
$wrapped = $this->service->wrapKey($this->service->generateRandomKey(), 'correct horse battery');
$this->service->unwrapKey($wrapped, 'wrong horse battery');
})->throws(RuntimeException::class, 'Unwrapping failed');