A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.
Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.
- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
ciphertext and tag per chunk. Each nonce holds the chunk index and a
last-chunk flag (the STREAM construction), so cut or reordered files
fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
wrapped with Argon2id (sodium, libsodium's interactive limits) in
shares.wrapped_key, which names its parameters. Password shares from
before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
pending share of its own, lists the files with their progress, retries
a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
abort. UploadChunkController only accepts chunks from the session that
started the share: a repeat is acknowledged, a skip gets 409 with the
count stored. Chunks go out as Blobs, which Chromium sends about eight
times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
needed and takes no files. The Docker image gains AUTO_HTTPS, which
serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
ZIP64) instead of decrypting whole files into memory and writing the
archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
endpoint's answers, completing a share, the streamed ZIP, cleanup,
and in Chromium a real chunked upload and the HTTPS warning; the
selected-files overflow test runs again. README, website, CHANGELOG
and .ai/rules follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
452 lines
14 KiB
PHP
452 lines
14 KiB
PHP
<?php
|
|
|
|
namespace App\Services;
|
|
|
|
use Generator;
|
|
use RuntimeException;
|
|
|
|
/**
|
|
* The encrypted file formats and the keys behind them.
|
|
*
|
|
* New files are `SEALCHK2`, written chunk by chunk as the uploader's browser sends them:
|
|
*
|
|
* [8 bytes: "SEALCHK2" magic]
|
|
* [4 bytes: chunk size S, uint32 big-endian]
|
|
* [7 bytes: random nonce prefix]
|
|
* Per chunk i: [ciphertext (S bytes, fewer on the last chunk)][16 bytes: GCM tag]
|
|
*
|
|
* Chunk i's nonce is the prefix, i as uint32 big-endian and a byte that is 1 on the last chunk
|
|
* and 0 on every other (the STREAM construction), so dropping, reordering or appending chunks
|
|
* fails authentication. The browser encrypts with the same layout (resources/js/share-uploader.js).
|
|
*
|
|
* `SEALCHK1` (a tag before each chunk, the index XORed into a 12-byte nonce, no last-chunk flag)
|
|
* and the single-block legacy format are still read for shares created before.
|
|
*/
|
|
class FileEncryptionService
|
|
{
|
|
public const HEADER_LENGTH = 19;
|
|
|
|
public const TAG_LENGTH = 16;
|
|
|
|
private const CIPHER = 'aes-256-gcm';
|
|
|
|
private const PBKDF2_ITERATIONS = 100000;
|
|
|
|
private const KEY_LENGTH = 32;
|
|
|
|
private const NONCE_LENGTH = 12;
|
|
|
|
private const NONCE_PREFIX_LENGTH = 7;
|
|
|
|
private const MAGIC = 'SEALCHK2';
|
|
|
|
private const LEGACY_CHUNKED_MAGIC = 'SEALCHK1';
|
|
|
|
private const WRAPPED_KEY_ALGORITHM = 'argon2id';
|
|
|
|
/**
|
|
* Derive a key from a password and salt using PBKDF2-SHA256, as shares created before
|
|
* envelope encryption were keyed.
|
|
*/
|
|
public function deriveKey(string $password, string $salt): string
|
|
{
|
|
return hash_pbkdf2('sha256', $password, hex2bin($salt), self::PBKDF2_ITERATIONS, self::KEY_LENGTH, true);
|
|
}
|
|
|
|
/**
|
|
* Generate a random hex salt (32 bytes = 64 hex chars).
|
|
*/
|
|
public function generateSalt(): string
|
|
{
|
|
return bin2hex(random_bytes(32));
|
|
}
|
|
|
|
/**
|
|
* Generate a random encryption key (32 bytes, returned as hex).
|
|
*/
|
|
public function generateRandomKey(): string
|
|
{
|
|
return bin2hex(random_bytes(self::KEY_LENGTH));
|
|
}
|
|
|
|
/**
|
|
* Wrap a share's data key with a key derived from its password (Argon2id).
|
|
*
|
|
* The result names its algorithm and parameters, so they can be raised later without breaking
|
|
* shares wrapped before: `argon2id$<opslimit>$<memlimit>$<salt>$<nonce>$<box>`, in hex.
|
|
*/
|
|
public function wrapKey(string $dataKeyHex, string $password): string
|
|
{
|
|
$salt = random_bytes(SODIUM_CRYPTO_PWHASH_SALTBYTES);
|
|
$opslimit = SODIUM_CRYPTO_PWHASH_OPSLIMIT_INTERACTIVE;
|
|
$memlimit = SODIUM_CRYPTO_PWHASH_MEMLIMIT_INTERACTIVE;
|
|
|
|
$wrappingKey = $this->deriveWrappingKey($password, $salt, $opslimit, $memlimit);
|
|
$nonce = random_bytes(SODIUM_CRYPTO_SECRETBOX_NONCEBYTES);
|
|
$box = sodium_crypto_secretbox(hex2bin($dataKeyHex), $nonce, $wrappingKey);
|
|
|
|
sodium_memzero($wrappingKey);
|
|
|
|
return implode('$', [self::WRAPPED_KEY_ALGORITHM, $opslimit, $memlimit, bin2hex($salt), bin2hex($nonce), bin2hex($box)]);
|
|
}
|
|
|
|
/**
|
|
* Unwrap a share's data key with its password; returns the key as hex.
|
|
*/
|
|
public function unwrapKey(string $wrappedKey, string $password): string
|
|
{
|
|
$parts = explode('$', $wrappedKey);
|
|
|
|
if (count($parts) !== 6 || $parts[0] !== self::WRAPPED_KEY_ALGORITHM) {
|
|
throw new RuntimeException('Unsupported wrapped key');
|
|
}
|
|
|
|
[, $opslimit, $memlimit, $salt, $nonce, $box] = $parts;
|
|
|
|
$wrappingKey = $this->deriveWrappingKey($password, hex2bin($salt), (int) $opslimit, (int) $memlimit);
|
|
$dataKey = sodium_crypto_secretbox_open(hex2bin($box), hex2bin($nonce), $wrappingKey);
|
|
|
|
sodium_memzero($wrappingKey);
|
|
|
|
if ($dataKey === false) {
|
|
throw new RuntimeException('Unwrapping failed - wrong password or corrupted key');
|
|
}
|
|
|
|
return bin2hex($dataKey);
|
|
}
|
|
|
|
/**
|
|
* The header a new encrypted file starts with, with a fresh random nonce prefix.
|
|
*/
|
|
public function createHeader(int $chunkSize): string
|
|
{
|
|
return self::MAGIC.pack('N', $chunkSize).random_bytes(self::NONCE_PREFIX_LENGTH);
|
|
}
|
|
|
|
/**
|
|
* Read a `SEALCHK2` header.
|
|
*
|
|
* @return array{chunkSize: int, noncePrefix: string}
|
|
*/
|
|
public function parseHeader(string $header): array
|
|
{
|
|
if (strlen($header) !== self::HEADER_LENGTH || ! str_starts_with($header, self::MAGIC)) {
|
|
throw new RuntimeException('Invalid encrypted file header');
|
|
}
|
|
|
|
return [
|
|
'chunkSize' => unpack('N', substr($header, 8, 4))[1],
|
|
'noncePrefix' => substr($header, 12, self::NONCE_PREFIX_LENGTH),
|
|
];
|
|
}
|
|
|
|
/**
|
|
* How many chunks a file of this size is sent in; an empty file is one empty chunk.
|
|
*/
|
|
public function chunkCount(int $size, int $chunkSize): int
|
|
{
|
|
return max(1, intdiv($size + $chunkSize - 1, $chunkSize));
|
|
}
|
|
|
|
/**
|
|
* Where chunk `$index` starts in the encrypted file.
|
|
*/
|
|
public function chunkOffset(int $index, int $chunkSize): int
|
|
{
|
|
return self::HEADER_LENGTH + $index * ($chunkSize + self::TAG_LENGTH);
|
|
}
|
|
|
|
/**
|
|
* Encrypt one chunk: its ciphertext followed by its tag, as WebCrypto returns it.
|
|
*/
|
|
public function encryptChunk(string $plaintext, string $key, string $noncePrefix, int $index, bool $isLast): string
|
|
{
|
|
$tag = '';
|
|
|
|
$ciphertext = openssl_encrypt(
|
|
$plaintext,
|
|
self::CIPHER,
|
|
$this->normalizeToBinaryKey($key),
|
|
OPENSSL_RAW_DATA,
|
|
$this->chunkNonce($noncePrefix, $index, $isLast),
|
|
$tag,
|
|
'',
|
|
self::TAG_LENGTH,
|
|
);
|
|
|
|
if ($ciphertext === false) {
|
|
throw new RuntimeException('Encryption failed at chunk '.$index);
|
|
}
|
|
|
|
return $ciphertext.$tag;
|
|
}
|
|
|
|
/**
|
|
* Decrypt one chunk, which fails unless its index and last-chunk flag are the ones it was
|
|
* encrypted with.
|
|
*/
|
|
public function decryptChunk(string $chunk, string $key, string $noncePrefix, int $index, bool $isLast): string
|
|
{
|
|
if (strlen($chunk) < self::TAG_LENGTH) {
|
|
throw new RuntimeException('Invalid encrypted file: truncated chunk '.$index);
|
|
}
|
|
|
|
$plaintext = openssl_decrypt(
|
|
substr($chunk, 0, -self::TAG_LENGTH),
|
|
self::CIPHER,
|
|
$this->normalizeToBinaryKey($key),
|
|
OPENSSL_RAW_DATA,
|
|
$this->chunkNonce($noncePrefix, $index, $isLast),
|
|
substr($chunk, -self::TAG_LENGTH),
|
|
);
|
|
|
|
if ($plaintext === false) {
|
|
throw new RuntimeException('Decryption failed - wrong key or corrupted data');
|
|
}
|
|
|
|
return $plaintext;
|
|
}
|
|
|
|
/**
|
|
* Encrypt a file on the server in the `SEALCHK2` format.
|
|
*/
|
|
public function encryptFile(string $sourcePath, string $destPath, string $key, int $chunkSize): void
|
|
{
|
|
$source = fopen($sourcePath, 'rb');
|
|
|
|
if ($source === false) {
|
|
throw new RuntimeException("Cannot read source file: {$sourcePath}");
|
|
}
|
|
|
|
$dest = fopen($destPath, 'wb');
|
|
|
|
if ($dest === false) {
|
|
fclose($source);
|
|
|
|
throw new RuntimeException("Cannot write encrypted file: {$destPath}");
|
|
}
|
|
|
|
try {
|
|
$header = $this->createHeader($chunkSize);
|
|
$noncePrefix = $this->parseHeader($header)['noncePrefix'];
|
|
$chunkCount = $this->chunkCount((int) filesize($sourcePath), $chunkSize);
|
|
|
|
fwrite($dest, $header);
|
|
|
|
for ($index = 0; $index < $chunkCount; $index++) {
|
|
$plaintext = (string) fread($source, $chunkSize);
|
|
|
|
fwrite($dest, $this->encryptChunk($plaintext, $key, $noncePrefix, $index, $index === $chunkCount - 1));
|
|
}
|
|
} catch (RuntimeException $e) {
|
|
fclose($source);
|
|
fclose($dest);
|
|
@unlink($destPath);
|
|
|
|
throw $e;
|
|
}
|
|
|
|
fclose($source);
|
|
fclose($dest);
|
|
}
|
|
|
|
/**
|
|
* Stream decrypted file content directly to output (echo).
|
|
*/
|
|
public function streamDecryptedFile(string $encryptedPath, string $key): void
|
|
{
|
|
foreach ($this->decryptedChunks($encryptedPath, $key) as $chunk) {
|
|
echo $chunk;
|
|
flush();
|
|
}
|
|
}
|
|
|
|
/**
|
|
* The decrypted content of a file in any of the three formats, chunk by chunk.
|
|
*
|
|
* @return Generator<int, string>
|
|
*/
|
|
public function decryptedChunks(string $encryptedPath, string $key): Generator
|
|
{
|
|
$magic = (string) file_get_contents($encryptedPath, false, null, 0, 8);
|
|
|
|
if ($magic === self::MAGIC) {
|
|
yield from $this->decryptChunks($encryptedPath, $key);
|
|
} elseif ($magic === self::LEGACY_CHUNKED_MAGIC) {
|
|
yield from $this->decryptLegacyChunks($encryptedPath, $key);
|
|
} else {
|
|
yield $this->decryptLegacy($encryptedPath, $key);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Normalize a hex key to binary.
|
|
*/
|
|
private function normalizeToBinaryKey(string $key): string
|
|
{
|
|
return strlen($key) === 64 ? hex2bin($key) : $key;
|
|
}
|
|
|
|
private function deriveWrappingKey(string $password, string $salt, int $opslimit, int $memlimit): string
|
|
{
|
|
return sodium_crypto_pwhash(
|
|
SODIUM_CRYPTO_SECRETBOX_KEYBYTES,
|
|
$password,
|
|
$salt,
|
|
$opslimit,
|
|
$memlimit,
|
|
SODIUM_CRYPTO_PWHASH_ALG_ARGON2ID13,
|
|
);
|
|
}
|
|
|
|
/**
|
|
* A `SEALCHK2` chunk's nonce: the file's prefix, the chunk index and the last-chunk flag.
|
|
*/
|
|
private function chunkNonce(string $noncePrefix, int $index, bool $isLast): string
|
|
{
|
|
return $noncePrefix.pack('N', $index).($isLast ? "\x01" : "\x00");
|
|
}
|
|
|
|
/**
|
|
* Decrypt a `SEALCHK2` file; the chunk count comes from the file's length, so a file cut short
|
|
* at a chunk boundary fails on its new last chunk.
|
|
*
|
|
* @return Generator<int, string>
|
|
*/
|
|
private function decryptChunks(string $encryptedPath, string $key): Generator
|
|
{
|
|
$handle = fopen($encryptedPath, 'rb');
|
|
|
|
if ($handle === false) {
|
|
throw new RuntimeException("Cannot read encrypted file: {$encryptedPath}");
|
|
}
|
|
|
|
try {
|
|
['chunkSize' => $chunkSize, 'noncePrefix' => $noncePrefix] = $this->parseHeader((string) fread($handle, self::HEADER_LENGTH));
|
|
|
|
$storedChunkSize = $chunkSize + self::TAG_LENGTH;
|
|
$payloadLength = (int) filesize($encryptedPath) - self::HEADER_LENGTH;
|
|
$chunkCount = intdiv($payloadLength + $storedChunkSize - 1, $storedChunkSize);
|
|
|
|
if ($chunkCount === 0) {
|
|
throw new RuntimeException('Invalid encrypted file: no chunks');
|
|
}
|
|
|
|
for ($index = 0; $index < $chunkCount; $index++) {
|
|
$chunk = (string) fread($handle, $storedChunkSize);
|
|
|
|
yield $this->decryptChunk($chunk, $key, $noncePrefix, $index, $index === $chunkCount - 1);
|
|
}
|
|
} finally {
|
|
fclose($handle);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Decrypt a `SEALCHK1` file.
|
|
*
|
|
* @return Generator<int, string>
|
|
*/
|
|
private function decryptLegacyChunks(string $encryptedPath, string $key): Generator
|
|
{
|
|
$handle = fopen($encryptedPath, 'rb');
|
|
|
|
if ($handle === false) {
|
|
throw new RuntimeException("Cannot read encrypted file: {$encryptedPath}");
|
|
}
|
|
|
|
try {
|
|
fread($handle, 8);
|
|
|
|
$chunkSize = unpack('N', (string) fread($handle, 4))[1];
|
|
$baseNonce = (string) fread($handle, self::NONCE_LENGTH);
|
|
|
|
if (strlen($baseNonce) !== self::NONCE_LENGTH) {
|
|
throw new RuntimeException('Invalid chunked file: truncated header');
|
|
}
|
|
|
|
$binaryKey = $this->normalizeToBinaryKey($key);
|
|
$chunkIndex = 0;
|
|
|
|
while (! feof($handle)) {
|
|
$tag = fread($handle, self::TAG_LENGTH);
|
|
|
|
if ($tag === false || strlen($tag) === 0) {
|
|
break;
|
|
}
|
|
|
|
if (strlen($tag) !== self::TAG_LENGTH) {
|
|
throw new RuntimeException('Invalid chunked file: truncated tag at chunk '.$chunkIndex);
|
|
}
|
|
|
|
$ciphertext = fread($handle, $chunkSize);
|
|
|
|
if ($ciphertext === false || $ciphertext === '') {
|
|
throw new RuntimeException('Invalid chunked file: missing ciphertext at chunk '.$chunkIndex);
|
|
}
|
|
|
|
$plaintext = openssl_decrypt(
|
|
$ciphertext,
|
|
self::CIPHER,
|
|
$binaryKey,
|
|
OPENSSL_RAW_DATA,
|
|
$this->legacyChunkNonce($baseNonce, $chunkIndex),
|
|
$tag,
|
|
);
|
|
|
|
if ($plaintext === false) {
|
|
throw new RuntimeException('Decryption failed - wrong key or corrupted data');
|
|
}
|
|
|
|
yield $plaintext;
|
|
$chunkIndex++;
|
|
}
|
|
} finally {
|
|
fclose($handle);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* A `SEALCHK1` chunk's nonce: the chunk index XORed into the last 4 bytes of the base nonce.
|
|
*/
|
|
private function legacyChunkNonce(string $baseNonce, int $chunkIndex): string
|
|
{
|
|
$nonce = $baseNonce;
|
|
$indexBytes = pack('N', $chunkIndex);
|
|
|
|
for ($i = 0; $i < 4; $i++) {
|
|
$nonce[self::NONCE_LENGTH - 4 + $i] = $nonce[self::NONCE_LENGTH - 4 + $i] ^ $indexBytes[$i];
|
|
}
|
|
|
|
return $nonce;
|
|
}
|
|
|
|
/**
|
|
* Decrypt a legacy single-block encrypted file.
|
|
* Format: [12-byte nonce][16-byte auth tag][ciphertext]
|
|
*/
|
|
private function decryptLegacy(string $encryptedPath, string $key): string
|
|
{
|
|
$data = file_get_contents($encryptedPath);
|
|
|
|
if ($data === false) {
|
|
throw new RuntimeException("Cannot read encrypted file: {$encryptedPath}");
|
|
}
|
|
|
|
$plaintext = openssl_decrypt(
|
|
substr($data, self::NONCE_LENGTH + self::TAG_LENGTH),
|
|
self::CIPHER,
|
|
$this->normalizeToBinaryKey($key),
|
|
OPENSSL_RAW_DATA,
|
|
substr($data, 0, self::NONCE_LENGTH),
|
|
substr($data, self::NONCE_LENGTH, self::TAG_LENGTH),
|
|
);
|
|
|
|
if ($plaintext === false) {
|
|
throw new RuntimeException('Decryption failed - wrong key or corrupted data');
|
|
}
|
|
|
|
return $plaintext;
|
|
}
|
|
}
|