A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.
Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.
- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
ciphertext and tag per chunk. Each nonce holds the chunk index and a
last-chunk flag (the STREAM construction), so cut or reordered files
fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
wrapped with Argon2id (sodium, libsodium's interactive limits) in
shares.wrapped_key, which names its parameters. Password shares from
before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
pending share of its own, lists the files with their progress, retries
a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
abort. UploadChunkController only accepts chunks from the session that
started the share: a repeat is acknowledged, a skip gets 409 with the
count stored. Chunks go out as Blobs, which Chromium sends about eight
times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
needed and takes no files. The Docker image gains AUTO_HTTPS, which
serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
ZIP64) instead of decrypting whole files into memory and writing the
archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
endpoint's answers, completing a share, the streamed ZIP, cleanup,
and in Chromium a real chunked upload and the HTTPS warning; the
selected-files overflow test runs again. README, website, CHANGELOG
and .ai/rules follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
404 lines
14 KiB
PHP
404 lines
14 KiB
PHP
<?php
|
|
|
|
namespace App\Services;
|
|
|
|
use App\Models\Setting;
|
|
use App\Models\Share;
|
|
use App\Models\ShareFile;
|
|
use Illuminate\Database\Eloquent\ModelNotFoundException;
|
|
use Illuminate\Http\UploadedFile;
|
|
use Illuminate\Support\Facades\Hash;
|
|
use Illuminate\Support\Facades\Storage;
|
|
use Illuminate\Support\Str;
|
|
use Illuminate\Validation\ValidationException;
|
|
use InvalidArgumentException;
|
|
use League\MimeTypeDetection\FinfoMimeTypeDetector;
|
|
use RuntimeException;
|
|
|
|
/**
|
|
* A share's life: files registered into a pending share, their encrypted chunks stored as the
|
|
* uploader's browser sends them, and the share completed with its options.
|
|
*/
|
|
class ShareService
|
|
{
|
|
public function __construct(
|
|
private FileEncryptionService $encryptionService,
|
|
) {}
|
|
|
|
/**
|
|
* Register a file the uploader's browser is about to send, in the given pending share or in a
|
|
* new one, and write its encrypted file's header.
|
|
*
|
|
* @throws ValidationException when the file breaks an admin limit
|
|
*/
|
|
public function registerFile(?Share $pendingShare, string $name, int $size, ?string $relativePath): ShareFile
|
|
{
|
|
$maxFileSize = (int) Setting::get('max_file_size', 100 * 1024 * 1024);
|
|
$maxFilesPerShare = (int) Setting::get('max_files_per_share', 50);
|
|
$maxSizePerShare = (int) Setting::get('max_size_per_share', 2 * 1024 * 1024 * 1024);
|
|
|
|
if ($name === '' || mb_strlen($name) > 255 || $size < 0) {
|
|
$this->rejectFile(__('The file could not be added.'));
|
|
}
|
|
|
|
if ($size > $maxFileSize) {
|
|
$this->rejectFile(__('":name" is too large (:size MB). Maximum file size is :max MB.', [
|
|
'name' => $name,
|
|
'size' => round($size / (1024 * 1024), 1),
|
|
'max' => intdiv($maxFileSize, 1024 * 1024),
|
|
]));
|
|
}
|
|
|
|
if ($pendingShare && $pendingShare->files()->count() >= $maxFilesPerShare) {
|
|
$this->rejectFile(__('Too many files. Maximum :max files allowed per share.', ['max' => $maxFilesPerShare]));
|
|
}
|
|
|
|
if (($pendingShare?->total_size ?? 0) + $size > $maxSizePerShare) {
|
|
$this->rejectFile(__('Total file size exceeds the maximum allowed per share.'));
|
|
}
|
|
|
|
if ($this->getTotalUsedSpace() + $size > $this->getMaxStorageQuota()) {
|
|
$this->rejectFile(__('Storage is full. Please contact the administrator.'));
|
|
}
|
|
|
|
$share = $pendingShare ?? Share::query()->create([
|
|
'token' => $this->generateUniqueToken(),
|
|
'encryption_key' => $this->encryptionService->generateRandomKey(),
|
|
'total_size' => 0,
|
|
]);
|
|
|
|
$storedName = Str::uuid().'.enc';
|
|
|
|
Storage::disk('shares')->makeDirectory($share->token);
|
|
Storage::disk('shares')->put($share->token.'/'.$storedName, $this->encryptionService->createHeader((int) config('uploads.chunk_size')));
|
|
|
|
$file = $share->files()->create([
|
|
'original_name' => $name,
|
|
'relative_path' => $this->sanitizeRelativePath($relativePath),
|
|
'stored_path' => 'shares/'.$share->token.'/'.$storedName,
|
|
'file_size' => $size,
|
|
]);
|
|
|
|
$share->increment('total_size', $size);
|
|
|
|
return $file;
|
|
}
|
|
|
|
/**
|
|
* Verify the encrypted chunk that comes next for a file and write it into place; returns how
|
|
* many of the file's chunks are stored. The plaintext only exists in memory, to be checked.
|
|
*
|
|
* @throws InvalidArgumentException when the chunk has the wrong length or fails authentication
|
|
* @throws ModelNotFoundException when the file was removed meanwhile
|
|
*/
|
|
public function storeChunk(ShareFile $file, int $index, string $chunk): int
|
|
{
|
|
$share = $file->share;
|
|
$handle = @fopen($this->storedFilePath($file), 'r+b');
|
|
|
|
if ($handle === false) {
|
|
throw (new ModelNotFoundException)->setModel(ShareFile::class, [$file->id]);
|
|
}
|
|
|
|
try {
|
|
['chunkSize' => $chunkSize, 'noncePrefix' => $noncePrefix] = $this->encryptionService->parseHeader(
|
|
(string) fread($handle, FileEncryptionService::HEADER_LENGTH),
|
|
);
|
|
|
|
$chunkCount = $this->encryptionService->chunkCount($file->file_size, $chunkSize);
|
|
$isLast = $index === $chunkCount - 1;
|
|
|
|
if ($index >= $chunkCount) {
|
|
throw new InvalidArgumentException('Chunk '.$index.' is beyond the end of the file');
|
|
}
|
|
|
|
$plaintextLength = $isLast ? $file->file_size - $index * $chunkSize : $chunkSize;
|
|
|
|
if (strlen($chunk) !== $plaintextLength + FileEncryptionService::TAG_LENGTH) {
|
|
throw new InvalidArgumentException('Chunk '.$index.' has the wrong length');
|
|
}
|
|
|
|
try {
|
|
$plaintext = $this->encryptionService->decryptChunk($chunk, $share->encryption_key, $noncePrefix, $index, $isLast);
|
|
} catch (RuntimeException) {
|
|
throw new InvalidArgumentException('Chunk '.$index.' failed authentication');
|
|
}
|
|
|
|
$mimeType = $index === 0
|
|
? ((new FinfoMimeTypeDetector)->detectMimeType($file->original_name, $plaintext) ?? 'application/octet-stream')
|
|
: $file->mime_type;
|
|
|
|
unset($plaintext);
|
|
|
|
if (fseek($handle, $this->encryptionService->chunkOffset($index, $chunkSize)) !== 0
|
|
|| fwrite($handle, $chunk) !== strlen($chunk)
|
|
|| ! fflush($handle)) {
|
|
throw new RuntimeException('Cannot write chunk '.$index.' of file '.$file->id);
|
|
}
|
|
} finally {
|
|
fclose($handle);
|
|
}
|
|
|
|
// Counted only once, even when a retry of the same chunk raced this request.
|
|
$stored = ShareFile::query()
|
|
->whereKey($file->id)
|
|
->where('uploaded_chunks', $index)
|
|
->update([
|
|
'uploaded_chunks' => $index + 1,
|
|
'mime_type' => $mimeType,
|
|
'completed_at' => $isLast ? now() : null,
|
|
]);
|
|
|
|
if ($stored === 0) {
|
|
return ShareFile::query()->findOrFail($file->id)->uploaded_chunks;
|
|
}
|
|
|
|
$share->touch();
|
|
|
|
return $index + 1;
|
|
}
|
|
|
|
/**
|
|
* Remove a file from a pending share, whether or not its upload finished.
|
|
*/
|
|
public function removeFile(ShareFile $file): void
|
|
{
|
|
Storage::disk('shares')->delete($file->share->token.'/'.basename($file->stored_path));
|
|
|
|
$file->share->decrement('total_size', $file->file_size);
|
|
$file->delete();
|
|
}
|
|
|
|
/**
|
|
* Complete a pending share once every file has arrived: with a password the data key is
|
|
* wrapped and no longer stored as it is.
|
|
*
|
|
* @param array{password?: string|null, expires_at?: mixed, max_downloads?: int|null} $options
|
|
*
|
|
* @throws ValidationException when files are missing, unfinished or break an admin limit
|
|
*/
|
|
public function completeShare(Share $share, array $options = []): Share
|
|
{
|
|
$files = $share->files()->get();
|
|
$maxFilesPerShare = (int) Setting::get('max_files_per_share', 50);
|
|
$maxSizePerShare = (int) Setting::get('max_size_per_share', 2 * 1024 * 1024 * 1024);
|
|
|
|
if ($files->isEmpty()) {
|
|
$this->rejectFile(__('Please select at least one file to upload.'));
|
|
}
|
|
|
|
if ($files->contains(fn (ShareFile $file): bool => $file->completed_at === null)) {
|
|
$this->rejectFile(__('Wait until every file has finished uploading, or remove the ones that failed.'));
|
|
}
|
|
|
|
if ($files->count() > $maxFilesPerShare) {
|
|
$this->rejectFile(__('Too many files. Maximum :max files allowed per share.', ['max' => $maxFilesPerShare]));
|
|
}
|
|
|
|
if ($files->sum('file_size') > $maxSizePerShare) {
|
|
$this->rejectFile(__('Total file size exceeds the maximum allowed per share.'));
|
|
}
|
|
|
|
$password = $options['password'] ?? null;
|
|
|
|
$share->update([
|
|
'password' => $password ? Hash::make($password) : null,
|
|
'wrapped_key' => $password ? $this->encryptionService->wrapKey($share->encryption_key, $password) : null,
|
|
'encryption_key' => $password ? null : $share->encryption_key,
|
|
'expires_at' => $options['expires_at'] ?? null,
|
|
'max_downloads' => $options['max_downloads'] ?? null,
|
|
'total_size' => $files->sum('file_size'),
|
|
'completed_at' => now(),
|
|
]);
|
|
|
|
return $share;
|
|
}
|
|
|
|
/**
|
|
* Create a share from files already on the server, through the same steps an upload from the
|
|
* browser takes. Used by tests and demo data.
|
|
*
|
|
* @param array<int, array{file: UploadedFile, relativePath: string|null}> $files
|
|
* @param array{password?: string|null, expires_at?: mixed, max_downloads?: int|null} $options
|
|
*/
|
|
public function createShare(array $files, array $options = []): Share
|
|
{
|
|
$share = null;
|
|
|
|
foreach ($files as $fileData) {
|
|
$file = $fileData['file'];
|
|
$shareFile = $this->registerFile($share, $file->getClientOriginalName(), $file->getSize(), $fileData['relativePath'] ?? null);
|
|
$share = $shareFile->share;
|
|
|
|
$header = $this->readHeader($shareFile);
|
|
$source = fopen($file->getRealPath(), 'rb');
|
|
|
|
for ($index = 0; $index < $header['chunkCount']; $index++) {
|
|
$plaintextLength = min($header['chunkSize'], $shareFile->file_size - $index * $header['chunkSize']);
|
|
|
|
// A fake upload reports a size its content does not have: zeros make up the rest.
|
|
$chunk = $this->encryptionService->encryptChunk(
|
|
str_pad($plaintextLength > 0 ? (string) fread($source, $plaintextLength) : '', $plaintextLength, "\0"),
|
|
$share->encryption_key,
|
|
$header['noncePrefix'],
|
|
$index,
|
|
$index === $header['chunkCount'] - 1,
|
|
);
|
|
|
|
$this->storeChunk($shareFile->refresh(), $index, $chunk);
|
|
}
|
|
|
|
fclose($source);
|
|
}
|
|
|
|
if ($share === null) {
|
|
$this->rejectFile(__('Please select at least one file to upload.'));
|
|
}
|
|
|
|
return $this->completeShare($share, $options);
|
|
}
|
|
|
|
/**
|
|
* Generate a unique share token with retry on collision.
|
|
*/
|
|
private function generateUniqueToken(): string
|
|
{
|
|
for ($i = 0; $i < 5; $i++) {
|
|
$token = Str::random(16);
|
|
|
|
if (! Share::query()->where('token', $token)->exists()) {
|
|
return $token;
|
|
}
|
|
}
|
|
|
|
throw new RuntimeException('Unable to generate a unique share token');
|
|
}
|
|
|
|
/**
|
|
* Delete a share and its files from disk.
|
|
*/
|
|
public function deleteShare(Share $share): void
|
|
{
|
|
Storage::disk('shares')->deleteDirectory($share->token);
|
|
|
|
$share->delete();
|
|
}
|
|
|
|
/**
|
|
* Get the decryption key for a share: unwrapped with the password, derived from it for shares
|
|
* created before key wrapping, or stored for shares without a password.
|
|
*/
|
|
public function getDecryptionKey(Share $share, ?string $password = null): string
|
|
{
|
|
if ($share->isPasswordProtected()) {
|
|
if (! $password) {
|
|
throw new RuntimeException('Password required for this share');
|
|
}
|
|
|
|
if ($share->wrapped_key !== null) {
|
|
return $this->encryptionService->unwrapKey($share->wrapped_key, $password);
|
|
}
|
|
|
|
return bin2hex($this->encryptionService->deriveKey($password, $share->encryption_salt));
|
|
}
|
|
|
|
return $share->encryption_key;
|
|
}
|
|
|
|
/**
|
|
* Verify a password against a share's stored hash.
|
|
*/
|
|
public function verifyPassword(Share $share, string $password): bool
|
|
{
|
|
if (! $share->isPasswordProtected()) {
|
|
return true;
|
|
}
|
|
|
|
return Hash::check($password, $share->password);
|
|
}
|
|
|
|
/**
|
|
* Record a download and auto-delete if limit reached.
|
|
*/
|
|
public function recordDownload(Share $share): void
|
|
{
|
|
$share->increment('download_count');
|
|
|
|
if ($share->hasReachedDownloadLimit()) {
|
|
$this->deleteShare($share);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Get total used space in bytes, files still being uploaded included.
|
|
*/
|
|
public function getTotalUsedSpace(): int
|
|
{
|
|
return (int) Share::query()->sum('total_size');
|
|
}
|
|
|
|
/**
|
|
* Check if storage is full based on admin-configured max quota.
|
|
*/
|
|
public function isStorageFull(): bool
|
|
{
|
|
return $this->getTotalUsedSpace() >= $this->getMaxStorageQuota();
|
|
}
|
|
|
|
/**
|
|
* Get the maximum storage quota in bytes.
|
|
*/
|
|
public function getMaxStorageQuota(): int
|
|
{
|
|
return (int) Setting::get('max_storage_quota', 20 * 1024 * 1024 * 1024);
|
|
}
|
|
|
|
/**
|
|
* A registered file's chunk size, nonce prefix and chunk count, from its encrypted file's header.
|
|
*
|
|
* @return array{chunkSize: int, noncePrefix: string, chunkCount: int}
|
|
*/
|
|
public function readHeader(ShareFile $file): array
|
|
{
|
|
$header = $this->encryptionService->parseHeader(
|
|
(string) file_get_contents($this->storedFilePath($file), false, null, 0, FileEncryptionService::HEADER_LENGTH),
|
|
);
|
|
|
|
return [...$header, 'chunkCount' => $this->encryptionService->chunkCount($file->file_size, $header['chunkSize'])];
|
|
}
|
|
|
|
/**
|
|
* Where a file's encrypted content is stored on disk.
|
|
*/
|
|
public function storedFilePath(ShareFile $file): string
|
|
{
|
|
return Storage::disk('shares')->path($file->share->token.'/'.basename($file->stored_path));
|
|
}
|
|
|
|
/**
|
|
* A relative path from a dropped folder, or null when it could reach outside the share.
|
|
*/
|
|
private function sanitizeRelativePath(?string $relativePath): ?string
|
|
{
|
|
if ($relativePath === null) {
|
|
return null;
|
|
}
|
|
|
|
$relativePath = str_replace('\\', '/', $relativePath);
|
|
|
|
if (str_starts_with($relativePath, '/') || str_contains($relativePath, '..')) {
|
|
return null;
|
|
}
|
|
|
|
return $relativePath;
|
|
}
|
|
|
|
/**
|
|
* @throws ValidationException
|
|
*/
|
|
private function rejectFile(string $message): never
|
|
{
|
|
throw ValidationException::withMessages(['files' => $message]);
|
|
}
|
|
}
|