Files
SealShare/app/Services/ShareService.php
T
Andreas Reinhold / reiniandClaude Opus 5 40e35bab0e Encrypt uploads in the browser and send them in chunks
A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.

Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.

- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
  ciphertext and tag per chunk. Each nonce holds the chunk index and a
  last-chunk flag (the STREAM construction), so cut or reordered files
  fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
  wrapped with Argon2id (sodium, libsodium's interactive limits) in
  shares.wrapped_key, which names its parameters. Password shares from
  before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
  pending share of its own, lists the files with their progress, retries
  a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
  abort. UploadChunkController only accepts chunks from the session that
  started the share: a repeat is acknowledged, a skip gets 409 with the
  count stored. Chunks go out as Blobs, which Chromium sends about eight
  times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
  needed and takes no files. The Docker image gains AUTO_HTTPS, which
  serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
  it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
  never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
  ZIP64) instead of decrypting whole files into memory and writing the
  archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
  dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
  for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
  default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
  UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
  endpoint's answers, completing a share, the streamed ZIP, cleanup,
  and in Chromium a real chunked upload and the HTTPS warning; the
  selected-files overflow test runs again. README, website, CHANGELOG
  and .ai/rules follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:49:17 +02:00

404 lines
14 KiB
PHP

<?php
namespace App\Services;
use App\Models\Setting;
use App\Models\Share;
use App\Models\ShareFile;
use Illuminate\Database\Eloquent\ModelNotFoundException;
use Illuminate\Http\UploadedFile;
use Illuminate\Support\Facades\Hash;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;
use Illuminate\Validation\ValidationException;
use InvalidArgumentException;
use League\MimeTypeDetection\FinfoMimeTypeDetector;
use RuntimeException;
/**
* A share's life: files registered into a pending share, their encrypted chunks stored as the
* uploader's browser sends them, and the share completed with its options.
*/
class ShareService
{
public function __construct(
private FileEncryptionService $encryptionService,
) {}
/**
* Register a file the uploader's browser is about to send, in the given pending share or in a
* new one, and write its encrypted file's header.
*
* @throws ValidationException when the file breaks an admin limit
*/
public function registerFile(?Share $pendingShare, string $name, int $size, ?string $relativePath): ShareFile
{
$maxFileSize = (int) Setting::get('max_file_size', 100 * 1024 * 1024);
$maxFilesPerShare = (int) Setting::get('max_files_per_share', 50);
$maxSizePerShare = (int) Setting::get('max_size_per_share', 2 * 1024 * 1024 * 1024);
if ($name === '' || mb_strlen($name) > 255 || $size < 0) {
$this->rejectFile(__('The file could not be added.'));
}
if ($size > $maxFileSize) {
$this->rejectFile(__('":name" is too large (:size MB). Maximum file size is :max MB.', [
'name' => $name,
'size' => round($size / (1024 * 1024), 1),
'max' => intdiv($maxFileSize, 1024 * 1024),
]));
}
if ($pendingShare && $pendingShare->files()->count() >= $maxFilesPerShare) {
$this->rejectFile(__('Too many files. Maximum :max files allowed per share.', ['max' => $maxFilesPerShare]));
}
if (($pendingShare?->total_size ?? 0) + $size > $maxSizePerShare) {
$this->rejectFile(__('Total file size exceeds the maximum allowed per share.'));
}
if ($this->getTotalUsedSpace() + $size > $this->getMaxStorageQuota()) {
$this->rejectFile(__('Storage is full. Please contact the administrator.'));
}
$share = $pendingShare ?? Share::query()->create([
'token' => $this->generateUniqueToken(),
'encryption_key' => $this->encryptionService->generateRandomKey(),
'total_size' => 0,
]);
$storedName = Str::uuid().'.enc';
Storage::disk('shares')->makeDirectory($share->token);
Storage::disk('shares')->put($share->token.'/'.$storedName, $this->encryptionService->createHeader((int) config('uploads.chunk_size')));
$file = $share->files()->create([
'original_name' => $name,
'relative_path' => $this->sanitizeRelativePath($relativePath),
'stored_path' => 'shares/'.$share->token.'/'.$storedName,
'file_size' => $size,
]);
$share->increment('total_size', $size);
return $file;
}
/**
* Verify the encrypted chunk that comes next for a file and write it into place; returns how
* many of the file's chunks are stored. The plaintext only exists in memory, to be checked.
*
* @throws InvalidArgumentException when the chunk has the wrong length or fails authentication
* @throws ModelNotFoundException when the file was removed meanwhile
*/
public function storeChunk(ShareFile $file, int $index, string $chunk): int
{
$share = $file->share;
$handle = @fopen($this->storedFilePath($file), 'r+b');
if ($handle === false) {
throw (new ModelNotFoundException)->setModel(ShareFile::class, [$file->id]);
}
try {
['chunkSize' => $chunkSize, 'noncePrefix' => $noncePrefix] = $this->encryptionService->parseHeader(
(string) fread($handle, FileEncryptionService::HEADER_LENGTH),
);
$chunkCount = $this->encryptionService->chunkCount($file->file_size, $chunkSize);
$isLast = $index === $chunkCount - 1;
if ($index >= $chunkCount) {
throw new InvalidArgumentException('Chunk '.$index.' is beyond the end of the file');
}
$plaintextLength = $isLast ? $file->file_size - $index * $chunkSize : $chunkSize;
if (strlen($chunk) !== $plaintextLength + FileEncryptionService::TAG_LENGTH) {
throw new InvalidArgumentException('Chunk '.$index.' has the wrong length');
}
try {
$plaintext = $this->encryptionService->decryptChunk($chunk, $share->encryption_key, $noncePrefix, $index, $isLast);
} catch (RuntimeException) {
throw new InvalidArgumentException('Chunk '.$index.' failed authentication');
}
$mimeType = $index === 0
? ((new FinfoMimeTypeDetector)->detectMimeType($file->original_name, $plaintext) ?? 'application/octet-stream')
: $file->mime_type;
unset($plaintext);
if (fseek($handle, $this->encryptionService->chunkOffset($index, $chunkSize)) !== 0
|| fwrite($handle, $chunk) !== strlen($chunk)
|| ! fflush($handle)) {
throw new RuntimeException('Cannot write chunk '.$index.' of file '.$file->id);
}
} finally {
fclose($handle);
}
// Counted only once, even when a retry of the same chunk raced this request.
$stored = ShareFile::query()
->whereKey($file->id)
->where('uploaded_chunks', $index)
->update([
'uploaded_chunks' => $index + 1,
'mime_type' => $mimeType,
'completed_at' => $isLast ? now() : null,
]);
if ($stored === 0) {
return ShareFile::query()->findOrFail($file->id)->uploaded_chunks;
}
$share->touch();
return $index + 1;
}
/**
* Remove a file from a pending share, whether or not its upload finished.
*/
public function removeFile(ShareFile $file): void
{
Storage::disk('shares')->delete($file->share->token.'/'.basename($file->stored_path));
$file->share->decrement('total_size', $file->file_size);
$file->delete();
}
/**
* Complete a pending share once every file has arrived: with a password the data key is
* wrapped and no longer stored as it is.
*
* @param array{password?: string|null, expires_at?: mixed, max_downloads?: int|null} $options
*
* @throws ValidationException when files are missing, unfinished or break an admin limit
*/
public function completeShare(Share $share, array $options = []): Share
{
$files = $share->files()->get();
$maxFilesPerShare = (int) Setting::get('max_files_per_share', 50);
$maxSizePerShare = (int) Setting::get('max_size_per_share', 2 * 1024 * 1024 * 1024);
if ($files->isEmpty()) {
$this->rejectFile(__('Please select at least one file to upload.'));
}
if ($files->contains(fn (ShareFile $file): bool => $file->completed_at === null)) {
$this->rejectFile(__('Wait until every file has finished uploading, or remove the ones that failed.'));
}
if ($files->count() > $maxFilesPerShare) {
$this->rejectFile(__('Too many files. Maximum :max files allowed per share.', ['max' => $maxFilesPerShare]));
}
if ($files->sum('file_size') > $maxSizePerShare) {
$this->rejectFile(__('Total file size exceeds the maximum allowed per share.'));
}
$password = $options['password'] ?? null;
$share->update([
'password' => $password ? Hash::make($password) : null,
'wrapped_key' => $password ? $this->encryptionService->wrapKey($share->encryption_key, $password) : null,
'encryption_key' => $password ? null : $share->encryption_key,
'expires_at' => $options['expires_at'] ?? null,
'max_downloads' => $options['max_downloads'] ?? null,
'total_size' => $files->sum('file_size'),
'completed_at' => now(),
]);
return $share;
}
/**
* Create a share from files already on the server, through the same steps an upload from the
* browser takes. Used by tests and demo data.
*
* @param array<int, array{file: UploadedFile, relativePath: string|null}> $files
* @param array{password?: string|null, expires_at?: mixed, max_downloads?: int|null} $options
*/
public function createShare(array $files, array $options = []): Share
{
$share = null;
foreach ($files as $fileData) {
$file = $fileData['file'];
$shareFile = $this->registerFile($share, $file->getClientOriginalName(), $file->getSize(), $fileData['relativePath'] ?? null);
$share = $shareFile->share;
$header = $this->readHeader($shareFile);
$source = fopen($file->getRealPath(), 'rb');
for ($index = 0; $index < $header['chunkCount']; $index++) {
$plaintextLength = min($header['chunkSize'], $shareFile->file_size - $index * $header['chunkSize']);
// A fake upload reports a size its content does not have: zeros make up the rest.
$chunk = $this->encryptionService->encryptChunk(
str_pad($plaintextLength > 0 ? (string) fread($source, $plaintextLength) : '', $plaintextLength, "\0"),
$share->encryption_key,
$header['noncePrefix'],
$index,
$index === $header['chunkCount'] - 1,
);
$this->storeChunk($shareFile->refresh(), $index, $chunk);
}
fclose($source);
}
if ($share === null) {
$this->rejectFile(__('Please select at least one file to upload.'));
}
return $this->completeShare($share, $options);
}
/**
* Generate a unique share token with retry on collision.
*/
private function generateUniqueToken(): string
{
for ($i = 0; $i < 5; $i++) {
$token = Str::random(16);
if (! Share::query()->where('token', $token)->exists()) {
return $token;
}
}
throw new RuntimeException('Unable to generate a unique share token');
}
/**
* Delete a share and its files from disk.
*/
public function deleteShare(Share $share): void
{
Storage::disk('shares')->deleteDirectory($share->token);
$share->delete();
}
/**
* Get the decryption key for a share: unwrapped with the password, derived from it for shares
* created before key wrapping, or stored for shares without a password.
*/
public function getDecryptionKey(Share $share, ?string $password = null): string
{
if ($share->isPasswordProtected()) {
if (! $password) {
throw new RuntimeException('Password required for this share');
}
if ($share->wrapped_key !== null) {
return $this->encryptionService->unwrapKey($share->wrapped_key, $password);
}
return bin2hex($this->encryptionService->deriveKey($password, $share->encryption_salt));
}
return $share->encryption_key;
}
/**
* Verify a password against a share's stored hash.
*/
public function verifyPassword(Share $share, string $password): bool
{
if (! $share->isPasswordProtected()) {
return true;
}
return Hash::check($password, $share->password);
}
/**
* Record a download and auto-delete if limit reached.
*/
public function recordDownload(Share $share): void
{
$share->increment('download_count');
if ($share->hasReachedDownloadLimit()) {
$this->deleteShare($share);
}
}
/**
* Get total used space in bytes, files still being uploaded included.
*/
public function getTotalUsedSpace(): int
{
return (int) Share::query()->sum('total_size');
}
/**
* Check if storage is full based on admin-configured max quota.
*/
public function isStorageFull(): bool
{
return $this->getTotalUsedSpace() >= $this->getMaxStorageQuota();
}
/**
* Get the maximum storage quota in bytes.
*/
public function getMaxStorageQuota(): int
{
return (int) Setting::get('max_storage_quota', 20 * 1024 * 1024 * 1024);
}
/**
* A registered file's chunk size, nonce prefix and chunk count, from its encrypted file's header.
*
* @return array{chunkSize: int, noncePrefix: string, chunkCount: int}
*/
public function readHeader(ShareFile $file): array
{
$header = $this->encryptionService->parseHeader(
(string) file_get_contents($this->storedFilePath($file), false, null, 0, FileEncryptionService::HEADER_LENGTH),
);
return [...$header, 'chunkCount' => $this->encryptionService->chunkCount($file->file_size, $header['chunkSize'])];
}
/**
* Where a file's encrypted content is stored on disk.
*/
public function storedFilePath(ShareFile $file): string
{
return Storage::disk('shares')->path($file->share->token.'/'.basename($file->stored_path));
}
/**
* A relative path from a dropped folder, or null when it could reach outside the share.
*/
private function sanitizeRelativePath(?string $relativePath): ?string
{
if ($relativePath === null) {
return null;
}
$relativePath = str_replace('\\', '/', $relativePath);
if (str_starts_with($relativePath, '/') || str_contains($relativePath, '..')) {
return null;
}
return $relativePath;
}
/**
* @throws ValidationException
*/
private function rejectFile(string $message): never
{
throw ValidationException::withMessages(['files' => $message]);
}
}