noNameWEB/SealShare
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Encrypt uploads in the browser and send them in chunks

Browse Source 
A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.

Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.

- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
  ciphertext and tag per chunk. Each nonce holds the chunk index and a
  last-chunk flag (the STREAM construction), so cut or reordered files
  fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
  wrapped with Argon2id (sodium, libsodium's interactive limits) in
  shares.wrapped_key, which names its parameters. Password shares from
  before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
  pending share of its own, lists the files with their progress, retries
  a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
  abort. UploadChunkController only accepts chunks from the session that
  started the share: a repeat is acknowledged, a skip gets 409 with the
  count stored. Chunks go out as Blobs, which Chromium sends about eight
  times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
  needed and takes no files. The Docker image gains AUTO_HTTPS, which
  serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
  it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
  never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
  ZIP64) instead of decrypting whole files into memory and writing the
  archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
  dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
  for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
  default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
  UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
  endpoint's answers, completing a share, the streamed ZIP, cleanup,
  and in Chromium a real chunked upload and the HTTPS warning; the
  selected-files overflow test runs again. README, website, CHANGELOG
  and .ai/rules follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Reinhold / reini
2026-09-16 20:49:17 +02:00
co-authored by Claude Opus 5
parent 504971ad7f
commit 40e35bab0e
53 changed files with 2280 additions and 946 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/Livewire/FileUploader.php
+96 -118
View File
@@ -3,26 +3,27 @@
namespace App\Livewire;
use App\Models\Setting;
use App\Models\Share;
use App\Services\PasswordGeneratorService;
use App\Services\ShareService;
use Illuminate\Support\Facades\Crypt;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
use Illuminate\Validation\ValidationException;
use Livewire\Attributes\Layout;
use Livewire\Attributes\Locked;
use Livewire\Component;
use Livewire\Features\SupportFileUploads\TemporaryUploadedFile;
use Livewire\WithFileUploads;
/**
* The upload page. The browser encrypts each file chunk by chunk and sends the chunks to
* UploadChunkController (resources/js/share-uploader.js); this component registers the files
* into a pending share, lists them and completes the share with its options.
*/
#[Layout('layouts.app')]
class FileUploader extends Component
{
use WithFileUploads;
/** @var array<int, TemporaryUploadedFile> */
public array $files = [];
/** @var array<int, string|null> */
public array $relativePaths = [];
/** The pending share this page uploads into: created with the first file, one per page load. */
#[Locked]
public ?string $pendingToken = null;
public bool $usePassword = false;
@@ -40,67 +41,70 @@ class FileUploader extends Component
}
/**
* Handle an upload the temporary upload endpoint did not accept.
* Register the files a visitor chose and hand the browser what it encrypts and sends them
* with. A file an admin limit refuses gets `null` in its place and the reason under `files`.
*
* Validation errors (a 422) mean the whole file reached the server and was
* rejected there, so the real reason is logged for the administrator rather
* than guessed at in front of the user. Anything else is a transport failure.
* @param array<int, array{name?: mixed, size?: mixed, path?: mixed}> $files
* @return array<int, array{id: int, url: string, key: string, noncePrefix: string, chunkSize: int, chunkCount: int}|null>
*/
public function _uploadErrored($name, $errorsInJson, $isMultiple): void
public function registerFiles(array $files, ShareService $shareService): array
{
$this->dispatch('upload:errored', name: $name)->self();
$this->resetErrorBag('files');
$errors = is_null($errorsInJson) ? null : (json_decode($errorsInJson, true)['errors'] ?? null);
$targets = [];
if ($errors) {
Log::warning('File upload rejected by the temporary upload endpoint.', ['errors' => $errors]);
foreach ($files as $file) {
try {
$shareFile = $shareService->registerFile(
$this->pendingShare(),
(string) ($file['name'] ?? ''),
(int) ($file['size'] ?? -1),
isset($file['path']) ? (string) $file['path'] : null,
);
} catch (ValidationException $e) {
if (! $this->getErrorBag()->has('files')) {
$this->addError('files', $e->errors()['files'][0]);
}
throw ValidationException::withMessages([
'files' => __('Upload failed: the server could not accept the file. Please try again or contact the administrator.'),
]);
$targets[] = null;
continue;
}
if ($this->pendingToken !== $shareFile->share->token) {
$this->pendingToken = $shareFile->share->token;
session()->push('pending_shares', $this->pendingToken);
}
$header = $shareService->readHeader($shareFile);
$targets[] = [
'id' => $shareFile->id,
'url' => Str::beforeLast(route('upload.chunk', ['shareFile' => $shareFile, 'index' => 0]), '/'),
'key' => $shareFile->share->encryption_key,
'noncePrefix' => bin2hex($header['noncePrefix']),
'chunkSize' => $header['chunkSize'],
'chunkCount' => $header['chunkCount'],
];
}
$maxFileSizeMb = (int) ((int) Setting::get('max_file_size', 100 * 1024 * 1024) / (1024 * 1024));
throw ValidationException::withMessages([
'files' => __('Upload failed: file may be too large (max :max MB) or the connection was interrupted.', ['max' => $maxFileSizeMb]),
]);
return $targets;
}
/**
* Validate a freshly uploaded batch of files.
* Take files out of the pending share, whether or not their upload finished.
*
* Dispatches `files-processed` so the front end can drop its "uploading" state.
* This runs for every batch, including additional files added to an existing
* selection, which a one-off `x-init` on the file list cannot cover.
* @param array<int, mixed> $fileIds
*/
public function updatedFiles(): void
public function removeFiles(array $fileIds, ShareService $shareService): void
{
$this->dispatch('files-processed')->self();
$files = $this->pendingShare()?->files()->whereIn('id', array_map('intval', $fileIds))->get() ?? [];
$maxFileSize = (int) Setting::get('max_file_size', 100 * 1024 * 1024);
$maxFileSizeMb = $maxFileSize / (1024 * 1024);
$maxFilesPerShare = (int) Setting::get('max_files_per_share', 50);
foreach ($files as $file) {
$shareService->removeFile($file);
}
$this->resetErrorBag('files');
if (count($this->files) > $maxFilesPerShare) {
$this->addError('files', __('Too many files. Maximum :max files allowed per share.', ['max' => $maxFilesPerShare]));
return;
}
foreach ($this->files as $file) {
if ($file->getSize() > $maxFileSize) {
$this->addError('files', __('":name" is too large (:size MB). Maximum file size is :max MB.', [
'name' => $file->getClientOriginalName(),
'size' => round($file->getSize() / (1024 * 1024), 1),
'max' => (int) $maxFileSizeMb,
]));
return;
}
}
}
/**
@@ -126,27 +130,11 @@ class FileUploader extends Component
$this->resetErrorBag('password');
}
public function removeFile(int $index): void
{
unset($this->files[$index], $this->relativePaths[$index]);
$this->files = array_values($this->files);
$this->relativePaths = array_values($this->relativePaths);
}
public function createShare(ShareService $shareService): void
{
$maxFilesPerShare = (int) Setting::get('max_files_per_share', 50);
$maxSizePerShare = (int) Setting::get('max_size_per_share', 2 * 1024 * 1024 * 1024);
$maxFileSize = (int) Setting::get('max_file_size', 100 * 1024 * 1024);
$rules = [];
$rules = [
'files' => ['required', 'array', 'min:1', 'max:'.$maxFilesPerShare],
'files.*' => ['required', 'file', 'max:'.($maxFileSize / 1024)],
];
$allowNeverExpire = (bool) Setting::get('allow_never_expire', false);
if (! $allowNeverExpire) {
if (! Setting::get('allow_never_expire', false)) {
$rules['expiration'] = ['required', 'string', 'in:1h,24h,48h,7d,14d,30d'];
}
@@ -154,61 +142,36 @@ class FileUploader extends Component
$rules['password'] = ['required', 'string', 'min:8'];
}
$this->validate($rules, [
'expiration.required' => __('An expiration time is required.'),
'files.required' => __('Please select at least one file to upload.'),
'files.max' => __('Too many files. Maximum :max files allowed per share.'),
'files.*.max' => __('A file exceeds the maximum size of :max KB.'),
]);
if ($rules !== []) {
$this->validate($rules, [
'expiration.required' => __('An expiration time is required.'),
]);
}
if ($shareService->isStorageFull()) {
$this->addError('files', __('Storage is full. Please contact the administrator.'));
$pendingShare = $this->pendingShare();
if ($pendingShare === null) {
$this->addError('files', __('Please select at least one file to upload.'));
return;
}
$totalSize = collect($this->files)->sum(fn ($file) => $file->getSize());
if ($totalSize > $maxSizePerShare) {
$this->addError('files', __('Total file size exceeds the maximum allowed per share.'));
return;
}
$fileData = [];
foreach ($this->files as $index => $file) {
$relativePath = $this->relativePaths[$index] ?? null;
if ($relativePath !== null) {
$relativePath = str_replace('\\', '/', $relativePath);
if (str_starts_with($relativePath, '/') || str_contains($relativePath, '..')) {
$relativePath = null;
}
}
$fileData[] = [
'file' => $file,
'relativePath' => $relativePath,
];
}
$expiresAt = match ($this->expiration) {
'1h' => now()->addHour(),
'24h' => now()->addDay(),
'48h' => now()->addDays(2),
'7d' => now()->addWeek(),
'14d' => now()->addDays(14),
'30d' => now()->addMonth(),
default => null,
};
$share = $shareService->createShare($fileData, [
$share = $shareService->completeShare($pendingShare, [
'password' => $this->usePassword ? $this->password : null,
'expires_at' => $expiresAt,
'expires_at' => match ($this->expiration) {
'1h' => now()->addHour(),
'24h' => now()->addDay(),
'48h' => now()->addDays(2),
'7d' => now()->addWeek(),
'14d' => now()->addDays(14),
'30d' => now()->addMonth(),
default => null,
},
'max_downloads' => $this->maxDownloads ?: null,
]);
session()->put('pending_shares', array_values(array_diff(session('pending_shares', []), [$share->token])));
// The page the upload leads to offers the password once more, next to the link; it is
// never stored in the clear, so this flash is the only way it gets there.
if ($this->usePassword) {
@@ -224,8 +187,11 @@ class FileUploader extends Component
public function render(): mixed
{
$shareService = app(ShareService::class);
$pendingFiles = $this->pendingShare()?->files()->orderBy('id')->get() ?? collect();
return view('livewire.file-uploader', [
'pendingFiles' => $pendingFiles,
'allFilesUploaded' => $pendingFiles->isNotEmpty() && $pendingFiles->every(fn ($file): bool => $file->completed_at !== null),
'isStorageFull' => $shareService->isStorageFull(),
'siteTitle' => Setting::get('site_title'),
'siteDescription' => Setting::get('site_description'),
@@ -234,4 +200,16 @@ class FileUploader extends Component
'passwordGeneratorMode' => app(PasswordGeneratorService::class)->mode(),
]);
}
/**
* This page's pending share, while it is still pending and this session started it.
*/
private function pendingShare(): ?Share
{
if ($this->pendingToken === null || ! in_array($this->pendingToken, session('pending_shares', []), true)) {
return null;
}
return Share::query()->where('token', $this->pendingToken)->whereNull('completed_at')->first();
}
}