noNameWEB/SealShare
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Encrypt uploads in the browser and send them in chunks

Browse Source 
A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.

Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.

- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
  ciphertext and tag per chunk. Each nonce holds the chunk index and a
  last-chunk flag (the STREAM construction), so cut or reordered files
  fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
  wrapped with Argon2id (sodium, libsodium's interactive limits) in
  shares.wrapped_key, which names its parameters. Password shares from
  before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
  pending share of its own, lists the files with their progress, retries
  a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
  abort. UploadChunkController only accepts chunks from the session that
  started the share: a repeat is acknowledged, a skip gets 409 with the
  count stored. Chunks go out as Blobs, which Chromium sends about eight
  times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
  needed and takes no files. The Docker image gains AUTO_HTTPS, which
  serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
  it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
  never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
  ZIP64) instead of decrypting whole files into memory and writing the
  archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
  dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
  for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
  default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
  UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
  endpoint's answers, completing a share, the streamed ZIP, cleanup,
  and in Chromium a real chunked upload and the HTTPS warning; the
  selected-files overflow test runs again. README, website, CHANGELOG
  and .ai/rules follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Reinhold / reini
2026-09-16 20:49:17 +02:00
co-authored by Claude Opus 5
parent 504971ad7f
commit 40e35bab0e
53 changed files with 2280 additions and 946 deletions
Download Patch File Download Diff File Expand all files Collapse all files
resources/views/livewire/file-uploader.blade.php
+40 -89
View File
@@ -12,85 +12,33 @@
</x-stack>
</x-stack>
@if ($isStorageFull)
{{-- Files this page already uploaded count towards the quota: they can still become a share. --}}
@if ($isStorageFull && $pendingFiles->isEmpty())
<x-alert color="warning" :title="__('Storage is full. Uploads are temporarily disabled.')" />
@else
<x-form
wire:submit="createShare"
x-data="{
uploading: false,
progress: 0,
dragging: false,
handleDrop(e) {
this.dragging = false;
const items = e.dataTransfer.items;
const files = [];
for (let i = 0; i < items.length; i++) {
const entry = items[i].webkitGetAsEntry?.();
if (entry) {
this.traverseEntry(entry, '', files);
} else if (items[i].kind === 'file') {
files.push({ file: items[i].getAsFile(), path: null });
}
}
setTimeout(() => {
if (! files.length) {
return;
}
const dt = new DataTransfer();
const paths = [];
files.forEach(f => {
dt.items.add(f.file);
paths.push(f.path);
});
$wire.relativePaths = [...($wire.relativePaths ?? []), ...paths];
this.uploading = true;
this.progress = 0;
$wire.uploadMultiple(
'files',
dt.files,
() => this.progress = 100,
() => this.resetUpload(),
(event) => this.progress = event.detail.progress,
() => this.resetUpload(),
);
}, 500);
},
resetUpload() {
this.uploading = false;
this.progress = 0;
},
traverseEntry(entry, path, files) {
if (entry.isFile) {
entry.file(file => {
files.push({ file, path: path ? path + '/' + file.name : null });
});
} else if (entry.isDirectory) {
const reader = entry.createReader();
reader.readEntries(entries => {
entries.forEach(e => this.traverseEntry(e, path ? path + '/' + entry.name : entry.name, files));
});
}
}
}"
x-init="$wire.$on('files-processed', () => resetUpload())"
x-on:livewire-upload-start="uploading = true; progress = 0"
x-on:livewire-upload-finish="progress = 100"
x-on:livewire-upload-cancel="resetUpload()"
x-on:livewire-upload-error="resetUpload()"
x-on:livewire-upload-progress="progress = $event.detail.progress"
x-data="shareUploader({
csrfToken: {{ \Illuminate\Support\Js::from(csrf_token()) }},
messages: {{ \Illuminate\Support\Js::from([
'queued' => __('Waiting'),
'uploaded' => __('Uploaded'),
'failed' => __('Upload failed'),
'sessionExpired' => __('Your session expired. Reload the page to upload again.'),
]) }},
})"
x-on:beforeunload.window="warnBeforeLeaving($event)"
>
{{-- WebCrypto, which encrypts the files in the browser, only exists on HTTPS (or localhost). --}}
<div x-show="! secure" x-cloak data-test="insecure-context">
<x-alert color="warning" :title="__('Uploads need a secure connection (HTTPS).')" :description="__('Ask the administrator to serve this site over HTTPS.')" />
</div>
{{-- Drop zone: the shape behind the icon turns into a burst while files are over it. --}}
<div
class="upload-drop-zone"
x-bind:data-dragging="dragging ? 'true' : 'false'"
x-bind:aria-disabled="uploading ? 'true' : 'false'"
x-bind:aria-disabled="secure ? 'false' : 'true'"
x-on:dragover.prevent="dragging = true"
x-on:dragleave.prevent="dragging = false"
x-on:drop.prevent="handleDrop($event)"
@@ -108,25 +56,21 @@
<p class="md-type-body-md md-ink-variant md-text-center">{{ __('or click to browse') }}</p>
</x-stack>
{{-- The button is the tab stop and opens the browser's own picker; the input only carries the upload. --}}
<x-button :label="__('Browse Files')" icon="folder_open" variant="outlined" x-on:click="$refs.picker.click()" x-bind:disabled="uploading" />
<input type="file" wire:model="files" multiple hidden x-ref="picker" x-bind:disabled="uploading" />
{{-- The button is the tab stop and opens the browser's own picker; the input only carries the selection. --}}
<x-button :label="__('Browse Files')" icon="folder_open" variant="outlined" x-on:click="$refs.picker.click()" x-bind:disabled="! secure" />
<input type="file" multiple hidden x-ref="picker" x-on:change="choose($event)" x-bind:disabled="! secure" data-test="file-input" />
</x-stack>
</div>
{{-- Upload progress --}}
<div x-show="uploading" x-cloak data-test="upload-progress">
<x-stack x-show="progress < 100" gap="space100">
{{-- Upload progress, over every file still to send --}}
<div x-show="busy" x-cloak data-test="upload-progress">
<x-stack gap="space100">
<x-row justify="between">
<span class="md-type-label-lg">{{ __('Uploading...') }} <span x-text="Math.round(progress)"></span>%</span>
<x-button :label="__('Cancel')" size="xs" x-on:click="$wire.cancelUpload('files')" />
<x-button :label="__('Cancel')" size="xs" x-on:click="cancel()" />
</x-row>
<x-progress bind="progress" wavy :label="__('Uploading')" />
</x-stack>
<x-row x-show="progress >= 100" gap="space100">
<x-loading class="upload-processing-indicator" :label="false" />
<span class="md-type-label-lg">{{ __('Processing files...') }}</span>
</x-row>
</div>
@error('files')
@@ -134,21 +78,28 @@
@enderror
{{-- Selected files --}}
@if (count($files))
@if ($pendingFiles->isNotEmpty())
<x-stack gap="space100">
<h2 class="md-type-title-lg">{{ __('Selected Files') }} ({{ count($files) }})</h2>
<h2 class="md-type-title-lg">{{ __('Selected Files') }} ({{ $pendingFiles->count() }})</h2>
<div class="upload-file-list">
<x-list segmented :label="__('Selected Files')">
@foreach ($files as $index => $file)
@foreach ($pendingFiles as $file)
<x-list-item
:title="$relativePaths[$index] ?? $file->getClientOriginalName()"
:title="$file->relative_path ?? $file->original_name"
icon="description"
wire:key="selected-file-{{ $index }}"
wire:key="selected-file-{{ $file->id }}"
data-test="selected-file"
>
<x-slot:description><span class="md-tabular">{{ Number::fileSize($file->getSize()) }}</span></x-slot:description>
<x-slot:description>
<span class="md-tabular">{{ Number::fileSize($file->file_size) }}</span>
· <span class="md-tabular" x-text="statusOf({{ $file->id }}, {{ $file->completed_at ? 'true' : 'false' }})" data-test="file-status"></span>
</x-slot:description>
<x-slot:end>
<x-button icon="close" :aria-label="__('Remove')" wire:click="removeFile({{ $index }})" />
<span x-show="uploads[{{ $file->id }}]?.state === 'failed'" x-cloak>
<x-button icon="refresh" :aria-label="__('Retry')" x-on:click="retry({{ $file->id }})" />
</span>
<x-button icon="close" :aria-label="__('Remove')" x-on:click="remove({{ $file->id }})" />
</x-slot:end>
</x-list-item>
@endforeach
@@ -217,7 +168,7 @@
size="md"
icon="link"
spinner="createShare"
x-bind:disabled="uploading || {{ count($files) === 0 ? 'true' : 'false' }}"
x-bind:disabled="busy || {{ $allFilesUploaded ? 'false' : 'true' }}"
data-test="create-share"
/>
</x-slot:actions>