Encrypt uploads in the browser and send them in chunks

A 6 GB upload kept a customer waiting long after its progress bar
reached 100%. The server wrote every upload three times: PHP's
temporary file, Livewire's copy of it ("Processing files...") and the
encrypted file ("Create Share Link"), each a full rewrite of a slow
disk. The unencrypted copy also stayed behind in livewire-tmp.

Now the uploader's browser encrypts each file in 16 MB chunks with
WebCrypto and PUTs them one at a time; the server checks each chunk in
memory and writes it once, already encrypted. Creating the share only
wraps its key and saves the options. A 200 MB upload through the
Docker image took 2.8 s, and its download matched byte for byte.

- SEALCHK2: a 19-byte header (chunk size, 7-byte nonce prefix), then
  ciphertext and tag per chunk. Each nonce holds the chunk index and a
  last-chunk flag (the STREAM construction), so cut or reordered files
  fail to decrypt. SEALCHK1 and the single-block format still read.
- Envelope encryption: one random key per share. With a password it is
  wrapped with Argon2id (sodium, libsodium's interactive limits) in
  shares.wrapped_key, which names its parameters. Password shares from
  before keep their PBKDF2-derived key.
- The upload page registers each selection with FileUploader into a
  pending share of its own, lists the files with their progress, retries
  a failed chunk after 1-16 s, then offers Retry; Remove and Cancel
  abort. UploadChunkController only accepts chunks from the session that
  started the share: a repeat is acknowledged, a skip gets 409 with the
  count stored. Chunks go out as Blobs, which Chromium sends about eight
  times faster than ArrayBuffers.
- Uploads need a secure context: over plain HTTP the page says HTTPS is
  needed and takes no files. The Docker image gains AUTO_HTTPS, which
  serves Let's Encrypt on 443 for SERVER_NAME and redirects 80; without
  it the container stays on HTTP 80 behind a proxy. docker/Caddyfile was
  never loaded and is gone; docker/healthcheck.sh covers both modes.
- "Download all" streams the ZIP with maennchen/zipstream-php (STORE,
  ZIP64) instead of decrypting whole files into memory and writing the
  archive unencrypted to /tmp.
- Pending shares count towards the quota, stay out of the admin
  dashboard and 404 everywhere else. shares:cleanup deletes uploads idle
  for 4 hours and Livewire temporary files older than that.
- PHP's upload limits no longer cap the admin's max file size and
  default to 64M; LIVEWIRE_MAX_UPLOAD_TIME is gone and
  UPLOAD_CHUNK_SIZE_MB is new.
- Tests cover the format, key wrapping, registration limits, the chunk
  endpoint's answers, completing a share, the streamed ZIP, cleanup,
  and in Chromium a real chunked upload and the HTTPS warning; the
  selected-files overflow test runs again. README, website, CHANGELOG
  and .ai/rules follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Reinhold / reini
2026-09-16 20:49:17 +02:00
co-authored by Claude Opus 5
parent 504971ad7f
commit 40e35bab0e
53 changed files with 2280 additions and 946 deletions
-14
View File
@@ -1,14 +0,0 @@
{
frankenphp
order php_server before file_server
admin off
}
{$SERVER_NAME:localhost} {
root * /app/public
encode zstd gzip
request_body {
max_size 4gb
}
php_server
}
+2 -2
View File
@@ -6,8 +6,8 @@ cd /app
# Generate PHP ini from environment variables (with defaults)
echo "[dev] Configuring PHP settings..."
cat > /usr/local/etc/php/conf.d/99-uploads.ini <<EOF
upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE:-4G}
post_max_size = ${PHP_POST_MAX_SIZE:-4G}
upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE:-64M}
post_max_size = ${PHP_POST_MAX_SIZE:-64M}
max_execution_time = ${PHP_MAX_EXECUTION_TIME:-300}
max_input_time = ${PHP_MAX_INPUT_TIME:-300}
memory_limit = ${PHP_MEMORY_LIMIT:-512M}
+15 -3
View File
@@ -15,8 +15,8 @@ fi
# Generate PHP ini from environment variables (with defaults)
echo "[entrypoint] Configuring PHP settings..."
cat > /usr/local/etc/php/conf.d/99-uploads.ini <<EOF
upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE:-4G}
post_max_size = ${PHP_POST_MAX_SIZE:-4G}
upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE:-64M}
post_max_size = ${PHP_POST_MAX_SIZE:-64M}
max_execution_time = ${PHP_MAX_EXECUTION_TIME:-300}
max_input_time = ${PHP_MAX_INPUT_TIME:-300}
memory_limit = ${PHP_MEMORY_LIMIT:-512M}
@@ -33,5 +33,17 @@ php artisan config:cache
php artisan route:cache
php artisan view:cache
echo "[entrypoint] Starting Octane (FrankenPHP)..."
# Uploads are encrypted in the browser, which browsers only allow over HTTPS: either this container
# fetches a certificate for SERVER_NAME itself, or a reverse proxy in front terminates TLS.
if [ "${AUTO_HTTPS:-false}" = "true" ]; then
if [ -z "$SERVER_NAME" ]; then
echo "[entrypoint] AUTO_HTTPS=true needs SERVER_NAME, the domain to fetch a certificate for." >&2
exit 1
fi
echo "[entrypoint] Starting Octane (FrankenPHP) with automatic HTTPS for $SERVER_NAME..."
exec php artisan octane:frankenphp --host="$SERVER_NAME" --port=443 --https --http-redirect
fi
echo "[entrypoint] Starting Octane (FrankenPHP) on HTTP..."
exec php artisan octane:frankenphp --host=0.0.0.0 --port=80
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
# Healthy when the application answers /up: over HTTP on port 80, or over HTTPS for SERVER_NAME when
# AUTO_HTTPS is on (port 80 then only redirects). The certificate is not checked, so a container
# still waiting for Let's Encrypt is judged by the application, not by its certificate.
if [ "${AUTO_HTTPS:-false}" = "true" ]; then
exec curl --silent --fail --insecure --resolve "$SERVER_NAME:443:127.0.0.1" "https://$SERVER_NAME/up"
fi
exec curl --silent --fail http://localhost/up
+2 -2
View File
@@ -2,8 +2,8 @@
; These are default values — overridden at runtime by the entrypoint
; when PHP_UPLOAD_MAX_FILESIZE / PHP_POST_MAX_SIZE / etc. env vars are set.
upload_max_filesize = 4G
post_max_size = 4G
upload_max_filesize = 64M
post_max_size = 64M
max_execution_time = 300
max_input_time = 300
memory_limit = 512M