The schedule definitions are bootstrap wiring, not route definitions, so
bootstrap/scheduledtasks.php is a better home. Updated the require path in
bootstrap/app.php. Verified with schedule:list.
Move the schedule definitions out of the withSchedule() closure in
bootstrap/app.php into a dedicated routes/scheduledtasks.php, required with
the Schedule instance in scope. Behavior-preserving; verified with
schedule:list.
The upload/delete/read paths now self-heal stale storage_used counters and
the upgrade backfill migration repairs existing accounts, so the weekly
reconciler is no longer required. Gate it behind pixelfed.account_storage_
reconcile (ACCOUNT_STORAGE_RECONCILE), defaulting off, so operators can opt
in to the background hygiene job without editing source.
Now that the upload/delete hot path and get() self-heal stale counters, the
scheduled reconciler is a background drift safety net rather than the primary
unblock mechanism, so weekly is sufficient.
users.storage_used only ever grew: uploads incremented it but no deletion
path decremented it, so users hit the account size limit even when their
real media usage was well below it.
- Decrement storage_used in MediaDeletePipeline when media is removed
- Add UserStorageService::increaseStorageUsed / decrementStorageUsed as the
fast, symmetric hot-path counter updates (floor-based, clamped at zero)
- Refactor the 6 upload call sites to use increaseStorageUsed instead of
duplicated inline writes (also fixes ceil/floor drift vs the reconciler)
- Add (user_id, size) covering index so per-user SUM(size) is not a full
table scan (INPLACE/LOCK=NONE, skipped on sqlite)
- Add user:storage:recalculate command to repair affected accounts, with a
daily --stale=168 scheduled reconciler to correct any drift
- Add regression tests for the pipeline and UserStorageService
Replace the remcache GC (GarbageCollectorRemcache / gc:remcache) with a
broader storage:maintenance command that sweeps stale remcache temp files and
recursively removes the random empty directories accumulated under the media,
story, avatar and import trees (--hours/--only/--except/--dry-run), scheduled
daily.
Fix the root causes so flows clean up after themselves rather than relying on
the sweep:
- MediaDeletePipeline removes its own emptied m/_v2 leaf dir
- AvatarOptimize logs the previously-swallowed exception, still cleans up the
old avatar on failure, and removes the old file's now-empty splayed dir
- AvatarController::deleteAvatar removes the emptied splayed dir
- StoryExpire/StoryDelete remove the story's own emptied leaf dir
- TransformImports removes imports/{userId} once its files are moved out
- StoryFetch cleans up its remcache temp file in a finally block
Back out all emoji cloud-storage work from staging so it can be reworked and
re-landed separately (the URL resolution flips to cloud on a global config
flag, which created a broken-URL window, and the migration approach needs
revisiting).
Reverts to pre-emoji state:
- CustomEmoji model URL/storage helpers (urlForPath, storageTarget, storeMedia,
storeMediaFromFile, deleteMedia, url) and callers in ImportEmojis,
CustomEmojiService, AdminController
- admin custom-emoji blade views back to local /storage URLs
- Remove admin:EmojiMoveStorageLocalToCloud command
- Remove the deploy migration and its scheduler entry
Media (and the already-reverted story) scheduler entries are untouched.
Back out the story local->cloud migration so we can land and verify the
emoji cloud work first, one change at a time.
Reverts:
- 9f110bb74 feat: migrate local story media to cloud storage
- 842681b99 fix: schedule StoryMoveStorageLocalToCloud command
Removes StoryMoveStorageLocalToCloud command, its scheduler entry, and the
StoryExpire explicit-disk changes. Remcache and emoji work are untouched.
Will revisit story once emoji is confirmed.
The story cloud-migration command was merged (#6958) but its scheduler
entry was dropped when the emoji branch (based off staging before that
merge) later merged and overwrote the scheduler block. Restore the hourly
schedule so local story media is migrated to cloud automatically.
Change --limit default to 0 (no limit) so the emoji migration processes
every local emoji in a single run instead of capping at 1000, and drop the
limit from the scheduled invocation. Avoids a multi-run window where
not-yet-migrated emoji resolve to missing cloud URLs.
Custom emoji were always written locally and served via hardcoded /storage
URLs, so they never used S3 even on cloud instances.
- CustomEmoji: centralize URL + storage on the active disk (cloud when
pixelfed.cloud_storage is enabled, else local public/ disk) via
urlForPath/url/storageTarget/storeMedia/storeMediaFromFile/deleteMedia
- Route emoji writes/deletes and URL generation (scan, CustomEmojiService::all)
through those helpers in ImportEmojis, CustomEmojiService::import and
AdminController
- Add admin:EmojiMoveStorageLocalToCloud to migrate existing local emoji to
cloud: copy, verify by size, delete local, bust caches
- Schedule it daily when cloud storage is enabled
Ensure story media lands on and stays on cloud storage for S3 instances.
- StoryExpire: archive expiring story media on the same explicit disk the
media lives on (S3 move is a server-side copy+delete), with error handling
- Add admin:StoryMoveStorageLocalToCloud to migrate local story media
(active + story_archives) to cloud: copy, verify by size, then delete local
- --orphans option relocates untracked story_archives/ files to cloud using
the same copy/verify/delete flow (media is moved, never discarded)
- Schedule it hourly alongside the media migration when cloud storage is on
The remote avatar/media fetchers wrote temp files to storage/app/remcache/
and only unlinked them on the happy path. Any exception between the write
and the unlink (e.g. a cloud upload failure) leaked the file, and nothing
swept the directory.
- Wrap post-write logic in fetchAvatar() and remoteToCloud() in try/finally
so the temp file is always removed, even on failure
- Add gc:remcache command to delete stale remcache files (default >24h old,
preserves .gitignore, supports --hours and --dry-run)
- Schedule gc:remcache daily to clean up any stragglers
StoryFetch already handled cleanup via try/catch and was left unchanged.
Add admin:MediaMoveStorageLocalToCloud and admin:MediaMoveStorageCloudToLocal:
- Copy media (+thumbnail) between local and cloud disks, verify by size (and
sha256 against original_sha256 when present) before deleting the source.
- Integrated GC: delete the verified source copy (local on upload, cloud on
download), set version=4 / reset to 3, and bust MediaService/StatusService
caches. --keep-local / --keep-cloud opt out.
- Manage PF_ENABLE_CLOUD in .env AND the live runtime + config cache so new
uploads route to the correct backend mid-migration on a hot server. Uses the
installer's atomic .env writer (shared ManagesMediaStorageEnv trait).
- --limit / --dry-run / --force.
Replaces media:migrate2cloud (CloudMediaMigrate) and media:s3gc
(MediaS3GarbageCollector); scheduler now runs MediaMoveStorageLocalToCloud
hourly for straggler upload + GC. Keeps media:fix-nonlocal-driver.
Adds feature tests (download+GC, --keep-cloud, dry-run, env-flag flip both
directions, unknown-disk guard).
Prepares for Laravel 13 where VerifyCsrfToken is deprecated in favor
of PreventRequestForgery. The old class remains as an alias in v13 but
will be removed in a future version.
Adds a global rate limiter (240 req/min per user or IP) to all API
routes. Previously rate limiting was only applied ad-hoc on individual
routes, leaving some endpoints unprotected.
Laravel Passport 13 renamed:
- CheckScopes → CheckToken (verifies ALL listed scopes)
- CheckForAnyScope → CheckTokenForAnyScope (verifies ANY listed scope)
The old class names no longer exist, causing BindingResolutionException
on all /api/v1/admin/* routes that use the 'scope' or 'scopes' middleware
aliases.
Delete 4 middleware wrapper classes that added no custom logic:
- EncryptCookies (empty $except)
- TrimStrings ($except matches framework default)
- VerifyCsrfToken (exceptions moved to validateCsrfTokens() in bootstrap)
- TrustProxies (headers matched framework default)
CSRF exceptions (/api/v1/*, oauth/token) are now configured via
$middleware->validateCsrfTokens(except: [...]) in bootstrap/app.php.
All 107 tests pass.
Your Name
Daniel Supernault
Shlee