queueDelivery() runs synchronously from the v1 follow/unfollow endpoints
(via Helpers::sendSignedObject), which commit local state before delivery
and have no try/catch. After the Http::send() rewrite, a ConnectionException
from a momentarily-unreachable remote was rethrown out of queueDelivery(),
turning a best-effort delivery into a 500 for the user after the follow/
unfollow was already persisted. For unfollows, a retry then hit the
isFollowing==false branch and never re-sent the Undo, diverging state.
Treat transport failures (ConnectionException) as best-effort on this
single-delivery path: log, record host health, and return without
propagating. Other exception types (invalid sender/destination, signing,
serialization) still throw, matching pre-rewrite precondition behavior.
Also widen SendUpdateActor's per-user catch from HttpException to Throwable
so a single bad host no longer aborts a fleet-wide actor update (the old
HttpException catch is dead for ConnectionException/invalid-destination).
original_sha256 is the pre-optimization upload hash and can never match the
optimized local file, so comparing against it is not a valid integrity
check for migration. Drop the --verify-sha256 option and the checksum step
entirely; verify now relies on cloud-object existence and size parity.
The cloud migration failure (stale original_sha256 verify) is resolved, so
the temporary default-on debug is no longer needed. Restore --debug to a
bare, off-by-default flag and remove the debugEnabled() string-parsing
helper.
original_sha256 is the hash of the file as originally uploaded, but the
async optimize pipeline (ImageResize/ImageUpdate) rewrites the local file
in place and never updates that column. Verifying the current local bytes
against it made every optimized image fail with sha256_mismatch, so the
migration reported moved=0 and exited 1.
Add a --verify-sha256 flag (off by default) that gates the checksum step.
By default verify relies on cloud-object existence and size parity, which
is the only signal that actually describes the uploaded copy. Add tests
covering both the default (migrates) and opt-in (fails) paths.
The scheduled command only surfaced 'exit code 1' with no cause. Add
structured Log::error entries at every failure path (cloud disk
unresolvable/unconfigured, cloud storage disabled, invalid --before-id,
candidate fetch failure, per-media failure with full context, and a
run-level summary when failures occur).
Also make --debug default to true (now a valued option) so production
runs emit verbose routing detail while this is investigated; pass
--debug=false to silence.
The media storage migration commands read/parsed the .env file directly to
check and flip PF_ENABLE_CLOUD. In containerized deploys there is no .env on
disk (config is injected via env vars), so updateEnvFile() threw
'file_get_contents(.env): Failed to open stream' and the scheduled command
exited 1.
- Check the live setting via config_cache('pixelfed.cloud_storage') like the
rest of the app, instead of parsing .env.
- Make the .env write best-effort in ManagesMediaStorageEnv: skip gracefully
when the file is missing or read-only, and still apply the runtime + DB
config-cache updates (the load-bearing changes on a hot server).
- Apply the same fix to the sibling unstable:MediaMoveStorageCloudToLocal.
- Add a regression test covering the no-.env container scenario.
Adds CustomEmojiService::resync() which re-fetches a remote custom emoji's
media from its origin (image_remote_url) via the SSRF-hardened
SecureMediaFetchService and stores it locally under public/{media_path},
reusing the existing headCheck validation and cache busting.
The admin:resyncemoji command takes a comma-separated list of emoji
filenames, looks each up by media_path, and resyncs remote ones. Supports
--missingonly, --dry-run and --force.
MediaMoveStorageLocalToCloud is stable, so move it back out of the Unstable
namespace: restore App\Console\Commands\Admin\MediaMoveStorageLocalToCloud and
its admin:MediaMoveStorageLocalToCloud signature, and update the scheduler,
README, and feature test. CloudToLocal and CloudToCloud remain under unstable:.
- Rename App\Console\Commands\Admin\MigrateLocalS3MediaURL to MediaUpdateS3CDNUrl
(class + filename only; the admin:MigrateLocalS3MediaURL signature is unchanged)
- Move the three MediaMoveStorage{LocalToCloud,CloudToLocal,CloudToCloud} commands
into App\Console\Commands\Admin\Unstable and change their signatures from
admin: to unstable:
- Update the scheduler in bootstrap/app.php, the command README, the
config/filesystems.php reference comment, and the affected feature tests
Back out all emoji cloud-storage work from staging so it can be reworked and
re-landed separately (the URL resolution flips to cloud on a global config
flag, which created a broken-URL window, and the migration approach needs
revisiting).
Reverts to pre-emoji state:
- CustomEmoji model URL/storage helpers (urlForPath, storageTarget, storeMedia,
storeMediaFromFile, deleteMedia, url) and callers in ImportEmojis,
CustomEmojiService, AdminController
- admin custom-emoji blade views back to local /storage URLs
- Remove admin:EmojiMoveStorageLocalToCloud command
- Remove the deploy migration and its scheduler entry
Media (and the already-reverted story) scheduler entries are untouched.
Process-level workers plateaued at ~7.5 uploads/sec against Fastly Object
Storage because each PUT is high-latency and only a handful ran concurrently.
Add --concurrency=N which uses the AWS SDK CommandPool to keep N PutObject
requests in flight from a single process. A successful PutObject response is
the confirmation (no separate HEAD verify), and the local file is deleted on
success. Commands are yielded lazily so memory stays flat over large runs.
--no-acl escape hatch for S3-compatible stores that reject the ACL header.
- Live 'up/s' rate shown on the progress bar during single-worker runs
- Final summary reports elapsed time and uploads/sec
- Parallel runs tally moved across workers and report aggregate uploads/sec
Makes it easy to compare --workers counts and decide whether async S3
(option 2) is worth pursuing.
The migration was ~1-2s/file due to sequential S3 round-trips (HEAD + PUT +
verify HEAD). Speed it up:
- --workers=N spawns N child processes, each handling a strided slice of the
files (index % N == shard) for real concurrency on the I/O-bound uploads
- --skip-cloud-check skips the upfront HEAD (always upload, idempotent)
- --skip-verify skips the post-upload size re-check
- --offset for manual chunking
Storage/Flysystem has no batch or async upload API, so process-level
concurrency is the pragmatic lever here.
The frontend renders a hardcoded /storage/emoji/missing.png local onerror
fallback for emoji, so that placeholder must stay on local disk. Skip it in
the migration so it is never moved to cloud or deleted locally.
The migration was DB-driven (whereNull('uri')), which excluded federated
emoji whose media is stored locally but have a uri set -> the disk was never
scanned, resulting in moved=0.
- Drive the migration by enumerating local files under public/emoji/ instead
of a DB query; the local file is the source of truth for what needs moving
- Add --debug to print config, custom_emoji table breakdown, local emoji dir
contents, and per-file decisions
Back out the story local->cloud migration so we can land and verify the
emoji cloud work first, one change at a time.
Reverts:
- 9f110bb74 feat: migrate local story media to cloud storage
- 842681b99 fix: schedule StoryMoveStorageLocalToCloud command
Removes StoryMoveStorageLocalToCloud command, its scheduler entry, and the
StoryExpire explicit-disk changes. Remcache and emoji work are untouched.
Will revisit story once emoji is confirmed.
Two issues prevented emoji from serving/migrating correctly on cloud:
- Admin custom-emoji views hardcoded url('storage/'.media_path), so they
always showed local URLs and bypassed cloud resolution. Use $emoji->url().
- The migration/command guard relied solely on config_cache('pixelfed.cloud_storage'),
which is DB/12h-cached and can read stale-false right after cloud is
enabled, causing the migration to silently no-op. Treat cloud as enabled
when either live config() or config_cache() is true.
Change --limit default to 0 (no limit) so the emoji migration processes
every local emoji in a single run instead of capping at 1000, and drop the
limit from the scheduled invocation. Avoids a multi-run window where
not-yet-migrated emoji resolve to missing cloud URLs.
Custom emoji were always written locally and served via hardcoded /storage
URLs, so they never used S3 even on cloud instances.
- CustomEmoji: centralize URL + storage on the active disk (cloud when
pixelfed.cloud_storage is enabled, else local public/ disk) via
urlForPath/url/storageTarget/storeMedia/storeMediaFromFile/deleteMedia
- Route emoji writes/deletes and URL generation (scan, CustomEmojiService::all)
through those helpers in ImportEmojis, CustomEmojiService::import and
AdminController
- Add admin:EmojiMoveStorageLocalToCloud to migrate existing local emoji to
cloud: copy, verify by size, delete local, bust caches
- Schedule it daily when cloud storage is enabled
Ensure story media lands on and stays on cloud storage for S3 instances.
- StoryExpire: archive expiring story media on the same explicit disk the
media lives on (S3 move is a server-side copy+delete), with error handling
- Add admin:StoryMoveStorageLocalToCloud to migrate local story media
(active + story_archives) to cloud: copy, verify by size, then delete local
- --orphans option relocates untracked story_archives/ files to cloud using
the same copy/verify/delete flow (media is moved, never discarded)
- Schedule it hourly alongside the media migration when cloud storage is on
- Add return type hints (void) and final class markers
- Guard null returns from newestBackup() and putFileAs() in BackupToCloud
- Type ask() default values as strings
- Fix uses_left fallback condition for null/zero max_uses
- Annotate AdminInvite::whereInviteCode and cast Str::uuid() to string
- Ignore local redis-data and mysql-9-data dev directories
The resynced summary printed all three counts unconditionally, which
made an untouched metric (e.g. an already-correct comments count) look
like it had been resynced. Drive the summary from the drifted set and
show before->after values, so it matches the drift detection exactly.
reply_count is a nullable column, so NULL rendered as an empty string in
the resync summary. Cast the summary output to int so a null/absent
comment count prints as 0. No behavior change to the reconcile logic.
FixProfileCounts and FixPostCounts use the admin: signature prefix and
are operator-run maintenance tools, so move them from FixBugs/ to Admin/
(namespace updated) and refresh the README tables to match.
Group console commands into Admin, Dev, FixBugs, Install, Internal, and
User subfolders (matching the earlier reorganization), and add a new
Status subfolder for the status:user, status:profile, and status:post
debug commands. Namespaces updated to match; command signatures and the
total command count are unchanged.
Your Name
Shlee
Daniel Supernault