Commit Graph

21 Commits (0aa4da98e371c23bbfca8900a48ed263f3555f9c)

Author SHA1 Message Date
Your Name 341351c47e Remove sha256 verification from MediaMoveStorageLocalToCloud
original_sha256 is the pre-optimization upload hash and can never match the
optimized local file, so comparing against it is not a valid integrity
check for migration. Drop the --verify-sha256 option and the checksum step
entirely; verify now relies on cloud-object existence and size parity.
2 weeks ago
Your Name 441a94e731 Make original_sha256 verify opt-in in MediaMoveStorageLocalToCloud
original_sha256 is the hash of the file as originally uploaded, but the
async optimize pipeline (ImageResize/ImageUpdate) rewrites the local file
in place and never updates that column. Verifying the current local bytes
against it made every optimized image fail with sha256_mismatch, so the
migration reported moved=0 and exited 1.

Add a --verify-sha256 flag (off by default) that gates the checksum step.
By default verify relies on cloud-object existence and size parity, which
is the only signal that actually describes the uploaded copy. Add tests
covering both the default (migrates) and opt-in (fails) paths.
2 weeks ago
Your Name d51cf4ccc9 Fix isDomainCompatible throwing on non-json beagle response 2 weeks ago
Your Name 6b14b229d1 Fix media storage migration crash when no .env file exists
The media storage migration commands read/parsed the .env file directly to
check and flip PF_ENABLE_CLOUD. In containerized deploys there is no .env on
disk (config is injected via env vars), so updateEnvFile() threw
'file_get_contents(.env): Failed to open stream' and the scheduled command
exited 1.

- Check the live setting via config_cache('pixelfed.cloud_storage') like the
  rest of the app, instead of parsing .env.
- Make the .env write best-effort in ManagesMediaStorageEnv: skip gracefully
  when the file is missing or read-only, and still apply the runtime + DB
  config-cache updates (the load-bearing changes on a hot server).
- Apply the same fix to the sibling unstable:MediaMoveStorageCloudToLocal.
- Add a regression test covering the no-.env container scenario.
3 weeks ago
Your Name a81270770c refactor: keep MediaMoveStorageLocalToCloud as a stable admin command
MediaMoveStorageLocalToCloud is stable, so move it back out of the Unstable
namespace: restore App\Console\Commands\Admin\MediaMoveStorageLocalToCloud and
its admin:MediaMoveStorageLocalToCloud signature, and update the scheduler,
README, and feature test. CloudToLocal and CloudToCloud remain under unstable:.
4 weeks ago
Your Name 41c9b8830f refactor: rename MigrateLocalS3MediaURL class and move media move-storage commands to unstable
- Rename App\Console\Commands\Admin\MigrateLocalS3MediaURL to MediaUpdateS3CDNUrl
  (class + filename only; the admin:MigrateLocalS3MediaURL signature is unchanged)
- Move the three MediaMoveStorage{LocalToCloud,CloudToLocal,CloudToCloud} commands
  into App\Console\Commands\Admin\Unstable and change their signatures from
  admin: to unstable:
- Update the scheduler in bootstrap/app.php, the command README, the
  config/filesystems.php reference comment, and the affected feature tests
4 weeks ago
Your Name 078380723f style: import DB facade in FixPostCounts test (pint) 4 weeks ago
Your Name ec5be52418 test: add feature tests for admin:fixPostCounts
Cover source-of-truth resync of likes/boosts/comments, dry-run, no-op on
correct data, --type restriction, argument validation, and bulk --all
mode. Includes regression tests for the two reporting bugs: the summary
now lists only drifted metrics, and a null reply_count renders as 0.
4 weeks ago
Your Name 70b4a05b5c Add admin:MediaMoveStorageCloudToCloud for cold S3->S3 migration
Cold-migrate existing media from an old S3 bucket to the current cloud
bucket, one media row at a time (like MigrateLocalS3MediaURL):
- Source = --sourceDisk (default s3-old, reads AWS_OLD_*); destination = the
  current cloud disk (config filesystems.cloud). No .env editing: operators
  point AWS_* at the new bucket first (restarting workers as usual) so new
  uploads/downloads land on the new bucket, then run this to backfill old data.
- Copies media (+thumbnail) source->destination, verifies by size and by
  sha256 of the freshly-written destination object (against original_sha256),
  rewrites cdn_url/optimized_url/thumbnail_url to the destination host, and
  GCs the source objects (unless --keep-source). Busts caches.
- Only touches rows whose cdn_url still points at the source host; idempotent.
- --sourceDisk / --limit / --dry-run / --force.
- Adds the s3-old disk (AWS_OLD_*) to config/filesystems.php and feature tests.
4 weeks ago
Your Name 6ff9ffbbb8 Add media storage migration commands (local<->cloud) with integrated GC
Add admin:MediaMoveStorageLocalToCloud and admin:MediaMoveStorageCloudToLocal:
- Copy media (+thumbnail) between local and cloud disks, verify by size (and
  sha256 against original_sha256 when present) before deleting the source.
- Integrated GC: delete the verified source copy (local on upload, cloud on
  download), set version=4 / reset to 3, and bust MediaService/StatusService
  caches. --keep-local / --keep-cloud opt out.
- Manage PF_ENABLE_CLOUD in .env AND the live runtime + config cache so new
  uploads route to the correct backend mid-migration on a hot server. Uses the
  installer's atomic .env writer (shared ManagesMediaStorageEnv trait).
- --limit / --dry-run / --force.

Replaces media:migrate2cloud (CloudMediaMigrate) and media:s3gc
(MediaS3GarbageCollector); scheduler now runs MediaMoveStorageLocalToCloud
hourly for straggler upload + GC. Keeps media:fix-nonlocal-driver.

Adds feature tests (download+GC, --keep-cloud, dry-run, env-flag flip both
directions, unknown-disk guard).
4 weeks ago
Your Name 34d6fb31f9 Fix MigrateLocalS3MediaUrl tests failing in CI
config_cache() falls through to config() when instance.enable_cc is off
(ENABLE_CONFIG_CACHE=false, as in CI/.env.testing), so ConfigCacheService::put()
alone did not toggle pixelfed.cloud_storage and the command's cloud-enabled
guard aborted with exit 1. Set the underlying config value too (both in
beforeEach and the local-storage refusal test).
4 weeks ago
Your Name da9e73dd22 Rename to admin:MigrateLocalS3MediaURL and drop --avatars
Rename the command (and test) to admin:MigrateLocalS3MediaURL to reflect its
scope: rewriting stale S3/cloud media URLs only. Remove avatar handling and
the --avatars option; the command now focuses solely on status media
(cdn_url, thumbnail_url, optimized_url).
4 weeks ago
Your Name 04536a6e32 Add admin:MigrateLocalMediaURL; replace media:cloud-url-rewrite
Rebuilds stale local media URLs (cdn_url, thumbnail_url, optimized_url) and
avatar cdn_urls from their storage paths via the configured cloud disk.

- Default target host comes from the configured cloud disk (AWS_URL);
  requires confirmation (or --force) and can be overridden with --newDomain.
- Optional --oldDomain filters to a single old backend host; by default all
  stale hosts are rewritten.
- Refuses to run when PF_ENABLE_CLOUD is false (local storage) and, when
  auto-detecting, refuses a target equal to the app domain — so local-storage
  instances are never rewritten.
- Single status id / post URL, --all, --avatars; --dry-run; busts
  MediaService/StatusService caches for affected statuses.
- Removes the superseded media:cloud-url-rewrite command.
- Adds feature tests covering rewrite/skip/dry-run/oldDomain/newDomain/
  remote-skip/local-storage-refusal.
4 weeks ago
Your Name bcd5a5bd7b Require --scope (local/remote/both) for admin:fixProfileCounts --all
Bulk --all reconciliation previously scanned both local and remote profiles
implicitly. Now --all requires an explicit --scope of local, remote, or
both. --active stays local-only and rejects a non-local --scope. Adds the
BelongsTo return type to Profile::user() so the whereHas('user') scope
filter passes Larastan, and adds tests for scope requirement/validation and
local/remote filtering.
4 weeks ago
Your Name 96f26405f1 Rename to admin:fixProfileCounts, make --active its own mode, add --type
- Rename command signature fix:profilecounts -> admin:fixProfileCounts.
- --active is now its own bulk mode (recently-active local accounts),
  mutually exclusive with --all and a single id.
- Add --type=followers|following|statuses to restrict reconciliation to a
  single metric (validated).
- Update/extend tests for the new name, --type restriction and invalid-type
  rejection.
4 weeks ago
Your Name 698ba224e3 Schedule weekly profile-count reconcile and add reconciliation tests
- Add --force flag to fix:profilecounts for unattended runs and schedule
  'fix:profilecounts --all --force' weekly (Sun 03:37) as a safety-net
  reconcile. Kept as a low-frequency full scan rather than a new event-driven
  dirty-set; it only writes profiles that actually drifted.
- Add Feature tests for AccountStatService recompute helpers and
  reconcileProfileCounts (media-type status_count semantics, follower/
  following counts, drift/no-drift/no-write, metric restriction, missing
  profile) plus fix:profilecounts command behavior (silent-when-synced,
  dry-run makes no changes).
4 weeks ago
Your Name c0cde2f682 refactor: move 52 legacy models from App\ to App\Models\
Move all Eloquent models from the app/ root directory to app/Models/
for consistency with modern Laravel conventions. The project already had
54 models in App\Models; this migrates the remaining 52 legacy models.

Changes:
- Move 52 model files from app/ to app/Models/
- Update namespace declarations in each model
- Update all ~1000 import references across the codebase
- Add Relation::morphMap() in AppServiceProvider for backward
  compatibility with existing polymorphic database records
- Add missing HasSnowflakePrimary imports for models that relied
  on same-namespace resolution
4 weeks ago
Your Name 412c29bb47 Improve test assertions and imports 4 weeks ago
Your Name 0f3820e7bf test: add notification, search, compose, report, and collection tests (332 total)
Api/NotificationTest: notification isolation, correct user filtering
Api/SearchTest: v2 search auth, structure, account lookup
Api/CollectionTest: self/user collections, auth requirement
Compose/ComposeTest: page access, settings, media validation, autocomplete
Account/ReportTest: report creation, type validation, auth requirement
4 weeks ago
Your Name e1f883a41b test: add status, timeline, federation, and privacy tests (309 total)
Api/StatusTest: get/delete statuses, favourite/unfavourite, bookmark,
  status creation validation, ownership checks
Api/TimelineTest: public/home/hashtag timelines, private exclusion,
  pagination support
Federation/NodeInfoTest: nodeinfo, webfinger, host-meta endpoints
Account/PrivacyTest: private profile visibility, blocked user access,
  privacy settings toggle
4 weeks ago
Your Name 891e282808 test: add settings, mute/block, and follow tests (278 total)
Settings/ProfileUpdateTest: profile name, bio, website validation,
  password change flow with Mail::fake assertion
Account/MuteBlockTest: mute/unmute, block/unblock, self-protection,
  admin block protection, validation
Account/FollowTest: follow/unfollow via API, self-follow rejection,
  followers/following list endpoints
4 weeks ago