Commit Graph

93 Commits (ce4343e3e26073abd0d9107e7ca02bd8799eba03)

Author SHA1 Message Date
Your Name ce4343e3e2 Replace custom register token with spatie/laravel-honeypot
Swap the custom 'rt' register token anti-spam mechanism for
spatie/laravel-honeypot on the registration and parental-controls
invite flows.

- Add spatie/laravel-honeypot and publish config/honeypot.php
- Remove getRegisterToken() and the rt validation rule from RegisterController
- Replace the rt hidden field with the @honeypot directive in both forms
- Attach ProtectAgainstSpam middleware to POST /register and the
  parental-controls invite register route
- Update RegisterTest to disable honeypot for the valid registration case
2 weeks ago
Your Name 6d8ad3885a Convert string class references to ::class
Applies the ::class conversion from pixelfed-staging PR #9 (patch 1/21),
formatted with Pint (short imported ::class form). Excludes the
ModelNamespaceMigrationTest namespace assertions, which intentionally
compare against literal namespace strings.
3 weeks ago
Your Name 6b14b229d1 Fix media storage migration crash when no .env file exists
The media storage migration commands read/parsed the .env file directly to
check and flip PF_ENABLE_CLOUD. In containerized deploys there is no .env on
disk (config is injected via env vars), so updateEnvFile() threw
'file_get_contents(.env): Failed to open stream' and the scheduled command
exited 1.

- Check the live setting via config_cache('pixelfed.cloud_storage') like the
  rest of the app, instead of parsing .env.
- Make the .env write best-effort in ManagesMediaStorageEnv: skip gracefully
  when the file is missing or read-only, and still apply the runtime + DB
  config-cache updates (the load-bearing changes on a hot server).
- Apply the same fix to the sibling unstable:MediaMoveStorageCloudToLocal.
- Add a regression test covering the no-.env container scenario.
3 weeks ago
Shlee 003953eb3e
Merge pull request #7045 from pixelfed/perf/follower-service-following-ids
Deduplocation: add FollowerService::getFollowingIds for common function
3 weeks ago
Your Name 667f6e2fc9 Extract following-ids lookup into FollowerService::getFollowingIds
The Cache::remember('profile:following:'.$pid, ...) block that plucks
following_id and appends the caller's own id was copy-pasted across four
call sites, with inconsistent TTLs (1440 minutes vs 1209600 seconds).

Add FollowerService::getFollowingIds($pid), which owns the cache key that
add()/remove() already invalidate, and use it from InternalApiController,
PublicApiController, ApiV1Controller and HashtagUnfollowPipeline. Removes
the now-unused Follower/Cache imports left behind.

Adds a test covering the followed-ids-plus-self result and the
follows-nobody case.
3 weeks ago
Your Name c4e5b96d25 Stream deletions with cursor and batch notification lookups in delete jobs
The status- and account-deletion jobs loaded whole collections with
->get() and then looped, running a per-row Notification lookup inside
each iteration.

- StatusDelete / RemoteStatusDelete: resolve associated DirectMessage and
  MediaTag ids, fetch their notifications in a single whereIn query,
  clear each via cursor (NotificationService::del must run per row for
  cache/redis cleanup), then bulk delete the DMs and media tags.
- DeleteAccountPipeline / DeleteRemoteProfilePipeline: stream Story and
  Collection deletions with cursor() instead of loading every row into
  memory. Per-row file unlink and item deletes are preserved.

Adds StatusDeleteCleanupTest covering DM + notification cleanup, media
tag + notification cleanup, and the no-associations case.
3 weeks ago
Your Name e4e12fad7c Extract duplicated blocked-id and duplicate-shortcode query patterns
Two query patterns were copy-pasted across several call sites:

- The 'users who blocked me, plus myself' list used to filter profile
  search (UserFilter::whereFilterableId($pid)->pluck('user_id')->push($pid))
  appeared in ComposeController (x2) and DirectMessageController. Extracted
  to UserFilterService::searchExcludedProfileIds(). Note this is the
  inverse of blocks() (who I blocked), so it is a distinct method.

- CustomEmoji duplicate detection (groupBy('shortcode')->havingRaw(
  'count(*) > 1')) appeared three times in AdminController. Extracted to a
  CustomEmoji::duplicateShortcodes() query scope.

Adds tests for both. No behaviour change.
3 weeks ago
Shlee 7ad550d2f7
Merge pull request #7037 from pixelfed/perf/seasonal-sql-aggregation
Compute Year-in-Review averages in SQL instead of in PHP
3 weeks ago
Your Name cd873c8976 Compute Year-in-Review averages in SQL instead of in PHP
SeasonalController::getData computed the average posts/likes per profile
by grouping in SQL, then pulling every grouped row into a collection and
calling ->pluck('count')->avg() in PHP. This loaded one row per profile
into memory just to average.

Wrap the grouped per-profile counts in a subquery and let the database
compute AVG(count), returning a single value. Also drops the invalid
SELECT * with GROUP BY (ONLY_FULL_GROUP_BY) by selecting count(*) only.

Adds a test verifying the average-of-per-profile-counts and its
exclusions (remote, wrong type, out-of-range date), plus the empty case.
3 weeks ago
Shlee 660880eac7
Merge pull request #7036 from pixelfed/fix/remote-auth-http-timeout
Add timeout, retry and error handling to remote auth HTTP calls
3 weeks ago
Your Name 2ad6e28318 Add timeout, retry and error handling to remote auth HTTP calls
RemoteAuthService::getVerifyCredentials, getFollowing and getToken made
outbound HTTP requests to a user-controlled remote instance during the
Mastodon login flow with no timeout, no retry and no exception handling.
A slow or hostile instance could hang the request or surface an uncaught
exception.

Wrap all three in timeout(20)->retry(3, 750) with try/catch that returns
false on failure, matching the existing pattern in isDomainCompatible().
Callers already treat a falsy return as a failure; add the missing guard
at the one verify_credentials call site that accessed the result array
without checking it first.

Adds RemoteAuthServiceTest covering connection failure, server error and
success paths.
3 weeks ago
Your Name fbfd26d775 Mark direct messages read with a single bulk update
DirectMessageController@read fetched every matching DirectMessage and
saved each one individually in a loop, issuing one UPDATE per row. On an
active thread this is N queries.

Pluck the matching ids and perform a single bulk update, preserving the
existing response (the list of affected message ids) and updated_at
behaviour.

Adds regression tests covering the marked-read ids, the status_id lower
bound, and sender isolation.
3 weeks ago
Your Name 29280cd950 Fix silent failure in avatar upload endpoints
AvatarController@store and BaseApiController@avatarUpdate wrapped the
upload flow in an empty catch(\Exception) block and returned a success
response even when the upload or save failed.

Log the exception and return a real error response (500 JSON for the
API endpoint, a redirect with validation errors for the web endpoint).

Adds regression tests covering the failure path, the success path, and
non-image rejection.
3 weeks ago
Daniel Supernault c19fd269b1
Lint 3 weeks ago
Daniel Supernault 584ce27f71
Add Sanctum support 3 weeks ago
Your Name 9214e9680a Add admin:resyncemoji command to re-download remote emoji locally
Adds CustomEmojiService::resync() which re-fetches a remote custom emoji's
media from its origin (image_remote_url) via the SSRF-hardened
SecureMediaFetchService and stores it locally under public/{media_path},
reusing the existing headCheck validation and cache busting.

The admin:resyncemoji command takes a comma-separated list of emoji
filenames, looks each up by media_path, and resyncs remote ones. Supports
--missingonly, --dry-run and --force.
3 weeks ago
Your Name d456b7b64a Add structured metadata to MediaDeletePipeline skip/failure logs
Replace interpolated log strings with structured context (media/status/profile/
user ids, mime, size, order, paths, hls_path, remote flag, timestamps) so
operators can trace why orphan-purge deletions are skipped or fail.
3 weeks ago
Your Name 0df4c7117d Add domain-mismatch metadata to Announce status fetch and stop noisy ERROR logs
storeStatus() now throws with JSON metadata (checked id/url hosts, expected
rule, and the full activity payload) when status domains mismatch. The Announce
inbox handler catches this, logs the context at debug level, and returns
gracefully instead of surfacing a full production ERROR stack trace.
3 weeks ago
Your Name b6d645a4d4 fix: delete superseded image/thumbnail files instead of orphaning them
Image::handleImageTransform derives the output filename from the current
media_path and applies the encoder's output extension. When that differs from
what is already stored (heic/avif -> jpg, or a thumbnail regenerated to a new
extension), the new file landed at a different path and the previous file was
left orphaned in the media directory — the source of the leftover _thumb files
under public/m/_v2.

Capture the path each transform supersedes and delete it after a successful
write (only when the new output path differs, so we never delete what we just
wrote). Remove the stale MediaDeleteLeafCleanupTest whose source change is not
in the tree.
3 weeks ago
Your Name fce75030e0 refactor: simplify storage:maintenance flags and make it quiet by default
- Drop --except (--only already covers task selection)
- Quiet by default; per-root/per-item and summary lines now require -v/--verbose
- Errors are always shown regardless of verbosity
- Document the command in the console README
3 weeks ago
Your Name 400f00c5e1 feat: storage:maintenance command + in-flow cleanup of emptied dirs
Replace the remcache GC (GarbageCollectorRemcache / gc:remcache) with a
broader storage:maintenance command that sweeps stale remcache temp files and
recursively removes the random empty directories accumulated under the media,
story, avatar and import trees (--hours/--only/--except/--dry-run), scheduled
daily.

Fix the root causes so flows clean up after themselves rather than relying on
the sweep:
- MediaDeletePipeline removes its own emptied m/_v2 leaf dir
- AvatarOptimize logs the previously-swallowed exception, still cleans up the
  old avatar on failure, and removes the old file's now-empty splayed dir
- AvatarController::deleteAvatar removes the emptied splayed dir
- StoryExpire/StoryDelete remove the story's own emptied leaf dir
- TransformImports removes imports/{userId} once its files are moved out
- StoryFetch cleans up its remcache temp file in a finally block
3 weeks ago
Your Name a81270770c refactor: keep MediaMoveStorageLocalToCloud as a stable admin command
MediaMoveStorageLocalToCloud is stable, so move it back out of the Unstable
namespace: restore App\Console\Commands\Admin\MediaMoveStorageLocalToCloud and
its admin:MediaMoveStorageLocalToCloud signature, and update the scheduler,
README, and feature test. CloudToLocal and CloudToCloud remain under unstable:.
3 weeks ago
Your Name 41c9b8830f refactor: rename MigrateLocalS3MediaURL class and move media move-storage commands to unstable
- Rename App\Console\Commands\Admin\MigrateLocalS3MediaURL to MediaUpdateS3CDNUrl
  (class + filename only; the admin:MigrateLocalS3MediaURL signature is unchanged)
- Move the three MediaMoveStorage{LocalToCloud,CloudToLocal,CloudToCloud} commands
  into App\Console\Commands\Admin\Unstable and change their signatures from
  admin: to unstable:
- Update the scheduler in bootstrap/app.php, the command README, the
  config/filesystems.php reference comment, and the affected feature tests
3 weeks ago
Your Name b52c3d7659 perf: fix N+1 queries; fix ComposeController lint and test namespace
Performance:
- TrendingHashtagService: batch-load hashtags with whereIn/keyBy instead
  of Hashtag::find() per trending row.
- DirectMessageController: eager-load status.media and read the in-memory
  collection instead of firstMedia() issuing a query per DM message.
- GroupsSearchController: batch Profile/Follower/GroupInvitation lookups
  with whereIn instead of per-invitee queries.

Lint/tests:
- ComposeController: whitespace formatting (Pint).
- ComposeControllerTest: correct App\User to App\Models\User, fixing the
  larastan class.notFound error and import ordering.

Full suite: 547 passed. Pint and PHPStan clean.
3 weeks ago
Shlee d9817840c1
Merge pull request #6955 from hohoho1886/location-country-search
Add country filtering to location search
3 weeks ago
Nguyen Ninh Dao 129778ef2e implement search by country 4 weeks ago
Your Name 4056747666 Revert hashtags collation migration from #6098
Migration failed with a duplicate entry error: recollating to
utf8mb4_unicode_520_ci causes previously-distinct hashtag names/slugs
to collide on the unique indexes. Reverting until the data is
de-duplicated first.
4 weeks ago
Shlee 336f5f899f
Merge pull request #6098 from bpeel/hashtags-outside-bmp
Fix hashtags usings scripts outside the BMP on MySQL
4 weeks ago
Shlee 27768cd69c
Create HashtagCollationTest.php 4 weeks ago
Your Name b7c15dc7ca Fix larastan errors in RemoteOidcTest: import Test attribute and RefreshDatabase, replace removed str_random helper 4 weeks ago
Shlee a82dc91295
Merge branch 'staging' into dev 4 weeks ago
Your Name 078380723f style: import DB facade in FixPostCounts test (pint) 4 weeks ago
Your Name ec5be52418 test: add feature tests for admin:fixPostCounts
Cover source-of-truth resync of likes/boosts/comments, dry-run, no-op on
correct data, --type restriction, argument validation, and bulk --all
mode. Includes regression tests for the two reporting bugs: the summary
now lists only drifted metrics, and a null reply_count renders as 0.
4 weeks ago
Your Name 0d01d5a963 Fix duplicate-key violation when importing remote media attachments
Helpers::importNoteAttachment unconditionally inserted a new Media row per
attachment, so re-importing a remote status (an Announce racing another
inbox job, a re-fetch, or a duplicate url within one activity) hit the
media_status_id_media_path_unique constraint and crashed the queue job with
a 1062 UniqueConstraintViolationException, dropping the boost/import.

Make createMediaAttachment idempotent on (status_id, media_path): skip when
a row already exists, and catch the unique-constraint violation as a
lost-race no-op, returning null so the caller skips re-dispatching storage.

Adds regression tests (re-import no-op, distinct urls still stored,
concurrent-insert returns null).
4 weeks ago
Your Name 70b4a05b5c Add admin:MediaMoveStorageCloudToCloud for cold S3->S3 migration
Cold-migrate existing media from an old S3 bucket to the current cloud
bucket, one media row at a time (like MigrateLocalS3MediaURL):
- Source = --sourceDisk (default s3-old, reads AWS_OLD_*); destination = the
  current cloud disk (config filesystems.cloud). No .env editing: operators
  point AWS_* at the new bucket first (restarting workers as usual) so new
  uploads/downloads land on the new bucket, then run this to backfill old data.
- Copies media (+thumbnail) source->destination, verifies by size and by
  sha256 of the freshly-written destination object (against original_sha256),
  rewrites cdn_url/optimized_url/thumbnail_url to the destination host, and
  GCs the source objects (unless --keep-source). Busts caches.
- Only touches rows whose cdn_url still points at the source host; idempotent.
- --sourceDisk / --limit / --dry-run / --force.
- Adds the s3-old disk (AWS_OLD_*) to config/filesystems.php and feature tests.
4 weeks ago
Your Name 6ff9ffbbb8 Add media storage migration commands (local<->cloud) with integrated GC
Add admin:MediaMoveStorageLocalToCloud and admin:MediaMoveStorageCloudToLocal:
- Copy media (+thumbnail) between local and cloud disks, verify by size (and
  sha256 against original_sha256 when present) before deleting the source.
- Integrated GC: delete the verified source copy (local on upload, cloud on
  download), set version=4 / reset to 3, and bust MediaService/StatusService
  caches. --keep-local / --keep-cloud opt out.
- Manage PF_ENABLE_CLOUD in .env AND the live runtime + config cache so new
  uploads route to the correct backend mid-migration on a hot server. Uses the
  installer's atomic .env writer (shared ManagesMediaStorageEnv trait).
- --limit / --dry-run / --force.

Replaces media:migrate2cloud (CloudMediaMigrate) and media:s3gc
(MediaS3GarbageCollector); scheduler now runs MediaMoveStorageLocalToCloud
hourly for straggler upload + GC. Keeps media:fix-nonlocal-driver.

Adds feature tests (download+GC, --keep-cloud, dry-run, env-flag flip both
directions, unknown-disk guard).
4 weeks ago
Your Name 34d6fb31f9 Fix MigrateLocalS3MediaUrl tests failing in CI
config_cache() falls through to config() when instance.enable_cc is off
(ENABLE_CONFIG_CACHE=false, as in CI/.env.testing), so ConfigCacheService::put()
alone did not toggle pixelfed.cloud_storage and the command's cloud-enabled
guard aborted with exit 1. Set the underlying config value too (both in
beforeEach and the local-storage refusal test).
4 weeks ago
Your Name da9e73dd22 Rename to admin:MigrateLocalS3MediaURL and drop --avatars
Rename the command (and test) to admin:MigrateLocalS3MediaURL to reflect its
scope: rewriting stale S3/cloud media URLs only. Remove avatar handling and
the --avatars option; the command now focuses solely on status media
(cdn_url, thumbnail_url, optimized_url).
4 weeks ago
Your Name 04536a6e32 Add admin:MigrateLocalMediaURL; replace media:cloud-url-rewrite
Rebuilds stale local media URLs (cdn_url, thumbnail_url, optimized_url) and
avatar cdn_urls from their storage paths via the configured cloud disk.

- Default target host comes from the configured cloud disk (AWS_URL);
  requires confirmation (or --force) and can be overridden with --newDomain.
- Optional --oldDomain filters to a single old backend host; by default all
  stale hosts are rewritten.
- Refuses to run when PF_ENABLE_CLOUD is false (local storage) and, when
  auto-detecting, refuses a target equal to the app domain — so local-storage
  instances are never rewritten.
- Single status id / post URL, --all, --avatars; --dry-run; busts
  MediaService/StatusService caches for affected statuses.
- Removes the superseded media:cloud-url-rewrite command.
- Adds feature tests covering rewrite/skip/dry-run/oldDomain/newDomain/
  remote-skip/local-storage-refusal.
4 weeks ago
Shlee 1ba3f8c9cf
Merge pull request #6928 from pixelfed/feature/user-status-command
Require --scope (local/remote/both) for admin:fixProfileCounts --all
4 weeks ago
Your Name bcd5a5bd7b Require --scope (local/remote/both) for admin:fixProfileCounts --all
Bulk --all reconciliation previously scanned both local and remote profiles
implicitly. Now --all requires an explicit --scope of local, remote, or
both. --active stays local-only and rejects a non-local --scope. Adds the
BelongsTo return type to Profile::user() so the whereHas('user') scope
filter passes Larastan, and adds tests for scope requirement/validation and
local/remote filtering.
4 weeks ago
Shlee 511527fc5a
Merge pull request #6923 from pixelfed/feature/user-status-command
Refactor: FixProfileCounts
4 weeks ago
Your Name 96f26405f1 Rename to admin:fixProfileCounts, make --active its own mode, add --type
- Rename command signature fix:profilecounts -> admin:fixProfileCounts.
- --active is now its own bulk mode (recently-active local accounts),
  mutually exclusive with --all and a single id.
- Add --type=followers|following|statuses to restrict reconciliation to a
  single metric (validated).
- Update/extend tests for the new name, --type restriction and invalid-type
  rejection.
4 weeks ago
Your Name 698ba224e3 Schedule weekly profile-count reconcile and add reconciliation tests
- Add --force flag to fix:profilecounts for unattended runs and schedule
  'fix:profilecounts --all --force' weekly (Sun 03:37) as a safety-net
  reconcile. Kept as a low-frequency full scan rather than a new event-driven
  dirty-set; it only writes profiles that actually drifted.
- Add Feature tests for AccountStatService recompute helpers and
  reconcileProfileCounts (media-type status_count semantics, follower/
  following counts, drift/no-drift/no-write, metric restriction, missing
  profile) plus fix:profilecounts command behavior (silent-when-synced,
  dry-run makes no changes).
4 weeks ago
Your Name f0e951dcce fix: stop caching raw Eloquent models to prevent incomplete-object 500s
Caching an Eloquent model in a Cache::remember closure could deserialize
into a __PHP_Incomplete_Class on read, throwing 'attempt to access a
property on an incomplete object' and returning a 500. This surfaced on
guest profile pages (ProfileController::buildProfile reading
$user->user->settings) and affected several other latent call sites.

Changes:
- ProfileController: cache a plain settings array instead of the
  UserSetting model; fall back to defaults when the settings row is missing
- StoryService::getById: fetch a live model instead of caching it
- InstanceService::getByDomain, CustomEmoji::scan: cache arrays
- Site/MobileController: cache Page data as an array via a shared
  ManagesCachedPages trait; update blade views to array access
- Add public-route smoke/regression tests covering the cache-read path
4 weeks ago
Your Name 302edf09d5 refactor: rename PixelfedUsername rule to ValidUsername
Pure rename of the App\Rules\PixelfedUsername validation rule to
App\Rules\ValidUsername for a clearer, more idiomatic name. Updates
the class, filename, test, and all 8 controller call sites. No
behavior change.
4 weeks ago
Your Name 7c5d93e96b refactor: consolidate username validation into PixelfedUsername rule
Replace 7 duplicated inline username validation closures across 6
controllers (ApiV1Dot1, RemoteAuth, CuratedRegister, AdminInvite x2,
AppRegister, Auth/Register) with the existing PixelfedUsername rule.

Add the 'must contain at least one alphabetical character' check to
the rule so all call sites share consistent, stricter validation.

Add PixelfedUsernameTest covering all validation branches.
4 weeks ago
Your Name 941c305104 fix: resolve larastan class.notFound errors
- Add missing FractalService import to Groups/GroupCommentService and
  Groups/GroupPostService (wrong namespace resolution)
- Update Inbox handler traits to use App\Models\* namespace instead of
  old App\* references (Status, Profile, DirectMessage, Media, Follower,
  Like, Instance, Story, User, FollowRequest, Notification, UserFilter,
  StoryView)
- Update HttpClientMigrationTest to use App\Models\* namespace
4 weeks ago
Your Name e7b70c6084 refactor: extract duplicate patterns into shared methods
1. Add FractalService with static item() and collection() helpers
   replacing 22 call sites that repeated the 4-line Fractal Manager
   + ArraySerializer boilerplate.

2. Add AccountInterstitial::createFromStatus() factory method
   consolidating 4 identical 15-line blocks that create interstitials
   with status metadata.

3. Add NotificationService::createNotification() to handle the
   repeated pattern of creating, caching, and registering a
   notification in the recipient's feed.

4. Add NotificationService::firstOrCreateNotification() for
   idempotent notifications (share/boost, mention) that should
   only notify once per actor+action+item combination.
4 weeks ago
Your Name 9c9e2a5a22 refactor: extract shared ActivityPub pool delivery into ActivityPubDeliveryService
Add ActivityPubDeliveryService::pool() using Laravel's Http::pool() to
consolidate the duplicated delivery pattern found across 10 jobs.

Updated jobs:
- StatusActivityPubDeliver
- StatusDelete
- StatusLocalUpdateActivityPubDeliverPipeline
- FanoutDeletePipeline
- SharePipeline
- UndoSharePipeline
- StoryFanout
- StoryExpire
- StoryDelete
- ProfileMigrationDeliverMoveActivityPipeline

The shared method accepts a Profile (sender), audience (inbox URLs),
and activity (payload array), handling signing, user-agent, timeout,
and concurrency in one place. No direct Guzzle usage remains in
app/Jobs/.
4 weeks ago