mastodon

Commit Graph

Author	SHA1	Message	Date
Claire	8b636a29c6	Improve automatic post cleanup worker performances (#24785 )	2 years ago
Matt Jankowski	f1c1dd0118	Rename `with_lock` to `with_redis_lock` to avoid confusion with ActiveRecord's method (#24741 )	2 years ago
Claire	32a030dd74	Rewrite import feature (#21054 )	2 years ago
Claire	5dc3173ef8	Fix AccountsStatusesCleanupScheduler not spreading deletes across accounts correctly (#24607 )	2 years ago
Claire	501d6197c4	Change automatic post deletion thresholds and load detection (#24614 )	2 years ago
Vyr Cossont	d5ad8b6422	Fix Redis client and type errors introduced in #24285 (#24342 )	2 years ago
Vyr Cossont	7646ad8a2b	IndexingScheduler: fetch and import in batches (#24285 ) Co-authored-by: Claire <claire.github-309c@sitedethib.com>	2 years ago
Christian Schmidt	eb38e9df31	Requeue expiration notification (#24311 )	2 years ago
Vyr Cossont	94cbd808b5	Webhooks for local status.create, status.update, account.update (#24133 )	2 years ago
Claire	1d0ad558ff	Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call (#24034 )	2 years ago
Jean byroot Boussier	922837dc96	Upgrade to latest redis-rb 4.x and fix deprecations (#23616 ) Co-authored-by: Jean Boussier <jean.boussier@gmail.com>	2 years ago
Nick Schonning	b00f945d92	Remove implied StandardError rescue (#23942 )	2 years ago
Claire	050f1669c6	Fix original account being unfollowed on migration before the follow request could be sent (#21957 )	2 years ago
Claire	ddde4e0d95	Change `ActivityPub::DeliveryWorker` retries to be spread out more (#21956 )	2 years ago
Claire	20b80c62ff	Change auto-deletion throttling constants to better scale with server size (#23320 )	2 years ago
Nick Schonning	717683d1c3	Autofix Rubocop remaining Layout rules (#23679 )	2 years ago
Nick Schonning	21bf326356	Autofix Rubocop Rails/Pluck (#23730 )	2 years ago
Nick Schonning	2177daeae9	Autofix Rubocop Style/RedundantBegin (#23703 )	2 years ago
Nick Schonning	a6f77aa28a	Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681 )	2 years ago
Claire	0c9eac80d8	Fix unbounded recursion in post discovery (#23506 ) * Add a limit to how many posts can get fetched as a result of a single request * Add tests * Always pass `request_id` when processing `Announce` activities --------- Co-authored-by: nametoolong <nametoolong@users.noreply.github.com>	2 years ago
Nick Schonning	cec005068f	Apply Rubocop Performance/RedundantBlockCall (#23440 )	2 years ago
Nick Schonning	ed570050c6	Autofix Rails/EagerEvaluationLogMessage (#23429 ) * Autofix Rails/EagerEvaluationLogMessage * Update spec for debug block syntax	2 years ago
Claire	9edefc779f	Fix `UserCleanupScheduler` crash when an unconfirmed account has a moderation note (#23318 ) * Fix `UserCleanupScheduler` crash when an unconfirmed account has a moderation note * Add tests	2 years ago
luzpaz	596923da4a	Fix typos in source documentation (#21046 ) Fixed 2 source comment/documentation typos	2 years ago
Eugen Rochko	f8e8e622e5	Change incoming activity processing to happen in `ingress` queue (#20264 )	2 years ago
Claire	5333447be0	Change account deletion requests to spread out over time (#20222 )	2 years ago
Eugen Rochko	e98833748e	Fix being able to spoof link verification (#20217 ) - Change verification to happen in `default` queue - Change verification worker to only be queued if there's something to do - Add `link` tags from metadata fields to page header of profiles	2 years ago
Claire	5925a31b78	Fix followers count not being updated when migrating follows (#19998 ) Fixes #19900	2 years ago
Eugen Rochko	d0c9ac3919	Fix indexing scheduler trying to index when Elasticsearch is disabled (#19805 ) Fix #19646	2 years ago
Eugen Rochko	5f9e47be34	Add caching for payload serialization during fan-out (#19642 )	2 years ago
Claire	6804228fdf	Fix N+1 on mentions in PushUpdateWorker (#19637 )	2 years ago
Claire	03d9618595	Fix UserCleanupScheduler crash when an unconfirmed account has a moderation note (#19629 ) Fixes #19109	2 years ago
Eugen Rochko	bf0ab3e0fa	Fix vacuum scheduler missing lock, locks never expiring (#19458 ) Remove vacuuming of orphaned preview cards	2 years ago
Takeshi Umeda	74ead7d106	Change featured tag updates to add/remove activity (#19409 ) * Change featured tag updates to add/remove activity * Fix to check for the existence of feature tag * Rename service and worker * Merge AddHashtagSerializer with AddSerializer * Undo removal of sidekiq_options	2 years ago
Takeshi Umeda	b0e3f0312c	Add synchronization of remote featured tags (#19380 ) * Add LIMIT of featured tag to instance API response * Add featured_tags_collection_url to Account * Add synchronization of remote featured tags * Deliver update activity when updating featured tag * Remove featured_tags_collection_url * Revert "Add featured_tags_collection_url to Account" This reverts commit `cff349fc27`. * Add hashtag sync from featured collections * Fix tag name normalize * Add target option to fetch featured collection * Refactor fetch_featured_tags_collection_service * Add LIMIT of featured tag to v1/instance API response	2 years ago
Eugen Rochko	5c9abdeff1	Add retention policy for cached content and media (#19232 )	2 years ago
Claire	8cf7006d4e	Refactor ActivityPub handling to prepare for non-Account actors (#19212 ) * Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService ActivityPub::FetchRemoteAccountService is kept as a wrapper for when the actor is specifically required to be an Account * Refactor SignatureVerification to allow non-Account actors * fixup! Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService * Refactor ActivityPub::FetchRemoteKeyService to potentially return non-Account actors * Refactor inbound ActivityPub payload processing to accept non-Account actors * Refactor inbound ActivityPub processing to accept activities relayed through non-Account * Refactor how Account key URIs are built * Refactor Request and drop unused key_id_format parameter * Rename ActivityPub::Dereferencer `signature_account` to `signature_actor`	2 years ago
Eugen Rochko	50948b46aa	Add ability to filter followed accounts' posts by language (#19095 )	2 years ago
Eugen Rochko	0b3e4fd5de	Remove digest e-mails (#17985 ) * Remove digest e-mails * Remove digest-related code	3 years ago
Eugen Rochko	c3f0621a59	Add ability to follow hashtags (#18809 )	3 years ago
Jeremy Kescher	25e076505e	ip_cleanup_scheduler: Make IP and session retention configurable (#18757 )	3 years ago
Eugen Rochko	a2871cd747	Add administrative webhooks (#18510 ) * Add administrative webhooks * Fix error when webhook is deleted before delivery worker runs	3 years ago
Jeong Arm	fed7380e9f	Prevent use locale with empty string (#18543 ) Somehow user's locale could be an empty string, And empty string itself are treated as true value.	3 years ago
Eugen Rochko	679b7158e3	Change search indexing to use batches to minimize resource usage (#18451 )	3 years ago
Eugen Rochko	6cf57c6765	Refactor how Redis locks are created (#18400 ) * Refactor how Redis locks are created * Fix autorelease duration on account deletion lock	3 years ago
Eugen Rochko	f6d35ed57d	Remove IP matching from e-mail domain blocks (#18190 ) Clear out e-mail domain blocks created from automatically resolved DNS records	3 years ago
Eugen Rochko	3917353645	Fix single Redis connection being used across all threads (#18135 ) * Fix single Redis connection being used across all Sidekiq threads * Fix tests	3 years ago
Jeong Arm	3349cbeb3e	Fix clean_discarded_statuses (#17822 )	3 years ago
Eugen Rochko	bd53dd5210	Change design of federation pages in admin UI (#17704 ) * Change design of federation pages in admin UI * Fix query performance in instance media attachments measure * Fix reblogs being included in instance languages dimension	3 years ago
Eugen Rochko	2ea754b861	Fix duplicate notifications being possible after poll expiration (#17697 )	3 years ago
Eugen Rochko	50ea54b3ed	Change authorized applications page (#17656 ) * Change authorized applications page * Hide revoke button for superapps and suspended accounts * Clean up db/schema.rb	3 years ago
Claire	3d60708508	Fix crash in EmailDomainBlockRefreshScheduler (#17649 )	3 years ago
Eugen Rochko	27965ce5ed	Add trending statuses (#17431 ) * Add trending statuses * Fix dangling items with stale scores in localized sets * Various fixes and improvements - Change approve_all/reject_all to approve_accounts/reject_accounts - Change Trends::Query methods to not mutate the original query - Change Trends::Query#skip to offset - Change follow recommendations to be refreshed in a transaction * Add tests for trending statuses filtering behaviour * Fix not applying filtering scope in controller	3 years ago
Eugen Rochko	a29a982eaa	Change e-mail domain blocks to block IPs dynamically (#17635 ) * Change e-mail domain blocks to block IPs dynamically * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>	3 years ago
Eugen Rochko	8f03b7a2fb	Add notifications when a reblogged status has been updated (#17404 ) * Add notifications when a reblogged status has been updated * Change wording to say "edit" instead of "update" and add missing controls * Replace previous update notifications with the most up-to-date one	3 years ago
Claire	da91b18a8b	Fix NoMethodError in StatusUpdateDistributionWorker (#17499 ) * Add tests * Fix NoMethodError in StatusUpdateDistributionWorker * Fix tests	3 years ago
Eugen Rochko	63002cde03	Add editing for published statuses (#17320 ) * Add editing for published statuses * Fix change of multiple-choice boolean in poll not resetting votes * Remove the ability to update existing media attachments for now	3 years ago
Eugen Rochko	35850f8195	Fix localization of cold-start follow recommendations (#17479 )	3 years ago
Claire	52c1b86964	Fix Ruby 2.5 incompatibility (#17465 )	3 years ago
Claire	0d2cf3cd4a	Fix errors when multiple Delete are received for a given actor (#17460 )	3 years ago
Claire	03d59340da	Fix Sidekiq warnings about JSON serialization (#17381 ) * Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.	3 years ago
Eugen Rochko	9eb775a9d1	Fix error when using raw distribution worker (#17334 ) Regression from #16697	3 years ago
Eugen Rochko	1060666c58	Add support for editing for published statuses (#16697 ) * Add support for editing for published statuses * Fix references to stripped-out code * Various fixes and improvements * Further fixes and improvements * Fix updates being potentially sent to unauthorized recipients * Various fixes and improvements * Fix wrong words in test * Fix notifying accounts that were tagged but were not in the audience * Fix mistake	3 years ago
Eugen Rochko	14f436c457	Add notifications for statuses deleted by moderators (#17204 )	3 years ago
Claire	d5c9feb7b7	Add support for private pinned posts (#16954 ) * Add support for private pinned toots * Allow local user to pin private toots * Change wording to avoid "direct message"	3 years ago
Eugen Rochko	8e84ebf0cb	Remove IP tracking columns from users table (#16409 )	3 years ago
Claire	7f803c41e2	Add ability to purge undeliverable domains from admin interface (#16686 ) * Add ability to purge undeliverable domains from admin interface * Add tests	3 years ago
Takeshi Umeda	bda8e4f815	Fix follow recommendation biased towards older accounts (#17126 )	3 years ago
Eugen Rochko	6e50134a42	Add trending links (#16917 ) * Add trending links * Add overriding specific links trendability * Add link type to preview cards and only trend articles Change trends review notifications from being sent every 5 minutes to being sent every 2 hours Change threshold from 5 unique accounts to 15 unique accounts * Fix tests	3 years ago
Claire	87085a5152	Fix AccountNote not having a maximum length (#16942 )	3 years ago
Claire	5efb1ff337	Fix followers synchronization mechanism not working when URI has empty path (#16510 ) * Fix followers synchronization mechanism not working when URI has empty path To my knowledge, there is no current implementation on the fediverse that can use bare domains (e.g., actor is at https://example.org instead of something like https://example.org/actor) that also plans to support the followers synchronization mechanism. However, Mastodon's current implementation would exclude such accounts from followers list. Also adds tests and rename them to reflect the proper method names. * Move url prefix regexp to its own constant	4 years ago
Claire	4ac78e2a06	Add feature to automatically delete old toots (#16529 ) * Add account statuses cleanup policy model * Record last inspected toot to delete to speed up successive calls to statuses_to_delete * Add service to cleanup a given account's statuses within a budget * Add worker to go through account policies and delete old toots * Fix last inspected status id logic All existing statuses older or equal to last inspected status id must be kept by the current policy. This is an invariant that must be kept so that resuming deletion from the last inspected status remains sound. * Add tests * Refactor scheduler and add tests * Add user interface * Add support for discriminating based on boosts/favs * Add UI support for min_reblogs and min_favs, rework UI * Address first round of review comments * Replace Snowflake#id_at_start with with_random parameter * Add tests * Add tests for StatusesCleanupController * Rework settings page * Adjust load-avoiding mechanisms * Please CodeClimate	4 years ago
Takeshi Umeda	590e1578bf	Fix when MoveWorker cannot get locale from remote account (#16576 )	4 years ago
Claire	1c21dcfa35	Fix newlines in accout notes added by the Move handler (#16415 ) * Fix newlines in account notes added by the move handler * Make MoveWorker more robust	4 years ago
Eugen Rochko	d174d12c83	Add authentication history (#16408 )	4 years ago
Claire	9a19227f17	Fix some RedisLocks auto-releasing too fast (#16276 ) * Fix Delete and Create-related locks expiring too fast Fixes #16238 By default, RedisLock expires after 10 seconds, which may not be enough to process statuses, especially when those have attached media files. This commit extends those 10 seconds to 15 minutes, which should be plenty enough to handle any status, while being short enough to not waste many sidekiq job retries in the exceedingly rare case in which a sidekiq process would crash when processing a `Create` or `Delete`. * Fix other RedisLock autorelease durations Fixes #15645 - things that only perform a few simple database queries (e.g. finding and saving a record) have been left unchanged, so they'll still use the default 10s duration - things that perform significantly more complex database queries have been changed to a 5 minutes timeout - things that perform multiple HTTP queries have been changed to a 15 minutes timeout	4 years ago
Takeshi Umeda	c403c3695b	Fix to be able to redownload avatar and header (#16190 ) * Fix to reset if header and avatar download fails * Add RedownloadAvatarWorker and RedownloadHeaderWorker	4 years ago
Claire	566fc90913	Add Ruby 3.0 support (#16046 ) * Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0 Also improve the Terrapin monkey-patch for the stderr/stdout issue. * Fix keyword argument handling throughout the codebase * Monkey-patch Paperclip to fix keyword arguments handling in validators * Change validation_extensions to please CodeClimate * Bump microformats from 4.2.1 to 4.3.1 * Allow Ruby 3.0 * Add Ruby 3.0 test target to CircleCI * Add test for admin dashboard warnings * Fix admin dashboard warnings on Ruby 3.0	4 years ago
Eugen Rochko	6d9ad30bf8	Fix media redownload worker retrying on unexpected response codes (#16111 )	4 years ago
Claire	d9ae3db8d5	Improve performance of follow recommendation scheduler (#16159 ) Express follow_recommendations in terms of account_summaries rather than accounts, integrate filters that are unconditionally used, and materialize the resulting view. This should result in the bulk of the computation being performed only once instead of once per recommendation language.	4 years ago
Eugen Rochko	fab65848d2	Fix empty home feed before first follow has finished processing (#16152 ) Change queue of merge worker from pull to default	4 years ago
Eugen Rochko	f78cbc0c32	Fix thread resolve worker retrying when status no longer exists (#16109 )	4 years ago
Eugen Rochko	f4b7c6b619	Fix nil error when removing status caused by race condition (#16099 )	4 years ago
Eugen Rochko	120965eb0b	Change Web Push API deliveries to use request pooling (#16014 )	4 years ago
Eugen Rochko	f7117646af	Add cold-start follow recommendations (#15945 )	4 years ago
Eugen Rochko	e89e976e92	Fix configuration for sidekiq-unique-jobs after 7.x upgrade (#15908 ) Remove locks from scheduled jobs	4 years ago
ThibG	f1f96ebf02	Fix being able to import more than allowed number of follows (#15384 ) * Fix being able to import more than allowed number of follows Without this commit, if someone tries importing a second list of accounts to follow before the first one has been processed, this will queue imports for the two whole lists, even if they exceed the account's allowed number of outgoing follows. This commit changes it so the individual queued imports aren't exempt from the follow limit check (they remain exempt from the rate-limiting check though). * Catch validation errors to not re-queue failed follows Co-authored-by: Claire <claire.github-309c@sitedethib.com>	4 years ago
ThibG	3249d35bdc	Improve account deletion performances further (#15407 ) * Delete status records by batches of 50 * Do not precompute values that are only used once * Do not generate redis events for removal of public toots older than two weeks * Filter reported toots a priori for polls and status deletion * Do not process reblogs when cleaning up public timelines As in Mastodon proper, reblogs don't appear in public TLs * Clean the deleted account's own feed in one go * Refactor Account#clean_feed_manager and List#clean_feed_manager * Delete instead of destroy a few more associations * Fix preloading Co-authored-by: Claire <claire.github-309c@sitedethib.com>	4 years ago
ThibG	7bf3c6e57b	Fix AccountDeletionWorker crashing and clogging sidekiq queues (#15380 ) * Fix account deletion workers being queued multiple times for a single account * Fix poll votes being unnecessarily instantiated on poll deletion * Fix favourites being unnecessarily instantiated on status deletion * Remove inaccurate comments * Delete polls instead of destroying them Co-authored-by: Claire <claire.github-309c@sitedethib.com>	4 years ago
ThibG	a60d9335d8	Fix resolving accounts sometimes creating duplicate records for a given AP id (#15364 ) * Fix ResolveAccountService accepting mismatching acct: URI * Set attributes that should be updated regardless of suspension * Fix key fetching * Automatically merge remote accounts with duplicate `uri` * Add tests * Add "tootctl accounts fix-duplicates" Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and merge them under the canonical `acct:` URI. Co-authored-by: Claire <claire.github-309c@sitedethib.com>	4 years ago
Eugen Rochko	216b85b053	Fix performance on instances list in admin UI (#15282 ) - Reduce duplicate queries - Remove n+1 queries - Add accounts count to detailed view - Add separate action log entry for updating existing domain blocks	4 years ago
ThibG	2f6831f318	Fix sending spurious Rejects when processing remote account deletion (#15104 ) * Fix sending spurious Rejects when processing remote account deletion * Make skip_side_effects imply skip_activitypub	4 years ago
Eugen Rochko	df1653174b	Add cache buster feature for media files (#15155 ) Nginx can be configured to bypass proxy cache when a special header is in the request. If the response is cacheable, it will replace the cache for that request. Proxy caching of media files is desirable when using object storage as a way of minimizing bandwidth costs, but has the drawback of leaving deleted media files for a configured amount of cache time. A cache buster can make those media files immediately unavailable. This especially makes sense when suspending and unsuspending an account.	4 years ago
ThibG	0a4d0e8320	Fix AccountDeletionWorker not accepting keyword arguments (#15152 )	4 years ago
Eugen Rochko	3134691948	Add support for reversible suspensions through ActivityPub (#14989 )	4 years ago
ThibG	c49805efb1	Fix poll ending notifications being created for each vote (#15071 ) On a poll ending, notifications were created for each vote instead of for each voter.	4 years ago
ThibG	ca56527140	Add follower synchronization mechanism (#14510 ) * Add support for followers synchronization on the receiving end Check the `collectionSynchronization` attribute on `Create` and `Announce` activities and synchronize followers from provided collection if possible. * Add tests for followers synchronization on the receiving end * Add support for follower synchronization on the sender's end * Add tests for the sending end * Switch from AS attributes to HTTP header Replace the custom `collectionSynchronization` ActivityStreams attribute by an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as the `Signature` header and the following fields: - `collectionId` to specify which collection to synchronize - `digest` for the SHA256 hex-digest of the list of followers known on the receiving instance (where “receiving instance” is determined by accounts sharing the same host name for their ActivityPub actor `id`) - `url` of a collection that should be fetched by the instance actor Internally, move away from the webfinger-based `domain` attribute and use account `uri` prefix to group accounts. * Add environment variable to disable followers synchronization Since the whole mechanism relies on some new preconditions that, in some extremely rare cases, might not be met, add an environment variable (DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and avoid followers being incorrectly removed. The current conditions are: 1. all managed accounts' actor `id` and inbox URL have the same URI scheme and netloc. 2. all accounts whose actor `id` or inbox URL share the same URI scheme and netloc as a managed account must be managed by the same Mastodon instance as well. As far as Mastodon is concerned, breaking those preconditions require extensive configuration changes in the reverse proxy and might also cause other issues. Therefore, this environment variable provides a way out for people with highly unusual configurations, and can be safely ignored for the overwhelming majority of Mastodon administrators. * Only set follower synchronization header on non-public statuses This is to avoid unnecessary computations and allow Follow-related activities to be handled by the usual codepath instead of going through the synchronization mechanism (otherwise, any Follow/Undo/Accept activity would trigger the synchronization mechanism even if processing the activity itself would be enough to re-introduce synchronization) * Change how ActivityPub::SynchronizeFollowersService handles follow requests If the remote lists a local follower which we only know has sent a follow request, consider the follow request as accepted instead of sending an Undo. * Integrate review feeback - rename X-AS-Collection-Synchronization to Collection-Synchronization - various minor refactoring and code style changes * Only select required fields when computing followers_hash * Use actor URI rather than webfinger domain in synchronization endpoint * Change hash computation to be a XOR of individual hashes Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues. * Marginally improve followers_hash computation speed * Further improve hash computation performances by using pluck_each	4 years ago
OSAMU SATO	96761752ec	Add duration parameter to muting. (#13831 ) * Adding duration to muting. * Remove useless checks	4 years ago
Eugen Rochko	5e1364c448	Add IP-based rules (#14963 )	4 years ago
Eugen Rochko	974b1b79ce	Add option to be notified when a followed user posts (#13546 ) * Add bell button Fix #4890 * Remove duplicate type from post-deployment migration * Fix legacy class type mappings * Improve query performance with better index * Fix validation * Remove redundant index from notifications	4 years ago

1 2 3 4 5 ...

414 Commits (36f9c96812c313c5f782a702df221ad4b57da0f3)