190 Commits (b3496583e0d85be80b84657dcfed9c354e5aebe2)

Author	SHA1	Message	Date
Eugen Rochko	7b0fe4aef9	Fix opening and closing Redis connections instead of using a pool (#18171) ... * Fix opening and closing Redis connections instead of using a pool * Fix Redis connections not being returned to the pool in CLI commands	3 years ago
Eugen Rochko	3917353645	Fix single Redis connection being used across all threads (#18135) ... * Fix single Redis connection being used across all Sidekiq threads * Fix tests	3 years ago
Claire	33cd80d69c	Fix instance actor being incorrectly created when running migrations (#18109) ... * Add migration test about instance actor key * Fix old migration * Work around incorrect database state	3 years ago
Eugen Rochko	71f2b95106	Fix edits with no actual changes being allowed (#17843) ... * Fix edits with no actual changes being allowed locally * Fix edits with no actual changes being allowed through ActivityPub * Fix false positive changes caused by description processing in model * Fix not recording poll expiration update * Fix test * Revert changes to ProcessStatusUpdateService * Various fixes and improvements * Fix code style issues * Various changes and improvements * Add guard clause	3 years ago
Claire	642528f455	Update fix-duplicates maintenance task (#17731) ... * Update fix-duplicates task to 2022_02_10_153119 Also add support for Appeal to AccountMerging#merge_with! * Update fix-duplicates task to 2022_03_07_094650 * Update fix-duplicates task to 2022_03_09_213005 * Update fix-duplicates task to 2022_03_07_083603 * Update fix-duplicates task to 2022_03_10_060626 * Update fix-duplicates script to 2022_03_07_083603 * Update fix-duplicates task to 2022_03_10_060706 * Update fix-duplicates task to 2022_03_10_060959 * Silence CodeClimate	3 years ago
chandrn7	a6ed6845c9	Allow login through OpenID Connect (#16221) ... * added OpenID Connect as an SSO option * minor fixes * added comments, removed an option that shouldn't be set * fixed Gemfile.lock * added newline to end of Gemfile.lock * removed tab from Gemfile.lock * remove chomp * codeclimate changes and small name change to make function's purpose clearer * codeclimate fix * added SSO buttons to /about page * minor refactor * minor style change * removed spurious change * removed unecessary conditional from ensure_valid_username and added support for auth.info.name in user_params_from_auth * minor changes	3 years ago
Eugen Rochko	14f436c457	Add notifications for statuses deleted by moderators (#17204)	3 years ago
Eugen Rochko	7de0ee7aba	Remove Keybase integration (#17045)	3 years ago
Takeshi Umeda	17f4e457b3	Add remove from followers api (#16864) ... * Add followed_by? to account_interactions * Add RemoveFromFollowersService * Fix AccountBatch to use RemoveFromFollowersService * Add remove from followers API	3 years ago
Claire	fc3ae1343d	Switch from unmaintained paperclip to kt-paperclip (#16724) ... * Switch from unmaintained paperclip to kt-paperclip * Drop some compatibility monkey-patches not required by kt-paperclip * Drop media spoof check monkey-patching It's broken with kt-paperclip and hopefully it won't be needed anymore * Fix regression introduced by paperclip 6.1.0 * Do not rely on pathname to call FastImage * Add test for ogg vorbis file with cover art * Add audio/vorbis to the accepted content-types This seems erroneous as this would be the content-type for a vorbis stream without an ogg container, but that's what the `marcel` gem outputs, so… * Restore missing for_as_default method * Refactor Attachmentable concern and delay Paperclip's content-type spoof check Check for content-type spoofing *after* setting the extension ourselves, this fixes a regression with kt-paperclip's validations being more strict than paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown extensions. * Please CodeClimate * Add audio/vorbis to the unreliable set It doesn't correspond to a file format and thus has no extension associated.	3 years ago
Claire	5efb1ff337	Fix followers synchronization mechanism not working when URI has empty path (#16510) ... * Fix followers synchronization mechanism not working when URI has empty path To my knowledge, there is no current implementation on the fediverse that can use bare domains (e.g., actor is at https://example.org instead of something like https://example.org/actor) that also plans to support the followers synchronization mechanism. However, Mastodon's current implementation would exclude such accounts from followers list. Also adds tests and rename them to reflect the proper method names. * Move url prefix regexp to its own constant	4 years ago
Claire	4ac78e2a06	Add feature to automatically delete old toots (#16529) ... * Add account statuses cleanup policy model * Record last inspected toot to delete to speed up successive calls to statuses_to_delete * Add service to cleanup a given account's statuses within a budget * Add worker to go through account policies and delete old toots * Fix last inspected status id logic All existing statuses older or equal to last inspected status id must be kept by the current policy. This is an invariant that must be kept so that resuming deletion from the last inspected status remains sound. * Add tests * Refactor scheduler and add tests * Add user interface * Add support for discriminating based on boosts/favs * Add UI support for min_reblogs and min_favs, rework UI * Address first round of review comments * Replace Snowflake#id_at_start with with_random parameter * Add tests * Add tests for StatusesCleanupController * Rework settings page * Adjust load-avoiding mechanisms * Please CodeClimate	4 years ago
Claire	763ab0c7eb	Fix owned account notes not being deleted when an account is deleted (#16579) ... * Add account_notes relationship * Add tests * Fix owned account notes not being deleted when an account is deleted * Add post-migration to clean up orphaned account notes	4 years ago
Eugen Rochko	d174d12c83	Add authentication history (#16408)	4 years ago
Claire	526332c545	Fix account deletion sometimes failing because of optimistic locks (#16317) ... * Fix account deletion sometimes failing because of optimistic locks In some rare occasions[1], deleting accounts would fail with a `StaleObjectError` exception. Indeed, account deletion manually sets the `AccountStat` values without handling cases where the optimistic locking on `AccountStat` would fail. To my knowledge, with the rewrite of account counters in #15913, the `DeleteAccountService` is now the only place that changes the counters in a way that is not atomic. Since in this specific case, we do not care about the previous values of the account counters, it appears we don't need locking at all for this table anymore. [1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602 * Bump MAX_SUPPORTED_VERSION in maintenance script	4 years ago
Claire	76064e6608	Update fix-duplicates maintenance script to support latest migrations (#16231) ... * Update maintenance script to support latest database migrations * Update Account#merge_with!	4 years ago
Takeshi Umeda	c403c3695b	Fix to be able to redownload avatar and header (#16190) ... * Fix to reset if header and avatar download fails * Add RedownloadAvatarWorker and RedownloadHeaderWorker	4 years ago
Eugen Rochko	fab65848d2	Fix empty home feed before first follow has finished processing (#16152) ... Change queue of merge worker from pull to default	4 years ago
Eugen Rochko	f7117646af	Add cold-start follow recommendations (#15945)	4 years ago
Claire	cbd0ee1d07	Update Mastodon to Rails 6.1 (#15910) ... * Update devise-two-factor to unreleased fork for Rails 6 support Update tests to match new `rotp` version. * Update nsa gem to unreleased fork for Rails 6 support * Update rails to 6.1.3 and rails-i18n to 6.0 * Update to unreleased fork of pluck_each for Ruby 6 support * Run "rails app:update" * Add missing ActiveStorage config file * Use config.ssl_options instead of removed ApplicationController#force_ssl Disabled force_ssl-related tests as they do not seem to be easily testable anymore. * Fix nonce directives by removing Rails 5 specific monkey-patching * Fix fixture_file_upload deprecation warning * Fix yield-based test failing with Rails 6 * Use Rails 6's index_with when possible * Use ActiveRecord::Cache::Store#delete_multi from Rails 6 This will yield better performances when deleting an account * Disable Rails 6.1's automatic preload link headers Since Rails 6.1, ActionView adds preload links for javascript files in the Links header per default. In our case, that will bloat headers too much and potentially cause issues with reverse proxies. Furhermore, we don't need those links, as we already output them as HTML link tags. * Switch to Rails 6.0 default config * Switch to Rails 6.1 default config * Do not include autoload paths in the load path	4 years ago
Eugen Rochko	af8fe6e1e9	WIP (#15222)	4 years ago
Claire	741d0952b1	Improve account counters handling (#15913) ... * Improve account counters handling * Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them Keep using string interpolation for `key` as it is safe and using “ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching bits of SQL in a way that is not more easily checked for safety. * Add migration hook to catch PostgreSQL versions earlier than 9.5	4 years ago
Claire	1b02d29be5	Fix not being able to change world filter expiration back to “Never” (#15858) ... Fixes #15849	4 years ago
Claire	acdeb162b8	Create instance actor if it hasn't been properly seeded (#15693) ... An uncommon but somewhat difficult to digagnose issue is dealing with improperly-seeded databases. In such cases, instance-signed fetches will fail with a ActiveRecord::RecordNotFound error, usually caught and handled as generic 404, leading people to think the remote resource itself has not been found, while it's the local instance actor that does not exist. This commit changes the code so that failure to find the instance actor automatically creates a new one, so that improperly-seeded databases do not cause any issue.	4 years ago
luigi	eb51e43fb4	Optimize some regex matching (#15528) ... * Use Regex#match? * Replace =~ too * Avoid to call match? from Nil * Keep value of Regexp.last_match	4 years ago
luigi	087ed84367	Optimize map { ... }.compact calls (#15513) ... * Optimize map { ... }.compact using Enumerable#filter_map, supported since Ruby 2.7 * Add poyfill for Enumerable#filter_map	4 years ago
ThibG	f1f96ebf02	Fix being able to import more than allowed number of follows (#15384) ... * Fix being able to import more than allowed number of follows Without this commit, if someone tries importing a second list of accounts to follow before the first one has been processed, this will queue imports for the two whole lists, even if they exceed the account's allowed number of outgoing follows. This commit changes it so the individual queued imports aren't exempt from the follow limit check (they remain exempt from the rate-limiting check though). * Catch validation errors to not re-queue failed follows Co-authored-by: Claire <claire.github-309c@sitedethib.com>	4 years ago
Eugen Rochko	216b85b053	Fix performance on instances list in admin UI (#15282) ... - Reduce duplicate queries - Remove n+1 queries - Add accounts count to detailed view - Add separate action log entry for updating existing domain blocks	4 years ago
ThibG	fbe7519243	Fix account merging in maintenance script (#15264) ... Also include AccountNote and AccountDeletionRequest to the list of classes needing to be reassigned the merged account.	4 years ago
ThibG	9136be480f	Fix followers hash cache not being invalidated on account merge (#15256) ... Also clear relationships cache.	4 years ago
Eugen Rochko	f844386809	Add `tootctl accounts merge` (#15201) ... * Add `tootctl accounts merge` * Update lib/mastodon/accounts_cli.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>	4 years ago
ThibG	ca56527140	Add follower synchronization mechanism (#14510) ... * Add support for followers synchronization on the receiving end Check the `collectionSynchronization` attribute on `Create` and `Announce` activities and synchronize followers from provided collection if possible. * Add tests for followers synchronization on the receiving end * Add support for follower synchronization on the sender's end * Add tests for the sending end * Switch from AS attributes to HTTP header Replace the custom `collectionSynchronization` ActivityStreams attribute by an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as the `Signature` header and the following fields: - `collectionId` to specify which collection to synchronize - `digest` for the SHA256 hex-digest of the list of followers known on the receiving instance (where “receiving instance” is determined by accounts sharing the same host name for their ActivityPub actor `id`) - `url` of a collection that should be fetched by the instance actor Internally, move away from the webfinger-based `domain` attribute and use account `uri` prefix to group accounts. * Add environment variable to disable followers synchronization Since the whole mechanism relies on some new preconditions that, in some extremely rare cases, might not be met, add an environment variable (DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and avoid followers being incorrectly removed. The current conditions are: 1. all managed accounts' actor `id` and inbox URL have the same URI scheme and netloc. 2. all accounts whose actor `id` or inbox URL share the same URI scheme and netloc as a managed account must be managed by the same Mastodon instance as well. As far as Mastodon is concerned, breaking those preconditions require extensive configuration changes in the reverse proxy and might also cause other issues. Therefore, this environment variable provides a way out for people with highly unusual configurations, and can be safely ignored for the overwhelming majority of Mastodon administrators. * Only set follower synchronization header on non-public statuses This is to avoid unnecessary computations and allow Follow-related activities to be handled by the usual codepath instead of going through the synchronization mechanism (otherwise, any Follow/Undo/Accept activity would trigger the synchronization mechanism even if processing the activity itself would be enough to re-introduce synchronization) * Change how ActivityPub::SynchronizeFollowersService handles follow requests If the remote lists a local follower which we only know has sent a follow request, consider the follow request as accepted instead of sending an Undo. * Integrate review feeback - rename X-AS-Collection-Synchronization to Collection-Synchronization - various minor refactoring and code style changes * Only select required fields when computing followers_hash * Use actor URI rather than webfinger domain in synchronization endpoint * Change hash computation to be a XOR of individual hashes Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues. * Marginally improve followers_hash computation speed * Further improve hash computation performances by using pluck_each	4 years ago
OSAMU SATO	96761752ec	Add duration parameter to muting. (#13831) ... * Adding duration to muting. * Remove useless checks	4 years ago
Eugen Rochko	5e1364c448	Add IP-based rules (#14963)	4 years ago
Eugen Rochko	974b1b79ce	Add option to be notified when a followed user posts (#13546) ... * Add bell button Fix #4890 * Remove duplicate type from post-deployment migration * Fix legacy class type mappings * Improve query performance with better index * Fix validation * Remove redundant index from notifications	4 years ago
Eugen Rochko	ed099d8bdc	Change account suspensions to be reversible by default (#14726)	4 years ago
tateisu	fa0c71f0d9	allow pagination by min_id and max_id (#14776) ... * allow pagination by min_id and max_id * also AccountConversation allows min_id,max_id pair * also home,list TL allows min_id,max_id pair	4 years ago
Akihiko Odaki	ae871c4d46	Make Array-creation behavior of Paginable more predictable (#14687) ... * Make Array-creation behavior of Paginable more predictable Paginable.paginate_by_id usually returns ActiveRecord::Relation, but it returns an Array if min_id option is present. The behavior caused problems fixed with the following commits: - 552e886b64 - b63ede5005 - 64ef37b89d To prevent from recurring similar problems, this commit introduces two changes: - The scope now always returns an Array whether min_id option is present or not. - The scope is renamed to to_a_paginated_by_id to clarify it returns an Array. * Transform Paginable.to_a_paginated_by_id from a scope to a class method https://api.rubyonrails.org/classes/ActiveRecord/Scoping/Named/ClassMethods.html#method-i-scope > The method is intended to return an ActiveRecord::Relation object, which > is composable with other scopes. Paginable.to_a_paginated_by_id returns an Array and is not appropriate as a scope.	5 years ago
Takeshi Umeda	7540e235a2	Fix movie width and frame_rate returning nil (#14357) ... * Fix movie width and frame_rate returning nil * Add StreamValidationError and raise * Fix code style	5 years ago
abcang	6a96af4d20	Fix rubocop warning (#14288) ... * Fix rubocop warning * use limit variable * use ContextCreatingMethods option	5 years ago
Eugen Rochko	7aaf2b44ec	Fix remote files not using Content-Type header, streaming (#14184)	5 years ago
ThibG	65506bac3f	Add user notes on accounts (#14148) ... * Add UserNote model * Add UI for user notes * Put comment in relationships entity * Add API to create user notes * Copy user notes to new account when receiving a Move activity * Address some of the review remarks * Replace modal by inline edition * Please CodeClimate * Button design changes * Change design again * Cancel note edition when pressing Escape * Fixes * Tweak design again * Move “Add note” item, and allow users to add notes to themselves * Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”	5 years ago
Eugen Rochko	1b198d6489	Fix trying to write non-existent image remote URL attribute on preview cards (#14181) ... Regression from #14145	5 years ago
Eugen Rochko	64aac30733	Add customizable thumbnails for audio and video attachments (#14145) ... - Change audio files to not be stripped of metadata - Automatically extract cover art from audio if it exists - Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id` - Add `icon` to represent it in attachments in ActivityPub - Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null - Fix duration of audio not being displayed on public pages until the file is loaded	5 years ago
Eugen Rochko	5d8398c8b8	Add E2EE API (#13820)	5 years ago
ThibG	51ff679b9d	Fix admin page crashing when trying to block an invalid domain name (#13884) ... * Fix admin page crashing when trying to block an invalid domain name Fixes #13880 * Fix trailing and leading spaces not being properly stripped for domain blocks	5 years ago
kaiyou	3511528e50	Only check locally when deduplicating usernames (#13581) ... When deduplicating account usernames for OAuthable users, the routine did check if any account was known with that username, including remote accounts. This caused some unnecessary deduplication, and usernames ending with unexpected trailing _1. This fixes #13580	5 years ago
Eugen Rochko	9660aa4543	Change local media attachments to perform heavy processing asynchronously (#13210) ... Fix #9106	5 years ago
Eugen Rochko	339ce1c4e9	Add specific rate limits for posting and following (#13172)	5 years ago
Eugen Rochko	ff07e80533	Fix remote convertible media attachments not being saved (#13032) ... Because the file name was set after loading the file, Paperclip was flagging the differing file extension as content type spoofing Fix #12938	5 years ago