1018 Commits (93e8a15415c35dbe4089b5d20c08161646ba76b3)

Author	SHA1	Message	Date
Claire	bb89f0af8a	Fix ActivityPub::ProcessingWorker error on incoming malformed JSON-LD (#23416)	2 years ago
Markus Unterwaditzer	f2a6e71bb6	Suppress AddressFamilyError in link verification (#23204) ... * Suppress AddressFamilyError * clarify comment	2 years ago
Claire	628dcbb732	Revert "Remove LDSignature on actor Delete activities (#21466)" (#23185) ... This reverts commit f4f2b062ec.	2 years ago
Claire	68dcbcb7bf	Add more specific error messages to HTTP signature verification (#21617) ... * Return specific error on failure to parse Date header * Add error message when preferredUsername is not set * Change error report to be JSON and include more details * Change error report to differentiate unknown account and failed refresh * Add tests	2 years ago
Claire	4b92e59f4f	Add support for editing media description and focus point of already-posted statuses (#20878) ... * Add backend support for editing media attachments of existing posts * Allow editing media attachments of already-posted toots * Add tests	2 years ago
Markus Unterwaditzer	0c689b9d01	fix: allow verification when page size exceeds 1MB (using HTML5 parser) (#22879) ... * fix: allow verification when page size exceeds 1MB Truncates the page after 1MB instead Closes #15316 * switch to HTML5 parser, fix rubocop errors * undo rubocop fixes Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>	2 years ago
Kaspar V	ae62e5fa53	Fix/remove calling private method with send in model (#22951) ... * fix(status): remove send usage for private unlink_from_conversations - make unlink_from_conversations public method - rename unlink_from_conversations to unlink_from_conversations! - fix send call on private method in statuses_vacuum and batched_remove_status_service * fix(feeds_vacuum): replace find_in_batches with in_batches because active record query results should be a little more efficient than itterating with map and each. Postgres can grasp such lists of ids much quicker than ruby can. Will probably make allmost no difference, but cannot hurt either.	2 years ago
Claire	18fb01ef7c	Fix possible race conditions when suspending/unsuspending accounts (#22363) ... * Fix possible race conditions when suspending/unsuspending accounts * Fix tests Tests were assuming SuspensionWorker and UnsuspensionWorker would do the suspending/unsuspending themselves, but this has changed.	2 years ago
Effy Elden	4dc9152b3e	Be more lenient with OEmbed detection and validation (#22533)	2 years ago
Claire	c1de673060	Fix suspension worker crashing on S3-compatible setups without ACL support (#22487)	2 years ago
Claire	bf1c7e2122	Ensure exact match is the first result in hashtag searches (#21315) ... Fixes #17494	2 years ago
Claire	2644a28cb3	Change remote media files to be downloaded outside of transactions (#21796)	2 years ago
Brian Campbell	2d12948220	Fix idempotency when database writes are slow (#21840) ... There is an idempotency key generated by clients when authoring a post, and stored in Redis, to ensure that if a user or client retries posting the same status, we don't get a duplicate. Hachyderm.io has been experiencing some filesystem and database performance issues, causing database writes to be slow. This can mean that there are successful posts, but the reverse proxy returns 504 Gateway Timeout before the idempotency status has been updated; users or clients who retry (such as Tusky which retries automatically, see tuskyapp/Tusky#2951) can re-try the same post with the same idempotency key before it has actually been recorded in Redis, leading to duplicate posts. To address this issue, move all of the database updates after the initial transaction that creates the status into the `postprocess_status!` method, so we can insert the idempotency key immediately after the status has been created, significantly reducing the window in which the status could be created but the idempotency key not yet stored. Note: this has not yet been tested; I'm submitting this PR for discussion and to offer to the Hachyderm.io admins to try out to fix the multiple posting problem. Co-authored-by: Brian Campbell <brcampbell@beta.team>	2 years ago
Claire	c8849d6cee	Fix unbounded recursion in account discovery (#22025) ... * Fix trying to fetch posts from other users when fetching featured posts * Rate-limit discovery of new subdomains * Put a limit on recursively discovering new accounts	2 years ago
Claire	f4f2b062ec	Remove LDSignature on actor Delete activities (#21466) ... They are currently not used for anything and represent more than half of the payload size.	2 years ago
Claire	625216d8e1	Fix attachments of edited statuses not being fetched (#21565) ... * Fix attachments of edited statuses not being fetched * Fix tests	2 years ago
Joshua Wood	daf6f3453e	Handle links with no href in VerifyLinkService (#20741) ... Before this change, the following error would cause VerifyAccountLinksWorker to fail: NoMethodError: undefined method `downcase' for nil:NilClass [PROJECT_ROOT]/app/services/verify_link_service.rb:31 :in `block in link_back_present?`	2 years ago
Hampton Lintorn-Catlin	147d8bd8fc	Support UTF-8 Characters in Domains During CSV Import (#20592) ... * Support UTF-8 Characters in Domains During Import * Update Changelong	2 years ago
James Tucker	78a6b871fe	Improve performance by avoiding regex construction (#20215) ... ```ruby 10.times { p /#{FOO}/.object_id } 10.times { p FOO_RE.object_id } ```	2 years ago
Claire	a5394980f2	Fix NameError in Webfinger redirect handling in ActivityPub::FetchRemoteActorService (#20260)	2 years ago
Eugen Rochko	e98833748e	Fix being able to spoof link verification (#20217) ... - Change verification to happen in `default` queue - Change verification worker to only be queued if there's something to do - Add `link` tags from metadata fields to page header of profiles	2 years ago
trwnh	b1a48e05b6	Change Report category to "violation" if rule IDs are provided (#20137) ... * Change Report category to "violation" if rule IDs are provided * Fix LiteralAsCondition * Add parentheses to conditional statement	2 years ago
Claire	bbf74498f5	Fix validation error in SynchronizeFeaturedTagsCollectionWorker (#20018) ... * Fix followers count not being updated when migrating follows Fixes #19900 * Fix validation error in SynchronizeFeaturedTagsCollectionWorker Also saves remote user's chosen case for hashtags * Limit remote featured tags before validation	2 years ago
Eugen Rochko	c4b92b1aee	Fix n+1 query during status removal (#19753)	2 years ago
Claire	c2170991c7	Fix reblogs being discarded after the reblogged status (#19731)	2 years ago
Eugen Rochko	5f9e47be34	Add caching for payload serialization during fan-out (#19642)	2 years ago
Claire	4fb0aae636	Change mentions of blocked users to not be processed (#19725) ... Fixes #19698	2 years ago
Claire	e0eb39d41b	Fix bookmark import stopping at the first failure (#19669) ... Fixes #19389	2 years ago
Eugen Rochko	dc5c86add7	Fix account migration form ever using outdated account data (#18429)	2 years ago
Eugen Rochko	f6bcf86caf	Fix wrong math function used in search query (#19481)	2 years ago
Eugen Rochko	7d25f72b9f	Fix negatives values in search index causing queries to fail (#19464)	2 years ago
Eugen Rochko	1ae508bf2f	Change unauthenticated search to not support pagination in REST API (#19326) ... - Only exact search matches for queries with < 5 characters - Do not support queries with `offset` (pagination) - Return HTTP 401 on truthy `resolve` instead of overriding to false	2 years ago
Yamagishi Kazutoshi	45d3b32488	Fix `Settings::FeaturedTagsController` (#19418) ... Regression from #19409	2 years ago
Takeshi Umeda	74ead7d106	Change featured tag updates to add/remove activity (#19409) ... * Change featured tag updates to add/remove activity * Fix to check for the existence of feature tag * Rename service and worker * Merge AddHashtagSerializer with AddSerializer * Undo removal of sidekiq_options	2 years ago
Yamagishi Kazutoshi	94feb2b93f	Fix `FetchFeaturedCollectionService` spec (#19401) ... Regression from #19380	2 years ago
Takeshi Umeda	b0e3f0312c	Add synchronization of remote featured tags (#19380) ... * Add LIMIT of featured tag to instance API response * Add featured_tags_collection_url to Account * Add synchronization of remote featured tags * Deliver update activity when updating featured tag * Remove featured_tags_collection_url * Revert "Add featured_tags_collection_url to Account" This reverts commit cff349fc27. * Add hashtag sync from featured collections * Fix tag name normalize * Add target option to fetch featured collection * Refactor fetch_featured_tags_collection_service * Add LIMIT of featured tag to v1/instance API response	2 years ago
Claire	cedcece0cc	Fix deleted pinned posts potentially counting towards the pinned posts limit (#19005) ... Fixes #18938	2 years ago
Eugen Rochko	55a2e9b5be	Fix translations not being formatted, other issues in web UI (#19245) ... Fix #19237	3 years ago
Claire	85890bc80f	Fix crash in FetchRemoteKeyService (#19225) ... Fix regression from #19212	3 years ago
Eugen Rochko	0d6b878808	Add user content translations with configurable backends (#19218)	3 years ago
Claire	8cf7006d4e	Refactor ActivityPub handling to prepare for non-Account actors (#19212) ... * Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService ActivityPub::FetchRemoteAccountService is kept as a wrapper for when the actor is specifically required to be an Account * Refactor SignatureVerification to allow non-Account actors * fixup! Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService * Refactor ActivityPub::FetchRemoteKeyService to potentially return non-Account actors * Refactor inbound ActivityPub payload processing to accept non-Account actors * Refactor inbound ActivityPub processing to accept activities relayed through non-Account * Refactor how Account key URIs are built * Refactor Request and drop unused key_id_format parameter * Rename ActivityPub::Dereferencer `signature_account` to `signature_actor`	3 years ago
Eugen Rochko	50948b46aa	Add ability to filter followed accounts' posts by language (#19095)	3 years ago
Claire	7b38cb88ca	Fix ProcessMentionService swallowing unprocessed mentions to unconfirmed/unapproved users (#19191)	3 years ago
Claire	1145dbd327	Improve error reporting and logging when processing remote accounts (#15605) ... * Add a more descriptive PrivateNetworkAddressError exception class * Remove unnecessary exception class to rescue clause * Remove unnecessary include to JsonLdHelper * Give more neutral error message when too many webfinger redirects * Remove unnecessary guard condition * Rework how “ActivityPub::FetchRemoteAccountService” handles errors Add “suppress_errors” keyword argument to avoid raising errors in ActivityPub::FetchRemoteAccountService#call (default/previous behavior). * Rework how “ActivityPub::FetchRemoteKeyService” handles errors Add “suppress_errors” keyword argument to avoid raising errors in ActivityPub::FetchRemoteKeyService#call (default/previous behavior). * Fix Webfinger::RedirectError not being a subclass of Webfinger::Error * Add suppress_errors option to ResolveAccountService Defaults to true (to preserve previous behavior). If set to false, errors will be raised instead of caught, allowing the caller to be informed of what went wrong. * Return more precise error when failing to fetch account signing AP payloads * Add tests * Fixes * Refactor error handling a bit * Fix various issues * Add specific error when provided Digest is not 256 bits of base64-encoded data * Please CodeClimate * Improve webfinger error reporting	3 years ago
Claire	c7147bab90	Fix incorrect and slow cache invalidation in ClearDomainMediaService (#19062) ... Fixes #19060	3 years ago
Eugen Rochko	d83faa1a89	Add ability to block sign-ups from IP (#19037)	3 years ago
Jeong Arm	6aa83b13ba	Properly delete remote account's avatar/header when fetch/update (#18973)	3 years ago
Eugen Rochko	c3f0621a59	Add ability to follow hashtags (#18809)	3 years ago
Eugen Rochko	44b2ee3485	Add customizable user roles (#18641) ... * Add customizable user roles * Various fixes and improvements * Add migration for old settings and fix tootctl role management	3 years ago
Claire	1b4054256f	Fix crash when a remote Flag activity mentions a private post (#18760) ... * Add tests * Fix crash when a remote Flag activity mentions a private post	3 years ago
Eugen Rochko	2936f42a14	Add notifications for new reports (#18697)	3 years ago
Eugen Rochko	a2871cd747	Add administrative webhooks (#18510) ... * Add administrative webhooks * Fix error when webhook is deleted before delivery worker runs	3 years ago
Eugen Rochko	52f4e834f2	Fix concurrent unfollowing decrementing follower count more than once (#18527)	3 years ago
Eugen Rochko	8a9acbe604	Fix being able to appeal a strike unlimited times (#18529) ... Peculiarity of the `has_one` association is that the convenience creation method deletes the previous association even if the new one is invalid	3 years ago
Eugen Rochko	c4d2c39a75	Fix being able to report otherwise inaccessible statuses (#18528)	3 years ago
Eugen Rochko	1ff4877945	Fix empty votes arbitrarily increasing voters count in polls (#18526)	3 years ago
Eugen Rochko	976cd6413e	Fix moderator leak in undo_mark_statuses_as_sensitive (#18525) ... Signed-off-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: 40826d <74816220+40826d@users.noreply.github.com>	3 years ago
Claire	440eb71310	Change unapproved and unconfirmed account to not be accessible in the REST API (#17530) ... * Change unapproved and unconfirmed account to not be accessible in the REST API * Change Account#searchable? to reject unconfirmed and unapproved users * Disable search for unapproved and unconfirmed users in Account.search_for * Disable search for unapproved and unconfirmed users in Account.advanced_search_for * Remove unconfirmed and unapproved accounts from Account.searchable scope * Prevent mentions to unapproved/unconfirmed accounts * Fix some old tests for Account.advanced_search_for * Add some Account.advanced_search_for tests for existing behaviors * Add some tests for Account.search_for * Add Account.advanced_search_for tests unconfirmed and unapproved accounts * Add Account.searchable tests * Fix Account.without_unapproved scope potentially messing with previously-applied scopes * Allow lookup of unconfirmed/unapproved accounts through /api/v1/accounts/lookup This is so that the API can still be used to check whether an username is free to use.	3 years ago
Claire	e0bdaeab65	Fix NoMethodError when resolving a link that redirects to a local post (#18314) ... * Fix NoMethodError when resolving a link that redirects to a local post * Fix tests	3 years ago
Eugen Rochko	6cf57c6765	Refactor how Redis locks are created (#18400) ... * Refactor how Redis locks are created * Fix autorelease duration on account deletion lock	3 years ago
Sara Golemon	336c23336a	Allow VerifyLinkService to accept backlinks with differing case (#18320)	3 years ago
Claire	0597e5db68	Fix ArgumentError when processing pinned posts from Friendica (#18260) ... Follow-up to #18254	3 years ago
Claire	71d02ffcf3	Fix compatibility with Friendica regarding pinned posts (#18254) ... * Fix multiple database queries when fetching pinned posts for remote account * Fix compatibility with Friendica regarding pinned posts Fixes #18066 * Add tests	3 years ago
Claire	9a3be0ad68	Fix error when looking handle with surrounding spaces (#18225)	3 years ago
Claire	84d991988e	Fix temporary network/remote server error prevent from interactions with remote accounts (#18161) ... * Fix temporary network/remote server error prevent from interactions with remote accounts * Fix and add tests	3 years ago
Eugen Rochko	3917353645	Fix single Redis connection being used across all threads (#18135) ... * Fix single Redis connection being used across all Sidekiq threads * Fix tests	3 years ago
Jeong Arm	1de748bf5e	Fix FetchFeaturedCollectionService (#18030)	3 years ago
Eugen Rochko	8e20e16cf0	Change e-mail notifications to only be sent when recipient is offline (#17984) ... * Change e-mail notifications to only be sent when recipient is offline Change the default for follow and mention notifications back on * Add preference to always send e-mail notifications * Change wording	3 years ago
Claire	8f91e304a5	Fix spurious edits and require incoming edits to be explicitly marked as such (#17918) ... * Change post text edit to not be considered significant if it's identical after reformatting * We don't need to clear previous change information anymore * Require status edits to be explicit, except for poll tallies * Fix tests * Add some tests * Add poll-related tests * Add HTML-formatting related tests	3 years ago
Claire	454ef42aab	Fix error when encountering invalid pinned posts (#17964)	3 years ago
Claire	abb11778d7	Fix inconsistency in error handling when removing a status (#17974) ... Not completely sure this could actually have any ill effect, but if `RemoveStatusService` fails to acquire a lock in an `ActivityPub::ProcessingWorker` job processing a `Delete`, the status is currently discarded and causes a job failure but the next time the job is attempted, it will skip deleting the status due to it being discarded. This commit makes the behavior of `RemoveStatusService` a bit more consistent in case of failure to acquire the lock.	3 years ago
Claire	2cc7ba2671	Refactor `response_to_recipient?` CTE (#17899) ... * Optimize and clean up `response_to_recipient?` CTE Marginally improve performances, and make the CTE much more readable * Limit max depth in `response_to_recipient?` CTE	3 years ago
Eugen Rochko	cefa526c6d	Refactor formatter (#17828) ... * Refactor formatter * Move custom emoji pre-rendering logic to view helpers * Move more methods out of Formatter * Fix code style issues * Remove Formatter * Add inline poll options to RSS feeds * Remove unused helper method * Fix code style issues * Various fixes and improvements * Fix test	3 years ago
Eugen Rochko	71f2b95106	Fix edits with no actual changes being allowed (#17843) ... * Fix edits with no actual changes being allowed locally * Fix edits with no actual changes being allowed through ActivityPub * Fix false positive changes caused by description processing in model * Fix not recording poll expiration update * Fix test * Revert changes to ProcessStatusUpdateService * Various fixes and improvements * Fix code style issues * Various changes and improvements * Add guard clause	3 years ago
Claire	92a86b958e	Fix issues with processing toot edits (#17748) ... * Fix searching for an already-known status by URL not working * Fix Update processing from statuses prior to 20220302232632 `ordered_media_attachment_ids_changed?` would return `true` when going from `nil` to anything (including `[]`). * Add tests	3 years ago
Claire	d3aa9cf774	Fix Updates being forwarded even when not processable or causing no change (#17699) ... * Fix Updates being forwarded even when not processable or causing no change * Refactor and ensure status edit is strictly newer than last known edit	3 years ago
Eugen Rochko	ddbe906c25	Fix not updating a status when newer version is fetched manually (#17745)	3 years ago
Eugen Rochko	b2cd34474b	Add rate limit for editing (#17728)	3 years ago
Eugen Rochko	d17fb70131	Change how changes to media attachments are stored for edits (#17696) ... * Change how changes to media attachments are stored for edits Fix not being able to re-order media attachments * Fix not broadcasting updates when polls/media is changed through ActivityPub * Various fixes and improvements * Update app/models/report.rb Co-authored-by: Claire <claire.github-309c@sitedethib.com> * Add tracking of media attachment description changes * Change poll in status edit to have a structure closer to the real one Co-authored-by: Claire <claire.github-309c@sitedethib.com>	3 years ago
Eugen Rochko	c0327ff31f	Fix invalid language resulting in no fallback being set on statuses (#17722)	3 years ago
Eugen Rochko	2ea754b861	Fix duplicate notifications being possible after poll expiration (#17697)	3 years ago
Eugen Rochko	02b8d63fce	Fix report category not being saved in REST API (#17682)	3 years ago
Eugen Rochko	25d3dc4373	Add ability to mark statuses as sensitive from reports in admin UI (#17668) ... * Add ability to mark statuses as sensitive from reports in admin UI * Allow mark as sensitive action on statuses with preview cards	3 years ago
Eugen Rochko	27965ce5ed	Add trending statuses (#17431) ... * Add trending statuses * Fix dangling items with stale scores in localized sets * Various fixes and improvements - Change approve_all/reject_all to approve_accounts/reject_accounts - Change Trends::Query methods to not mutate the original query - Change Trends::Query#skip to offset - Change follow recommendations to be refreshed in a transaction * Add tests for trending statuses filtering behaviour * Fix not applying filtering scope in controller	3 years ago
Claire	1c3e5e44e2	Fix new sign-up notification not working because of incorrect type name (#17629)	3 years ago
Eugen Rochko	7b816eb5ae	Add notifications for new sign-ups (#16953)	3 years ago
luzpaz	73f5e4a1d9	Fix various typos (#17621) ... Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,ro`	3 years ago
Eugen Rochko	564efd0651	Add appeals (#17364) ... * Add appeals * Add ability to reject appeals and ability to browse pending appeals in admin UI * Add strikes to account page in settings * Various fixes and improvements - Add separate notification setting for appeals, separate from reports - Fix style of links in report/strike header - Change approving an appeal to not restore statuses (due to federation complexities) - Change style of successfully appealed strikes on account settings page - Change account settings page to only show unappealed or recently appealed strikes * Change appealed_at to overruled_at * Fix missing method error	3 years ago
Eugen Rochko	8f03b7a2fb	Add notifications when a reblogged status has been updated (#17404) ... * Add notifications when a reblogged status has been updated * Change wording to say "edit" instead of "update" and add missing controls * Replace previous update notifications with the most up-to-date one	3 years ago
Eugen Rochko	c9a52833b6	Fix deletes not being signed in authorized fetch mode (#17484) ... Fix #17483	3 years ago
Claire	63854bee6c	Fix poll votes not being properly reset on poll change (#17498) ... * Fix poll votes not being properly reset on poll change * Fix and add tests * Fix poll update handling when the number of options changes	3 years ago
Eugen Rochko	1bfcb75105	Fix outdated iso-639 reference in update status service (#17496)	3 years ago
Eugen Rochko	63002cde03	Add editing for published statuses (#17320) ... * Add editing for published statuses * Fix change of multiple-choice boolean in poll not resetting votes * Remove the ability to update existing media attachments for now	3 years ago
Eugen Rochko	2f8159baad	Add `category` and `rule_ids` params to `POST /api/v1/reports` (#17492)	3 years ago
Eugen Rochko	b6d7726ecb	Remove language detection through cld3 (#17478) ... * Remove language detection through cld3 * Update app/helpers/languages_helper.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>	3 years ago
Claire	c8b1e72a4f	Fix compacted JSON-LD possibly causing compatibility issues on forwarding (#17428)	3 years ago
Claire	948235592a	Fix response_to_recipient? CTE (#17427)	3 years ago
Claire	d1ecc323e7	Compact JSON-LD signed incoming activities (#17426) ... Co-authored-by: Puck Meerburg <puck@puck.moe>	3 years ago
Claire	03d59340da	Fix Sidekiq warnings about JSON serialization (#17381) ... * Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.	3 years ago
Claire	166cc5b89d	Fix local distribution of edited statuses (#17380) ... Because `FanOutOnWriteService#update?` was broken, edits were considered as new toots and a regular `update` payload was sent.	3 years ago