mastodon

Commit Graph

Author	SHA1	Message	Date
Eugen Rochko	27965ce5ed	Add trending statuses (#17431 ) * Add trending statuses * Fix dangling items with stale scores in localized sets * Various fixes and improvements - Change approve_all/reject_all to approve_accounts/reject_accounts - Change Trends::Query methods to not mutate the original query - Change Trends::Query#skip to offset - Change follow recommendations to be refreshed in a transaction * Add tests for trending statuses filtering behaviour * Fix not applying filtering scope in controller	3 years ago
Eugen Rochko	a29a982eaa	Change e-mail domain blocks to block IPs dynamically (#17635 ) * Change e-mail domain blocks to block IPs dynamically * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>	3 years ago
Eugen Rochko	35850f8195	Fix localization of cold-start follow recommendations (#17479 )	3 years ago
Claire	52c1b86964	Fix Ruby 2.5 incompatibility (#17465 )	3 years ago
Claire	03d59340da	Fix Sidekiq warnings about JSON serialization (#17381 ) * Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.	3 years ago
Eugen Rochko	14f436c457	Add notifications for statuses deleted by moderators (#17204 )	3 years ago
Eugen Rochko	8e84ebf0cb	Remove IP tracking columns from users table (#16409 )	3 years ago
Takeshi Umeda	bda8e4f815	Fix follow recommendation biased towards older accounts (#17126 )	3 years ago
Eugen Rochko	6e50134a42	Add trending links (#16917 ) * Add trending links * Add overriding specific links trendability * Add link type to preview cards and only trend articles Change trends review notifications from being sent every 5 minutes to being sent every 2 hours Change threshold from 5 unique accounts to 15 unique accounts * Fix tests	3 years ago
Claire	4ac78e2a06	Add feature to automatically delete old toots (#16529 ) * Add account statuses cleanup policy model * Record last inspected toot to delete to speed up successive calls to statuses_to_delete * Add service to cleanup a given account's statuses within a budget * Add worker to go through account policies and delete old toots * Fix last inspected status id logic All existing statuses older or equal to last inspected status id must be kept by the current policy. This is an invariant that must be kept so that resuming deletion from the last inspected status remains sound. * Add tests * Refactor scheduler and add tests * Add user interface * Add support for discriminating based on boosts/favs * Add UI support for min_reblogs and min_favs, rework UI * Address first round of review comments * Replace Snowflake#id_at_start with with_random parameter * Add tests * Add tests for StatusesCleanupController * Rework settings page * Adjust load-avoiding mechanisms * Please CodeClimate	4 years ago
Eugen Rochko	d174d12c83	Add authentication history (#16408 )	4 years ago
Claire	d9ae3db8d5	Improve performance of follow recommendation scheduler (#16159 ) Express follow_recommendations in terms of account_summaries rather than accounts, integrate filters that are unconditionally used, and materialize the resulting view. This should result in the bulk of the computation being performed only once instead of once per recommendation language.	4 years ago
Eugen Rochko	f7117646af	Add cold-start follow recommendations (#15945 )	4 years ago
Eugen Rochko	e89e976e92	Fix configuration for sidekiq-unique-jobs after 7.x upgrade (#15908 ) Remove locks from scheduled jobs	4 years ago
ThibG	3249d35bdc	Improve account deletion performances further (#15407 ) * Delete status records by batches of 50 * Do not precompute values that are only used once * Do not generate redis events for removal of public toots older than two weeks * Filter reported toots a priori for polls and status deletion * Do not process reblogs when cleaning up public timelines As in Mastodon proper, reblogs don't appear in public TLs * Clean the deleted account's own feed in one go * Refactor Account#clean_feed_manager and List#clean_feed_manager * Delete instead of destroy a few more associations * Fix preloading Co-authored-by: Claire <claire.github-309c@sitedethib.com>	4 years ago
Eugen Rochko	216b85b053	Fix performance on instances list in admin UI (#15282 ) - Reduce duplicate queries - Remove n+1 queries - Add accounts count to detailed view - Add separate action log entry for updating existing domain blocks	4 years ago
Eugen Rochko	5e1364c448	Add IP-based rules (#14963 )	4 years ago
Eugen Rochko	ed099d8bdc	Change account suspensions to be reversible by default (#14726 )	4 years ago
Eugen Rochko	5d8398c8b8	Add E2EE API (#13820 )	5 years ago
Eugen Rochko	9014367bd8	Fix background jobs not using locks like they are supposed to (#13361 ) Also: - Fix locks not being removed when jobs go to the dead job queue - Add UI for managing locks to the Sidekiq dashboard - Remove unused Sidekiq workers Fix #13349	5 years ago
Eugen Rochko	401f32f9ee	Fix expired announcements being re-published (#12964 )	5 years ago
Eugen Rochko	f52c988e12	Add announcements (#12662 ) * Add announcements Fix #11006 * Add reactions to announcements * Add admin UI for announcements * Add unit tests * Fix issues - Add `with_dismissed` param to announcements API - Fix end date not being formatted when time range is given - Fix announcement delete causing reactions to send streaming updates - Fix announcements container growing too wide and mascot too small - Fix `all_day` being settable when no time range is given - Change text "Update" to "Announcement" * Fix scheduler unpublishing announcements before they are due * Fix filter params not being passed to announcements filter	5 years ago
abcang	c21386cff5	Improve IP cleanup query (#11871 )	6 years ago
Eugen Rochko	70ddef2654	Change trending hashtags to not disappear instantly after midnight (#11712 )	6 years ago
mayaeh	0d80f686d8	Add `tootctl preview_cards remove` (#11320 ) * Add `tootctl preview_cards remove` * fix code style * Remove `Scheduler::PreviewCardsCleanupScheduler` file * fix code style again Add exclude case where image_file_name is blank * Added a function to output confirmation if the specified number of days is less than 2 weeks	6 years ago
Eugen Rochko	bd87e66679	Remove WebSub subscriptions (#11303 )	6 years ago
Eugen Rochko	b3f44aa186	Add periodic removal of older thumbnails for preview cards (#11304 )	6 years ago
Eugen Rochko	23aeef52cc	Remove Salmon and PubSubHubbub (#11205 ) * Remove Salmon and PubSubHubbub endpoints * Add error when trying to follow OStatus accounts * Fix new accounts not being created in ResolveAccountService	6 years ago
ysksn	bcfff65195	Create Redisable#redis (#9633 ) * Create Redisable * Use #redis instead of Redis.current	6 years ago
Eugen Rochko	8ad57a0426	Fix PublishScheduledStatusWorker being queued with no arguments (#9724 )	6 years ago
Eugen Rochko	a49d43d112	Add scheduled statuses (#9706 ) Fix #340	6 years ago
Eugen Rochko	66436d0895	Improve e-mail digest (#9689 ) - Reduce time-to-digest from 20 to 7 days - Fetch mentions starting from +1 day since last login - Fix case when last login is more recent than last e-mail - Do not render all mentions, only 40, but show number in subject - Do not send digest to moved accounts - Do send digest to silenced accounts	6 years ago
Eugen Rochko	7f9dd92a27	Track historical space stats in PgHero to determine PostgreSQL growth (#8906 )	6 years ago
Eugen Rochko	93537c5560	Disable retries on all scheduled tasks (#8413 )	7 years ago
Eugen Rochko	d98de8ada7	Get rid of all batch order warnings (#8334 )	7 years ago
Eugen Rochko	91c929a42c	Keep scheduler jobs unique until they're done (#8287 )	7 years ago
Eugen Rochko	cb5b5cb5f7	Slightly reduce RAM usage (#7301 ) * No need to re-require sidekiq plugins, they are required via Gemfile * Add derailed_benchmarks tool, no need to require TTY gems in Gemfile * Replace ruby-oembed with FetchOEmbedService Reduce startup by 45382 allocated objects * Remove preloaded JSON-LD in favour of caching HTTP responses Reduce boot RAM by about 6 MiB * Fix tests * Fix test suite by stubbing out JSON-LD contexts	7 years ago
Eugen Rochko	f1867a7388	Adjust privacy policy (#6666 ) * Adjust privacy policy to be more specific to Mastodon Fix #6613 * Change data retention of IP addresses from 5 years to 1 year * Add even more information * Remove all (now invalid) translations of the privacy policy * Add information about archive takeout, remove pointless consent section * Emphasis on DM privacy * Improve wording * Add line about data use for moderation purposes	7 years ago
Eugen Rochko	61ed133fea	Account archive download (#6460 ) * Fix #201: Account archive download * Export actor and private key in the archive * Optimize BackupService - Add conversation to cached associations of status, because somehow it was forgotten and is source of N+1 queries - Explicitly call GC between batches of records being fetched (Model class allocations are the worst offender) - Stream media files into the tar in 1MB chunks (Do not allocate media file (up to 8MB) as string into memory) - Use #bytesize instead of #size to calculate file size for JSON (Fix FileOverflow error) - Segment media into subfolders by status ID because apparently GIF-to-MP4 media are all named "media.mp4" for some reason * Keep uniquely generated filename in Paperclip::GifTranscoder * Ensure dumped files do not overwrite each other by maintaing directory partitions * Give tar archives a good name * Add scheduler to remove week-old backups * Fix code style issue	7 years ago
Eugen Rochko	ed867eca9d	Move e-mail digest task to sidekiq, reduce workload, improve hint (#6252 )	7 years ago
Eugen Rochko	d68868ca14	Lists redis clean-up (#5886 ) * When list is deleted, remove feed from redis * Clean up list feeds of inactive users	7 years ago
aschmitz	554c2fd8af	Clean up reblog tracking keys, related improvements (#5428 ) * Clean up reblog-tracking sets from FeedManager Builds on #5419, with a few minor optimizations and cleanup of sets after they are no longer needed. * Update tests, fix multiply-reblogged case Previously, we would have lost the fact that a given status was reblogged if the displayed reblog of it was removed, now we don't. Also added tests to make sure FeedManager#trim cleans up our reblog tracking keys, fixed up FeedCleanupScheduler to use the right loop, and fixed the test for it.	7 years ago
Eugen Rochko	34118169ac	Keep references to all reblogs of a status on home feed (#5419 ) * Keep references to all reblogs of a status on home feed When inserting reblog: Add to set of reblogs of this status on the feed, if original status was present in the feed, add it to that set as well. When removing a reblog: Remove it from that set. Take random remaining item from the set. If one exists, re-insert it into feed, otherwise do not re-insert anything. Fix #4210 * When original is removed, toss out reblog references	7 years ago
sdukhovni	67559361e8	Add scheduled worker to purge old user IPs (#4951 ) * Add scheduled worker to purge old user IPs * Use ruby 1.9 hash syntax	8 years ago
Eugen Rochko	f391a4673a	Periodically remove expired PuSH subscribers (#4654 )	8 years ago
Eugen Rochko	6e9eda5331	ActivityPub migration procedure (#4617 ) * ActivityPub migration procedure Once one account is detected as going from OStatus to ActivityPub, invalidate WebFinger cache for other accounts from the same domain * Unsubscribe from PuSH updates once we receive an ActivityPub payload * Re-subscribe to PuSH unless already unsubscribed, regardless of protocol	8 years ago
Eugen Rochko	3d47154c20	Only PuSH-resubscribe to OStatus accounts (#4583 )	8 years ago
Eugen Rochko	467456f7a1	Move clean up of unconfirmed users to sidekiq-scheduler (#4336 ) * Move clean up of unconfirmed users to sidekiq-scheduler * mastodon:daily is now deprecated	8 years ago
Eugen Rochko	ed7dc1704d	Bind web UI access tokens to sessions (#3940 ) * Add overview of active sessions * Better display of browser/platform name * Improve how browser information is stored and displayed for sessions overview * Fix test * Fix #2347 - Bind web UI access token to session When you logout, session also destroys the access token, so it's no longer valid. If access token is destroyed some other way, the session is also destroyed, requiring a re-login. Fix #1681 - Add scheduler to remove revoked access tokens and grants * Fix test	8 years ago
takayamaki	3eedad2737	change sidekiq queueing to bulk push (#3536 )	8 years ago
Matt Jankowski	ec34ec63b1	Specs for cleanup workers (#3235 ) * Add spec files for feed and media cleanup workers * Add coverage for feed and media cleanup schedulers * Clean up feed and media cleanup workers	8 years ago
Eugen Rochko	ef900789bc	Replace mastodon:media:clear and mastodon:feeds:clear rake tasks with (#3180 ) sidekiq-scheduler jobs Resolves #2495	8 years ago
Eugen Rochko	b8e166894b	Fix Scheduler::SubscriptionsScheduler (#2834 ) * Fix Scheduler::SubscriptionsScheduler, add worker test for it * Change production log level of Sidekiq to "warn" instead of "info"	8 years ago
Eugen Rochko	81584779cb	More robust PuSH subscription refreshes (#2799 ) * Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron Fix an issue where / in domain would raise exception in TagManager#normalize_domain PuSH subscriptions refresh done in a round-robin way to avoid hammering a single server's hub in sequence. Correct handling of failures/retries through Sidekiq (see also #2613). Optimize Account#with_followers scope. Also, since subscriptions are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire refreshing operation halfway through Fix #2702 - Correct user agent header on outgoing http requests * Add test for SubscribeService * Extract #expiring_accounts into method * Make mastodon:push:refresh no-op * Queues are now defined in sidekiq.yml * Queues are now in sidekiq.yml	8 years ago

1 2 3

104 Commits (295ad6f19a016b3f16e1201ffcbb1b3ad6b455a2)