Commit Graph

1077 Commits (351c74459084ccffce1333b57c2af9a6b55cac8d)

Author SHA1 Message Date
Eugen Rochko c3ca3801f2
Add separate cache directory for non-local uploads (#12821) 5 years ago
Takeshi Umeda 46b2cc184f
Fix enable/disable relay failures (#13535) 5 years ago
ThibG 3825e1943f
Fix confusing error when failing to add an alias to an unknown account (#13480)
Follow-up to #13452, fixing broken `uri.nil?` test.

Also remove the separate check for `uri` presence, as that would result
in a “Please review 2 errors below” while only one would be listed.
5 years ago
Eugen Rochko 5edff32733
Change delivery failure tracking to work with hostnames instead of URLs (#13437) 5 years ago
ThibG 5524258da9
Fix “Email changed” notification sometimes having wrong e-mail (#13475)
* Fix “Email changed” notification sometimes having wrong e-mail

Fixes #6778

The root of the issue is that `send_devise_notification` was called before
the changes were properly commited to the database, causing the mailer to
pick previous values if running too early.

Devise's documentation provides guidance on how to handle that[1][2], however,
I have found it to not be working, as the following happens, in that order:
- `send_devise_notification` is called for the `email_changed` notification.
  In that case, `changed?` is false and `saved_changes?` is true, so
  if we use the former, we have the same issue.
- the `after_commit` hook is called
- `send_devise_notification` is called for the `confirmation_instructions`
  notification.
  In that case, `changed?` is still false, and `saved_changes?` still true,
  so if we use the latter, that second notification email is simply not
  going to be sent (as we would be queuing the notification *after*
  executing the after_commit hook).

This is because it may be called from either an `after_update` or
`after_commit` hook, the difference not being a call to `save` but the
transaction actually being committed to the database. This may arguably
be a bug in Devise, or Devise's notification.

The proposed workaround is inspired by Devise's documentation but checks
whether a transaction is open to make the call whether to immediately
send the notification or defer it to the `after_commit` hook.

[1]: https://www.rubydoc.info/github/plataformatec/devise/Devise%2FModels%2FAuthenticatable:send_devise_notification
[2]: 406915cb78/lib/devise/models/authenticatable.rb (L133-L194)

* Fix cases when sending notifications without changing the model

* Defer sending if and only if in transaction including current record
5 years ago
ThibG f7e011919e
Fix account aliases page (#13452)
* Fix error not being displayed when adding an account alias, add error for self-references

Co-Authored-By: Mélanie Chauvel (ariasuni) <perso@hack-libre.org>

* Add “You have no aliases.” note in confusing empty aliases table

Co-Authored-By: Mélanie Chauvel (ariasuni) <perso@hack-libre.org>

Co-authored-by: Mélanie Chauvel (ariasuni) <perso@hack-libre.org>
5 years ago
Eugen Rochko c9efb400b4
Add rate limit for reporting (#13390) 5 years ago
ThibG 89e28c7674
Fix PostgreSQL load when linking in announcements (#13250)
* Fix PostgreSQL load when linking in announcements

Fixes #13245 by caching status lookups

Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.

Since it's only used by announcements so far, there won't
be much statuses to cache.

* Perform status lookup when saving announcements, not when rendering them

* Change EntityCache#status to fetch URLs instead of looking into the database

* Move announcement link lookup to publishing worker

* Address issues pointed out during review
5 years ago
Eugen Rochko f65568f1d4
Add ability to filter audit log in admin UI (#13381) 5 years ago
Eugen Rochko 0c8945e5ff
Change `tootctl media remove-orphans` to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
5 years ago
Jeong Arm 8c42e0b53c
Make bookmarks also searchable (#13271) 5 years ago
ThibG 61f35c1a8a
Fix reported accounts not being whitelisted when resolving a spamcheck report (#13289) 5 years ago
Eugen Rochko bea0bb39d6
Add option to include resolved DNS records when blacklisting e-mail domains in admin UI (#13254)
* Add shortcuts to blacklist a user's e-mail domain in admin UI

* Add option to blacklist resolved MX and IP records for e-mail domains
5 years ago
Eugen Rochko f556f79b77
Add titles to warning presets in admin UI (#13252) 5 years ago
ThibG 2c6099125d
Fix videos with unsupported colorspace not being transcoded (#13242) 5 years ago
ThibG abd8394880
Fix MP4 (H264 + AAC) video files being needlessly re-encoded (#13239) 5 years ago
Eugen Rochko 8a2b8e0321
Change video uploads to always be converted to H264/MP4 (#13220)
Even if the container format is the same (.mp4), the codec could
be different and not playable in web browsers
5 years ago
Eugen Rochko dc15c81e67
Change video uploads to enforce certain limits (#13218)
- Dimensions at most 1920x1200
- Frame rate at most 60
5 years ago
ThibG b154428e14
Add federation support for the "hide network" preference (#11673)
* Change ActivityPub follower/following collections to not link first page

* Add support for hiding followers and following of remote users

* Switch to using a single `hide_collections` column

* Address code style remarks
5 years ago
Eugen Rochko 9660aa4543
Change local media attachments to perform heavy processing asynchronously (#13210)
Fix #9106
5 years ago
ThibG aa67036b41
Add support for links to statuses in announcements to be opened in web UI (#13212)
* Add support for links to public statuses in announcements to be opened in WebUI

* Please CodeClimate
5 years ago
ThibG 4a4cd686c1
Add sorting by username, creation and last activity in moderation view (#13076)
* Add ability to order accounts in moderation view

* Display last status date in “Most recent activity” for remote users
5 years ago
Eugen Rochko 339ce1c4e9
Add specific rate limits for posting and following (#13172) 5 years ago
Eugen Rochko d081a80cff
Fix "tootctl media remove-orphans" crashing when encountering invalid media (#13170)
Fixes #13168
5 years ago
ThibG 7b63c5469e
Fix announcements with fully-qualified mention to local user crashing WebUI (#13164) 5 years ago
Eugen Rochko ff07e80533
Fix remote convertible media attachments not being saved (#13032)
Because the file name was set after loading the file, Paperclip
was flagging the differing file extension as content type spoofing

Fix #12938
5 years ago
abcang 61a7390b66
Search account domain in lowercase (#13016)
* Search account domain in lowercase

* fix rubocop error

* fix spec/models/account_spec.rb
5 years ago
Eugen Rochko 663ea84b08
Add publish/unpublish controls to announcements in admin UI (#12967) 5 years ago
Eugen Rochko 401f32f9ee
Fix expired announcements being re-published (#12964) 5 years ago
Eugen Rochko b9d74d4076
Add streaming API updates for announcements being modified or deleted (#12963)
Change `all_day` to be a visual client-side cue only

Publish immediately if `scheduled_at` is in the past

Add `published_at` and `updated_at` to announcements JSON
5 years ago
Eugen Rochko 71921f6bc3
Fix user disabling changing activity timestamps, fix nil error (#12943) 5 years ago
Eugen Rochko daf71573d0
Fix password change/reset not immediately invalidating other sessions (#12928)
While making browser requests in the other sessions after a password
change or reset does not allow you to be logged in and correctly
invalidates the session making the request, sessions have API tokens
associated with them, which can still be used until that session
is invalidated.

This is a security issue for accounts that were already compromised
some other way because it makes it harder to throw out the hijacker.
5 years ago
Eugen Rochko ce1dee85b5
Fix relationships page not showing results in admin UI (#12934)
Follow-up to #12927
5 years ago
Eugen Rochko f52c988e12
Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
5 years ago
Eugen Rochko 81cc86bb1f
Fix media attachments without file being uploadable (#12562)
Fix #12554
5 years ago
ThibG 43daeccccb Add “account timeline” filter category (#12918)
* Add “account timeline” filter category

Previously, no filter category applied to account timelines.

* Rename “Account timelines” into “Profiles”
5 years ago
Eugen Rochko c0006a004d
Change followers page to relationships page in admin UI (#12927)
Allow browsing and filtering all relationships instead of just
followers, unify the codebase with the user-facing relationship
manager, add ability to see who the user invited
5 years ago
Eugen Rochko 02063c245c
Fix not all of account's active IPs showing up in admin UI (#12909) 5 years ago
Eugen Rochko ba16d4b413
Fix search by IP not using alternative browser sessions in admin UI (#12904) 5 years ago
Eugen Rochko 6feafb8802
Various fixes and improvements (#12878)
* Fix unused role routes being generated

* Remove unused JavaScript code

* Refactor filters code to be DRYer

* Fix `.count == 0` comparisons to `.empty?` in views

* Fix filters in views
5 years ago
Eugen Rochko 3287ec8ca3
Fix file names being obfuscated on update when file didn't change (#12857)
Fix #12849
5 years ago
ThibG 57e2833f6a Remove dependency on OStatus2 gem (#12822) 5 years ago
ysksn 6f8f018e3e Refactor StatusThreadingConcern (#9626)
* Remove #filter_from_context?

* Create scope Status.with_accounts

Retrieving AR objects should be
their model's scope
5 years ago
Alexander 05756c9a14 improve status title (#8596)
* improve shown status title, useful for atom/rss

* use single quotes to satisfy codeclimate

* fix tests, make message more pretty

* fix tests

* fix codestyle

* fix codestyle

* remove atom_serializer_spec

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
5 years ago
ThibG 51eb111503 Allow blocking TLDs, and fix TLD blocks not being editable (#12805)
Fixes #12795

It was already possible to create domain blocks for TLDs, but those
weren't enforced, nor editable. This commit changes it so that they
are enforced and editable.
5 years ago
Eugen Rochko 49b2f7c0a2
Fix base64-encoded file uploads not being possible (#12748)
Fix #3804, Fix #5776
5 years ago
Eugen Rochko f86ee4b59f
Fix IDN mentions not being processed, IDN domains not being rendered (#12715)
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.

Fix #7812, fix #12246
5 years ago
ThibG c0ed53b810 Fix custom emoji category creation silently erroring out on duplicate category (#12647)
Instead, just re-use the existing category if any.

Fixes #12608
5 years ago
ThibG da2143b308 Fixes featured hashtag setting page erroring out instead of rejecting invalid tags (#12436)
* Revert "Fix ignoring whole status because of one invalid hashtag (#11621)"

This reverts commit dff46b260b.

* Fix statuses being rejected because of invalid hashtag names

* Add spec for invalid hashtag names in statuses

* Add test for featured tags controller
5 years ago
scd31 3830c0b741 Increase max backup size (#12602)
* Increased max backup size

* partially reverted schema.rb
5 years ago
Alice Gaudon 668f698077 Admin setting to disable default follows (#12566) 5 years ago
Eugen Rochko f3d232381d
Add `tootctl media remove-orphans` (#12568) 5 years ago
Takeshi Umeda f43f1e0184 Add basic support for group actors (#12071)
* Show badge on group actor in WebUI

* Do not notify in case of  by following group actor

* If you mention group actor, also mention group actor followers

* Relax characters that can be used in username (same as Application)

* Revert "Relax characters that can be used in username (same as Application)"

This reverts commit 7e10a137b8.

* Delete display_name method
5 years ago
Yamagishi Kazutoshi 6be16d02cb Update ESLint and RuboCop in Code Climate (#12534) 5 years ago
Mathieu Brunot bd8dc9bd0c Add an LDAP Mail attribute config (#12053)
Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>
5 years ago
ThibG 911cc14481 Add follow_request notification type (#12198)
* Add follow_request notification type

The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.

Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.

Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.

* Show follow request link in single-column if there are pending requests, even if account isn't locked

* Push follow requests from notifications to the follow_requests list

* Offer to accept or reject follow request from the notification

* Redesign follow request notification
5 years ago
ThibG f60cd97638 Only normalize local polls (#12515)
Before this patch, if remote poll options have leading or trailing spaces,
the information stored locally won't match them, causing federated voting to
fail.
5 years ago
Mathieu Brunot d70268f099 Convert LDAP username (#12461)
*  Convert LDAP username #12021

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 🐛 Fix conversion var use

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 🐛 Fix LDAP uid conversion test

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 👌 Remove comments with ref to PR

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 👌 Remove unnecessary paranthesis

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 🔧 Move space in conversion string

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>
5 years ago
ThibG d8f96028c5 Add ability to filter reports by target account domain (#12154)
* Add ability to filter reports by target account domain

* Reword by_target_domain label
5 years ago
ThibG 2edf6d81cb Simplify SQL query when performing account search amongst followings (#12302) 5 years ago
ThibG dfea7368c9 Add bookmarks (#7107)
* Add backend support for bookmarks

Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.

* Add spec for bookmark endpoints

* Add front-end support for bookmarks

* Introduce OAuth scopes for bookmarks

* Add bookmarks to archive takeout

* Fix migration

* Coding style fixes

* Fix rebase issue

* Update bookmarked_statuses to latest UI changes

* Update bookmark actions to properly reflect status changes in state

* Add bookmarks item to single-column layout

* Make active bookmarks red
5 years ago
ThibG 65e13cfacf Add abilityto add oneself to lists (#12271)
* Add ability to add oneself to lists

* Change search results to include oneself when searching through followers

* Mark follow relation as optional in ListAccount
5 years ago
ThibG 650820d62d Fix remote media descriptions being cut off at 420 chars (#12262)
* Fix remote media descriptions being cut off at 420 chars

Fixes #12258

* Fix tests
5 years ago
Faye Duxovni 48f75b86ae Add setting for whether to crop images in unexpanded toots (#12126) 5 years ago
Hugo Gameiro 488dd0ff7a remove audio metadata (#12171) 5 years ago
Eugen Rochko 915f3712ae
Fix admin setting to auto-approve hashtags not affecting query (#12130)
Follow-up to #12122
6 years ago
Eugen Rochko b5f7e12817
Remove auto-silence behaviour from spam check (#12117)
Fix #12113
6 years ago
Eugen Rochko 354fdd317e
Fix attachment not being re-downloaded even if file is not stored (#12125)
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
6 years ago
Eugen Rochko c8bcf5cbfd
Add admin setting to auto-approve hashtags (#12122)
Change inaccurate labels on other admin settings
6 years ago
Eugen Rochko f665901e3c
Fix performance of home feed regeneration (#12084)
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive

Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
6 years ago
Hugo Gameiro a7917269d3 add loglevel fatal to video and audio styles (#12088) 6 years ago
Eugen Rochko ca22a22d7f
Fix performance of GIF re-encoding (#12057)
* Change animated GIF detection to not shell out to ImageMagick

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Change video encoding parameters to limit to 10800 video frames

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Limit GIF image size further

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Always strip metadata from video files

* Fix code style issues
6 years ago
Eugen Rochko 575dc11cb2
Fix needlessly indexing unsearchable statuses into ElasticSearch (#12041) 6 years ago
Eugen Rochko 62f60e86c2
Fix account counters being overwritten by parallel writes (#12045) 6 years ago
Eugen Rochko 5c42f47617
Fix records not being indexed sometimes (#12024)
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
6 years ago
ThibG 3babf8464b Add voters count support (#11917)
* Add voters count to polls

* Add ActivityPub serialization and parsing of voters count

* Add support for voters count in WebUI

* Move incrementation of voters count out of redis lock

* Reword “voters” to “people”
6 years ago
Eugen Rochko 163ed91af3
Add (back) option to set redirect notice on account without moving followers (#11994)
Fix #11913
6 years ago
Eugen Rochko 50af41a00d
Fix unscoped being used in the wrong place (#11987)
Regression from #11982
6 years ago
Eugen Rochko 3ec80c7aec
Fix preview card image not being re-fetched even if link is re-posted (#11981)
Fix #11956
6 years ago
Eugen Rochko 32ff78f749
Fix index not being used in Status.reblogs_map (#11982)
Regression from #11623
6 years ago
Eugen Rochko ab33c4df94
Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
6 years ago
abcang 07b057eabb Validate Web::PushSubscription (#11971) 6 years ago
Eugen Rochko add4d4118c
Fix relays UI being available in whitelist/secure mode (#11963)
Fix relays UI referencing relay that is not functional
6 years ago
Eugen Rochko a1f04c1e34
Fix authentication before 2FA challenge (#11943)
Regression from #11831
6 years ago
ThibG b359974d9b Show user what options they have voted (#11195)
* Add own_votes field to poll results in REST API

Fixes #10679

* Display user votes in WebUI

* Update styling

* Add vote checkmark to public pages
6 years ago
Takeshi Umeda b18aea91e3 Accept acct starting with @ in account migration (#11907) 6 years ago
Takeshi Umeda f497d14b19 Addition of update activity distribution by alias, minor correction (#11905)
* Addition of update activity distribution by alias, minor correction

* Distribute Update activity after adding alias
* Add uniqueness verification to alias uri
* accept acct starting with @

* fix double-quoted to single-quoted
6 years ago
Eugen Rochko 3ed94dcc1a
Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
6 years ago
Eugen Rochko d930eb88b6
Add table of contents to about page (#11885)
Move public domain blocks information to about page
6 years ago
Eugen Rochko e1066cd431
Add password challenge to 2FA settings, e-mail notifications (#11878)
Fix #3961
6 years ago
Eugen Rochko a4b60e9ba4
Fix TOTP codes not being filtered from logs during enabling/disabling (#11877)
Not a serious issue because they are meaningless past single use
6 years ago
han@highemelry 3919571c39 Fix audit log error when custom emoji is copied from remote server (#11876) 6 years ago
mayaeh 0e5b9e3ba0 Fix the sample account icon of Profile directory to local only (#11872) 6 years ago
Eugen Rochko 3d14b8f28f
Fix SSO login not using existing account when e-mail is verified (#11862)
Fix #11472
6 years ago
mayaeh ef0d22f232 Add search and sort functions to hashtag admin UI (#11829)
* Add search and sort functions to hashtag admin UI

* Move scope processing from tags_controller to tag_filter

* Fix based on method naming conventions

* Fixed not to get 500 errors for invalid requests
6 years ago
Eugen Rochko c707ef49d9
Fix 2FA challenge and password challenge for non-database users (#11831)
* Fix 2FA challenge not appearing for non-database users

Fix #11685

* Fix account deletion not working when using external login

Fix #11691
6 years ago
Eugen Rochko 0762258aec
Fix hashtags being split by ZWNJ character (#11821)
Fix #11761
6 years ago
Eugen Rochko 59da5ccb8e
Change unlisted custom emoji to not appear in autosuggestions (#11818)
Fix #11669
6 years ago
Eugen Rochko b6381bdc7d
Change max length of media descriptions from 420 to 1500 characters (#11819)
Fix #11658
6 years ago
Eugen Rochko c5d37f18cb
Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
6 years ago
Tao Bror Bojlén 4fe127664b add admin setting for default search engine indexing (fix #11750) (#11804) 6 years ago
Eugen Rochko 031ca25014
Add retry for failed media downloads and `tootctl media refresh` (#11775) 6 years ago
Eugen Rochko 8674814825
Change tootctl to use inline parallelization instead of Sidekiq (#11776)
- Remove --background option
- Add --concurrency(=5) option
- Add progress bars
6 years ago
Eugen Rochko 1110ea1a91
Add batch actions and categories to admin UI for custom emojis (#11793) 6 years ago
Eugen Rochko 261e52268c
Add batch approve/reject for pending hashtags in admin UI (#11791) 6 years ago
Eugen Rochko a75009a65e
Change half-life of trend decay (#11774) 6 years ago
Eugen Rochko e445a8af64
Add timeline read markers API (#11762)
Fix #4093
6 years ago
Eugen Rochko 70ddef2654
Change trending hashtags to not disappear instantly after midnight (#11712) 6 years ago
umonaca 7235f538c6 Add more media formats (#11687)
* Add more media formats

* Removed amr file type
6 years ago
Eugen Rochko 5f591d979c
Fix non-numbers being passed to Redis and causing an error (#11697) 6 years ago
Eugen Rochko b54b725d6b
Fix uncaught domain normalization error in remote follow (#11703) 6 years ago
Eugen Rochko cb447b28c4
Add profile directory to web UI (#11688)
* Add profile directory to web UI

* Add a line of bio to the directory
6 years ago
Eugen Rochko 73ca0bb925
Add option to include reported statuses in warning e-mail (#11639) 6 years ago
Eugen Rochko 282ea17078
Add soft delete for statuses for instant deletes through API (#11623)
* Add soft delete for statuses to allow them to appear instant

* Allow reporting soft-deleted statuses and show them in the admin UI

* Change index for getting an account's statuses
6 years ago
Eugen Rochko 97192d9a77
Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
6 years ago
Eugen Rochko dff46b260b
Fix ignoring whole status because of one invalid hashtag (#11621)
Fix #11618
6 years ago
ThibG cb62a83a71 Add invite comments (#10465) 6 years ago
ThibG 9b6a5ed109 Add public blocks to /about/blocks (#11298)
* Add automatic blocklist display in /about/blocks

Inspired by https://github.com/Gargron/mastodon.social-misc

* Add admin option to set who can see instance blocks

* Normalize locales files

* Rename “Sandbox” to “Silence” for consistency

* Disable /about/blocks when in whitelist mode

* Optionally display rationale for domain blocks

* Only display domain blocks that have user-facing limitations, and order them

* Redesign table of blocked domains to better handle long domain names and rationales

* Change domain blocks ordering now that rationales aren't displayed right away

* Only show explanation for block severities actually in use

* Reword instance block explanations and add disclaimer for public fetch mode
6 years ago
Eugen Rochko 96702e7f67
Add `tootctl cache recount` command (#11597) 6 years ago
Eugen Rochko cc0a55cf9a
Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
6 years ago
Stanislas 5d8ee24cd5 Remove WebP support (#11589) 6 years ago
Eugen Rochko 8fdff2748f
Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
6 years ago
Arnout Engelen d7bdddbeef Include max image dimensions in error (#11552) 6 years ago
kedama aebefc7ce9 Fix some emojis in profile metadata labels are not emojified. (#11534) 6 years ago
Eugen Rochko 7a1f8a58df
Fix crash when saving invalid domain name (#11528)
Fix #7629
6 years ago
Eugen Rochko 898dacf83e
Fix weekly usage not being displayed correctly in hashtag admin UI (#11524)
Fix percentages in usage breakdown having too many digits

Change trending hashtags to only ask for review if a hashtag enters
the top 3 position in the set, since it's the only items shown in
the default web UI
6 years ago
Eugen Rochko 41aac1ba3a
Add GIF and WebP support for custom emojis (#11519)
Fix #11466
6 years ago
Eugen Rochko f51c7c105f
Fix acct URIs with IDN domains not being resolved (#11520)
Fix #11494
6 years ago
ThibG bced70469a Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
6 years ago
Eugen Rochko 94c54997cf
Fix trending tags returning less items than requested sometimes (#11513)
Add better sorting defaults to the hashtags admin UI

Add "not reviewed" filter to hashtags admin UI
6 years ago
Eugen Rochko ac33f1aedd
Fix account tags not being saved correctly (#11507)
* Fix account tags not being saved correctly

Regression from f371b32

Fix Tag#discoverable not returning tags where listable is nil instead of true

Add notice when saving hashtags in admin UI

Change public hashtag and directory pages to return 404 for forbidden tags

* Remove unused locale string
6 years ago
Eugen Rochko a8958d07e2
Fix featured tags not finding the right tag on save (#11504)
Regression from f371b32
6 years ago
Eugen Rochko 9072fe5ab6
Add trends UI with admin and user settings (#11502) 6 years ago
Eugen Rochko 115dab78f1
Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
6 years ago
Eugen Rochko 29609c09f4
Fix tag score not being updated when the tag is trending (#11465) 6 years ago
Eugen Rochko 92de439c04
Change hashtag search to only return results that have trended in the past (#11448)
* Change hashtag search to only return results that have trended in the past

A way to eliminate typos and other one-off "junk" results

* Fix excluding exact matches that don't have a score

* Fix tests
6 years ago
Eugen Rochko 648cdbc04a
Add hashtag score for better sorting of autosuggestions (#11427)
* Add hashtag score for better sorting of autosuggestions

* Do not use `~<~` operator with no text_pattern_ops index
6 years ago
Eugen Rochko 24552b5160
Add whitelist mode (#11291) 6 years ago
Eugen Rochko e136112ab7
Fix tag normalization and migration not removing duplicate tags (#11441)
Fix #11428
6 years ago
Eugen Rochko b9b0313c78
Revert "Remove conversation URI (#11423)" (#11424)
This reverts commit 75f7f9930e.
6 years ago
Eugen Rochko 75f7f9930e
Remove conversation URI (#11423)
It is not part of ActivityPub and will free up a lot of space
6 years ago
Eugen Rochko f371b32137
Change hashtags to preserve first-used casing (#11416) 6 years ago
Eugen Rochko 501148ab91
Remove timestamps from converted images to make them deterministic (#11408) 6 years ago
ThibG 92569ffde8 Fix invites not being disabled upon account suspension (#11412)
* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account
6 years ago
Eugen Rochko 964ae8eee5
Change unconfirmed user login behaviour (#11375)
Allow access to account settings, 2FA, authorized applications, and
account deletions to unconfirmed and pending users, as well as
users who had their accounts disabled. Suspended users cannot update
their e-mail or password or delete their account.

Display account status on account settings page, for example, when
an account is frozen, limited, unconfirmed or pending review.

After sign up, login users straight away and show a simple page that
tells them the status of their account with links to account settings
and logout, to reduce onboarding friction and allow users to correct
wrongly typed e-mail addresses.

Move the final sign-up step of SSO integrations to be the same
as above to reduce code duplication.
6 years ago
Eugen Rochko bd87e66679
Remove WebSub subscriptions (#11303) 6 years ago
ThibG c37c1da41e Disallow numeric-only hashtags (#11363)
* Add spec covering numeric-only hashtags

* Fix hashtag regex
6 years ago
ThibG 730c4053d6 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
6 years ago
Eugen Rochko 84e988479e
Fix only one middle dot being recognized in hashtags (#11345)
Fix #10934
6 years ago
Eugen Rochko 4906cabc6b
Add aac, m4a, 3gp to allowed audio formats (#11342)
Fix #11186
6 years ago