300 Commits (af6d35627978047724612c027e60cb0a89cb2eef)

Author	SHA1	Message	Date
Eugen Rochko	8a0d677cde	Fix stoplight logging to stderr separate from Rails logger (#10624)	6 years ago
Eugen Rochko	0e8819f0e8	Add rate limit for media proxy requests (#10490) ... 30 per 30 minutes, like media uploads	6 years ago
Eugen Rochko	11fe293e1b	Remove unused ActivityPub `@context` values depending on response (#10378) ... Fix #8078	6 years ago
Eric	7169928f96	cas_options :validate_url should be :service_validate_url (#10328) ... Otherwise, no matter what is given for CAS_VALIDATE_URL the default /serviceValidate path would be used.	6 years ago
Eugen Rochko	99fa1ce93d	Add tight rate-limit for API deletions (#10042) ... Deletions take a lot of resources to execute and cause a lot of federation traffic, so it makes sense to decrease the number someone can queue up through the API. 30 per 30 minutes	6 years ago
Eugen Rochko	016ad37bc8	Fix URL linkifier grabbing full-width spaces and quotations (#9997) ... Fix #9993 Fix #5654	6 years ago
Eugen Rochko	4699cf853c	Add timeouts for S3 (#9842)	6 years ago
Moritz Heiber	ecf40d09ed	Disable Same-Site cookie implementation to fix SSO issues on WebKit browsers (#9819)	6 years ago
Nolan Lawson	f05eb67081	Enable immutable caching for S3 objects (#9722) ... I also added "public" here, as I can't think of a good reason not to add it. Perhaps it has some marginal benefit in that ISPs (or other proxies) can cache it for all users. The assets are certainly publicly available and the same for all users.	6 years ago
Eugen Rochko	5d2fc6de32	Add REST API for creating an account (#9572) ... * Add REST API for creating an account The method is available to apps with a token obtained via the client credentials grant. It creates a user and account records, as well as an access token for the app that initiated the request. The user is unconfirmed, and an e-mail is sent as usual. The method returns the access token, which the app should save for later. The REST API is not available to users with unconfirmed accounts, so the app must be smart to wait for the user to click a link in their e-mail inbox. The method is rate-limited by IP to 5 requests per 30 minutes. * Redirect users back to app from confirmation if they were created with an app * Add tests * Return 403 on the method if registrations are not open * Require agreement param to be true in the API when creating an account	6 years ago
Eugen Rochko	240c122767	Skip mailer job retries when a record no longer exists (#9590) ... Fix #8666	6 years ago
ThibG	3f12c07ff5	Use same CORS policy for /@:username and /users/:username (#9485) ... Fixes #8189 rack-cors being called before the application router, it does not follow the redirection, and we need a separate rule for /users/:username.	6 years ago
ThibG	84e5ed43e7	Preload common JSON-LD contexts (#9412) ... Fixes #9411	6 years ago
Ben Lubar	13e049d772	Allow cross-origin requests to /.well-known/* URLs. (#9083) ... Right now, this includes three endpoints: host-meta, webfinger, and change-password. host-meta and webfinger are publicly available and do not use any authentication. Nothing bad can be done by accessing them in a user's browser. change-password being CORS-enabled will only reveal the URL it redirects to (which is /auth/edit) but not anything about the actual /auth/edit page, because it does not have CORS enabled. The documentation for hosting an instance on a different domain should also be updated to point out that Access-Control-Allow-Origin: * should be set at a minimum for the /.well-known/host-meta redirect to allow browser-based non-proxied instance discovery.	6 years ago
Eugen Rochko	a38a452481	Add unread indicator to conversations (#9009)	6 years ago
ThibG	8ab081ec32	Add manifest_src to CSP, add blob to connect_src (#8967)	6 years ago
Eugen Rochko	edc7f895be	Fix CSP headers blocking media and development environment (#8962) ... Regression from #8957	6 years ago
ThibG	2d27c11061	Set Content-Security-Policy rules through RoR's config (#8957) ... * Set CSP rules in RoR's configuration * Override CSP setting in the embed controller to allow frames	6 years ago
Sascha	b2a57a5d6f	add ffmpeg initializer (#8855) ... * add ffmpeg initializer * use different expression to check for environment var	6 years ago
ashleyhull-versent	f194857ac9	rubocop issues - Cleaning up (#8912) ... * cleanup pass * undo mistakes * fixed. * revert	6 years ago
aus-social	0a4739c732	lint pass 2 (#8878) ... * Code quality pass * Typofix * Update applications_controller_spec.rb * Update applications_controller_spec.rb	6 years ago
aus-social	1f98eae1cf	Lint pass (#8876)	6 years ago
Yamagishi Kazutoshi	65f04e6046	Fix that Rails.cache information could not be sent via StatsD (#8831)	6 years ago
Eugen Rochko	f4d549d300	Redesign forms, verify link ownership with rel="me" (#8703) ... * Verify link ownership with rel="me" * Add explanation about verification to UI * Perform link verifications * Add click-to-copy widget for verification HTML * Redesign edit profile page * Redesign forms * Improve responsive design of settings pages * Restore landing page sign-up form * Fix typo * Support <link> tags, add spec * Fix links not being verified on first discovery and passive updates	6 years ago
luzpaz	40dd19be37	Misc. typos (#8694) ... Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`	6 years ago
Sorin Davidoi	6f3d934bc1	feat(cookies): Use the same-site attribute to lax (#8626) ... CSFR-prevention is already implemented but adding this doesn't hurt. A brief introduction to Same-Site cookies (and the difference between strict and lax) can be found at https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/ TLDR: We use lax since we want the cookies to be sent when the user navigates safely from an external site.	6 years ago
M Somerville	2bba6e582d	Rename S3_CLOUDFRONT_HOST to S3_ALIAS_HOST. (#8423) ... Still check for S3_CLOUDFRONT_HOST for existing installs.	6 years ago
ThibG	f06fa09962	Revert to using Paperclip's filesystem storage, and fix dangling records in remove_remote (#8339) ... * Fix uncaching worker * Revert to using Paperclip's filesystem backend instead of fog-local fog-local has lots of concurrency issues, causing failure to delete files, dangling file records, and spurious errors UncacheMediaWorker	6 years ago
Immae	b0f4fe456b	Add ldap search filter (#8151)	7 years ago
Eugen Rochko	018a9e4e7f	Add post-deployment migration system (#8182) ... Adopted from GitLab CE. Generate new migration with: rails g post_deployment_migration name_of_migration_here By default they are run together with db:migrate. To not run them, the env variable SKIP_POST_DEPLOYMENT_MIGRATIONS must be set Code by Yorick Peterse <yorickpeterse@gmail.com>, see also: 83c8241160	7 years ago
abcang	69bf116345	Add secure option to additional cookie (#8069)	7 years ago
Eugen Rochko	1f6ed4f86a	Add more granular OAuth scopes (#7929) ... * Add more granular OAuth scopes * Add human-readable descriptions of the new scopes * Ensure new scopes look good on the app UI * Add tests * Group scopes in screen and color-code dangerous ones * Fix wrong extra scope	7 years ago
MIYAGI Hikaru	ddd0bb69e1	Merge `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into `ALLOW_ACCESS_TO_HIDDEN_SERVICE` (#7901) ... If Mastodon accesses to the hidden service via transparent proxy, it's needed to avoid checking whether it's a private address, since `.onion` is resolved to a private address. I was previously using the `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` to provide that function. However, I realized that using `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` is redundant, since this specification is always used with `ALLOW_ACCESS_TO_HIDDEN_SERVICE`. Therefore, I decided to integrate the setting of `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into` ALLOW_ACCESS_TO_HIDDEN_SERVICE`.	7 years ago
Eugen Rochko	0df91c7b1e	Add dat, dweb, ipfs, ipns, ssb, gopher protocols to URL extractor (#7810) ... * Add dat:// and gopher:// to URL extractor Fix #6072 * Fix comment indent * Add dweb, ipfs, ipns, ssb	7 years ago
Eugen Rochko	53f0452b70	Remove rack-timeout (#7809) ... Timeout considered harmful due to leaving the app in a broken state, including unreaped database connections	7 years ago
Eugen Rochko	d87649db07	Disable AMS logging (#7623) ... Especially in production it's just noise and doesn't mix well with the log format	7 years ago
MIYAGI Hikaru	919eef3098	User agent for WebFinger (#7531) ... * User agent for WebFinger * local_domain → web_domain * 'http' is away accidentally...	7 years ago
Eugen Rochko	b4fb766b23	Add REST API for Web Push Notifications subscriptions (#7445) ... - POST /api/v1/push/subscription - PUT /api/v1/push/subscription - DELETE /api/v1/push/subscription - New OAuth scope: "push" (required for the above methods)	7 years ago
Hugo Gameiro	ea4e243303	Improve OpenStack v3 compatibility (#7392) ... * Update paperclip.rb * Update .env.production.sample * Update paperclip.rb	7 years ago
Akihiko Odaki	a7e71bbd08	Add a missing question mark in rack_attack.rb (#7338)	7 years ago
Akihiko Odaki	b1d4471e36	Throttle media post (#7337) ... The previous rate limit allowed to post media so fast that it is possible to fill up the disk space even before an administrator notices. The new rate limit is configured so that it takes 24 hours to eat 10 gigabytes: 10 * 1024 / 8 / (24 * 60 / 30) = 27 (which rounded to 30) The period is set long so that it does not prevent from attaching several media to one post, which would happen in a short period. For example, if the period is 5 minutes, the rate limit would be: 10 * 1024 / 8 / (24 * 60 / 5) = 4 This long period allows to lift the limit up.	7 years ago
Eugen Rochko	cb5b5cb5f7	Slightly reduce RAM usage (#7301) ... * No need to re-require sidekiq plugins, they are required via Gemfile * Add derailed_benchmarks tool, no need to require TTY gems in Gemfile * Replace ruby-oembed with FetchOEmbedService Reduce startup by 45382 allocated objects * Remove preloaded JSON-LD in favour of caching HTTP responses Reduce boot RAM by about 6 MiB * Fix tests * Fix test suite by stubbing out JSON-LD contexts	7 years ago
MIYAGI Hikaru	f58dcbc981	HTTP proxy support for outgoing request, manage access to hidden service (#7134) ... * Add support for HTTP client proxy * Add access control for darknet Supress error when access to darknet via transparent proxy * Fix the codes pointed out * Lint * Fix an omission + lint * any? -> include? * Change detection method to regexp to avoid test fail	7 years ago
Yamagishi Kazutoshi	50529cbceb	Upgrade Rails to version 5.2.0 (#5898)	7 years ago
Eugen Rochko	49bbef1202	Use RAILS_LOG_LEVEL to set log level of Sidekiq, too (#7079) ... Fix #3565 (oops)	7 years ago
Eugen Rochko	80a944c882	Log rate limit hits (#7096) ... Fix #7095	7 years ago
Eugen Rochko	d4de2239b0	Add a circuit breaker for ActivityPub deliveries (#7053)	7 years ago
Yamagishi Kazutoshi	28384c1771	Revert "Revert "Upgrade Paperclip to version 6.0.0" (#6807)" (#6808) ... This reverts commit 40871caa4b.	7 years ago
Eugen Rochko	ac49c7932d	Add LDAP_TLS_NO_VERIFY option, don't require LDAP_ENABLED outside .env (#6845) ... Fix #6816, fix #6790	7 years ago
Alexander	33ee347c99	rename pam email environment variable to something more understandable and default to LOCAL_DOMAIN (better fallback) (#6833)	7 years ago
Eugen Rochko	40871caa4b	Revert "Upgrade Paperclip to version 6.0.0" (#6807) ... * Revert "Bump version to 2.3.2rc1" This reverts commit cdf8b92fea. * Revert "Downgrade Dockerfile to Ruby 2.4.3 on Alpine 3.6 (#6806)" This reverts commit 0074cad44f. * Revert "Handle Mastodon::HostValidationError when pulling remoteable assets (#6782)" This reverts commit 4a0a19fe54. * Revert "Correct the reference to user's password in mastodon:add_user task (#6800)" This reverts commit 338bff8b93. * Revert "Upgrade Paperclip to version 6.0.0 (#6754)" This reverts commit b88fcd53f7.	7 years ago
Yamagishi Kazutoshi	b88fcd53f7	Upgrade Paperclip to version 6.0.0 (#6754)	7 years ago
Effy Elden	dd9d00d293	Add additional first_name and last_name SAML attribute statement options, and modify Omniauthable concern to use full_name or first_name + last_name if not available (#6669)	7 years ago
Alexander	42fe05dea1	fix logic for pam_controlled_service (#6599)	7 years ago
Eugen Rochko	47bdb9b33b	Fix #942: Seamless LDAP login (#6556)	7 years ago
Akihiko Odaki	2e8a492e88	Raise Mastodon::HostValidationError when host for HTTP request is private (#6410)	7 years ago
Ghislain Loaec	e668180044	New variable OAUTH_REDIRECT_AT_SIGN_IN + Ref #6538 (not only SAML strategies) (#6540)	7 years ago
Ghislain Loaec	3084fe4959	New env variable: SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED + fixes #6533 (#6538)	7 years ago
Eugen Rochko	02856073f7	Fix #6509: Use pull queue for chewy jobs (#6513)	7 years ago
Daniel King	6ef3874b2e	Fix URLs incorrectly having trailing hyphen removed (#6465) ... In cases where a URL has a trailing hyphen the FetchLinkCardService incorrectly removes the hyphen when it is parsed The hyphen is not a reserved character in the URI spec https://tools.ietf.org/html/rfc3986#section-2.2	7 years ago
Eugen Rochko	e20700fe8f	Fix Chewy trying to update index with the wrong strategy (#6464)	7 years ago
Eugen Rochko	3ebc0ad4d3	Full-text search for authorized statuses (#6423) ... * Add full-text search for authorized statuses - Search API will return statuses that match the query - Only for logged in users - Only if you are author of the status, - Or you were mentioned in it - Or you favourited or reblogged it - Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX` - Run `rails chewy:deploy` to create & populate index Fix #5880 Fix #4293 Fix #1152 * Add commented out docker-compose configuration for ES container * Optimize index import, filter search results * Add basic normalization to the index * Add better stemming and normalization to the index * Skip webfinger request if search query includes both @ and a space * Fix code style * Visually separate search result sections * Fix code style issues	7 years ago
Eugen Rochko	38e0133e1b	Make PAM gem optional, allow configuration over environment (#6415)	7 years ago
Eugen Rochko	26f21fd5a0	CAS + SAML authentication feature (#6425) ... * Cas authentication feature * Config * Remove class_eval + Omniauth initializer * Codeclimate review * Codeclimate review 2 * Codeclimate review 3 * Remove uid/email reconciliation * SAML authentication * Clean up code * Improve login form * Fix code style issues * Add locales	7 years ago
Alexander	04fef7b888	pam authentication (#5303) ... * add pam support, without extra column * bugfixes for pam login * document options * fix code style * fix codestyle * fix tests * don't call remember_me without password * fix codestyle * improve checks for pam usage (should fix tests) * fix remember_me part 1 * add remember_token column because :rememberable requires either a password or this column. * migrate db for remember_token * move pam_authentication to the right place, fix logic bug in edit.html.haml * fix tests * fix pam authentication, improve username lookup, add comment * valid? is sometimes not honored, return nil instead trying to authenticate with pam * update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests * update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user * codeconvention fixes * code convention fixes * fix idention * update dependency, explicit conflict check * fix disabled password updates if in pam mode * fix check password if password is present, fix templates * block registration if account is maintained by pam * Revert "block registration if account is maintained by pam" This reverts commit 8e7a083d650240b6fac414926744b4b90b435f20. * fix identation error introduced by rebase * block usernames maintained by pam * document pam settings better * fix code style	7 years ago
Eugen Rochko	5276c0a090	HTML e-mails for UserMailer (#6256) ... - premailer gem to turn CSS into inline styles automatically - rework UserMailer templates - reword UserMailer templates	7 years ago
Patrick Figel	537d2939b1	Suppress CSRF token warnings (#6240) ... CSRF token checking was enabled for API controllers in #6223, producing "Can't verify CSRF token authenticity" log spam. This disables logging of failed CSRF checks. This also changes the protection strategy for PushSubscriptionsController to use exceptions, making it consistent with other controllers that use sessions.	7 years ago
Eugen Rochko	921b781909	Increase rate limit on protected paths (#6229) ... Previously each protected path had a separate rate limit. Now they're all in the same bucket, so people are more likely to hit one with register->login. Increasing to 25 per 5 minutes should be fine.	7 years ago
Naoki Kosaka	8d51ce4290	Fix enforce HTTPS in production. (#6180)	7 years ago
Patrick Figel	04ecf44c2f	Add confirmation step for email changes (#6071) ... * Add confirmation step for email changes This adds a confirmation step for email changes of existing users. Like the initial account confirmation, a confirmation link is sent to the new address. Additionally, a notification is sent to the existing address when the change is initiated. This message includes instruction to reset the password immediately or to contact the instance admin if the change was not initiated by the account owner. Fixes #3871 * Add review fixes	7 years ago
nightpool	9592b5e31e	enforce LOCAL_HTTPS=true in production (#6061) ... * enforce https in production * note changes in production env sample * typo fix	7 years ago
Yamagishi Kazutoshi	6855baa0c5	Change streaming API URL when remote development (#5942) ... * Change streaming API URL when remote development * Use STREAMING_API_BASE_URL when dev env	7 years ago
Eugen Rochko	feed07227b	Apply a 25x rate limit by IP even to authenticated requests (#5948)	7 years ago
Naoki Kosaka	4bce376fdc	Missing require 'authorization_decorator'. (#5947)	7 years ago
Eugen Rochko	a865b62efc	Rate limit by user instead of IP when API user is authenticated (#5923) ... * Fix #668 - Rate limit by user instead of IP when API user is authenticated * Fix code style issue * Use request decorator provided by Doorkeeper	7 years ago
THE BOSS ♨	17e26f8afe	Fix typo in paperclip.rb (#5936)	7 years ago
Yamagishi Kazutoshi	b0db4dad79	Revert fog-aws (ref #5604) (#5934)	7 years ago
Eugen Rochko	42bcbd36b7	Remove rabl dependency (#5894) ... * Remove rabl dependency * Replicate old Oj configuration	7 years ago
Eugen Rochko	546257bc7f	Allow specifying STATSD_NAMESPACE (#5700)	7 years ago
MitarashiDango	cbbeec05be	Fix spell miss (SWIIFT_OBJECT_URL -> SWIFT_OBJECT_URL) (#5617)	7 years ago
Yamagishi Kazutoshi	47b0c61853	Unify file upload to using fog (#5604)	7 years ago
Jeong Arm	9d97054fe6	Remove timestamps on any option (#5282)	7 years ago
unarist	7fd66cf2fe	Fix migration failure due to StrongMigrations on production env (#5283)	7 years ago
Lynx Kotoura	32e8a87830	adjust public profile pages 2 (#5223)	7 years ago
Nishi, Keisuke	83ffc4dc07	Fix Paperclip::Fog always responds Not Found in OpenStack-v2 like ConoHa (#5155)	7 years ago
Eugen Rochko	35a8cafa35	Replace self-rolled statsd instrumention with localshred/nsa (#5118)	7 years ago
Eugen Rochko	db3ed498b0	When OAuth password verification fails, return 401 instead of redirect (#5111) ... Call to warden.authenticate! in resource_owner_from_credentials would make the request redirect to sign-in path, which is a bad response for apps. Now bad credentials just return nil, which leads to HTTP 401 from Doorkeeper. Also, accounts with enabled 2FA cannot be logged into this way.	7 years ago
Eugen Rochko	e528114c53	Follow-up to #4582 and #5027, removing dead code (#5101)	7 years ago
Eugen Rochko	b982d549f4	Add strong_migrations gem to warn when creating unsafe migrations (#5078)	7 years ago
Eugen Rochko	d68df88d4e	Disable private status federation over OStatus (#5027)	7 years ago
unarist	6db034a866	Re-allow underscore on valid_url_path_ending_chars (#4999) ... Limiting allowed characters in the last character of the URL is came from twitter-text, but underscore is allowed on there, and Mastodon before #4941.	7 years ago
ふぁぼ原	3816943e6b	Enable to recognize most kinds of characters as URL paths (#4941)	7 years ago
abcang	1aad015bbb	Revert unique retry job (#4937) ... * Revert "Enable UniqueRetryJobMiddleware even when called from sidekiq worker (#4836)" This reverts commit 6859d4c028. * Revert "Do not execute the job with the same arguments as the retry job (#4814)" This reverts commit be7ffa2d75.	7 years ago
Patrick Figel	3018043fc2	Add OpenStack Keystone V3 support (#4889) ... Keystone V2 is deprecated in favour of V3. This adds the necessary connection parameters for establishing a V3 connection. Connections to V2 endpoints are still possible and the configuration should remain compatible. This also introduces a SWIFT_REGION variable for multi-region OpenStack environments and a SWIFT_CACHE_TTL that controls how long tokens and other meta-data is cached for. Caching tokens avoids rate-limiting errors that would result in media uploads becoming unavailable during high load or when using tasks like media:remove_remote. fog-openstack only supports token caching for V3 endpoints, so a recommendation for using V3 was added.	7 years ago
abcang	6859d4c028	Enable UniqueRetryJobMiddleware even when called from sidekiq worker (#4836)	7 years ago
Adam Thurlow	6994664a13	swift-enable the paperclip! 📎 (#2322)	7 years ago
abcang	be7ffa2d75	Do not execute the job with the same arguments as the retry job (#4814)	7 years ago
Eugen Rochko	1b1e025b41	Use updated ActivityStreams context (added: sharedInbox) (#4764)	7 years ago
Eugen Rochko	00840f4f2e	Add handling of Linked Data Signatures in payloads (#4687) ... * Add handling of Linked Data Signatures in payloads * Add a way to sign JSON, fix canonicalization of signature options * Fix signatureValue encoding, send out signed JSON when distributing * Add missing security context	7 years ago
Eugen Rochko	cf615abbf9	Add configuration to disable private status federation over PuSH (#4582)	8 years ago