Commit Graph

30 Commits (3b60832214fdd9d8b5fd01e9a177a6dde5259907)

Author SHA1 Message Date
aschmitz 669fe9ee06 Change IDs to strings rather than numbers in API JSON output (#5019)
* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Back out RelationshipsController Change

This was made to make a test a bit less flakey, but has nothing to
do with this branch.

* Change internal streaming payloads to stringified IDs as well

Per
https://github.com/tootsuite/mastodon/pull/5019#issuecomment-330736452
we need these changes to send deleted status IDs as strings, not
integers.
7 years ago
Eugen Rochko 4c76402ba1 Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.

Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
7 years ago
unarist 7876aed134 Fix deletion of status which has been reblogged (#4728) 7 years ago
Eugen Rochko 2a2698e450 Add ActivityPub serializer for Undo of Announce (#4703) 7 years ago
Eugen Rochko 00840f4f2e Add handling of Linked Data Signatures in payloads (#4687)
* Add handling of Linked Data Signatures in payloads

* Add a way to sign JSON, fix canonicalization of signature options

* Fix signatureValue encoding, send out signed JSON when distributing

* Add missing security context
7 years ago
Eugen Rochko b7370ac8ba ActivityPub delivery (#4566)
* Deliver ActivityPub Like

* Deliver ActivityPub Undo-Like

* Deliver ActivityPub Create/Announce activities

* Deliver ActivityPub creates from mentions

* Deliver ActivityPub Block/Undo-Block

* Deliver ActivityPub Accept/Reject-Follow

* Deliver ActivityPub Undo-Follow

* Deliver ActivityPub Follow

* Deliver ActivityPub Delete activities

Incidentally fix #889

* Adjust BatchedRemoveStatusService for ActivityPub

* Add tests for ActivityPub workers

* Add tests for FollowService

* Add tests for FavouriteService, UnfollowService and PostStatusService

* Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService

* Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService

* Add tests for BatchedRemoveStatusService

* Deliver updates to a local account to ActivityPub followers

* Minor adjustments
7 years ago
Eugen Rochko 605e2a417c Fix regression from #3672 - Do not use pipeline around zscore (#3704) 7 years ago
Eugen Rochko ce812466c7 Fix removal of status sending the original status to mentioned users instead of delete Salmon (#3672)
* Fix removal of status sending the original status to mentioned users instead
of delete Salmon, add test

* Create remove_status_service_spec.rb
7 years ago
Eugen Rochko 51d7caaf19 Fix wrong pubsub channel on public timelines 8 years ago
Eugen Rochko dbd529109e Fix notifications delivered to wrong pubsub channel, optimized RemoveStatusService,
slightly optimized FanOutOnWriteService again
8 years ago
Eugen Rochko 5b95be1c42 Replace calls to FeedManager#inline_render and #broadcast 8 years ago
Eugen Rochko c77a54fe0a Fix #651 - Do not reinsert original status into all followers feeds
upon un-reblogging. Check if the reblog was in the feed in the first
place. It might have been filtered on distribution.
8 years ago
Eugen Rochko 446267d1bf Deduplicate delete salmons (send only one per mentioned-account domain) 8 years ago
Eugen Rochko 149887a0ff Make follow requests federate 8 years ago
Eugen Rochko d9ca46b464 Cleaning up format of broadcast real-time messages, removing
redis-backed "mentions" timeline as redundant (given notifications)
8 years ago
Eugen Rochko 5c7add2176 Fix #147 - Unreblogging will leave original status in feeds 8 years ago
Eugen Rochko 6de079a5af Removing external hub completely, fix #333 fixing digit-only hashtags,
removing web app capability from non-webapp pages
8 years ago
Eugen Rochko 2ef9f36cf2 Improve suspend account service 8 years ago
Eugen 1e99a2bb03 Fix trying to PuSH-publish updates of remote removals 8 years ago
Eugen Rochko 2d2c81765b Adding embedded PuSH server 8 years ago
Eugen Rochko fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 8 years ago
Eugen Rochko c5e03a2e0d Status removal is broadcast to public/hashtag timelines too 8 years ago
Eugen Rochko 0895ff414e Fix RemoveStatusService trying to send delete salmons on behalf of remote statuses 8 years ago
Eugen Rochko 244d1307a3 Fix remove status service sending salmons 8 years ago
Eugen Rochko 927333f4f8 Improve code style 8 years ago
Eugen Rochko d6a64f45fd Adding a notification stack for error messages 8 years ago
Eugen Rochko 05b0c985b4 Send "delete" Salmons to remote mentioned accounts on status removal
Fixes #27
8 years ago
Eugen Rochko 3cc47beb6e Refactored generation of unique tags, URIs and object URLs into own classes,
as well as formatting of content
8 years ago
Eugen Rochko 926eea89b5 RemoveStatusService fleshed out, still doesn't send Salmon slaps though 8 years ago
Eugen Rochko a289c1d52f Handle delete Salmons, todo: clean up timelines 8 years ago