Commit Graph

81 Commits

Author SHA1 Message Date
26b08a3c54 Add remote only to public timeline (#13504)
* Add remote only to public timeline

* Fix code style
2020-05-10 10:36:18 +02:00
0dfba0884e minor server-sent events fixes (#12945)
* Send output on the server-sent events stream immediately so the client sees that it was successfully opened even if it doesn't have any messages.

Fix transparent SSE streaming for the public:local and hashtag:local stream types.

* Tell caches to never store server-sent events.
2020-01-24 20:51:33 +01:00
24552b5160 Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
e7353c47db Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (#11302) 2019-07-15 05:56:35 +02:00
d63c3c0cef Improve streaming server security (#10818)
* Check OAuth token scopes in the streaming API

* Use Sec-WebSocket-Protocol instead of query string to pass WebSocket token

Inspired by https://github.com/kubevirt/kubevirt/issues/1242
2019-05-24 15:21:42 +02:00
ca6c93a2f5 Migrate from uws to cws (#10805) 2019-05-22 18:19:16 +02:00
13a7f05030 Fix streaming API always attempting to use SSL with Postgres (#10231)
Fix #10223
2019-03-11 00:51:23 +01:00
dbeab5a036 Fix SSL configuration regression in streaming API (#10225)
* Fix SSL configuration regression in streaming API

Fix #10223

* Fix code style issues and integrate #10219

* Fix dumb thing
2019-03-10 16:00:54 +01:00
42e733681a config: add DB_SSLMODE for managed/remote PG (#10210)
* config: add DB_SSLMODE for managed/remote PG

* streaming: set PG sslmode, defaults to prefer
2019-03-08 14:36:28 +01:00
8d70d3de38 Fix crash when using UNIX socket (#9036) 2018-10-21 16:41:33 +02:00
369cc5f555 Check if port/socket is available before forking in Streaming API (#9023)
Previously, the server would attempt taking port/socket in worker
process, and if it was taken, fail, which made the master process
create a new worker. This led to really high CPU usage if the
streaming API was started when the port or socket were not
available.

Now, before clustering (forking) into worker processes, a test
server is created and then removed to check if it can be done.
2018-10-20 02:25:25 +02:00
18e7ef6eda Add check for missing tag param in streaming API (#8955)
* Add check for missing tag param in streaming API

Fixes error:

```
TypeError: Cannot read property 'toLowerCase' of undefined
at app.get (.../streaming/index.js:493:50)
```

* Fix code style issues
2018-10-11 19:24:43 +02:00
774ac47373 Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
f37fafe30b Add health endpoint to streaming API (#8441)
GET /api/v1/streaming/health

Answers with OK. Fix #8337
2018-08-26 11:54:25 +02:00
a083604d1d Support UNIX domain socket for streaming service without using PORT (#8217)
* Support UNIX domain socket for streaming service without using PORT

The use of UNIX domain socket for streaming service was not officially supported,
but it was made unofficial to use by setting a path to PORT.
From now on, SOCKET will be used just like setting for puma.

* Hundle relative path
2018-08-24 18:16:53 +02:00
0180037dfb Fix streaming API still using filtered instead of chosen languages (#8009) 2018-07-14 03:59:31 +02:00
7403e5d306 Add media timeline (#6631) 2018-05-21 12:43:38 +02:00
156b916caf Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
609bf93029 Perform processing that does not use the database before connecting to the database (#7168) 2018-04-17 13:49:09 +02:00
ccf4f170de Make sure call done(); in the listener of public timeline for anonymous connection (#6009) 2017-12-13 14:27:36 +01:00
90e7da16a0 Fix the condition in streaming listener (#6008) 2017-12-13 13:42:16 +01:00
c986218c3a Improve error handling in streaming/index.js (#5968)
On an unhandled worker exception, we should log the exception
and exit with nonzero status, instead of letting workers
silently fail and restarting them in an endless loop.

Note: we previously tried to handle the `'error'` signal.
That's not a signal Node fires; my patch traps `'uncaughtException'`,
which is what the code was _trying_ to do.
2017-12-12 20:19:33 +01:00
cfea28216f make it possible to stream public timelines without authorization (#5977)
* make it possible to stream public timelines without authorization

* Fix

* Make eslint allow `value == null`

* Remove redundant line

* Improve style and revert .eslintrc.yml

* Fix streamWsEnd

* Show IP address instead of (anonymous user)

* Add missing semicolon
2017-12-12 15:13:24 +01:00
2864e5e077 Make it possible to bind streming service to 0.0.0.0 (#5744) 2017-11-18 04:44:19 +01:00
24cafd73a2 Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
0e0c6b1b4b use-DB_NAME-in-development (#5430) 2017-10-17 11:45:37 +02:00
909a6d4661 Fix streaming API double-JSON-encoding status IDs for deletes (#5069) 2017-09-24 15:31:03 +02:00
b6f3869f8d Fix streaming url to lowercase (#4804) 2017-09-04 12:52:06 +02:00
3f82d8b979 Gracefully stop streaming server (#4103) 2017-07-07 20:01:00 +02:00
ebd2dde688 Restore streaming API output format (#4100)
* Restore streaming API output format

Regression from #4090

* Remove whitespace
2017-07-07 16:56:52 +02:00
285038972b Stop using Babel with streaming server (#3950) 2017-06-26 04:49:39 +02:00
e5563843a2 Re-fix errorMiddleware (#3922) 2017-06-26 01:46:15 +02:00
c972e1ee1f Ignore DB_NAME for development env on streaming as well as rails side (#3948) 2017-06-26 01:45:50 +02:00
87efa38721 more free pgconfig by .env (#3909)
* more free pgconfig for streaming by .env

* fix wrong default values

* database.yml read ENV as same as streaming server
2017-06-25 18:13:31 +02:00
0c44316b22 Fix errorMiddleware to prevent "TypeError: res.writeHead is not a function" (#3913)
* Fix errorMiddleware

* Add "eslint-disable-line no-unused-vars"
2017-06-23 19:22:02 +02:00
eff9416469 Remove unused variables (#3906) 2017-06-23 16:05:04 +02:00
d8ec832806 Fix streaming server. Redis connection subscribe for each channel. (#3828) 2017-06-20 20:41:41 +02:00
0464602978 Fix issue with some Node.js versions not having Array.prototype.includes (#3667)
by using Array.prototype.indexOf instead
2017-06-09 19:46:33 +02:00
dab8fc4584 Execute PushUpdateWorker only for accounts who uses StreamingAPI just now. (#3278)
* Add redis key "subscribed:timeline:#{account.id}" to indicate active streaming API listeners exists.

* Add endpoint for notification only stream.

* Run PushUpdateWorker only for users uses Streaming API now.

* Move close hander streamTo(Http/Ws) -> stream(Http/Ws)End (Deal with #3370)

* Add stream type for stream start log message.
2017-06-03 20:50:53 +02:00
9a81be0d37 [RFC] Return 401 for an authentication error on WebSockets (#3411)
* Return 401 for an authentication error on WebSocket

* Use upgradeReq instead of a custom object
2017-05-29 18:20:53 +02:00
91c789ec63 Don't stream toots from users who have blocked the recipient user (#3300)
* Don't stream toots from users who have blocked the recipient user

This filter was already applied on the `/api/v1/timelines/public` API, but
not yet for the Streaming API.

* Boosted status' account_id doesn't need for filtering in streaming/index.js

This filtering is only for public/hashtag timelines, but boosts already filtered on FanOutOnWriteService because those timelines don't show boosts.
2017-05-29 18:01:08 +02:00
45837c533e Re-add stream end log for WebSocket (#3397) 2017-05-28 19:14:44 +02:00
425d02287a Improve streaming API cluster logging (#3370)
* Improve streaming API cluster logging

* Less verbose error middleware logging (stack trace useless there)

* Fix error logging

* Prevent potential issue

* Add missing "done()" in catch of Promise.all, websocket heartbeat re-implemented like in example

* I actually forgot a done(), the absolute madman
2017-05-28 16:25:26 +02:00
2e429c0c25 Reject revoked access_token on Streaming API. (#3367) 2017-05-27 23:27:54 +02:00
0734e1fe33 Language filtering in streaming API (#3339) 2017-05-27 00:53:48 +02:00
8e84177305 Use local default for postgres host in node streaming service (#3240)
This location varies across postgres installations, and it seems like the pg
package knows how to guess correctly on each system.
2017-05-22 20:27:24 -04:00
7fba4cb3d1 Allow access token in URI (#3208) 2017-05-21 21:13:11 +02:00
20c0054460 Adjust REDIS_URL usage in node_redis (#3183)
Resolves #2780
2017-05-20 21:06:09 +02:00
2e112e2406 Improve eslint rules (#3147)
* Add semi to ESLint rules

* Add padded-blocks to ESLint rules

* Add comma-dangle to ESLint rules

* add config/webpack and storyboard

* add streaming/

* yarn test:lint -- --fix
2017-05-20 17:31:47 +02:00
620d0d8029 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00