Commit Graph

1386 Commits

Author SHA1 Message Date
57b893d505 Fix spaces not being stripped in admin account search (#21324)
Fixes #21058

Regression from #18641
2022-11-27 20:47:29 +01:00
47f0d7021e refactor(vacuum statuses): reduce amount of db queries and load for each query - improve performance (#21487)
* refactor(statuses_vacuum): remove dead code - unused

Method is not called inside class and private.
Clean up dead code.

* refactor(statuses_vacuum): make retention_period present test explicit

This private method only hides functionality.
It is best practice to be as explicit as possible.

* refactor(statuses_vacuum): improve query performance

- fix statuses_scope having sub-select for Account.remote scope by
  `joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
  because it is inexplicit, bad practice and even slower than normal
  `.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
  active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
  because in_batches delivers a full blown active record query result,
  in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
  reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
  the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
  another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send

* refactor(status): simplify unlink_from_conversations

- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?

* refactor(status): searchable_by way less sub selects

These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.

This is how it was:

```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
  Status Load (1.6ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (1.5ms)  SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```

This is how it is with this change:

```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
  Status Load (1.7ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (0.7ms)  SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```
2022-11-27 20:41:18 +01:00
f17fc5742e Clear voter count when poll is reset (#21700)
When a poll is edited, we reset the poll and remove all previous
votes. However, prior to this commit, the voter count on the poll
was not reset. This leads to incorrect percentages being shown in
poll results.

Fixes #21696
2022-11-26 23:08:25 +01:00
5b2ff8d32d fix media uploads with ffmpeg 5 (#21191) 2022-11-25 16:20:47 +01:00
69378eac99 Don't allow URLs that contain non-normalized paths to be verified (#20999)
* Don't allow URLs that contain non-normalized paths to be verified

This stops things like https://example.com/otheruser/../realuser where
"/otheruser" appears to be the verified URL, but the actual URL being
verified is "/realuser" due to the "/../".

Also fix a test to use 'https', so it is testing the right thing, now
that since #20304 https is required.

* missing do
2022-11-20 19:28:13 +01:00
c373148b3d Support for import/export of instance-level domain blocks/allows for 4.x w/ additional fixes (#20597)
* Allow import/export of instance-level domain blocks/allows (#1754)

* Allow import/export of instance-level domain blocks/allows.
Fixes #15095

* Pacify circleci

* Address simple code review feedback

* Add headers to exported CSV

* Extract common import/export functionality to
AdminExportControllerConcern

* Add additional fields to instance-blocked domain export

* Address review feedback

* Split instance domain block/allow import/export into separate pages/controllers

* Address code review feedback

* Pacify DeepSource

* Work around Paperclip::HasAttachmentFile for Rails 6

* Fix deprecated API warning in export tests

* Remove after_commit workaround

(cherry picked from commit 94e98864e3)

* Add confirmation page when importing blocked domains (#1773)

* Move glitch-soc-specific strings to glitch-soc-specific locale files

* Add confirmation page when importing blocked domains

(cherry picked from commit b91196f4b7)

* Fix authorization check in domain blocks controller

(cherry picked from commit 7527937758)

* Fix error strings for domain blocks and email-domain blocks

Corrected issue with non-error message used for Mastodon:NotPermittedError in Domain Blocks
Corrected issue Domain Blocks using the Email Domain Blocks message on ActionContoller::ParameterMissing
Corrected issue with Email Domain Blocks using the not_permitted string from "custom emojii's"

* Ran i18n-tasks normalize to address test failure

* Removed unused admin.export_domain_blocks.not_permitted string

Removing unused string as indicated by Check i18n

* Fix tests

(cherry picked from commit 9094c2f52c)

* Fix domain block export not exporting blocks with only media rejection

(cherry picked from commit 26ff48ee48)

* Fix various issues with domain block import

- stop using Paperclip for processing domain allow/block imports
- stop leaving temporary files
- better error handling
- assume CSV files are UTF-8-encoded

(cherry picked from commit cad824d8f501b95377e4f0a957e5a00d517a1902)

Co-authored-by: Levi Bard <taktaktaktaktaktaktaktaktaktak@gmail.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-11-17 11:05:09 +01:00
00b2720ef0 Change automatic post deletion configuration to be accessible to redirected users (#20774)
Fixes #20550
2022-11-17 10:55:23 +01:00
0cc77263fc Change batch account suspension to create a strike (#20897) 2022-11-17 10:52:51 +01:00
552d69ad96 Fix error when invalid domain name is submitted (#19474)
Fix #19175
2022-11-14 08:07:14 +01:00
b31afc6294 Fix error when passing unknown filter param in REST API (#20626)
Fix #19156
2022-11-14 08:06:06 +01:00
5c826c408d Fix image type not being set after conversion for convertible image types (#20624) 2022-11-14 07:13:14 +01:00
2e2ba39abf Fix rules with same priority being sorted non-deterministically (#20623) 2022-11-14 06:28:19 +01:00
c4c1bee880 Fix trendable status without review (#20214) 2022-11-11 21:24:10 +01:00
96f51e593f Guard against error extracting body from URL (#20428)
If `Nokogiri::HTML(value).at_xpath('//body')` fails to find the `body` element, it will return `nil`. We can guard against that with an early return. Avoids calling `children` on `Nilclass` in those cases.
2022-11-11 21:22:28 +01:00
9bc0a6c861 Fix metadata scrubbing removing color profile from images (#20389) 2022-11-11 09:20:10 +01:00
86f6631d28 Remove dead code and refactor status threading code (#20357)
* Remove dead code

* Remove unneeded/broken parameters and refactor descendant computation
2022-11-10 22:30:00 +01:00
c6c7c6223d Change verification to only work for https links (#20304)
Fix #20242
2022-11-10 21:09:03 +01:00
9965a23b04 Change link verification to ignore IDN domains (#20295)
Fix #3833
2022-11-10 06:27:45 +01:00
78a6b871fe Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
0cd0786aef Revert filtering public timelines by locale by default (#20294) 2022-11-10 05:34:42 +01:00
e98833748e Fix being able to spoof link verification (#20217)
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
2022-11-09 08:24:21 +01:00
ca80beb653 Micro-optimization: use if/else instead of Array#compact and Array#min (#19906)
* Technically `if`/`else` is faster than using `[value1, value2].compact.min` to find the lesser of two values, one of which may be `nil`.
2022-11-08 03:50:47 +01:00
bbf74498f5 Fix validation error in SynchronizeFeaturedTagsCollectionWorker (#20018)
* Fix followers count not being updated when migrating follows

Fixes #19900

* Fix validation error in SynchronizeFeaturedTagsCollectionWorker

Also saves remote user's chosen case for hashtags

* Limit remote featured tags before validation
2022-11-07 22:35:53 +01:00
106648b456 Micro-optimization: only split acct into two Strings (#19901)
* Since `acct` is split by `@` and assigned to `username` and `domain`, we only need to split `acct` into two Strings.
2022-11-07 16:17:55 +01:00
54f0f1b9ef Skip Webfinger cache during migrations as well (#19883) 2022-11-07 03:31:38 +01:00
3151b260e2 Fix not using GIN index for account search queries (#19830) 2022-11-06 06:16:34 +01:00
c2170991c7 Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
b1a219552e Fix featured tags not saving preferred casing (#19732) 2022-11-04 16:08:29 +01:00
125322718b Fix inaccurate admin log entry for re-sending confirmation e-mails (#19674)
Fixes #19593
2022-11-02 18:50:21 +01:00
0f5e6dd02b Add support for AVIF uploads (#19647) 2022-11-01 22:08:41 +01:00
ae07cfb868 Add support for HEIC uploads (#19618) 2022-11-01 16:26:25 +01:00
15bae3e0e4 Change post-processing to be deferred only for large media types (#19617) 2022-11-01 15:27:58 +01:00
d0ba77047e Change max. thumbnail dimensions to 640x360px (360p) (#19619) 2022-11-01 13:01:39 +01:00
26478f461c Remove language filtering from hashtag timelines (#19563) 2022-10-30 21:29:23 +01:00
ac9fb0d654 Add reputation and followers score boost to SQL-only account search (#19251) 2022-10-30 13:23:05 +01:00
40c7f3e830 Fix account action type validation (#19476)
* Fix account action type validation

Fix #19143

* Fix #19145

* Fix code style issues
2022-10-30 02:44:32 +02:00
3b024c563c Fix not being able to input featured tag with # (#19535) 2022-10-30 02:43:20 +02:00
dc5c86add7 Fix account migration form ever using outdated account data (#18429) 2022-10-29 01:31:45 +02:00
d9d722d74b Change admin announcement edition interface to use datetime-local (#18321)
* Change admin announcement edition interface to use datetime-local

* Dynamically set announcement stop date as required if start date is set, set minimum date for stop date

* Change `all_day` to not be bound to presence of time-range

* Add pattern and placeholder as minimal fallback for browsers not supporting datetime-local

* Display datetime-local inputs as local time

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-10-28 12:56:32 +02:00
07cc201acc Fix using wrong policy on status-related actions in admin UI (#19490) 2022-10-28 00:48:30 +02:00
5f733ad83a Remove unused method searchable? on accounts (#19489)
It called the wrong methods, but nothing uses it
2022-10-27 19:30:08 +02:00
d2eb726962 Fix notifications about deleted reports not being also deleted (#19475)
* Fix notifications about deleted reports not being also deleted

* Fix notification with empty report crashing web UI

Fix #18909
2022-10-27 02:10:54 +02:00
52ebfb7792 Store integer settings as integer (#19478) 2022-10-26 22:14:07 +02:00
f8ca3bb2a1 Add ability to view previous edits of a status in admin UI (#19462)
* Add ability to view previous edits of a status in admin UI

* Change moderator access to posts to be controlled by a separate policy
2022-10-26 13:42:29 +02:00
487d81fb92 Fix IP blocks not having a unique index (#19456) 2022-10-25 21:43:44 +02:00
74ead7d106 Change featured tag updates to add/remove activity (#19409)
* Change featured tag updates to add/remove activity

* Fix to check for the existence of feature tag

* Rename service and worker

* Merge AddHashtagSerializer with AddSerializer

* Undo removal of sidekiq_options
2022-10-22 18:30:55 +02:00
53e86747e4 Fix duplicate featured tags (#19403)
* Fix duplicate featured tags

* Add unique tag name validator

* Fix error message
2022-10-22 14:30:59 +02:00
1d34eff63f Add featured tag add/remove activity handler (#19408) 2022-10-22 11:49:41 +02:00
7c152acb2c Change settings area to be separated into categories in admin UI (#19407)
And update all descriptions
2022-10-22 11:44:41 +02:00
839f893168 Change public accounts pages to mount the web UI (#19319)
* Change public accounts pages to mount the web UI

* Fix handling of remote usernames in routes

- When logged in, serve web app
- When logged out, redirect to permalink
- Fix `app-body` class not being set sometimes due to name conflict

* Fix missing `multiColumn` prop

* Fix failing test

* Use `discoverable` attribute to control indexing directives

* Fix `<ColumnLoading />` not using `multiColumn`

* Add `noindex` to accounts in REST API

* Change noindex directive to not be rendered by default before a route is mounted

* Add loading indicator for detailed status in web UI

* Fix missing indicator appearing while account is loading in web UI
2022-10-20 14:35:29 +02:00