Account archive download (#6460)

* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
This commit is contained in:
Eugen Rochko
2018-02-21 23:21:32 +01:00
committed by GitHub
parent c1e77b56a9
commit 61ed133fea
27 changed files with 374 additions and 7 deletions

View File

@ -301,6 +301,7 @@ GEM
mini_mime (>= 0.1.1)
mario-redis-lock (1.2.0)
redis (~> 3, >= 3.0.5)
memory_profiler (0.9.10)
method_source (0.9.0)
microformats (4.0.7)
json
@ -664,6 +665,7 @@ DEPENDENCIES
link_header (~> 0.0)
lograge (~> 0.7)
mario-redis-lock (~> 1.2)
memory_profiler
microformats (~> 4.0)
mime-types (~> 3.1)
nokogiri (~> 1.8)