1,035
edits
Storage: Difference between revisions
→Cache Pruning
(Created page with "Up to: Part Of::Tech WG so Mastodon takes up space, right? How do we manage that? Currently we just have all our storage on the same Linode compute instance that we run the mastodon instance from. That should probably change at some point to using S3-compatible object storage. But for now, what the heck takes up all the space? == Database == The database is relatively small since it stores == Media Storage == The instance is currently configured to remov...") |
|||
| (4 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
But for now, what the heck takes up all the space? | But for now, what the heck takes up all the space? | ||
== | == What Gets Stored == | ||
=== Database === | |||
== Media Storage == | The database is relatively small since it mostly stores text! For reference, as of August 15 (roughly 8 months after the instance started), it is 13.5GB, with statuses maching up 4.5GB of that. | ||
=== Media Storage === | |||
The instance is currently configured to remove media (images, video, audio) from other instances from the cache after '''1 day''' - after that, the media is redownloaded from the hosting instance when accessed. | The instance is currently configured to remove media (images, video, audio) from other instances from the cache after '''1 day''' - after that, the media is redownloaded from the hosting instance when accessed. | ||
Of course, we host all media from our members indefinitely, and actively encourage them to post big images and videos because that fills up their stat counters | Of course, we host all media from our members indefinitely, and actively encourage them to post big images and videos because that fills up their stat counters | ||
[[File:Sims stats.jpg|class=img-responsive]] | |||
=== Cache === | |||
Data from other instances that isn't status and other text data is stored in the system cache: <code>mastodon/live/public/system/cache</code> - this currently makes up most of the storage needs. | |||
It includes | |||
* account data, including avatars and header images, which are stored indefinitely and thus need to be [[#Pruning|pruned]] | |||
* media from other instances (images, videos in posts) | |||
* custom emojis from other instances | |||
* link preview cards for embedded links | |||
== Managing Storage == | |||
=== Media === | |||
Media storage is configured from the Mastodon admin interface - https://neuromatch.social/admin/settings/content_retention | |||
'''Warning:''' Do not set the "Content cache retention period" as it will remove posts and bosts from other servers! that includes removing the bookmarks, favorites, and boosts of our members! bad to do! | |||
=== Cache Pruning === | |||
See [[Maintenance#Cache Pruning]] | |||
We use [[tootctl]] to periodically prune data from the server by running these commands every month. At the moment this keeps our storage in a sustainable range: | |||
<syntaxhighlight lang="bash"> | |||
# remote cover images from accounts that nobody on the instance is following | |||
tootctl media remove --remove-headers | |||
</syntaxhighlight> | |||
We might also want to add these commands that make sense to do, but dont' necessarily contribute a ton to our storage burden. Want to check with the rest of [[Tech WG]] first before I do these (-[[Jonny]] 23-08-15) | |||
<syntaxhighlight lang="bash"> | |||
# remove remote accounts that no longer exist | |||
# excludes accounts with confirmed activity in the last week in case the server is down | |||
tootctl accounts cull | |||
# remove files that do not belong to media attachments. | |||
tootctl media remove-orphans | |||
</syntaxhighlight> | |||
== Reference == | |||
* Helpful info on managing disk usage with [[tootctl]] - https://ricard.dev/improving-mastodons-disk-usage/ | |||