1,035
edits
Performance: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 6: | Line 6: | ||
The changes we have actually made from the default configuration, each is either described below or on a separate page: | The changes we have actually made from the default configuration, each is either described below or on a separate page: | ||
* Split out sidekiq queues into separate service files | |||
* Optimized postgres using pgtune | |||
=== Archive === | |||
Olde changes that aren't true anymore | |||
* Increase [[Sidekiq]] <code>DB_POOL</code> and <code>-c</code> values from from 25 to 75 | * Increase [[Sidekiq]] <code>DB_POOL</code> and <code>-c</code> values from from 25 to 75 | ||
** Replaced with separate sidekiq service files | |||
== [[Sidekiq]] == | == [[Sidekiq]] == | ||
| Line 20: | Line 28: | ||
=== Configuration === | === Configuration === | ||
==== Default ==== | |||
By default, the <code>mastodon-sidekiq</code> service is configured with 25 threads, the full service file is as follows: | By default, the <code>mastodon-sidekiq</code> service is configured with 25 threads, the full service file is as follows: | ||
< | <syntaxhighlight lang="ini"> | ||
[Unit] | [Unit] | ||
Description=mastodon-sidekiq | Description=mastodon-sidekiq | ||
| Line 77: | Line 87: | ||
[Install] | [Install] | ||
WantedBy=multi-user.target | WantedBy=multi-user.target | ||
</ | </syntaxhighlight> | ||
==== Separate Services ==== | |||
We want to split up sidekiq into multiple processes using separate systemd service files. We want to a) make the site responsive by processing high-priority queues quickly but also b) use all our available resources by not having processes sit idle. So we give each of the main queues one service file that has that queue as the top prioriry, and mix the other queues in as secondary priorities - sidekiq will try and process items from the first queue first, second queue second, and so on. | |||
So we allocate 25 threads (and 25 db connections) each to four service files with the following priority orders. | |||
* default, ingress, pull, push | |||
* ingress, default, push, pull | |||
* push, pull, default, ingress | |||
* pull, push, default, ingress | |||
And two additional service files that give 5 threads to the lower-priority queues: | |||
* mailers | |||
* scheduler | |||
(each service file looks like this:) | |||
<syntaxhighlight lang="ini"> | |||
Environment="DB_POOL=25" | |||
ExecStart=/home/mastodon/.rbenv/shims/bundle exec sidekiq -q push -q pull -q default -q ingress -c 25 | |||
</syntaxhighlight> | |||
and is located in <code>/etc/systemd/system</code> with the name of its primary queue (eg. <code>/etc/systemd/system/mastodon-sidekiq-default.service</code>) | |||
Then we make one meta-service file <code>mastodon-sidekiq.service</code> that can control the others: | |||
<syntaxhighlight lang="ini"> | |||
[Unit] | |||
Description=mastodon-sidekiq | |||
After=network.target | |||
Wants=mastodon-sidekiq-default.service | |||
Wants=mastodon-sidekiq-ingress.service | |||
Wants=mastodon-sidekiq-mailers.service | |||
Wants=mastodon-sidekiq-pull.service | |||
Wants=mastodon-sidekiq-push.service | |||
Wants=mastodon-sidekiq-scheduler.service | |||
[Service] | |||
Type=oneshot | |||
ExecStart=/bin/echo "mastodon-sidekiq exists only to collectively start and stop mastodon-sidekiq-* instances, shimmi> | |||
RemainAfterExit=yes | |||
[Install] | |||
WantedBy=multi-user.target | |||
</syntaxhighlight> | |||
and make the subsidiary service dependent on the main service | |||
<syntaxhighlight lang="ini"> | |||
[Install] | |||
WantedBy=multi-user.target mastodon-sidekiq.service | |||
</syntaxhighlight> | |||
== [[Postgresql]] == | == [[Postgresql]] == | ||