Tech WG/Ops Diary: Difference between revisions

sidekiq scheduler queue issues
(ES restart)
(sidekiq scheduler queue issues)
Line 237: Line 237:
}}
}}


== 24-04-17 ==
== 24-04-17 Elastic Search issues ==


Manisha: Saw a warning in the admin panel:
Manisha: Saw a warning in the admin panel:
Line 256: Line 256:


Had to wait a tiny bit before [https://discord.com/channels/1049136631065628772/1049184335514828860/1230220033481773087 the warning in the admin panel] disappeared.
Had to wait a tiny bit before [https://discord.com/channels/1049136631065628772/1049184335514828860/1230220033481773087 the warning in the admin panel] disappeared.
== 24-04-30 Sidekiq scheduler queue issues ==
Noticed this warning about sidekiq [[File:Sidekiq issues.png|frameless]]
And saw that we were hitting our swap limits.
Restarted services. Sidekiq status is active but we are still near our swap limits and the sidqkiq warning in the admin panel hasn't gone away. [https://discord.com/channels/1049136631065628772/1049184335514828860/1235046062171885568 Link to discussion on the technical-wg channel].
[https://github.com/mastodon/mastodon/issues/20710#issuecomment-1329680431 Maybe this github discussion would help?] but on checking the sidekiq dashboard, while the no. of processed requests did plummet today, I do see sidekiq running. The warning about the scheduler queue(s) is puzzling as there are scheduled jobs in the sidekiq container! hmmm