Upgrade Mastodon: Difference between revisions

← Older edit

Upgrade Mastodon (view source)

Revision as of 03:26, 1 December 2023

9,485 bytes added , 03:26, 1 December 2023

no edit summary

Jonny

Bureaucrats, Interface administrators, Administrators

1,035

edits

@@ Line 3: / Line 3: @@
 == Stages ==
-=== Merging ===
+=== Steps to be followed for '''merging glitch + upstream stable release + instance's stable custom features''' ===
+* Sync the <code>[https://github.com/NeuromatchAcademy/mastodon/tree/glitch-clean-sync glitch-clean-sync]</code> branch to <code>glitch-soc/mastodon:main</code>: Open the branch page and click "'''sync fork'''"
+** The purpose of this branch is solely to serve as a replica of the parent <code>glitch-soc/mastodon:main</code> branch and so <code>[https://github.com/glitch-soc/mastodon/tree/glitch-clean-sync glitch-clean-sync]</code> is a locked read-only branch by design to protect it from force-pushes or merges from other branches.
+*Create a new branch called custom-glitch(insert version number) from the <code>[https://github.com/NeuromatchAcademy/mastodon/tree/glitch-clean-sync glitch-clean-sync]</code> branch. For example, if we are upgrading to v4.2, the branch will be <code>[https://github.com/NeuromatchAcademy/mastodon/tree/custom-glitch4.2 custom-glitch4.2]</code>
+<syntaxhighlight lang="bash">git checkout origin/glitch-clean-sync
+git checkout -b custom-glitch4.2 </syntaxhighlight>
+* Merge <code>[https://github.com/NeuromatchAcademy/mastodon/tree/dev-stable dev-stable]</code> to the custom glitch branch
+<syntaxhighlight lang="bash">git merge origin/dev-stable </syntaxhighlight>
+* All the stable features stay on the <code>[https://github.com/NeuromatchAcademy/mastodon/tree/dev-stable dev-stable]</code> branch. To ensure that this branch stays clean, branch protections have been enabled.
+* Any features that have passed all the tests, have no errors (including linting errors), received approval from at least 1 reviewer, and are ready to be merged should be merged from the feature branch to the dev-stable branch in a separate process. See [[Mastodon/Hacking]] for important guidelines for contributing custom features to dev-stable.
+** Briefly: dev-stable branch should ideally have no merge conflicts if all the steps to keep the branch clean and stable have been followed.
+*** If there are merge conflicts, try and identify which features caused merge conflicts. Then request the feature developer to :
+**** merge upstream (glitch-soc:main) to conflicting feature branches
+**** create a PR to merge conflicting feature branches with dev-stable
+**** ensure that all checks pass, there are no linting errors, and review the code to ensure that it does what the feature is intended to do before merging to dev-stable
+==== History: Steps that were followed for merging glitch + 4.2RC + our instance's features ====
 all branches refer to those in [https://github.com/NeuromatchAcademy/mastodon NeuromatchAcademy/mastodon] unless specified otherwise.
-https://github.com/NeuromatchAcademy/mastodon/compare/glitch-soc-main...glitch-soc:mastodon:main
+* Sync the <code>[https://github.com/NeuromatchAcademy/mastodon/tree/glitch-soc-main glitch-soc-main]</code> branch to <code>glitch-soc/mastodon:main</code>: Open the branch page and click "sync fork"
+** If there are any local changes that make a conflict, they can be discarded - this branch is supposed to just be a local copy of the upstream <code>main</code> branch that ''doesn't'' reflect our local changes, we will handle that in the next step.
+[[File:Upgrading-sync-branch.png|thumb]]
+* [https://github.com/NeuromatchAcademy/mastodon/compare/merge-upstream...NeuromatchAcademy:mastodon:glitch-soc-main Create a pull request] from <code>glitch-soc-main</code> to <code>merge-upstream</code>
+** Resolve conflicts, ensuring that [[Mastodon/Hacking|changes we have made locally]] are preserved when possible.
+** If it is not possible to preserve a hack, eg. because the relevant feature has been deprecated, update the corresponding feature page and include that in the pull request description
+** Don't try and fix failing tests here - that would edit the <code>glitch-soc-main</code> branch, which will then be overwritten on the next fork sync. NOTE from Manisha --  The glitch-soc-main branch had commits from merge-upstream even after syncing the the branch. So I created a new branch called glitch-clean-sync from glitch-soc's main branch. Let's keep this branch clean and not merge anything to this branch. Only use the sync feature. I also tried to follow the rest of the process but it seemed a bit complicated/unclear. I've tried to simplify it in a separate section above for glitch+4.2+our instance's features
+* Merge <code>dev</code> into <code>merge-upstream</code> if necessary
+** It seems like this is the way [[glitch-soc]] does it, merge all from upstream masto into a branch from dev, then merge dev into the branch to ensure all glitch modifications go on top of the upstream? not exactly sure -jonny
+* Fix any failing tests in the <code>merge-upstream</code> branch
+** Ruby Tests, One and Two step db migrations must pass, but linting failing is fine and normal
+* If any changes were made after merging <code>dev</code>, [https://github.com/NeuromatchAcademy/mastodon/compare/dev...NeuromatchAcademy:mastodon:merge-upstream Create a pull request] from <code>merge-upstream</code> to <code>dev</code>. Otherwise just merge <code>merge-upstream</code> to <code>dev</code>
+** We do this additional step before merging to main because we might have collected some hacks or additional code in <code>dev</code> from some other branches before deploying (and generally we want to play the upstream over dev)
+* Merge <code>dev</code> to <code>main</code>
+=== Deploying ===
+* Make a [[Backups|backup]] on the linode dashboard
+* Check for any release-specific instructions and follow those! If you are doing an upgrade through several versions, make sure you follow those too!
+** Base masto: https://github.com/mastodon/mastodon/releases
+** Glitch docs (mostly just the same as base masto): https://glitch-soc.github.io/docs/
+Otherwise, in general...
+Switch to the <code>mastodon</code> user and change to the masto directory
+<syntaxhighlight lang="bash">
+sudo su mastodon
+cd ~/live
+</syntaxhighlight>
+Fetch changes
+<syntaxhighlight lang="bash">
+git pull
+</syntaxhighlight>
+Install dependencies
+<syntaxhighlight lang="bash">
+bundle install
+yarn install --frozen-lockfile
+</syntaxhighlight>
+Run pre-deployment database migrations
+<syntaxhighlight lang="bash">
+RAILS_ENV=production SKIP_POST_DEPLOYMENT_MIGRATIONS=true bundle exec rails db:migrate
+</syntaxhighlight>
+Precompile assets
+<syntaxhighlight lang="bash">
+RAILS_ENV=production bundle exec rails assets:precompile
+</syntaxhighlight>
+Restart services (you may have to switch back to your original user with <code>exit</code> because the mastodon user cannot sudo for obvious reasons)
+<syntaxhighlight lang="bash">
+sudo systemctl reload mastodon-web
+sudo systemctl restart mastodon*
+</syntaxhighlight>
+Run post-deployment database migrations
+<syntaxhighlight lang="bash">
+RAILS_ENV=production bundle exec rails db:migrate
+</syntaxhighlight>
+== Problems ==
+(and solutions)
+=== Precompilation Failed ===
+You might see something like this:
+<pre>
+mastodon@localhost:~/live$ RAILS_ENV=production bundle exec rails assets:precompile
+DEPRECATION WARNING: Support for `config.active_support.cache_format_version = 6.1` has been deprecated and will be removed in Rails 7.2.
+Check the Rails upgrade guide at https://guides.rubyonrails.org/upgrading_ruby_on_rails.html#new-activesupport-cache-serialization-format
+for more information on how to upgrade.
+ (called from <main> at /home/mastodon/live/config/environment.rb:7)
+Compiling...
+Compilation failed:
+`isModuleDeclaration` has been deprecated, please migrate to `isImportOrExportDeclaration`
+    at isModuleDeclaration (/home/mastodon/live/node_modules/babel-plugin-lodash/node_modules/@babel/types/lib/validators/generated/index.js:2740:35)
+    at PluginPass.Program (/home/mastodon/live/node_modules/babel-plugin-lodash/lib/index.js:102:44)
+</pre>
+and assume that the compilation fails because of the deprecation error that follows immediately after "compilation failed:" but it is not so.
+This is an OOM error, see: https://github.com/glitch-soc/mastodon/issues/2423
+To remedy:
+) Switch back to your user (from the <code>mastodon</code> user) that has root permissions and stop the [[ElasticSearch]] process. Return to the mastodon user.
+<syntaxhighlight lang="bash">
+# previously...
+# sudo su mastodon
+exit # assuming you're the mastodon user
+sudo systemctl stop elasticsearch
+# return to mastodon user
+sudo su mastodon
+</syntaxhighlight>
+) Since disrupting the asset precompilation may leave you in a state where webpack thinks it has successfully compiled, you need to clobber the previous assets
+<syntaxhighlight lang="bash">
+cd ~/live
+RAILS_ENV=production bundle exec rails assets:clobber
+</syntaxhighlight>
+) Proceed with precompilation...
+<syntaxhighlight lang="bash">
+RAILS_ENV=production bundle exec rails assets:precompile
+</syntaxhighlight>
+) Remember to restart elasticsearch afterwards!
+<syntaxhighlight lang="bash">
+exit
+sudo systemctl start elasticsearch
+</syntaxhighlight>
-* Ensure any changes in <code>main</code> have been merged into <code>glitch-soc-main</code>
-* [https://github.com/NeuromatchAcademy/mastodon/compare/glitch-soc-main...glitch-soc:mastodon:main Create a pull request] to merge changes from <code>glitch-soc/mastodon:main</code> to <code>NeuromatchAcademy/mastodon:glitch-soc-main</code>:
 == Discord ==
@@ Line 31: / Line 168: @@
 https://discord.com/channels/1049136631065628772/1049184335514828860/1109355378178789396
 |Link=https://discord.com/channels/1049136631065628772/1052452072806686751/1109356466130931765
+}}{{Message
+|Author=mannazsci
+|Avatar=https://cdn.discordapp.com/avatars/540104559079456770/12ce492294d6a3cd6f1ea00cbe8edb04.png?size=1024
+|Date Sent=23-10-01 07:44:40
+|Channel=techwg-ops
+|Text=[[Tech WG/Ops Diary#23-10-01]]  I am ready to deploy v4.2 but I encountered a few issues while following the process mentioned in [[Upgrade_Mastodon]]. The glitch-soc-main branch had commits from merge-upstream even after syncing the the branch. So I created a new branch called glitch-clean-sync from glitch-soc's main branch and added protections to this branch. This let's us keep this branch clean and not merge any custom features to this branch. We'd only use the sync feature. I also tried to follow the rest of the process but it seemed a bit complicated/unclear. I've tried to simplify it in a separate section above for glitch+4.2+our instance's features. I've also created a clean and stable dev-stable branch for custom features that we add. I've added branch protections here as well and PRs will require reviews on this branch.
+|Link=https://discord.com/channels/1049136631065628772/1052452072806686751/1157946171432632383
 }}