Gitlab Upgrade

2-minute read

Below is the process I’ve ironed out during my contracts for various clients, where I was supporting self-hosted version of GitLab.

Pre-upgrade

GitLab create new releases on the 22th of each month.
Sometimes a few days later they issue patch release, if they find any bugs.
So it is recommended to wait a couple of weeks, and for example do Upgrades first Friday of every month.

T - 1 week:

  • review current versions: https://<GITLAB_SERVER_NAME>/admin
  • find new release at: https://about.gitlab.com/releases/
  • create a new Issue
  • reserve a time slot
  • review Release Notes for new release
  • review Upgrade recommendations
  • pay special attention if multi-step upgrade is needed, see all available versions, example for Debian\Ubuntu: apt-get update && apt list -a gitlab-ee
  • review top-level upgrade process
  • review versions compatibility between PostgreSQL and Omnibus GitLab, and plan PostgreSQL upgrade if needed and not part of GitLab upgrade

T - 1 day:

  • announce service down time in Slack \ email
  • announce as a Broadcast Message at https://<GITLAB_SERVER_NAME>/admin/broadcast_messages

During Upgrade

  • check that instance snapshot is created last night, for example when hosted in GCP see snapshots
  • check that there are no remaining migrations
  • create backup: sudo gitlab-backup create SKIP=repositories,artifacts,lfs,registry, see doc
  • backup /etc/gitlab/{gitlab.rb,gitlab-secrets.json} files separately somewhere, as they contain sensitive data and are not included in the backup
  • make sure that sudo gitlab-ctl reconfigure can run successfully
  • if PostgreSQL needs to be upgraded separately, follow doc, also per this you might want to specify version as well, e.g.: sudo gitlab-ctl pg-upgrade -V 11
  • upgrade GitLab according to GitLab update documentation:
    • sudo apt-get update
    • sudo apt-get install gitlab-ee - to upgrade to the latest version
    • sudo apt-get upgrade gitlab-ee=12.0.12-ee.0 - specify version in case of multi-step upgrade
  • [optional] upgrade other packages in the system: sudo apt-get upgrade
  • restart server and check if GitLab comes back up

Post-Upgrade

  • do some testing (push/pull, major repositories, runners/pipelines/jobs, boards)
  • announce in Slack that Upgrade was successful
  • add another Broadcast Message at https://<GITLAB_SERVER_NAME>/admin/broadcast_messages for few hours that upgrade was successful