Updating from 22.01 to 23.0 with Ansible

Galaxy introduced a number of changes in 22.05 and 23.0 that are extremely important to be aware of during the upgrade process. Namely a new database migration system, and a new required running environment (gunicorn instead of uwsgi).

The scripts to migrate to the new database migration system are only compatible with release 22.05, and then were subsequently removed, so it is mandatory to upgrade to 22.05 if you want to go further.

Here is the recommended update procedure with ansible:

1. Update to 22.01 normally
2. Change the release to 22.05, and run the upgrade
   1. Galaxy will probably not start correctly here, ignore it (even if the build fail, this if fine, just ignore).

   2. Run the database migration manually (with the galaxy user with the venv activated)

      GALAXY_CONFIG_FILE=/srv/galaxy/config/galaxy.yml sh /srv/galaxy/server/manage_db.sh -c /srv/galaxy/config/galaxy.yml upgrade
      

3. Update your system’s ansible, you probably need something with a major version of at least 2.

4. Set the release to 23.0 and make other required changes. There are a lot of useful changes, but the easiest procedure is probably something like:

   1. git clone https://github.com/hexylena/git-gat/
   2. cd git-gat
   3. git checkout c2e7bf6d3584fbf3281fb57d8024a9189f957e0e (this corresponds to the version of the repo after the 23.0 integration without too much customization and after potential bug fixes)

   4. Diff and sync (e.g. vimdiff group_vars/galaxyservers.yml git-gat/group_vars/galaxyservers.yml) for the main configuration files:

      • group_vars/all.yml
      • group_vars/dbservers.yml
      • galaxy.yml
      • requirements.yml (and don’t forget to install the new role versions)
      • hosts
      • templates/nginx/galaxy.j2

   But the main change is the swap from uwsgi to gravity+gunicorn

   -  uwsgi:
   -    socket: 127.0.0.1:8080
   -    buffer-size: 16384
   -    processes: 1
   -    threads: 4
   -    offload-threads: 2
   -    static-map:
   -      - /static=/static
   -      - /favicon.ico=/static/favicon.ico
   -    static-safe: client/galaxy/images
   -    master: true
   -    virtualenv: ""
   -    pythonpath: "/lib"
   -    module: galaxy.webapps.galaxy.buildapp:uwsgi_app()
   -    thunder-lock: true
   -    die-on-term: true
   -    hook-master-start:
   -      - unix_signal:2 gracefully_kill_them_all
   -      - unix_signal:15 gracefully_kill_them_all
   -    py-call-osafterfork: true
   -    enable-threads: true
   -    mule:
   -      - lib/galaxy/main.py
   -      - lib/galaxy/main.py
   -    farm: job-handlers:1,2
   +  gravity:
   +    process_manager: systemd
   +    galaxy_root: "/server"
   +    galaxy_user: ""
   +    virtualenv: ""
   +    gunicorn:
   +      # listening options
   +      bind: "unix:/gunicorn.sock"
   +      # performance options
   +      workers: 2
   +      # Other options that will be passed to gunicorn
   +      # This permits setting of 'secure' headers like REMOTE_USER (and friends)
   +      # https://docs.gunicorn.org/en/stable/settings.html#forwarded-allow-ips
   +      extra_args: '--forwarded-allow-ips="*"'
   +      # This lets Gunicorn start Galaxy completely before forking which is faster.
   +      # https://docs.gunicorn.org/en/stable/settings.html#preload-app
   +      preload: true
   +    celery:
   +      concurrency: 2
   +      loglevel: DEBUG
   +    handlers:
   +      handler:
   +        processes: 2
   +        pools:
   +          - job-handlers
   +          - workflow-schedulers
   

   Some other important changes include:

   • uchida.miniconda is replaced with galaxyproject.conda
   • usegalaxy_eu.systemd is no longer needed
   • galaxy_user_name is defined in all.yml in the latest git-gat
   • the galaxy_job_config needs to have a database handling specified - assign set to db-skip-locked
   • git-gat also separates out the DB serving into a dbservers.yml host group
5. Backup your venv, mv /srv/galaxy/venv/ /srv/galaxy/venv-old/, as your NodeJS is probably out of date and Galaxy doesn’t handle that gracefully
6. Do any local customs for luck (knocking on wood, etc.)
7. Run the playbook
8. Things might go wrong with systemd units
   • try running galaxyctl -c /srv/galaxy/config/galaxy.yml update as root
   • you may also need to rm /etc/systemd/system/galaxy.service which is then no longer needed
   • you’ll have a galaxy.target and you can instead systemctl daemon-reload and systemctl start galaxy.target
9. You may need to restart galaxy manually with sudo galaxyctl restart

Persistent URL

Resource purlPURL: https://gxy.io/GTN:F00357

Still have questions?

Gitter Chat Support

Galaxy Help Forum

Information for GTN Tutorial Authors