Background Info

Over the last year I’ve been working more with ansible, I’ve even used IronicBadger’s Infra playbook as a base to my own personal infrastructure playbook.

However one problem that has come up recently was with the release of docker-compose 2.0 because it was rewritten in go.

Don’t get me wrong, so far the go version of docker-compose has been fantastic, it appears to be faster to run and I love the changes they have done to the terminal output.

However due to the change from python to go, docker-compose no longer works in ansible

The Problem as it stands

Currently because the Ansible module imports the docker-compose python code directly it is incompatible with docker-compose releases 2.0.0 or later.

There is an issue raised on github to solve this, however the problem is that someone needs to write a python module that will interact with docker-compose via the cli.

Understandably no-one has yet to tackle this since it would be quite the undertaking to get this to work and be 100% compatible.

Unfortunately for me, a good 80-90% of my ansible roles I’ve written for my private infrastructure rely on docker compose.

So I was faced with two ways forward:

1. Rewriting everything to be under the docker network and docker run modules, or
2. Find a way to get docker-compose’s python module reinstalled at the same time as the latest docker compose release.

Option 1 I wasn’t looking forward to since I would be basically doing what docker-compose does in one ansible play step myself over multiple ansible steps. But Option 2 could lead to broken python installs if I am not careful.

However recently I was reminded of a fantastic feature/module of python, virtual environments

The workaround: virtualenv

Virtual Python Environments, the perfect way of installing python modules via pip in a safe way.

For the uninitiated, virtual environments, or venvs are a python substructure that allows you to manage your own user-managed python install independent of the system’s installed python modules. Very useful if you want to use pip for installing python modules, but also don’t want to cross-contaminate your system with pip managed modules. Many package managers will complain about finding files not owned by a package when attempting to install said module system wide.

The remaining question is how easy can Ansible be setup to use it?

After some searching, it is indeed quite easy, thanks to the work Matt Schlobohm showed in this stack overflow question

And after some testing with the ansible_python_interpreter variable, I got it working with minor tweaking.

Here are the relevant files I have in the role that sets up the virtual environment on my Arch/Manjaro boxes

./roles/python_venv_setup/tasks/main.yml

- name: Ensure dependencies are installed
  community.general.pacman:
    name:
      - docker
      - docker-compose
      - python-docker
      - python-pip
      - python-pyaml
      - python-setuptools
      - python-yaml
      - python-virtualenv
    state: present
    update_cache: true
- name: Setup Python virtualenv
  ansible.builtin.pip:
    virtualenv: '/var/local/docker-appdata/.venv/'
    state: present
    name:
      - docker
      - docker-compose
      - PyYAML
      - pyaml
- name: Copy pyvenv file
  ansible.builtin.copy:
    src: pyvenv
    dest: '/var/local/docker-appdata/.venv/bin/pyvenv'
    owner: root
    group: root
    mode: '775'

./roles/python_venv_setup/files/pyvenv

#!/bin/bash
source "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/activate"
python $@

Main Playbook YAML file

- hosts: all
  vars:
    ansible_python_interpreter: /usr/bin/python3
  roles:
    - python_venv_setup
- hosts: all
  roles:
    - docker_compose_project_1
    - docker_compose_project_2
    - docker_compose_project_3

Finally in the inventory file, you set ansible_python_interpreter to the location of the pyvenv file,
In the example above, that would be /var/local/docker-appdata/.venv/bin/pyvenv

What all of this does is first run through the python dependencies and ensures everything is there for it to work, then through the ansible builtin module pip it sets up the virtualenv and installs docker-compose inside of it.

Finally the pyvenv file auto sources & activates the virtualenv’s python and passes all python calls to the virtualenv’s version of python.

With this it has all of the tools needed to get docker-compose working again while keeping the up to date copy of docker-compose installed.

One note for Python

Any remote python modules that are required now need to be installed into the virtualenv, in my case I have both PyYAML and pyaml because some of the modules I use need both.