GOV.UK is a complex ecosystem of apps and services that jointly provide a single experience for users, from the backend publishing tools to the public-facing parts of the site.
Having multiple apps that interact with each other is a challenge for development and testing, whether it’s fixing a bug or developing a new feature.
Building on local machines
When a developer makes a change to an app, they will need the app and its dependencies to work, or ‘build’, on their local machine. This could be to run the tests, or to manually investigate the behaviour of the app in a web browser.
Getting an app to build on a particular machine can be quite a challenge, due to subtle variations, such as the versions of packages, or the shell a developer is using. Even the operating system (OS) can vary. The challenge grows if we then consider multiple GOV.UK apps that need to interact with each other.
Until recently, we used an Ubuntu virtual machine (VM) to provide a consistent environment for developers to build all the apps on GOV.UK. A new starter would create their own VM from a clean Ubuntu image. Next, they would run GOV.UK Puppet against their VM to set up all the apps - we also use Puppet to provision the live apps of GOV.UK. Once all the apps were set up, the final step would be to replicate GOV.UK production data into local databases on the VM, in order to simulate a realistic experience.
Using a VM had its drawbacks. Apart from taking up loads of disk space, the setup would often fail, which was particularly frustrating for new starters. The failures were partly due to the sheer number of steps to get from a clean Ubuntu VM to a ‘GOV.UK Dev VM’.
The likelihood of one of the steps failing was high, and the level of complexity would make it hard to debug the issue. On a more fundamental level, while GOV.UK Puppet was designed to accommodate production and development use cases, the reality was that developers usually made changes with only production in mind - only later would a change be applied in development, when other developers tried to update their VM.
Moving to Docker containers
To improve the developer experience on GOV.UK, we’re moving away from a ‘Dev VM’ and are starting to use Docker containers for local development. A Docker container reuses the host OS to run a program with its own isolated resources, including a separate, layered filesystem. This reuse makes containers lightweight, compared to a whole Ubuntu system with all the trimmings. Running a Docker container looks like this:
# run bash in a debian-like environment, with Ruby 2.6.5 installed
docker run -it ruby:2.6.5 bash
The initial state of a Docker container is determined by the ‘image’ it’s based on, such as ‘ruby:2.6.5’.
We use our own custom image to provide a generic environment for building any GOV.UK app. Although we could make a specific, fully functional image for every app, we prefer to use various Docker techniques to handle specific needs at runtime. For example, instead of baking ever-changing Ruby gems into an image, we use a shared ‘volume’ and install them at the point of use, like you would on a local machine.
A more stable and consistent experience
Docker containers provide a stable and consistent environment to build a single GOV.UK app. In reality, we often need to work with multiple apps that interact with each other, such as a publishing app and the Publishing API. With the ‘Dev VM’, all apps were built on the local host, using Bowler to manage the processes. With Docker, we use Docker Compose to orchestrate the containers for each app. Compose allows us to go further and vary the set of dependencies for each app to conserve resources. We call these variations ‘stacks’.
Moving to Docker has been a learning curve for many of the developers on GOV.UK. In order to cope with this, we created our own Docker training that’s specific to how we use it. While our new development environment is quite different to the old production-like VM, overall we’ve found using Docker is simpler and less error-prone. We’ve officially deprecated the VM approach, and work is ongoing to completely migrate all GOV.UK apps.
You can read more about how GOV.UK uses Docker in our GitHub repo.
3 comments
Comment by Dilwoar Hussain posted on
Amazing read, thanks for posting Ben!
Comment by Servile Civet posted on
really? why not just use a CMS?
Comment by Alex Newton posted on
Thanks Ben, useful to have the background on why we moved to Docker