Reproducible computational environments using containers

Key Points

Introducing containers
  • Almost all software depends on other software components to function, but these components have independent evolutionary paths.

  • Projects involving many software components can rapidly run into a combinatoric explosion in the number of software version configurations available, yet only a subset of possible configurations actually works as desired.

  • Containers collect software components together and can help avoid software dependency problems.

  • Virtualisation is an old technology that container technology makes more practical.

  • Docker is just one software platform that can create containers and the resources they use.

Introducing the Docker command line
  • A toolbar icon indicates that Docker is ready to use.

  • You will typically interact with Docker using the command line.

Creating containers
  • Containers are usually created using command line invocations.

  • The docker run command creates containers from images.

  • The docker image command can list and manage images that are (now) on your computer.

  • The docker container command can list and manage containers that have been created.

  • The -d -t options to docker run allow you to create a container running in the background.

  • The docker exec command allows you to execute commands in a running container, including getting interactive access.

Visiting the Docker Hub
  • The Docker Hub is an online repository of container images.

  • Many Docker Hub images are public, and may be officially endorsed.

  • Each Docker Hub page about an image provides structured information and subheadings

  • Most Docker Hub pages about images contain sections that provide examples of how to use those images.

Creating your own container images
  • Dockerfiles specify what is within Docker images.

  • The docker build command is used to build an image from a Dockerfile

  • You can share your Docker images through the Docker Hub so that others can create Docker containers from your images.

  • You can include files from your Docker host into your Docker images by using the COPY instruction in your Dockerfile.

  • Docker allows containers to read and write files from the Docker host.

  • You can export and import images from files.

Example: Containers used in generating this lesson
  • By using the container we avoid all the difficult installation issues of installing Jekyll.

Using containers to improve reproducibility
  • Container images allow us to encapsulate the computation (and data) we have used in our research.

  • Using a service such as Docker Hub allows us to easily share computational work we have done.

  • Using container images along with a DOI service such as Zenodo allows us to capture our work and enables reproducibility.