Background

Several projects in the Tails development and operations workflows make use of container images in GitLab CI jobs to test, build and deploy different kinds of resources, and many of these jobs can directly affect the code and documentation that is made available to end users. Because of that, we need proper security measures and workflows that can prevent malicious injection of data that could compromise the security of Tails users, while at the same time allowing developers and service admins to do their jobs.

Policy

Scope and affected users

This proposal applies to projects in the Tails GitLab instance whose repositories or CI/CD pipelines can directly affect the contents of the Tails OS image or website.

Affected users are members of such projects who have permissions to do at least one of the following:

  • push images to the project's container registry
  • push to protected branches of the project's repositories

These are project members that have at least Developer privileges.

Requirements

Must have

  • Sysadmin guidelines for configuring trusted GitLab runners
  • Developer guidelines for building, storing and using container images for projects that deal with sensitive credentials
  • Enough guarantees that only trusted runners have access to sensitive variables

Non-goals

  • Technically restrict the registration of new Runners.
  • Establish a general policy for runners in the Tails GitLab instance.

Definitions

This section provides definitions for what's considered trusted and sensitive in the context of this proposal.

Trusted software stack

The software stack of a GitLab Runner is considered trusted if all of the following apply:

  • all software installed and running on the hypervisor and the virtual machine (including operating systems, the GitLab Runner application and the container runtime) are either:
    • installed from Debian's official APT repositories, or
    • open source and installed using APT from a repository maintained by upstream which always has the latest security patches applied
  • upgrades are installed automatically
  • the need for manual system reboots and service restarts are periodically notified and acted upon

Trusted container images

A trusted container image is a container image that is:

  • either reproducible or based on a reproducible image,
  • stored in the Tails GitLab registry, and
  • peer-reviewed

The Tails sysadmin Team maintains a set of container images in the sysadmin-team/container-images project which are trusted and meant to either be used directly or serve as the basis for other trusted container images.

Trusted runners

A trusted GitLab runner is one that:

  • runs on systems which only Tails sysadmins have access to,
  • runs on top of trusted software stack, and
  • is restricted to only run trusted container images.

Sensitive credentials

Sensitive credentials are those that give write access to at least one of the following:

  • any project's repository
  • any project's container registry
  • the production website.

Examples of sensitive credentials are:

  • GitLab Access Tokens with Developer permissions or higher for tails/tails
  • SSH credentials with write permissions to the production website.

Sensitive jobs

A sensitive CI job is a GitLab CI job that can directly affect either the contents of the Tails OS image or of the website. In general, sensitive jobs are one of the following:

  • jobs that run for the protected branches of tails/tails (such jobs have access to sensitive credentials)
  • jobs that run with Developer permissions for any branch of tails/tails (such jobs can push container images to the project's container registry)
  • jobs in any other project that have access to any sensitive credential

Compromise of sensitive jobs could cause compromise of the contents of the Tails image and documentation, and because of that they should only execute on trusted runners.

Examples of sensitive jobs are:

  • build-website: gets an Access Tokens with Maintainer permissions in order to be able to push new commits back to protected branches of the tails/tails repository.
  • deploy-website: gets SSH credentials so it can push static data to our website mirrors.
  • Ensure MR from master into stable: gets an Access Tokens with Maintainer permissions in order to be able to create MRs to protected branches of tails/tails that will be merged automatically when the pipeline succeeds.

Guidelines for developers

Protect sensitive credentials:

  • Protect CI/CD variables that contain sensitive credentials
  • Only project Maintainers and trusted runners should have access to sensitive credentials

Use trusted container images for sensitive jobs:

  • Request the activation of your project's container registry to Sysadmins
  • Start with one of the base images
  • Use the Podman image to build and push images (see examples)
  • Build and push container images to your project's container registry using the CI/CD
  • Make sure the images you push to your project's registry are peer-reviewed

Use trusted runners for sensitive jobs:

Guidelines for sysadmins

  • Only install runners on VMs whose hosts we control
  • Install gitlab-runner from the upstream APT repository
  • Restrict runners to only use images from the Tails GitLab registry

References