Build VMs

A job with requires: [build] runs on a managed build VM: an amd64 Ubuntu 24.04 machine with 16 cores, 32 GiB of memory, 200 GiB of disk and KVM, started for your run when it needs one. Your steps run as a normal user, runner, with passwordless sudo, so they can install what they need with apt or Nix.

jobs:
  build:
    requires: [build]
    steps:
      - name: deps
        run: |
          sudo apt-get -o DPkg::Lock::Timeout=120 update
          sudo apt-get -o DPkg::Lock::Timeout=120 install -y --no-install-recommends protobuf-compiler
      - name: build
        run: make release

On a fresh VM, Ubuntu’s own package updates may hold the apt lock for a while; the lock timeout waits for them instead of failing.

Build VMs come with the Team and Business plans. How many your organization may have at once, and how their time is counted, is in Agents.

One run per VM

A build VM belongs to the run that started it. It takes that run’s jobs one after another, including the jobs of child workflows the run starts, and never a job of another run, another project or a fork. When the run has no more work for it, it exits after about two idle minutes, or as soon as it next asks for work once the run has ended. Jobs of a run that are ready at the same time get a VM each. A run started with spool spawn or call is a run of its own and gets its own VMs.

So the first build job of a run can wait a minute or two for a VM to boot, and starts with an empty disk. Later jobs of the same run often land on a VM that is already up and has whatever the earlier jobs left in /home/runner, /tmp and /nix. Don’t rely on that: a job may get a fresh VM, so each job still checks out its own code.

What’s on it

• The Ubuntu 24.04 cloud image: bash, coreutils, git, curl, openssh, python3 and apt. Anything else comes from apt or Nix.
• runner, with HOME=/home/runner, passwordless sudo, and access to /dev/kvm, so builds that boot a VM run accelerated.
• What rootless podman needs: subordinate ids, newuidmap, a container policy and cgroupfs. Podman itself isn’t installed; sudo apt-get install -y podman or nix shell nixpkgs#podman gets it. sudo podman runs rootful containers.
• No Nix. Install it in the job, below.
• The public internet only. Private addresses, link-local and cloud metadata addresses are blocked, and so are the outbound mail ports.

A step runs at most as long as the VM lives: an hour by default, longer when a job declares a longer timeout:, up to the ceiling in Agents.

Installing Nix

Each step is a fresh shell, so run the installer in a step of its own at the start of every job that needs Nix. /nix/var/nix/profiles/default/bin is already on the PATH of every step, so the steps after it find nix without sourcing anything.

      - name: nix
        run: |
          set -euo pipefail
          if [ ! -x /nix/var/nix/profiles/default/bin/nix ]; then
            curl -fsSL https://install.determinate.systems/nix |
              sudo sh -s -- install --determinate --no-confirm
          fi
      - name: build
        run: nix build .#default

The check makes the step cheap on a VM that already has Nix, which a later job of the same run often does. On a fresh VM the install takes about half a minute.

Using your own binary caches

runner isn’t a trusted Nix user, so the Nix daemon ignores substituters the job asks for itself through NIX_CONFIG, --option or cachix use. Put them in the daemon’s own config with sudo and restart it:

      - name: caches
        run: |
          set -euo pipefail
          printf '%s\n' \
            'extra-substituters = https://nix-community.cachix.org' \
            'extra-trusted-public-keys = nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=' \
            | sudo tee -a /etc/nix/nix.custom.conf >/dev/null
          sudo systemctl restart nix-daemon

That adds the nix-community cache; put your own cache’s URL and key in its place. The same step can add extra-trusted-users = runner instead, if you would rather let the job choose its own caches.

What sudo means for secrets

With sudo a step can read anything on the VM, including the credential the VM uses to take jobs. That credential only reaches jobs of the same run, and stops working when the run ends. So a step that misbehaves, or a dependency that does, can take over another job of its own run and see the secrets and job credentials that job is given, but nothing of any other run.

If some jobs of a run hold credentials the rest of the run must not reach, such as a deploy key used only by a deploy job, ask support to give your organization one environment per job. Every job then gets a VM of its own that can take no other job.