Deploying agents

An agent is a long-running process that dials out to the server, claims activities matching its capabilities, runs them, and reports back. It never accepts inbound connections, so it runs behind a firewall or inside a private network as long as it can reach the server over HTTPS.

This guide covers minting a token, running an agent by hand, installing it as a systemd service, running one in a container, self-update, and what happens when an agent dies.

Mint a token

An org owner issues an agent token, naming the agent id and the capabilities the token is allowed to advertise:

spool agents issue build-runner-1 \
  --name "build runner (hel1)" \
  --project acme/web \
  --expires-in 90d \
  --cap shell --cap polyglot --cap build

The command prints the token plaintext once. Store it now; it isn’t recoverable later. Agent tokens are prefixed csat_.

One token for several replicas

Running the same agent on several machines, or scaling a Kubernetes Deployment past one pod, does not need a token each. Mint one for a pool:

spool agents issue --pool build-runner \
  --name "build runners" \
  --project acme/web \
  --cap shell --cap polyglot --cap build

That token authenticates as any agent id beginning build-runner-, so each process still has an id of its own and the fleet can tell them apart. The pool name itself is not a usable id: it names the group.

Give each process a distinct id under the pool and it needs no other configuration:

spool agent run https://spool.example.com build-runner-$(hostname) --cap shell

On Kubernetes the pod name already has the right shape, since it is the Deployment’s name plus a suffix. Name the pool after the Deployment and pass the pod name straight through:

env:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
args: ["agent", "run", "https://spool.example.com", "$(POD_NAME)", "--cap", "shell"]

spool agents ls then shows the pool as one line with a replica count rather than N unrelated lines.

Capabilities, scope and expiry mean exactly what they mean for a single-agent token. A pool widens which id may authenticate and nothing else. What it costs is revocation granularity: one credential backs every replica, so revoking it stops all of them, and --expires-in is the only thing that bounds a leak.

Pool names collide with agent ids, in both directions, because both key the same registry. Minting the pool build-runner is refused while an agent called build-runner-1 exists, and vice versa. Forget the old one first.

An agent token has two independent bounds: capability and scope.

The token’s capabilities are a ceiling. An agent can advertise a subset of them at runtime but can’t widen past them, so a leaked token can’t claim work the operator didn’t grant.

Scope decides whose work the agent may claim, and therefore whose secrets it receives:

• --project <org/project> binds the token to that one project. A bare --project <name> with no org is rejected as ambiguous.
• --org <slug> binds it to every project in that org: the runner claims any of the org’s runs and receives each run’s project secrets.
• Omitting both mints a deployment-wide token. That agent can claim work and read secrets in every org and project on the instance.

Who may mint follows the scope. Either scoped form needs the owner role in the org named. Minting unscoped is a deployment operator’s call, since the credential reaches every tenant on the instance. An operator who needs a scoped token for an org they hold no membership in passes --admin, which mints through the instance route and skips the org role check.

Operators are the addresses in HYPERSPOOL_INSTANCE_ADMINS, a comma-separated list on the server.

While it is empty, operator actions fall back to accepting any org’s owner. That is the same person on a deployment with one org, and is not once there are two, so the fallback applies only while a single org exists. Create a second and every operator action is refused until the list is set and the server restarted, including for whoever created the first org.

Set it before you onboard anyone. Creating an org for a customer is what turns “the owner of the only org” into “any tenant”, and that is your own onboarding step rather than anything unusual: the customer’s owner would otherwise be able to mint a credential that reads every other customer’s secrets.

The list is read once at startup, so changing it needs a restart.

The unscoped form is the shared-fleet choice, worth making on purpose rather than by typing less. --expires-in <dur> puts a clock on the credential either way. See teams and projects for how org and project scope relate.

The agent id is claimed instance-wide, so a scoped mint naming an id another org already holds comes back as a conflict - pick another rather than reusing one. Re-minting an id your own scope holds is how you rotate its token.

Revoking a token (spool agents revoke <token-id>) is a deployment operator’s action today, so ask one to pull a leaked credential. --expires-in keeps that window short in the meantime.

Run an agent

export HYPERSPOOL_AGENT_TOKEN=csat_...
spool agent run https://spool.example.com build-runner-1 \
  --cap shell --cap polyglot --cap build

The two positional arguments are the server URL and the agent id. The token comes from $HYPERSPOOL_AGENT_TOKEN (the agent scrubs it from its own environment right after reading it, so jobs it runs never inherit the server credential).

Capabilities resolve in this order: repeated --cap flags win; otherwise $HYPERSPOOL_AGENT_CAPABILITIES (comma-separated); otherwise the default shell,polyglot. Whatever you pick is still capped by the token.

The languages a job uses (bash, python, node, go, …) must be present on the agent host’s PATH. The agent binary itself is a single static executable with no runtime dependencies.

The job environment

Jobs run in a scrubbed environment. The agent clears its own env and re-adds only a small allowlist (HOME, USER, LANG, TERM, the TZ/locale vars, and the TLS cert paths), then layers on PATH, the pipeline’s env: block, and the run’s SPOOL_* vars. Everything else the agent process carries stays out of the job, so an operator var or an injected credential can’t leak into workflow code. This is deliberately fail-closed: a var you forgot about is dropped, not forwarded.

Two ways to widen it when a job needs more:

• Per workflow: set it in the pipeline’s env: block. That flows through to the job as-is.
• Per agent: set SPOOL_ENV_PASSTHROUGH on the agent to a comma-separated list of names or globs - MYAPP_*,REGION - and any matching var in the agent’s environment is forwarded to every job it runs. Use this when a runner image or host sets env you want all its jobs to see. The bearer token is scrubbed from the agent’s env at startup, so it is never forwardable, even by *.

Either way the value reaches the job’s process, read as $NAME. It is not visible to a workflow’s expressions: a job’s if: is evaluated before the job reaches an agent, and two agents can hold different values for the same name, so a gate reading one could not decide the same way twice. A constant a gate needs belongs in the pipeline’s vars: block instead.

The agent also forwards PYTHONPATH and NODE_PATH by default (they’re search paths, not secrets), so a runner image that vendors the language SDKs at those paths makes them importable without any extra config.

Install as a systemd service

The install script fetches the binary and, with --service, registers a systemd unit:

curl -fsSL https://hyperspool.com/install.sh | sh -s -- \
  --service \
  --server https://api.hyperspool.dev \
  --id build-runner-1 \
  --token csat_... \
  --caps shell,polyglot,build

Without --service it installs the spool binary and prints the manual spool agent run command. --service needs systemd (Linux); on other platforms it prints the manual command instead.

The unit runs as a throwaway DynamicUser by default. A build runner that installs its own toolchain in-flow (nix, devenv) needs a persistent, privileged home, so pass --user root (or a real account) to pin a fixed user.

Run one in a container

There is no published agent image. The agent is a single static binary, so the image is yours to build and the Dockerfile is short:

FROM docker.io/library/debian:stable-slim
RUN apt-get update \
 && apt-get install -y --no-install-recommends ca-certificates bash python3 \
 && rm -rf /var/lib/apt/lists/*
ADD https://get.hyperspool.dev/linux/spool-amd64 /usr/local/bin/spool
RUN chmod +x /usr/local/bin/spool
ENTRYPOINT ["spool", "agent", "run"]

docker run -d --restart unless-stopped \
  -e HYPERSPOOL_AGENT_TOKEN=csat_... \
  my-registry/cryo-agent:2026-09-12 \
  https://api.hyperspool.dev build-runner-1 --cap shell --cap polyglot

The interpreters go in the image. A job’s interpreter (bash, python3, node, go, …) is spawned by the agent in its own filesystem, so a capability the image cannot run is a claim the agent will fail on when a job arrives. --cap has to match what you installed, and it is coarser than the toolchain: the example above advertises polyglot on the strength of python3 alone, so a python script runs and a node one fails on that agent even though the capability matched. Install the interpreters your flows actually name, or keep the image’s caps narrow enough that the server sends it only work it can do.

Keep the token out of the image. It is a server credential, and an image layer is not a secret; pass it at run time (-e, or your orchestrator’s secret mechanism). Once read it is scrubbed from the agent’s own environment, so the jobs it runs never inherit it.

Two settings from the host install do not belong here. Don’t pass --service: it writes a systemd unit, and the container runtime is already the supervisor. And leave HYPERSPOOL_AGENT_UPDATE_URL unset: self-update works by exiting (see below), and a restart re-runs the same image, so the agent would drain and come back on the byte-identical binary. Roll the image tag instead.

Self-update

The service unit fetches the published binary on every start and verifies it against a .sha256 manifest before launching.

A running agent polls that manifest every 5 minutes only when HYPERSPOOL_AGENT_UPDATE_URL points at it. When a newer build ships, the agent drains (finishes the in-flight job) and exits; whatever supervises it restarts it, the pre-start fetch pulls the new binary, and the agent comes back on the new build. An unreachable download host or a checksum mismatch keeps the current binary, so a restart always succeeds.

install.sh --service sets HYPERSPOOL_AGENT_UPDATE_URL for you. An agent you run by hand, or under your own supervisor (nomad, docker, a hand-written unit), has to set it as well as be supervised by something that restarts and re-fetches on exit: exiting is the update, so with nothing to restart it the drain just stops the agent.

Manifests are signed (minisign) and both consumers check the signature against a public key compiled into spool before acting: the poll won’t drain and the pre-start fetch won’t swap the binary on a missing or invalid signature, so a compromised download host can’t push code onto the fleet. The pre-start check runs through the already-installed binary (spool agent verify-manifest <manifest> <sig>), which also rejects a manifest published for a different os/arch.

Draining and agent death

Ask an agent to drain and exit cleanly:

spool agents shutdown build-runner-1

spool agents ls shows the registry: each agent’s id, status, the scope whose work it may claim, capabilities, last build id, and last-seen time. An agent you asked to drain reads draining until it goes, and a pool reads as one line carrying how many replicas are running.

A build id marked outdated is one that differs from the agent seen most recently. That is a comparison within your fleet, not against a published release, so during a rollout it tells you which agents have not picked the new binary up yet, and once they agree nothing is marked.

The registry holds two different things, and they disappear for different reasons. A token is something you created, so it stays until you revoke or forget it. A running process is something the server observed, so its entry goes when the process does: an agent shutting down cleanly says so and its entry is dropped at once, and one that dies without saying anything is dropped after a day of silence. HYPERSPOOL_AGENT_REGISTRY_TTL sets that window (24h by default, off to keep entries until you remove them by hand; anything under five minutes is refused, since it would expire agents that are still running).

So an agent with no last-seen time is one nothing is running under right now. That covers a token you minted and never started, and equally one whose agent died long enough ago to have been dropped. The token being listed at all is what tells you the credential still exists.

A running agent reports at least every 25 seconds, so being idle never costs it its entry. Only a process that is gone goes quiet.

When an agent dies without draining, its in-flight activity holds a 30-second lease. Once the lease expires, the activity returns to the queue and another capable agent claims it; the activity re-runs from the start. A durable spool step cache makes that replay fast because completed steps return their recorded output instead of re-running.

An agent started with spool agent run also enforces a per-activity timeout of one hour. A job that wedges past the ceiling is abandoned so it can’t hold the agent’s claim slot forever. The in-process agents that back spool run and spool dev don’t apply this timeout - they’re interactive, and Ctrl-C is right there.