YAML reference
Hyperspool reads two document formats, and both can be written in YAML:
- A pipeline is a file with
jobs:at its top level. Jobs with nothing ordering them run in parallel, and each job is one durable boundary. There is no version marker to write; the run is recorded under the workflow typehyperspool/pipeline/v1. Most of this page is about pipelines. - A graph starts
version: hyperspool-graph/v1and lists nodes and edges. The canvas stores one, and you can write one by hand in JSON or YAML. See Graph documents.
A pipeline compiles to a graph when it runs, and spool workflow render <file> prints that
graph; there is no conversion the other way. Pipelines and graphs
covers which to use.
spool workflow check <file> validates either format offline, with nearly every check a submit
runs (a few admission values, such as a throttle with nothing to count under, are only refused
at submit). A graph can also be tried without side effects: Try it
on the canvas runs a dry run that stubs every effect.
Keys at a glance
Section titled “Keys at a glance”Top level of a pipeline:
| Key | Type | Default | Section |
|---|---|---|---|
name |
string | none | Top level |
tags |
list of strings | none | Top level |
env |
map of strings | none | env: and vars: |
vars |
map of JSON values | none | env: and vars: |
credentials |
map, ENV_VAR: credential |
none | Credentials |
input |
any JSON | {"type": "manual"} |
Document-level input: |
if |
CEL string | always run | Top-level if: |
paths, paths_ignore |
list of patterns | none | paths: |
concurrency |
{group, policy, queue_max} |
none | Run admission |
throttle |
{limit, period, burst, key, policy, queue_max} |
none | Run admission |
cancel_on |
{events: [...], accept_untrusted} |
none | Run admission |
outputs |
map of values with ${{ }} holes |
none | Document-level outputs: |
jobs |
map of jobs | required | Job |
A job declares exactly one kind (steps:, approval:, wait:, wait_for_event:,
workflow:, script:) plus any of:
| Key | Applies to | Section |
|---|---|---|
depends_on |
every kind | Dependencies |
if |
every kind | Templating |
continue_on_error |
every kind | continue_on_error: |
outputs |
every kind | Outputs |
concurrency |
every kind | Job-level concurrency: |
requires, agent |
steps:, script: |
Where a job runs |
timeout |
steps:, script: |
timeout: |
retry |
steps:, script: |
retry: |
env, credentials |
steps:, script: |
env: and vars:, Credentials |
optional_when_untrusted |
steps:, script: |
Credentials |
matrix |
steps:, script: |
matrix: |
The activity-only keys are refused on any other kind, since nothing would read them. An
approval:, wait: signal: or wait_for_event: job takes its deadline inside its own block
(approval: { timeout: 1h }).
Top level
Section titled “Top level”name: releasetags: [ci, release] # on every run; merged with `spool submit --tag`env: REGION: us-east # $REGION in shell and script jobsvars: MIN_COVERAGE: 0.9 # ${{ vars.MIN_COVERAGE }} in expressionsinput: ref: refs/heads/main # used when a run is started with no inputoutputs: sha: ${{ steps.build.output.sha }}jobs: build: outputs: sha: sha steps: - name: build run: spool output set sha="$(git rev-parse HEAD)"name: is what groups runs into one workflow in the console. The order of jobs: is the
tiebreaker when several jobs are ready at once.
Document-level input:
Section titled “Document-level input:”input: pins a default input for a run started by hand. A trigger’s event, or
spool submit --input <json>, replaces it. Without either, a hand-run’s input is
{"type": "manual"}. spool workflow render --summary says input: pinned by the document
when one is set.
Run admission
Section titled “Run admission”Admission decides whether a run starts now, waits, or is turned away, before any job runs.
Pipelines and graphs take the same blocks at their top level: tags:, concurrency:,
throttle: and cancel_on:. They bind however the run was submitted: spool submit, a
handler, a schedule tick, a rerun, a spool spawn, or a push to a connected repo. A schedule
tick never queues: when its group is busy or its throttle is spent the tick is skipped, while a
manual “run now” of the schedule waits its turn.
One path skips admission: a child started by a workflow: job or a graph workflow node goes
straight to the engine, so put the group on the parent. The graph example
below carries a concurrency: block.
A script or a single node has no document, so it takes the request-level equivalents. A request’s declaration beats the document’s:
spool submit probe.sh --concurrency-group probes --concurrency-policy skipspool submit reindex.sh --throttle 20/1m --throttle-key reindexA script has no name: to count a budget under, so --throttle without --throttle-key is
refused.
Top-level concurrency: (one running run per group)
Section titled “Top-level concurrency: (one running run per group)”At most one run in a group runs at a time. policy: decides what happens when a run arrives
while the group is busy:
| Policy | What happens |
|---|---|
queue (default) |
The new run waits in a FIFO queue and starts when its turn comes. |
cancel-running |
The new run cancels the running one (superseded by <run-id>) and starts now. |
cancel-queued |
Waiting runs are cancelled; the new run takes the one slot behind the running one. |
skip |
No run is created. spool submit prints no id and exits 0. |
skip suits work where a second run is noise: a reconcile, a cache warm. A spool spawn of a
document that skips fails with “the spawned pipeline was refused”, since the script expected a
child id; don’t wrap that spawn in a retry.
queue_max: caps how many runs may wait (default 100). A submit past it is refused.
group: takes ${{ … }}, rendered against input and vars when the run is created, which
gives one group per branch:
concurrency: group: 'ci-${{ has(input.ref) ? input.ref : "manual" }}' policy: cancel-runningGuard optional fields with has(): a hand-run carries {"type": "manual"} with no ref, and
a missing field is an error. A group that renders empty is refused.
A waiting run shows as queued with its position and carries the concurrency-group:<group>
tag, so the runs list filters to one group. Cancel it to drop it from the queue. A signal sent
to a queued run is held and delivered when it starts.
Top-level throttle: (how many runs start per period)
Section titled “Top-level throttle: (how many runs start per period)”concurrency: bounds how many run at once. throttle: bounds how many start over time,
across the project.
name: reindex # the budget's name when `key:` is absentthrottle: limit: 20 period: 1m burst: 5 # default 1 key: '${{ input.tenant }}' # default: one budget for this document policy: queue # queue (default) | drop queue_max: 100 # queue policy onlyburst:1 is strict spacing:limit: 20, period: 1mstarts one every three seconds. A higher burst lets that many start together after a quiet stretch; idle time never banks more than the burst.key:renders likeconcurrency.group, which gives a budget per tenant. Without it the budget is counted under the document’sname:, and a throttle on a document with neither is refused.policy: droprefuses a run over the rate (HTTP 429 withRetry-After). Use it for work that is worthless late, such as a health probe.- A waiting run shows as
queuedand carries thethrottle-key:<key>tag.
A rate the engine can’t honour is refused rather than reinterpreted: a zero period, a limit
finer than a millisecond apart, or queue_max: 0 with policy: queue. A limit that doesn’t
divide the period rounds the spacing up.
throttle: does not pace the items inside one fan-out; that is the node’s own
rate.
The order the gates run in
Section titled “The order the gates run in”--idempotency-key, if given. A repeat is answered with the first run and costs nothing.throttle:. A run held here has not met its group yet.concurrency:. A run held here has spent its rate token and keeps it, since it will run.
A run the group refuses outright (full queue, skip) gets its token back. A run is never
waiting in both queues at once.
Top-level cancel_on: (abort on a bus event)
Section titled “Top-level cancel_on: (abort on a bus event)”cancel_on: events: [pr.closed, deploy.freeze]The run is cancelled when an event of one of these types lands on the project bus, with the
reason cancel_on event "<type>". Matching also uses the event’s subject: a run inherits the
subject of the event that started it, so a run started for acme/web is cancelled only by
acme/web’s events. An event emitted without a subject (spool event emit deploy.freeze)
reaches every run watching that type. A run started by a direct submit has no subject and matches on
type alone. See Subjects.
An untrusted event (a fork’s pull request, an outside contributor’s comment) doesn’t cancel the
run unless the block sets accept_untrusted: true, or the run is untrusted itself. See
Untrusted events.
Trigger filters
Section titled “Trigger filters”These decide whether an event a handler delivers straight to the pipeline
(spool handler set ci ci.yaml --on github.push) becomes a run at all. A hand-run, a schedule
tick and a spawn always run, and so does a build started by spool repo connect, which is
spawned: filter those pushes with spool repo connect --when (see
CI from a repo).
Top-level if: (trigger predicate)
Section titled “Top-level if: (trigger predicate)”A CEL expression over the delivered event, evaluated on the server before any run exists. False means no run:
if: 'event.ref == "refs/heads/main" || event.ref.startsWith("refs/heads/release/")'| Field | Example |
|---|---|
event.type |
github.push, github.pull_request.opened |
event.key |
the subject, owner/name for a forge delivery |
event.ref |
refs/heads/main (pushes) |
event.sha |
the pushed or PR head commit |
event.head_ref, event.base_ref |
pull-request branches |
event.repo |
owner/name |
event.sender |
the forge user who caused it |
event.paths |
files a push touched, as a list |
event.paths_known |
false when the forge reported no file list |
event.payload.body |
the raw delivery body, for anything not listed |
A field that isn’t there is an error, and an erroring predicate runs the pipeline and logs the
failure, so a broken guard misfires loudly instead of dropping pushes. Once a run exists, a job
reads the same fields under input.
paths: and paths_ignore:
Section titled “paths: and paths_ignore:”paths: [api/**, "*.rs", Cargo.lock] # run only if one of these changedpaths_ignore: [docs/**] # skip when these are ALL that changedBoth are ANDed with each other and with if:.
| Pattern | Matches |
|---|---|
dir/** |
anything under that directory |
*.ext / **/*.ext |
by extension, at any depth |
**/name |
that filename in any directory |
Cargo.lock |
that exact path |
A * anywhere else is refused. Both fail open when event.paths_known is false (a large or
force push), so an unknown file list builds instead of silently skipping.
Filtering one job
Section titled “Filtering one job”paths: decides whether the run happens. For one job, write its if: over input.paths:
jobs: lint: steps: [{ name: lint, run: make lint }] web: depends_on: [lint] if: ${{ success() && (!has(input.paths_known) || !input.paths_known || input.paths.exists(p, p.startsWith("web/"))) }} steps: [{ name: build, run: make web }]success()keeps the implicit gate that a hand-writtenif:replaces.!has(input.paths_known)lets a hand-run through; it has nopathsat all.!input.paths_knownbuilds when the forge didn’t say what changed.
Keep it on one line; a folded block keeps its newlines in errors. Other shapes are
p.endsWith(".rs"), p == "Cargo.lock" and p == "Cargo.toml" || p.endsWith("/Cargo.toml").
A job skipped this way still satisfies its dependents, so gate them the same way.
Each job’s keys are listed in Keys at a glance. This section covers the ones shared across kinds; each kind has its own section below.
Where a job runs
Section titled “Where a job runs”A steps: job needs an agent with the shell capability and a script: job needs
polyglot. requires: adds more, and the job goes only to an agent advertising all of them.
On the service, work needing shell, polyglot, connector or build runs in a managed
environment started for it: a Linux amd64 container with bash, git, curl, go, node 22 and
python3 (with the Python SDK) for the first three, or a 16-core amd64 VM with KVM and sudo for
build (Build VMs). Anything else (arm64, macOS, a GPU, a private network, tools the image lacks) needs an
agent you run that advertises it. A job nothing can serve shows as
“no agent can run this” and names the capabilities it asked for.
agent: <id> pins the job to one agent, on top of requires:, for when the machine is the
subject (reading its journal, checking its disk). A pinned job waits for that agent and never
fails over. On a matrix: job the pin covers every combination.
timeout:
Section titled “timeout:”A wall-clock ceiling for the job’s process (30m, 2h). The default is one hour, and an
override is capped at 24 hours. On expiry the process group is killed and the job fails,
subject to retry:. A managed environment lives at most an hour, so a longer job belongs on
an agent you run. See Retries and timeouts.
retry:
Section titled “retry:”| Field | Meaning |
|---|---|
attempts |
total attempts, counting the first |
backoff |
wait before attempt 2; doubles after that |
max_backoff |
cap on any single wait |
exit_codes |
retry only when the process exits with one of these |
attempts: 6, backoff: 1m waits 1, 2, 4, 8 and 16 minutes. exit_codes: [75] retries a
temporary failure and fails a malformed input (exit 64) on the first attempt. Only a process
exit matches: a job killed by its timeout or lost with its agent is never selectively retried.
An empty list is refused. A job that ran more than once is marked ×3 on the run page.
Credentials and untrusted runs
Section titled “Credentials and untrusted runs”credentials: maps an environment variable to a stored credential name, at the top level (every
job) or on a job (wins for that variable). Only the name is in the document; the value is
resolved when an agent claims the job and masked in logs. The name is a literal, not an
expression. A name that doesn’t resolve fails the job.
A run triggered by an untrusted actor (a pull request from a fork) gets no credentials and no
project secrets, and a job that declares a credential fails. optional_when_untrusted: true on a job lets it run without them:
jobs: lint: optional_when_untrusted: true steps: - { name: lint, run: make lint }It is job-level only. A top-level credentials: block reaches every job, so keep privileged
credentials on the job that needs them. See
Credentials in a pipeline.
continue_on_error: (absorb a job’s failure)
Section titled “continue_on_error: (absorb a job’s failure)”The job may fail without failing the run, and its dependents still run. Its own status stays
failed:
jobs: lint: continue_on_error: true steps: [{ name: run, run: ./lint.sh }] audit: depends_on: [lint] if: ${{ steps.lint.status == 'failed' }} steps: [{ name: file, run: ./open-ticket.sh }]success() and failure() below it read the absorbed failure as success, so branch on
steps.<job>.status when it matters.
Job-level concurrency: (serialize one job across runs)
Section titled “Job-level concurrency: (serialize one job across runs)”deploy: depends_on: [plan] concurrency: group: "deploy-${{ steps.plan.output.env }}" steps: …The job takes a durable lease on group right before it runs and holds it until the run ends.
Another run reaching the same job waits while its other jobs keep going; the console shows it as
“waiting for a lock”. The lease is the one a script’s spool lock <group> takes (see
Polyglot scripts), so the two exclude each other in a project. It takes
only group: and always queues in order.
The group renders just before the job runs, so it can read steps.*, input, vars and
run. Guard optional fields with has(). An empty group is refused, and so is matrix. in a
group, since the lease is taken once for the whole sweep.
Job kind: steps:
Section titled “Job kind: steps:”All steps run as one script, in one process, on one agent. Step names split the log; there is no durable boundary between steps.
build: outputs: tag: tag steps: - name: install run: apt-get install -y curl env: DEBIAN_FRONTEND: noninteractive - name: emit run: spool output set tag=v1.2.3A step has run, an optional name and an optional env:, exported only for that step. A
step without a name is named after the first line of its command, up to any ${{, cut to
40 characters. A step’s env: values are literal; a ${{ … }} there is refused, because the
value is quoted into the script. Put an interpolated value in job- or top-level env:, which
renders safely, and read it as a variable. That is also the safe way to use event text such as a
branch name:
deploy: env: REF: ${{ input.ref }} steps: - name: deploy run: ./deploy.sh "$REF"A ${{ … }} written straight into run: lands unescaped. Anything else that looks like a
template ({{ .Values.tag }}, jinja) passes through untouched; only ${{ is read. An unclosed
${{ is an error naming the job and step.
Side channels
Section titled “Side channels”spool output set stages a key and the job’s outputs: publishes it:
build: outputs: tag: tag built: built steps: - name: emit run: spool output set tag="$(git describe --tags)" built=trueship: depends_on: [build] if: ${{ steps.build.output.built }} steps: - name: deploy run: ./deploy.sh "${{ steps.build.output.tag }}"Values parse as JSON when they can (built=true is a boolean) and are committed only if the job
succeeds. A key the mapping doesn’t list stays inside the job, and reading it downstream fails
the run.
A step can also call spool annotate (a note on the run page, see
Annotations), spool event emit (a bus event), spool state and
spool artifact. Durable primitives that suspend are refused; see Anti-patterns.
env: and vars:
Section titled “env: and vars:”env: reaches the job’s process as $NAME and never appears in an expression. An agent you
run can pass its own variables through, so a name can differ per agent.
vars: is readable in any expression as vars.<NAME> and never reaches a process. Values are
JSON, so a number compares as a number. A job’s if: is decided before any agent is involved,
so a gate can read vars and never env.
env: REGISTRY: ghcr.iovars: MIN_COVERAGE: 0.9jobs: gate: if: 'vars.MIN_COVERAGE > 0.8' steps: - name: push run: echo "pushing to $REGISTRY"Values that change per run belong in the run’s input, read as input.<field>.
Built-in env
Section titled “Built-in env”Every shell job gets CI=true, CI_RUN_ID (the run id), CI_JOB_NAME and
SPOOL_INPUT_FILE, a file holding the run’s whole input as JSON. A run started by an event also
gets the event’s scalar fields as CI_TRIGGER_<FIELD>: CI_TRIGGER_EVENT is its type, and a
push adds CI_TRIGGER_REF, CI_TRIGGER_SHA and CI_TRIGGER_SHORT_SHA. CI, CI_RUN_ID and
CI_JOB_NAME are written last, so env: can’t shadow them; an env: key of the same name
does override a CI_TRIGGER_* value. Script jobs see more; see
Env exposed to scripts.
Secrets
Section titled “Secrets”Project secrets (spool secret set) reach every steps: and script: job as environment
variables; env: wins on a name clash. Their values are masked in job output. Masking catches
echoed values, not transformed ones (base64 and the like). See Secrets.
Job kind: approval:
Section titled “Job kind: approval:”A gate a person answers. It holds no agent while it waits.
review: depends_on: [build] outputs: env: env note: comment approval: prompt: "Deploy to production?" timeout: 3d fields: - { name: env, type: select, options: [staging, production] } - { name: comment, type: textarea, required: false, hint: "Why now?" }| Field key | Meaning |
|---|---|
name |
the key in the submission (required) |
type |
text, textarea, select, checkbox or number (required) |
required |
default true |
options |
the choices for select |
default |
prefilled value |
hint |
help text under the field |
min, max |
bounds for number |
The server checks the submission against the fields before the gate opens. The submission is
the job’s raw output, and outputs: picks from it. A field named timed_out is refused.
Answer it from the run page in the console, with
spool approve <run-id>, which asks for each field, or with
POST /api/v1/orgs/<org>/projects/<project>/runs/<run>/approvals/<job> and the field values as
the JSON body. A graph approval node takes the same fields and the same answers.
Rejecting
Section titled “Rejecting”Reject cancels the run and the children it was waiting on, and records the reject and an
optional reason on the gate. It is on the run page, or POST …/approvals/<job>/reject with
{"reason": "..."}. The approve route refuses a decision key unless the gate declares a
field with that name. See Approvals.
When “no” should run something instead of stopping everything, make the decision a field and
branch on it. The person answering picks hold and presses Approve:
review: approval: prompt: "Ship 1.4.0?" fields: - { name: decision, type: select, options: [ship, hold] }deploy: depends_on: [review] if: ${{ steps.review.output.decision == 'ship' }} steps: [{ name: deploy, run: ./deploy.sh }]tell-the-author: depends_on: [review] if: ${{ steps.review.output.decision == 'hold' }} steps: [{ name: notify, run: ./post-to-chat.sh "1.4.0 held" }]A gate whose timeout: runs out succeeds with timed_out: true, and jobs behind it go ahead as
if approved. Guard them with if: ${{ !steps.review.output.timed_out }} when that must not
happen; see Bounded waits.
Job kind: wait:
Section titled “Job kind: wait:”A timer or a wait for a signal. No agent is held.
soak: depends_on: [deploy] wait: duration: 30m # asleep for this long, then succeed
promote: depends_on: [deploy] wait: signal: deploy-ok # until `spool run signal <run-id> deploy-ok` timeout: 1hExactly one of duration: and signal:. timeout: bounds a signal wait and is refused beside
duration:.
Bounded waits
Section titled “Bounded waits”wait: signal:, wait_for_event: and approval: wait forever unless they have a timeout:.
On expiry the job succeeds, and its output carries timed_out: true, so a dependent branches on
it:
soak: depends_on: [deploy] wait_for_event: { event: alerts.fired, timeout: 14d }rollback: depends_on: [soak] if: ${{ !steps.soak.output.timed_out }} steps: [{ name: undo, run: ./deploy.sh staging --rollback }]promote: depends_on: [soak] if: ${{ steps.soak.output.timed_out }} approval: { prompt: "Promote to production?", timeout: 3d }timed_out exists only on a job that declares a timeout:; reading it on any other job is
refused at submit. When the signal wins, the flag is merged into an object payload (an approval
keeps steps.gate.output.<field>); a non-object payload becomes
{"timed_out": false, "payload": <value>}. The wait resolves once: a signal after the deadline
changes nothing.
An expired wait also publishes run.wait_expired on the bus; see
Lifecycle events.
Job kind: wait_for_event:
Section titled “Job kind: wait_for_event:”Waits until a matching event lands on the project bus. No agent is held.
hold: wait_for_event: event: order.placed when: ${{ event.payload.qty > 10 }} timeout: 1dWithout when: the first event of that type resumes the job. when: is a predicate over the
event envelope; an event it declines leaves the job waiting and still reaches every other
listener. A downstream if: can’t do this, since by then the event is taken. timeout: runs
from the first wait and works as in Bounded waits. Without one, the job
resumes only when the event arrives.
The wait is registered when the run is submitted, so an event that arrives while earlier jobs
run still counts. It matches the run’s subject the way cancel_on: does, and an event emitted
without a subject resumes any run waiting on that type. To unblock it by hand, emit the event
(spool event emit order.placed --payload '{"qty": 12}'). The job’s output is the event
envelope. spool run get <run-id> -o json shows the wait under substate.name as event:<job>, with
anything but letters, digits and _ in the job name turned into _ (event:wait_for_ship).
An untrusted event (a fork’s pull request, an outside contributor’s comment) resumes the job only
with accept_untrusted: true, or when the run is untrusted itself; otherwise the job keeps
waiting for a trusted one. The output of a job resumed by one carries untrusted: true. See
Untrusted events.
A script can wait on the same type with spool wait event (see
Polyglot scripts); each waiter gets the event.
Job kind: workflow:
Section titled “Job kind: workflow:”Runs a child as the body of this job and waits for it. See Composition.
canary: depends_on: [build] outputs: verdict: verdict workflow: type: hyperspool/script/v1 # or hyperspool/pipeline/v1 script: | #!/usr/bin/env bash spool step "smoke" -- bash -c 'echo OK' spool output set verdict=okscript: is rendered against this pipeline’s scope before the child starts. The child starts
clean, so this pipeline’s env: doesn’t reach it. A hyperspool/pipeline/v1 child returns what
its document-level outputs: declares.
A job can call a stored graph or one node kind instead:
notify: workflow: node: kind: matrix.send_message # a built-in or <connector>.<action> config: { room: "!ops:example.com" } input: { body: "deploy done" }
deploy: depends_on: [build] workflow: graph: name: deploy version: 3 # optional; the published version otherwise input: ref: ${{ steps.build.output.sha }} retries: ${{ vars.MAX_RETRIES }} # a whole-value hole keeps its typeExactly one of type:, graph: and node:. input: and a node’s config: render against this
pipeline, and every hole is checked at submit: an unknown job or a root other than steps,
vars, run or input fails then.
Letting the child answer a marker
Section titled “Letting the child answer a marker”To have a single node resolve a value against its own input:, wrap the expression in
$defer:
call_api: workflow: node: kind: http config: url: { $defer: { $expr: "input.endpoint" } } # the child answers this input: endpoint: https://api.example.com/v1/itemsA deferred expression may read only input and run. $defer is refused anywhere but a
node: target’s config:. spool workflow check catches a misplaced one.
Job kind: script:
Section titled “Job kind: script:”A durable polyglot script run inline in this run, with the spool toolkit (step,
sleep, wait signal, activity run, …) and suspend-and-replay. No child run is created.
analyze: depends_on: [build] outputs: verdict: verdict script: | #!/usr/bin/env bash set -eu score=$(spool step "scan" -- bash -c 'echo 0.98') spool sleep 30s spool output set verdict=ok score="$score"The body renders against the pipeline’s scope. Its spool output set values are the job’s raw
output, and a non-zero exit fails the job. Use workflow: instead when you want a separate run
with its own run page. See Polyglot scripts.
Templating
Section titled “Templating”${{ … }} holes and if: conditions are CEL. Plain ${VAR} and bare
{{ … }} are left alone. A hole is compiled and its references checked at submit, so a syntax
error, an unknown job, a scope root that doesn’t exist, or an output key a job’s outputs:
doesn’t list fails before the run starts. spool workflow check reports the same offline.
A value is rendered just before it is needed and can read what exists by then:
| Moment | In scope | Fields |
|---|---|---|
| the event arrives, no run yet | event |
top-level if:, paths:, paths_ignore: |
| the run is created | input, vars |
top-level concurrency.group, throttle.key |
| a job is about to run | + steps.*, run |
job if:, env:, approval.prompt, job concurrency.group, a workflow: job’s input:/config: |
| one matrix combination | + matrix |
the body, requires: |
| every job has finished | all steps.* |
document-level outputs: |
Reading from a later row is an error. wait.signal and wait_for_event.event take no holes.
Roots:
input.<field>- the run’s input: the event that started it, the--inputgiven, the document’sinput:, or{"type": "manual"}. Never size-capped.vars.<NAME>- the top-levelvars:.steps.<job>.output.<key>andsteps.<job>.status(success,skipped,failed). The job has to be upstream of this one, throughdepends_ondirectly or further up; reading any other job is refused when the pipeline is checked or run, naming thedepends_onto add. The document-leveloutputs:may read any job.matrix.<name>- inside a matrix job.event.<field>- in the top-levelif:only.
Functions in if::
success()- no upstream job failed and every dependency succeeded (a skipped one makes it false).failure()- some upstream job failed.always()- true.cancelled()- always false.
A job with no if: is skipped only when an upstream job failed, so a skipped dependency still
lets it run. Writing if: success() also skips it when a dependency was skipped.
String helpers, for text you didn’t write: trim(), lowerAscii(), upperAscii(),
replace(from, to), indexOf(s), lastIndexOf(s), charAt(i), substring(start),
truncate(max) and urlEncode(), beside CEL’s size(), contains, startsWith, endsWith and
matches(regex). Offsets count characters. There is no split.
deploy: depends_on: [build] if: ${{ success() && steps.build.output.branch.startsWith("release/") }} steps: - name: deploy run: ./deploy --tag "${{ steps.build.output.tag }}"Outputs
Section titled “Outputs”A job’s outputs: maps a name to a key in its raw output:
| Job kind | Raw output |
|---|---|
steps: |
spool output set values |
script: |
spool output set values, plus exit_code and duration_ms |
approval: |
the submission (plus timed_out with a timeout:) |
wait: |
nothing for a timer; timed_out for a signal wait with a timeout: |
wait_for_event: |
the event envelope (plus timed_out with a timeout:) |
workflow: |
the child’s output |
Downstream jobs read ${{ steps.<job>.output.<name> }}. A key the job’s outputs: doesn’t
declare is refused at submit; with no outputs: block, a missing key is a hard error when read.
Document-level outputs: (what the run returns)
Section titled “Document-level outputs: (what the run returns)”The top-level outputs: is what the whole run hands back to a caller: a workflow: job running
it as a child, or a graph workflow node running it from a repo (from_repo:).
outputs: sha: ${{ steps.build.output.sha }} image: "registry.example.com/app:${{ steps.build.output.sha }}" report: passed: ${{ steps.test.output.passed }}jobs: build: outputs: { sha: sha } steps: - name: build run: spool output set sha="$(git rev-parse HEAD)" test: depends_on: [build] outputs: { passed: passed } steps: - name: test run: ./test.sh && spool output set passed=trueHoles may be nested at any depth. A whole-value hole keeps its type; text around a hole makes a
string. A job must publish a key in its own outputs: before the document can return it. The
outputs sit beside the run’s nodes status map, so a graph caller reads
nodes.<child>.output.sha, and an output named nodes is refused. Only a successful run has
outputs, and an output that can’t be computed fails the run.
Dependencies
Section titled “Dependencies”depends_on: [a, b] runs a job after both. A job with no if: is skipped when anything upstream
failed. A cleanup or notify job opts out with if: always() or if: failure(); running it
doesn’t clear the failure for jobs below it:
jobs: build: steps: [{ name: b, run: make }] cleanup: depends_on: [build] if: always() steps: [{ name: c, run: ./cleanup.sh }] deploy: depends_on: [cleanup] # skipped when build failed steps: [{ name: d, run: ./deploy.sh }]A failure holds back only the jobs that depend on it, directly or further down. Every other job
runs as it would in a green run, a deploy included, so a job that must stop the rollout belongs in
the deploy’s depends_on. Approvals, waits and child runs beside the failure are waited for too,
so a failed run can stay running until someone answers a gate. Reject the gate or cancel the
run to end it sooner. A gate timeout: also ends the wait, but jobs behind it then go ahead as
if approved (see approval:). A failure() job below the failure does
not wait for a gate, timer or child run beside the failure, though it can wait for jobs that
were already running beside it. Once nothing more can run, the run ends failed if any job
failed without continue_on_error:.
A skipped job still satisfies depends_on. It reports status == "skipped" and has no outputs,
so gate on its status rather than reading an output.
matrix: (one job per combination)
Section titled “matrix: (one job per combination)”Runs a steps: or script: job once per combination, readable as ${{ matrix.<name> }}:
jobs: test: matrix: values: python: ["3.11", "3.12"] suite: [unit, integration] exclude: - { python: "3.11", suite: integration } max_parallel: 2 steps: - name: test run: ./test.sh --python ${{ matrix.python }} --suite ${{ matrix.suite }} publish: depends_on: [test] steps: [{ name: publish, run: ./publish.sh }]Three combinations run. Dimensions keep document order, the last varying fastest. Values are JSON, so one object-valued dimension can carry several settings together.
| Key | Meaning |
|---|---|
values |
dimension name → list of values; a literal list, no holes |
exclude |
partial combinations to drop; an omitted dimension matches anything |
max_parallel |
how many run at once |
rate: { limit, period } |
start limit combinations, then wait one period after they finish before the next batch |
fail_fast |
default true: the first failure fails the job; false runs them all |
An unknown dimension, a value not in the dimension, an empty exclude entry, an empty dimension,
excluding every combination, or more than 100 combinations is refused. The matrix stays one job: depends_on: [test] waits for
every combination, and the job’s output is the list of per-combination outputs. retry:,
timeout: and requires: apply per combination.
requires: may read the combination, which spreads one job across platforms:
build: matrix: values: target: - { cap: linux-amd64, triple: x86_64-unknown-linux-gnu } - { cap: linux-arm64, triple: aarch64-unknown-linux-gnu } requires: ["${{ matrix.target.cap }}"] steps: - name: build run: cargo build --release --target ${{ matrix.target.triple }}Each combination goes to an agent advertising its capability. These capability names are ones
your own agents advertise; the managed pool is amd64 Linux only. A hole in requires: without a
matrix: is refused. spool workflow render shows the compiled form, a single fan_out node.
More patterns are in Repeating work.
Dynamic pipelines
Section titled “Dynamic pipelines”A run’s job set is fixed when it starts, and matrix.values takes no holes. To decide work at
run time, have a script: job write a graph document and run it as an
awaited child with spool call <file|->:
plan: script: | #!/usr/bin/env bash set -eu ./generate-graph.py > work.yaml spool call work.yamlThe child has its own run page, the job waits for it, and spool call prints its output.
See A child computed at run time.
Build only what changed is a worked example that generates one job per
changed package.
Graph documents
Section titled “Graph documents”A graph is the format the canvas edits. It is JSON or YAML, holds at most 512 steps (a pipeline
has the same ceiling), and version: hyperspool-graph/v1 marks it:
version: hyperspool-graph/v1name: deploynote: Build, then ship once someone approves.concurrency: group: 'deploy-${{ has(input.ref) ? input.ref : "manual" }}' policy: cancel-runningvars: REGION: eu-westnodes: - id: build type: shell config: run: spool output set tag=v1.2.3 - id: review type: approval config: prompt: Ship it? - id: ship type: shell config: run: $template: "echo shipping {{ nodes.build.output.tag }} to {{ vars.REGION }}"edges: - { from: build, to: review } - { from: review, to: ship }outputs: tag: { $expr: nodes.build.output.tag }Where a graph goes:
spool workflow check file.yamlvalidates it andspool submit file.yamlruns it once.spool workflow set <name> file.yaml --publishstores it;spool workflow export <name>prints a stored one as YAML. The workflow’s Code view in the console edits the same document.- A repo file, such as the one
spool repo connectbuilds, may be a graph or a pipeline;version:tells them apart. A connected repo’s push hands the graphinput.sha,input.refandinput.repo. spool call <file>, aworkflownode’s inlinedefinition:, and afrom_repo:child all take one.
Top-level keys:
| Key | Meaning |
|---|---|
version |
hyperspool-graph/v1 (required) |
name, note |
display name; what the graph is for |
nodes |
the steps (required); order breaks ties between ready nodes |
edges |
{from, to}, plus when: naming a switch branch |
vars |
constants, vars.<name> in expressions |
outputs |
what the run returns, as $expr/$template values |
input_schema |
what input the graph takes: description, properties, required |
tags, concurrency, throttle, cancel_on |
as in Run admission |
Node fields:
| Key | Meaning |
|---|---|
id |
unique, [a-zA-Z_][a-zA-Z0-9_]*; read as nodes.<id> |
type |
shell, code, http, workflow, transform, switch, fan_out, wait, approval, event, emit, state, trigger, or <connector>.<action> |
config |
the type’s settings |
note |
why the step is there |
if |
CEL condition; the node is skipped unless true |
continue_on_error |
as for a job |
retry, requires, agent, timeout |
as for a job; nodes that run on an agent only |
concurrency |
a lock group string, with {{ … }} holes |
Dynamic values in config are markers: { $expr: "<cel>" } keeps the value’s type, and
{ $template: "text {{ cel }}" } makes a string. Expressions read nodes.<id>.output,
nodes.<id>.status, input, vars and run. Unknown keys are refused. What each type’s
config takes, and what it returns, is in the node reference.
Anti-patterns
Section titled “Anti-patterns”- Don’t split work that needs a shared filesystem across jobs. Each job may land on a different
agent; put the commands in one job’s
steps:. - Don’t call durable primitives (
spool step,sleep,wait signal,wait event,activity,spawn,call,lock) in asteps:job. A shell step can’t suspend, so the agent refuses them. Use await:,approval:,script:orworkflow:job. Calls that never suspend work there:spool annotate,state,event emit,output set,artifact,run get. See Atomic and durable activities. - Don’t loop on
spool sleepfor work that recurs; use Schedules.