Concepts

The words the docs and the console use, and what each one means.

One split explains most of the rest. The server keeps the record of every run and decides what happens next. The work itself runs on agents: processes that dial out to the server and claim work they can do. On hyperspool.dev those are managed environments started on demand, or agents you run yourself (see Agents). spool dev has one built in.

Workflow

A workflow is the definition a run executes. It comes in three forms:

• A pipeline (hyperspool/pipeline/v1): YAML with jobs: and depends_on, usually in a file next to your code.
• A script (hyperspool/script/v1): a program in bash, Python or another language, using spool commands for its checkpoints. Reach for it when the control flow is easier written as code than declared.
• A graph (hyperspool-graph/v1): nodes and edges, drawn on the canvas, written as YAML or JSON, or computed by an earlier step.

Any of them can start any other as a child run, and all three can emit events and be started by them: see Composition and Connecting pipelines, graphs and scripts. Pipelines and graphs compares the two document formats.

Run

A run is one execution of a workflow. It has an id (<org>:<project>:run-<random>), an event log, a status (queued, running, completed, failed or cancelled), and zero or more child runs. queued means a cap, a concurrency group or a rate limit is holding it before it starts.

A definition can declare outputs: at the top level: named values taken from the finished run and handed to whoever called it. That’s how a child run returns something to its parent. Only a successful run produces them.

While a run waits it says what for: an approval, a signal, an event, an agent. A run on a timer is asleep. None of these hold an agent.

Replay

This is the idea the rest of Hyperspool is built on.

A run’s log records the result of every durable call: a spool step, a sleep, a wait, a child run, and the like (the full list). Each recorded result is a checkpoint. When a run resumes, after a sleep, a crash or a restart, its code runs again from the top, and every durable call that already has a checkpoint returns the recorded result instead of doing the work again. Like this:

n="$(spool step pick -- bash -c 'echo $(( RANDOM % 100 ))')"
spool sleep 1d
echo "still $n"

spool sleep ends the process. A day later the script starts over, pick returns the number it drew the first time, and the echo prints it. That’s why a run can sleep for a month, or survive a crash, without holding a machine.

Code between durable calls runs again on every pass, so it has to reach the same durable calls in the same order. The authoring model has the rules, and when replay goes wrong shows what happens when a change breaks them.

Step

A step is one unit of work in a run: a job in a pipeline, a node in a graph, a spool step in a script. The console shows a run as its steps.

The formats keep their own words. A pipeline’s YAML has jobs:, and a shell job can list several named commands under steps:; each runs in its own shell, on the same agent, as one checkpoint. A graph’s document has nodes:. The docs say job and node when they talk about those formats, and “step” everywhere else.

A pipeline job is one of six kinds:

• steps: - shell commands, run by an agent.
• script: - a script run inline, with the whole spool toolkit (step, sleep, wait signal, activity).
• approval: - a form a person answers.
• wait: - a timer or a named signal.
• wait_for_event: - waits for a matching event on the project bus.
• workflow: - starts a child run as the body of this job.

The last four never touch an agent.

Agent

An agent is a long-running process that dials out to the server over HTTP, claims work matching its capabilities, runs it, and reports back. It opens no inbound port, so one can run behind a firewall.

Agents declare capabilities (shell, polyglot, connector, build, or tags of your own), and the server hands each piece of work to an agent whose capabilities cover what the step requires:. An agent takes one at a time; more agents give more concurrency.

Agents are interchangeable. A script that suspends is dispatched again on wake, and any agent with the right capabilities can take it, on any machine, in any week. What a run carries forward is its checkpoints. Local files, unsaved variables and the working directory stay behind.

Activity

An activity is one unit of work an agent claims, runs and reports back on. Its scheduling and result are recorded in the run’s log. A shell job runs as a shell_run activity, which needs an agent with the shell capability; a script runs as a script_run activity, which needs polyglot. spool activity submit and spool activity wait let a script fan out into child activities, and the parent’s agent is freed while it waits for them.

Atomic and durable activities

The two kinds differ in whether the activity can suspend and replay. A shell step runs once, start to finish. A script can suspend: it exits, frees the agent, and runs again from the top on wake, replaying its checkpoints.

So the calls that suspend (spool step, sleep, wait signal, wait event, activity, spawn, call, lock) are refused in a shell step, with an error naming the job kinds that offer them. The calls that never suspend work in both: spool annotate, spool state, spool output set, spool event emit, spool artifact, spool run get. A shell step can write a state cell, since that’s a single round trip, but it can’t spool sleep.

Signal

A signal is a named message with a payload, delivered to one run by id (spool run signal <run> <name>). A run waits for one with a wait: job, a wait node, or spool wait signal <name> in a script. Common uses: callbacks from another system and coordination between runs.

Approvals

An approval is a step a person answers, in the console, with spool approve, or from spool run follow. Approving resumes the run with the answer as the step’s output. Rejecting cancels the run and records the reject and its reason on the approval. Approvals are signals underneath.

Events

A project has an event bus. A run emits with spool event emit, and named handlers each match the event types they care about (github.push, run.failed, run.*) and start work. Every top-level run also emits a lifecycle event when it finishes (run.succeeded, run.failed, run.cancelled), so a handler can react to completions. Webhooks put deliveries from other systems on the same bus. See events and handlers.

Schedule

A schedule starts a fresh run at a set cadence: an interval (1m, 5h, 1d) or a cron expression. See Schedules.

Durable state cells

A state cell is named JSON state that survives replay, read and written directly instead of being rebuilt from the log. A cell is either private to one run (the default) or, with --entity <name>, shared by every run in the project that names the same entity. Entity scope is what makes a poll cursor work across scheduled runs:

since="$(spool state get last-seen --entity digest)"
latest="$(spool step "poll" -- ./fetch-since.sh "${since:-0}")"
spool state set last-seen "$latest" --entity digest

get then set can race between two runs, so add, incr and cas settle it: claim a key, count, or write only if the cell is unchanged. Cells are encrypted at rest with the org’s key, and the CLI, the SDKs and a canvas state node all reach the same cells. Durable state covers the operations in full.

Pipelines and graphs

Both are documents you can write in YAML, and both run on the same engine:

	Pipeline	Graph
Top level	jobs:	version: hyperspool-graph/v1, nodes:, edges:
Where it lives	a file, usually in your repo	stored in the project with a draft and a published version, or a file in your repo
How it starts	spool submit <file>, a push, a schedule, a handler	by name (spool submit -w <name>), spool submit <file>, a push, a schedule, a handler, or its own trigger nodes
Reference	YAML pipeline reference	Authoring on the canvas

Pick by what you’re doing. Work that belongs with your code and runs on a push is usually a pipeline. Work that reacts to events, branches, or gets edited on the canvas is usually a graph.

A pipeline becomes the same node graph the canvas draws, which is why a run of either renders the same way; spool workflow render prints it. One job becomes one node: steps: a shell node, script: a code node, wait_for_event: an event node, and approval:, wait: and workflow: keep their names. A matrix: job is one fan_out node. A graph doesn’t convert back into a pipeline.

Both formats take the same admission blocks at the top level: tags:, concurrency:, throttle: and cancel_on:. Two things are pipeline-only: a top-level if:, which gates a run on the event that delivered it, and several named commands in one job. A graph places switch, state, emit, fan_out and connector actions directly as nodes, and an edge out of a switch can name the branch it belongs to (see branch routing).