Concepts

attention.run works on pixels and input. It sees a screen as frames, sends clicks and keys to it, and recognises the parts you named by template matching: finding a small image in a frame. No DOM, selectors or test IDs are needed to run a flow, so it works on anything that draws pixels: web apps, canvas and WebGL, desktop apps, remote desktops.

Targets

A target is a screen attention.run can see and drive. There are three kinds:

Chrome, over the DevTools protocol. Either a Chrome the runner starts (headless by default, optionally with a start URL and a named profile that keeps its logins), or a Chrome you started with a DevTools endpoint (ws://…). A Chrome target has a fixed viewport (1366×768 unless you set another) at device pixel ratio 1, because a template only matches at the size it was cut.
A desktop over VNC: a host and port (5900 by default), with no password, a VNC password, or Apple Remote Desktop credentials. Windows, macOS, Linux, kiosks and remote desktops all work through a VNC server. A VNC target serves one session at a time, since two drivers on one desktop would fight.
SSH targets: a VNC server or a Chrome on your own machine, connected with one ssh -R command or attn share, without opening a port. See SSH targets.

Targets are made in the web app (Targets, then New target) or through the API (POST /api/v1/targets).

Sessions and recordings

A session is a connection to a target. In a live session the web app mirrors the screen and your clicks and typing go to it; a script drives one through the SDK and an agent through MCP. Each run opens a session of its own.

A recorded session (the default) keeps every frame, losslessly, and every input on one clock. The recording replays in the web app: scrub the timeline, see each click and key, and cut elements from any frame. A live session with no viewer and no input closes after two minutes on a default hub. Sessions you delete go to a trash and can be restored.

Elements and libraries

An element is a named part of a screen, such as login/sign-in. Slugs are lowercase words joined by /. An element is recognised by one or more variants: template images such as its default look, its hover look, or a state like on and off. A flow can ask which variant is on screen, which is how it reads a checkbox or a toggle.

Cutting an element from a recording checks it against the whole recording: whether it is unique, the margin to the next best match, and the threshold it needs. “Grow to unique” suggests the smallest larger rectangle that is unique when a cut is ambiguous.
A mask marks the parts of a variant to ignore, such as a counter or a date inside a button.
The click point is where clicks and typing land, as an offset into the element. Cut from a click, it is that click.
Multiple elements are expected more than once on a screen (list rows, icons); flows pick one by index, by region, or within another found item.
Status: active elements are matched. proposed ones (from a vision model’s suggestions or an agent) wait until a person accepts them. archived ones are kept but not matched.
Other ways in: upload a PNG as a variant; or, on a hub with a vision model configured, Suggest elements on a recording frame proposes the elements the library lacks.

A library is a set of elements with names unique in it. An element can be a member of several libraries, and a library can be cloned or forked. A flow names the library its slugs resolve in, default unless it says otherwise. Renaming an element keeps its old slug as an alias, so flows that name it keep working.

Matching is deterministic: the same frame and the same library give the same answer. Repeated screens are answered from a cache keyed by their pixels, and no model is called while a flow runs.

Flows

A flow is a JSON document of steps that act on elements and wait for them: click, type, press, scroll, wait for a condition, assert, branch, loop, call another flow. The full list is in the flow JSON reference. Three rules make flows reliable without sleeps:

Auto-wait: a step that acts on an element waits for it to be on screen first, up to its timeout.
The frame barrier: a wait after an action only looks at frames from after that action. A toast from the previous click can’t satisfy the next wait.
No silent ambiguity: an element found in two places fails the step with AMBIGUOUS and both places, instead of clicking one of them.

Flows have typed params (inputs) and outputs (what a run returns), handlers for what can appear at any time (a cookie banner, a “session expired” dialog), and call to run another flow inline. Each save is a new version; pushing an unchanged flow saves nothing. The hub validates a flow when it is saved: the schema, the element slugs against the library, placeholders and calls.

In the web app a flow opens in a visual editor (a graph) beside its JSON. Debug… starts a debug run that pauses at breakpoints and at a failure; while paused you can look at the live screen, fix the library, and retry or skip the step.

Runs

A run is one version of a flow on one target. It opens its own recorded session (or uses a live session you pick), and its report lists each step with its status, time, attempts, the match it acted on, and for a failure, the error, the frame at that moment and the best candidate the matcher saw.

Statuses: queued (waiting for a free runner), leased and running, then one of passed, failed (a step failed, or the flow’s fail), error (the target or the setup broke) or cancelled.
Started by the web app (Run…), the CLI (attn run), the API and SDK, an agent’s flow_run (MCP), the SSH run command, and triggers.
Outputs and message: what the flow’s declared outputs held at the end, and the text of its pass step.
Concurrency: a hub runs up to four runs at once by default (ATTN_RUN_CONCURRENCY); more wait as queued.

Triggers

A trigger starts runs on its own: on an event, run a flow on a target, under a policy.

Events: a schedule (5-field cron with a time zone), a webhook (POST /api/v1/hooks/<token>), a target coming online (SSH targets), or another flow’s run finishing (to chain flows).
Params can take values from the event: {{event.body.user}} from a webhook’s JSON, {{event.run.outputs.total}} from the run that finished.
Policy: what happens while a run it started is still going (skip, queue or parallel), a cooldown, and retries for runs that fail.

Triggers live on a flow’s page (Add a trigger) and on the Triggers page. Webhooks covers the webhook trigger and the hub’s outgoing webhooks, which tell another system when a run ends. Not built yet: watchers, which would start a run when something appears on a screen.

The hub and runners

The hub holds everything above in its database and blob store, serves the web app and the API, and routes work to runners. A runner is where the screens are: it starts Chrome or connects to VNC, records frames, finds elements and sends input. A local hub (npm run dev) runs a runner in the same process; attn-runner runs one as its own process that connects out to a hub, so it can sit next to the screens it drives.