Tasks, Schedules, and Actions
Extend durable execution while preserving ownership, retries, human handoffs, and external-effect safety.
Add or Change Task Behavior
- Keep task input on the task specification rather than duplicating prompt payload in run requests.
- Allocate immutable organization-local identifiers transactionally from the owning department.
- Add dependencies only through the cycle-checking service.
- Keep dependency conditions explicit: success, completion, or artifact.
- Start only due, allowed, dependency-ready work with no conflicting active execution.
- Replace a failed required cross-root edge with one bounded recovery review. Keep same-owner recovery in that department and route cross-owner recovery to Operator.
- Treat an integration task input as required unless it explicitly sets
required: false.
A missing required integration does not run the model and does not consume aggregate run budget.
The worker creates one idempotent Human Inbox question and finalizes the attempt as
waiting_human; answering the blocker resumes through a fresh request. Optional integrations may
be absent without blocking execution.
Completion Contracts
Use backend-decidable criteria wherever the postcondition is mechanical:
output_filematches a safeoutputs/path and may check size, SHA-256, detected media type, bounded UTF-8 text, or RFC 6901 JSON Pointer values;command_exitmatches the exact command hash and expected exit code; andintegration_callmatches connector key, tool name, and canonical arguments when supplied.
The trusted worker derives sanitized receipts from actual harness events and synchronized files. The backend rechecks that a file receipt points inside the active run workspace, reads the canonical object, and evaluates the contract before the task transition. Text and JSON inspection is limited to 1 MiB. Media type comes from file bytes, never the extension or an agent claim.
The worker does not stage a second execution result before completion. The completion transaction
creates the immutable execution outcome from the exact post-sync summary and artifact list, then
evaluates criteria and advances the task. Agent runs may stage only branch-specific semantic
outcomes. An already-staged trusted execution result remains subject to exact equality with the
worker-retained result, so stale or conflicting output cannot be accepted.
An empty criteria list closes from a successful execution result and must not create a verify run.
Use it when the operation is self-verifying and no separate postcondition adds value. Legacy text,
metric, and artifact criteria remain compatible as display-only expectations. They neither gate
completion nor create another run. If semantic judgment must gate downstream work, create an
explicit review task with a bounded evidence contract.
Verification remains available for profile learning and retained pre-cutover task states. It uses the ordinary unrestricted harness, but the prompt prohibits workspace mutation and external effects; the manifest has no output directory, integration grants, MCP servers, or network; workspace sync rejects every change; and backend authorization accepts reads plus the one mode-legal verification outcome only.
The first failed completion report uses one automatic execution repair when budget remains. A repeated failure creates one idempotent Human Inbox repair request; answering queues a fresh versioned execution. Because this question follows completed execution, it does not refund that run’s consumed credit. Budget exhaustion produces a terminal failed result with the completion report instead of another loop.
Run Requests and Attempts
run_requests hold task-only mutable queue/retry intent. runs are immutable attempts. Workers
create attempts and increment attempt counts. Scheduler and runbeat never create attempts.
Every task attempt receives a backend-authored prompt that names its selected run mode and exact Task Runtime exit. If an agent-controlled semantic phase returns without its required typed outcome, the backend rejects completion and the worker records a retryable protocol failure. Runbeat retries that same phase only within its existing attempt and task-run budgets.
tasks.agentRunCount is total immutable attempt history. tasks.consumedAgentRunCount is the
aggregate budget counter used at claim time. Claim increments both in the same transaction;
successful human_interrupt and action_interrupt finalization refunds only the consumed counter.
Failures and backend-opened completion repair waits remain charged. Do not infer either count from
Redis delivery.
A deterministic required-integration blocker is also finalized through the worker boundary. The
backend derives its human_interrupt outcome from the exact waiting output and staged question in
the same transaction; no model run or competing outcome write is involved.
Do not accept maxCostMinorUnits until the runtime records actual metered cost. A migration removes
legacy values instead of enforcing a synthetic zero-cost counter.
Late writes must include and satisfy the relevant lease, state, and version checks. The backend issues lease expiry and heartbeat cadence; workers never submit their own expiry. Preserve failed and timed-out attempts.
Public run reads use a dedicated joined projection containing task/profile identity, attempt, mode, status, outcome, error, usage, and timestamps. Never return the execution prompt, hydrated context, working state, worker/lease fields, or workspace/checkpoint/trajectory/log locations through the owner API. Run-history cursors bind the sort key, direction, value, and run ID so changing sort order cannot reuse a stale cursor.
Admission locks the organization and profile rows before counting starting and running
attempts. A full organization or profile returns a capacity conflict before attempt insertion.
Runbeat selects active attempts by canonical leaseExpiresAt, closes an expired run, cancels its
staged questions/actions, removes its active task reference, and moves the request to backoff in one
PostgreSQL transaction. Exhaustion fails the request and task in that transaction. The same
transactional recovery also closes legacy running tasks whose current request already exhausted
before task finalization. Failed-run repair only transitions claimed requests once; the scheduler
owns later due publication. Background pagination must use the same key as its cursor.
Add a Schedule
- Define owner, cadence, run mode, and expected task outcome.
- Store
nextRunAtas the schedule cursor. - Let the scheduler create normal tasks and run requests.
- Publish a wakeup only after durable state changes.
- Make duplicate scans and wakeups harmless.
- Keep healthy polls quiet.
- Mark built-in schedules as
system; persist the creating profile and exact run/chat provenance foragentschedules. - Coalesce missed intervals into one catch-up firing, then advance to the next future occurrence.
- Back off firing failures exponentially. Disable an agent-created schedule after five consecutive failures. Keep a built-in system schedule enabled, mark it as requiring attention, and continue bounded retries.
- Test pause, resume, due calculation, duplicates, backoff, provenance, and restart recovery.
System schedules remain organization-owned and read-only to agent execution capabilities. Department agents may schedule only their own department, Operator may delegate any department, and Chat may schedule work only from an explicit active user request.
Backend startup reconciles required built-in schedules for each active organization. It creates missing schedules, repairs stale templates or disabled cursors, and disables orphaned profile learning schedules. During each scheduler tick, due run-request retries publish before schedule firing, and one failing phase cannot starve later task repair.
Schedule lists accept status, origin, profile, and department filters. Their opaque cursor binds the selected ordering and explicitly preserves PostgreSQL null ordering for optional next/last run times; never combine an ID-only cursor with a different sort column.
Add a Human Question
Stage a typed blocker during an active run. Open it only when run finalization moves the task to waiting-human. The response removes that exact blocker, advances the task version, and queues fresh execution only when no blockers remain.
Cancel Task Work
Cancellation is transactional across the requested task and every active descendant. Cancel their queued requests, active attempts, staged/open questions, and actions that have not started. Publish one best-effort abort signal per active run after commit; durable run state remains authoritative. Preserve terminal descendants and action outcomes that are already executing or uncertain.
Resolve outgoing dependency edges by their declared condition: cancellation satisfies
completion, fails success and artifact, and blocks a dependent only when an unresolved edge is
required. A later action reconciliation must never reopen a terminal task.
A failed required dependency between separate roots creates one version-bound review task. The
failed edge remains as non-required history and a required completion edge points to the recovery
review. Completing, failing, or cancelling that bounded review releases the dependent for a fresh
decision; it does not recursively create another recovery. Dependencies inside one failing root do
not create orphan recovery work because the root closes together.
Automatic replans stop at the task budget, which is at most three. An amendment at that boundary opens one idempotent human question and refunds that waiting run credit. If the human authorizes a final amendment, the next review may apply it without increasing the bounded replan counter and marks the override used. Any later amendment fails the task deterministically instead of opening another question or leaving run finalization in conflict.
Add a Consequential Action
- Define typed arguments and result.
- Establish stable idempotency.
- Snapshot authorization epoch, task version, risk, and policy.
- Wait for approval or acquire a fenced execution lease.
- Attempt the provider call once.
- Store a redacted receipt for confirmed outcomes.
- Mark uncertain outcomes unknown.
- Reconcile through a human without replaying the effect.
Serialize action admission for each task before enforcing its hard action budget. Separately count
organization action volume by the organization’s local calendar day. Reaching maxDailyActions
creates one idempotent, non-blocking notification for that day; it does not reject the action.
Serialize the daily count so concurrent actions cannot cross the threshold silently.
Recovery Tests
Cover duplicate wakeups, missed wakeups, stale heartbeats, exhausted attempts, late worker writes, refunded waits, output-sync failure, a 100-file output set, a rejected 101-file output set with zero uploads, missing required versus optional integrations, cross-organization capabilities, cross-department/root mutations, multiple blockers, exhausted replans, the single replan override, stale approvals, and abandoned action leases. Also cover cursor reuse under a different sort, nullable schedule-date pagination, and public run projection redaction. Completion coverage must include empty criteria, mixed formal and display-only criteria, absent or spoofed receipts, byte-detected media, the 1 MiB structured-file bound, exact static matching, one automatic repair, repeated-failure human repair, budget exhaustion, retained-state compatibility, and verification capability fencing.