Skip to content

Attention, Evaluation, and Operations ​

TL;DR ​

A standing agent's output is not a stream that someone watches. It is an occasional message to a person who was doing something else, and that person's attention is the scarcest resource the system consumes. Every run therefore ends in one of three ways: a message to the outbox, nothing to send, or a deferred request for input. The outbox is a component with a budget, and the default is quiet. An agent starts by saying little and earns more of its principal's attention as its messages prove worth reading.

Evaluation has to change shape as well, because there is no prompt to grade an answer against. We evaluate policies over event streams: recorded or synthetic timelines are replayed against the agent on a simulated clock, and we score what it did, what it should have done and did not, what it said, and what it cost. Long-horizon defects such as memory drift, growing cost per wake, and creeping notification volume only appear over weeks of simulated time.

Operationally the agent is a service. It has service-level objectives, traces per run, an alert on silence, a cost model in which idle time is free and overhead is paid on every wake, and a rollout process that treats a prompt or model change to ten thousand living agents as a fleet migration.


The Outbox ​

In a supervised session the user sees every step, so the agent's communication problem is solved by the interface. Background work inverts this. The user receives the outcome and not the process, and what the agent chooses to send is the whole of their experience of it.

A run can end three ways. It can produce a message, it can produce nothing, or it can stop on a deferred call that asks for a decision, as chapter 5 described. "Nothing" is a legitimate and common ending, and it must be an explicit one: the run records that it considered the situation and judged that nobody needed to hear about it. An agent that feels obliged to report every wake teaches its principal to ignore it within a week.

For many agents a single send tool is enough. A formal outbox, a durable table of pending messages and a dispatcher that delivers them, becomes worth building when any of the following is needed: pacing and batching, retry after a channel failure without sending twice, a budget enforced outside the model, or one view of everything waiting on a person.

Message Classes ​

Outbound messages fall into classes with different costs to the reader, and the outbox routes by class.

ClassMeaningChannelBudgeted
InterruptA human decision or action is needed nowPage or pushTightly: a few per day
Decision requestA deferred call awaiting approvalInbox card showing the exact actionYes
QuestionThe agent cannot proceed without informationInboxYes
ReportWork was done; here is the outcomeDigestBy size
Activity recordWhat the agent did and whyPull only: an activity pageNo

The interrupt class is the one to protect. The reasoning is the same as for paging in Alert Evaluation & Notification: an interrupt should be urgent, should require a human, and should be something the human can act on. Everything else waits for a digest or an inbox the person visits when they choose.

The Interruption Budget ​

The mandate's attention block is enforced by the dispatcher, outside the model, as a token bucket per recipient. When the budget is spent, further interrupts are demoted to the digest with a note that they were demoted. The agent is told its remaining budget each run so that it can plan: an agent that knows it has one interrupt left today behaves differently from one that believes interrupts are free.

Research on proactive agents supports the practice. Work that frames proactivity as a gated decision about whether and when to intervene measures it by precision, recall, and false-alarm rate. Work on interruption budgets reports that capping clarifying questions at a small number preserved most task reward while cutting attention cost substantially. Asking fewer, better questions is close to free in outcome terms and valuable in attention terms.

Three further mechanisms keep volume down.

Batching. Several pending items for one person become one message. A digest at a fixed time is the simplest batch, and most reports belong in it.

Deduplication across agents. A person served by four agents has one attention budget, not four. When several agents can reach the same person, a single dispatcher owns the budget, and it merges or suppresses messages about the same subject. Without a shared owner, each team tunes its own agent to be reasonable and the sum is not.

Start quiet. A new agent begins with a low budget and a digest-first policy. Budget increases are earned from evidence: the fraction of its messages that were opened and acted on, and explicit feedback. Trust built this way is durable, and trust lost to a noisy first week is hard to recover.

Questions That Go Unanswered ​

A standing agent will ask questions nobody answers. Every question and decision request therefore carries three things: a deadline, a default the agent will take when the deadline passes, and a statement of that default in the request itself, such as "I will leave the meeting as it is unless you reply by 17:00." The default is the conservative option. Open requests are listed in one place, withdrawn when they become moot, and counted, because a growing backlog of unanswered questions means the agent is asking the wrong person or asking too much.

Evaluating an Agent With No Prompt ​

LLM Evaluation covers grading outputs and trajectories against a task. A standing agent has no task in that sense. Its input is a timeline, its output is a set of actions and messages distributed over that timeline, and inaction is frequently the right answer. The unit of evaluation is the scenario: an initial state of memory and world, a mandate, a sequence of timestamped events, and a rubric.

Timeline Replay ​

The harness runs the real agent, with its real wake path, against a simulated environment and a virtual clock.

python
def run_scenario(agent, scenario) -> Scorecard:
    world = SimulatedWorld(scenario.initial_state)     # stub tools with state
    clock = VirtualClock(scenario.start)
    agent.restore(scenario.memory_snapshot, scenario.mandate)

    for event in scenario.timeline:                    # timestamped events
        clock.advance_to(event.time)                   # fires due timers and sweeps
        agent.ingress(event)
        agent.run_until_idle(world, clock)             # gate, turns, outbox, timers

    clock.advance_to(scenario.end)                     # let follow-ups fire
    return score(
        actions=world.effects,                         # what it did
        messages=agent.outbox.sent,                    # what it said, and to whom
        expected=scenario.rubric,                      # must-act, must-not-act, may-act
        spend=agent.ledger,                            # tokens and wakes
    )

The virtual clock is what makes this practical. Timers, sweeps, debounce windows, cache expiry, and approval timeouts all read the clock, so a simulated month runs in the time its turns take and not in a month. The rubric has three lists and not one: conditions where the agent must act, conditions where it must not, and a wide remainder where either is acceptable. Scoring only required actions rewards an agent that acts on everything.

What to Score ​

DimensionMeasureFailure it exposes
Action recallRequired actions taken, within their freshness boundTimidity; gate false negatives; lost events
Action precisionActions taken that were permitted by the rubricOverreach
Silence precisionRuns that correctly ended with nothing to sendNoise
Interrupt precisionInterrupts that a reviewer agrees needed a human thenFatigue
Commitment keepingFollow-ups the agent promised and performed on timeLost or mis-set timers
ContainmentInjection scenarios where no gated effect escapedSecurity regressions
CostTokens and wakes per scenario, and per useful outcomeHeartbeat waste; context bloat

Scenarios come from three sources. Production session logs, with consent and redaction, give realistic timelines and are replayed with the recorded observations. Hand-written scenarios cover the conditions that matter and are rare: the dropped webhook, the contradictory instruction, the hostile email, the principal who does not answer. Synthetic timelines generated by a model and filtered by people give volume. Message volume in particular should be tuned with evaluations before it is tuned on users.

Soak Tests ​

Some defects only exist at length. Core memory grows until it crowds the task. Consolidation slowly rewords a preference into something else. Cost per wake climbs as the always-loaded overhead accumulates. A timer leak adds a wake a day. None of these appears in a scenario of twenty events.

A soak test runs months of simulated time, with consolidation and compaction included, and tracks trends: tokens per wake, core memory size, pending timers, messages per day, and the score on a fixed probe set asked at intervals. The pass condition is that the trends are flat. Soak tests are slow and expensive, so they run on release candidates for the model, prompt, and memory system and not on every change.

Online Evaluation ​

Offline scores are a gate, and production is the measure. The signals available online are the principal's reactions: messages opened, approvals granted or denied, actions undone, explicit feedback, and budgets the owner raised or lowered. An action the user reverses is the closest thing a standing agent has to a labeled error. A sampled audit, in which a reviewer or a judge model reads a random selection of runs including silent ones, finds the missed actions that no reaction can reveal.

Observability ​

The trace unit is the run. One trace spans the wake event, the gate decision, each model call and tool call, the policy decisions, memory reads and writes, and the outcome, and it carries the agent, mandate version, session, and model version as attributes. Tracing & Telemetry Pipelines applies unchanged. The OpenTelemetry generative-AI semantic conventions define attribute names for model and agent spans. They were still marked as in development at the time of writing, with the agent and tool spans the least settled, so pin the convention version and expect to migrate.

A small set of metrics describes the health of a fleet.

SignalWhy it is watched
Wake completeness per agentSilence is the failure mode; alert when expected runs do not happen
Reaction freshness, by wake classThe primary user-facing latency
Gate skip rate and sampled skip recallCost control, and whether it is skipping real work
Tokens per wake, split into fixed overhead and taskOverhead growth is the slow cost leak
Cached-token fractionA drop means the prefix became unstable or cadence outran the cache
Runs ended by budget, by lease loss, by errorEach has a different owner
Deferred calls open, age, approval rate, decision timeFatigue and stuck work
Interrupts per recipient per day, and acted-on rateAttention spend
Memory writes per day, quarantined writes, core sizeDrift and poisoning
Pending timers per agentLeaks

The alert that matters most is for wake completeness, because a standing agent that stops waking fails silently, as chapter 1 argued. The checker that compares expected wakes with completed runs must not share the scheduler's failure domain, or the outage that stops the agent also stops the alert.

Economics ​

A sleeping agent spends no tokens. The bill is therefore set by how often it wakes, what every wake costs before any work is done, and how much real work there is.

python
def monthly_cost(wakes, gate_pass_rate, overhead_tokens, cached_fraction,
                 task_in, task_out, gate_cost, p_in, p_cached, p_out):
    full_turns = wakes * gate_pass_rate
    overhead = overhead_tokens * (cached_fraction * p_cached
                                  + (1 - cached_fraction) * p_in)
    per_turn = overhead + task_in * p_in + task_out * p_out
    return wakes * gate_cost + full_turns * per_turn

Four levers follow from the terms.

Wake fewer times. Events over heartbeats, a state-diff gate, and adaptive timers, from chapter 2, reduce wakes and gate_pass_rate.

Shrink the overhead. The system prompt, tool definitions, and always-loaded memory are paid on every run. An agent with forty tool definitions and a long memory block may spend more on overhead than on work. Load tool definitions and memory bodies on demand, and keep the fixed prefix lean.

Keep the overhead cached. A stable prefix read from cache is billed at a fraction of the input price. cached_fraction depends on the prefix not changing and on wakes arriving within the cache's lifetime. This produces a non-obvious interaction with cadence: an agent that wakes just outside the cache lifetime pays full price every time, and it can be cheaper to wake slightly more often, or to batch work into fewer, longer runs, than to sit at that boundary.

Tier the model. The gate, routine triage, and memory housekeeping run on a small model. The large model is for runs that need judgment. Model prices have also been falling quickly for the same capability, and published comparisons between successive model generations show large reductions in cost per solved task, so a cost model should be re-derived when the model changes and not carried forward.

Report cost per useful outcome and not per token or per run. An agent that costs twice as much per run and interrupts half as often may be the better system.

Operating a Fleet ​

Rollouts. A change to the prompt, model, tools, or memory system goes out by cohort, as chapter 3 described, with the service-level indicators compared against a control cohort. Progressive Delivery supplies the mechanics. The observation window has to cover the agents' natural cycle, and attention metrics are guardrails: a version that completes more work by interrupting twice as often is a regression.

Configuration drift. Agents provisioned at different times accumulate different prompt versions, tool sets, and memory formats. An inventory records, for each agent, its owner, mandate version, model and prompt versions, granted scopes, last run, and mandate expiry. The inventory is how a fleet stop is targeted and how orphaned agents are found.

Incidents. The first action in an agent incident is to pause, by agent or by cohort, which is safe because the mailbox keeps accepting events. Then the session log and gateway log establish what was done, and the provenance fields from chapter 4 establish what was learned from the offending input. Remediation includes reversing effects, retracting memory records derived from the cause, and cancelling timers set during the affected runs. Resume is staged, with a coalesced catch-up and not a replay of every missed wake. Incident Command & Learning covers the process around this.

Capacity. Fleet demand is the sum of many low-duty-cycle agents, which is smooth on average and correlated in bursts: a shared upstream outage wakes every agent that watches it. Capacity planning uses the wake-rate distribution per mandate type and a correlated-wake scenario, and the scheduler's priority classes decide what is delayed when the scenario occurs.

Failure Modes ​

FailureMechanismDetectionMitigation
Notification fatigueReports sent as interrupts; every run sends somethingActed-on rate falls; users mute the agentMessage classes; dispatcher-enforced budget; start quiet
Over-silenceAgent suppresses something that needed a humanSampled audit; late discovery by the principalMust-act scenarios; interrupt recall tracked with precision
Stuck on a questionUnanswered request blocks work indefinitelyOpen request ageDeadline and stated default on every request
Attention fragmentationSeveral agents each within budget; sum is notPer-person totals across agentsOne dispatcher owns the recipient's budget
Duplicate sendDispatcher retries after an unknown delivery resultRecipients see repeatsIdempotency key per outbox row; channel-side dedupe
Evaluation rewards actionRubric lists only required actionsHigh recall, low precision in productionMust-not-act and may-act lists; silence precision
Slow driftMemory, overhead, or volume grows over monthsSoak-test trends; production trend alertsSoak tests on release candidates; caps
Silent fleet failureScheduler fault stops wakes; dashboards look quietIndependent expected-versus-actual checkAbsence alerts outside the scheduler's failure domain
Cost cliffCadence crosses the cache lifetime, or overhead growsCached fraction drops; tokens per wake riseStable prefix; cadence chosen with cache lifetime in mind
Rollout regression in attentionNew version interrupts more to finish moreGuardrail metrics by cohortAttention metrics as rollout gates

Decision Framework ​

Decide the attention design before the agent has users. A single send tool with a daily digest is right for one agent and one principal. Move to an outbox with a dispatcher when there are budgets to enforce, channels that fail, or more than one agent per person.

Invest in evaluation in proportion to authority. An agent that only reads and reports needs scenario tests for recall and noise. An agent that acts needs must-not-act scenarios and injection scenarios before it is given the tools. An agent with memory and a long life needs soak tests before each model or memory change.

Set objectives from the mandate. Reaction freshness comes from what the principal loses when the agent is late; the interrupt budget comes from what they will tolerate; the cost ceiling comes from what the outcomes are worth. An agent that cannot meet all three at once has the wrong mandate, and narrowing the mandate is usually the right fix.

Key Takeaways ​

  • Human attention is the resource a standing agent spends. Runs end with a message, with nothing, or with a deferred request, and silence is a recorded, legitimate outcome.
  • Classify outbound messages, route by class, and enforce an interruption budget in a dispatcher outside the model. Start quiet and earn more.
  • One dispatcher should own each person's attention across all the agents that can reach them.
  • Every question carries a deadline and a stated conservative default.
  • Evaluate policies over timelines, on a virtual clock, with must-act, must-not-act, and may-act rubrics. Score silence and interrupts as well as actions.
  • Soak-test for drift over simulated months before changing the model, prompt, or memory system.
  • Trace per run, alert on absent runs from outside the scheduler's failure domain, and watch fixed overhead and cached fraction as the leading cost signals.
  • Idle is free. Cost is wakes times per-wake overhead plus real work, and cadence interacts with cache lifetime.
  • Operate the fleet with cohort rollouts, an inventory with owners and expiries, and pause as the first incident action.

References ​


A practical reference for distributed system design. Released under the MIT License.