Long-horizon agents and durable execution

The agentic loop

Most agents, from coding agents such as Claude Code to personal
assistants such as OpenClaw, are built around the same loop, usually
called the agentic loop. It has four parts:

• a goal: the outcome we want the agent to achieve. For a
  coding agent, this might be a program to write; for Stock
  Replenisher, a part kept in stock.
• tools through which the agent acts in the world. Think of
  read_file, edit_file, and
  bash for a coding agent; get_quotes and
  place_order for Stock Replenisher.
• state: everything the agent knows about its progress
  toward the goal so far.
• a model that, at each turn, looks at the current state and
  chooses which tool to use next, and what to pass it.

At each turn, the model chooses a tool, and the code around it runs
the tool and updates the state with the result. That code is often
called the harness; Claude Code, for example, is a harness
around Claude. The loop ends when the agent reaches the goal or gives
up. When the agent needs input from a person or an external source, it
asks and waits for an answer.

Here is Stock Replenisher’s loop in code. (All code examples are in
Python.)

GOAL = ("Bring stock back above safety stock: order from a suitable "
        "supplier, get every price approved by the manager before money "
        "is committed, and see the order through to delivery.")
MAX_TURNS = 30

TOOLS = [
    GetQuotes(), DraftPurchaseOrder(), RequestApproval(),
    WaitForApproval(), PlaceOrder(), WaitForSupplierReply(),
    ReadSupplierReply(), WaitForGoods(), ClosePurchaseOrder(), Done(),
    ...
]

class Replenish:
    async def run(self, stock: StockLevel) -> ReplenishmentResult:
        state = State(stock.sku, stock.on_hand, stock.safety_stock)
        for turn in range(MAX_TURNS):
            allowed_tools = [t for t in TOOLS if t.allowed(state)]
            tool = await self.decider.decide(GOAL, state, allowed_tools)
            if isinstance(tool, Done):
                break
            if isinstance(tool, Wait):
                try:
                    payload = await asyncio.wait_for(
                        self.inbox[tool.event].get(),
                        timeout=timedelta(days=tool.days).total_seconds())
                    tool.apply(state, payload)
                except TimeoutError:
                    tool.apply(state, None)
            else:
                result = await tool.execute(state, self.context)
                tool.apply(state, result)
        return outcome(state)

decider.decide is the call to the model (in our case, a
classification model Jev); it picks the tool to run in
the current turn based on the GOAL, the current state of the replenishment run
(state), and a prefiltered list of allowed tools
allowed_tools. The loop ends when the model selects
Done or after MAX_TURNS. The
outcome shows how the replenishment concluded: received,
cancelled, rejected by the manager, or given up.

Each tool has a name and description, which the model uses to make
its selection. The allowed method states whether the tool
can be used given the replenishment’s current state. This way the
business rules that matter are in code, not in the prompt. For
instance, the place_order tool isn’t available until the
manager approves the price on the table. The execute
method runs the tool, e.g., makes API calls, writes an email, etc., and
apply updates state based on the
result of execute.

class PlaceOrder(Call):
    name = "place_order"
    description = "Send the approved purchase order to the supplier."

    def allowed(self, state):
        return (state.po is not None and state.order is None
                and state.approved_up_to is not None
                and state.price <= state.approved_up_to)

    async def execute(self, state, context):
        return await context.supplier.place_order(state.po)

    def apply(self, state, ack):
        state.order = ack

state holds everything the agent knows about one
replenishment: the quotes, purchase order, approved price range, and
order. Only the apply method updates it based on the
result of each turn.

@dataclass
class State:
    sku: str
    on_hand: int
    safety_stock: int
    quotes: list[Quote] | None = None
    po: PurchaseOrder | None = None
    price: float | None = None
    approved_up_to: float | None = None
    order: OrderAck | None = None
    goods: GoodsReceipt | None = None
    ...

Wait is also a tool that waits for a message, such as the
manager’s decision, or until its time runs out. This loop works as
long as the process running it stays up, but a lot can go wrong during
long wait times. And the agent needs to recover and continue its
task.

What can go wrong

It helps to think of a long-horizon agent as a distributed system.
Many of the failure modes we are familiar with in distributed systems
happen here too: calls to other systems fail, processes crash while the
agent waits, and so on.

The first four failures are the same in any domain. A durable
execution runtime handles them with general-purpose primitives so that
the harness can focus on domain-specific logic. Without the runtime,
you would have to write a lot of glue code to handle the failures
yourself: record what has been done, retry, set timers, and pick up
where you left off.

But treating the agent as just another distributed system misses
something. What makes it a different beast is the model at its heart,
which can decide how to recover from failures and keep the task going.
When a failure occurs, the harness’s job is to provide the model with
the right input so it can decide what to do next.

The next sections look at durable execution in detail. After that,
we return to the failures the harness must handle explicitly.

Durable execution

Durable execution is a programming model in which the
runtime keeps a record of the result of each step that interacts with
the outside world so that, when a failure occurs, the program runs
again, completed steps are skipped, and the program’s state is rebuilt
from the record.

Temporal, Restate, Inngest, DBOS, and Cloudflare Workflows all
implement it. We built Stock Replenisher on two of them, Temporal and
Cloudflare Workflows. The code in this article comes from the Temporal version.
Cloudflare appears where the two differ materially.

The definition rests on three ideas: steps, a log, and replay.

A step is a call to the outside world that the runtime
makes on the program’s behalf: the LLM call, a call to the ERP, a
message to the supplier. Anything that interacts with the outside world
goes through a step. The code between steps only reads from and writes
to local state. In Stock Replenisher, decider.decide and each
tool’s execute are steps; allowed and
apply are not.

Figure 2: Every call
to the outside world is a step

Before the program proceeds to the next step, the runtime writes
both the step’s input and output to a persistent log.

Figure 3: A step,
close-up

After a crash, the program restarts from the top (i.e., it is
replayed). Any step with a recorded output returns it from
the log rather than running again. The code
between steps runs again and rebuilds the program state, which in
Stock Replenisher is state. The first step without a
record runs for real.

Figure 4: After a crash, the program runs again; recorded steps don’t

This way the program can just rebuild its state from the log after
a crash.

  from temporalio import workflow
  
  @workflow.defn
  class Replenish:
      @workflow.run
      async def run(self, stock: StockLevel) -> ReplenishmentResult:
          state = State(stock.sku, stock.on_hand, stock.safety_stock)
          for turn in range(MAX_TURNS):
              allowed_tools = [t for t in TOOLS if t.allowed(state)]
              tool = await workflow.execute_activity(
                  Decider.decide, args=[GOAL, state, allowed_tools],
                  activity_id=f"decide-{turn}", ...)
              if isinstance(tool, Done):
                  break
              if isinstance(tool, Wait):
                  try:
                      await workflow.wait_condition(
                          lambda: self.inbox[tool.event],
                          timeout=timedelta(days=tool.days))
                      tool.apply(state, self.inbox[tool.event].pop(0))
                  except TimeoutError:
                      tool.apply(state, None)
              else:
                  result = await workflow.execute_activity(
                      run_tool, args=[tool, state],
                      activity_id=f"{tool.name}-{turn}", ...)
                  tool.apply(state, result)
          return outcome(state)

The entity and the workflow

We've seen the durable agentic loop. The next question is where
that loop lives and how it maps to the business domain.

Take stock replenishment: there are two loops at play. One handles
replenishment for a single SKU and follows the work through until the
goods are received. The other is the SKU-level loop. It checks stock
against the replenishment policy and triggers replenishment when
needed, one at a time. It is also the SKU's point of contact: messages
from the manager, the suppliers, and the warehouse come to it. The SKU
will be replenished many times over its life.

Figure 5: One
agent per SKU; one workflow at a time

This is the distinction between a process workflow and an
entity workflow.

A process workflow is a durable workflow that carries
out one piece of work (e.g., a replenishment) and then finishes.

An entity workflow is a durable workflow that
represents one business entity (e.g., a SKU) for the lifetime of that
entity. It maintains the entity's state and initiates process
workflows.

In a subscription business, customers fit the same pattern. The
customer's entity workflow maintains the customer context and
initiates the appropriate process workflows (onboarding, renewal,
support, etc.). Each of those process workflows completes, but the
customer entity workflow continues.

The entity workflow's first job is to watch the stock and trigger
replenishments.

the entity workflow: one per SKU

  @workflow.defn
  class SkuAgent:
      @workflow.run
      async def run(self, sku: str, outcomes: list[ReplenishmentResult]):
          stock = await workflow.execute_activity(get_stock, sku, ...)
          if stock.on_hand < stock.safety_stock:
              self.active = f"replenish-{sku}-{len(outcomes) + 1}"
              try:
                  result = await workflow.execute_child_workflow(
                      Replenish.run, stock, id=self.active)
              except ChildWorkflowError as e:
                  result = ReplenishmentResult(sku, "failed", str(e.cause))
              outcomes.append(result)
              self.active = None
          else:
              await workflow.sleep(CHECK_EVERY)
          workflow.continue_as_new(args=[sku, outcomes])

Each run of the entity workflow checks the stock once. If the SKU is
below safety stock, it starts a replenishment and waits for it to
finish; otherwise it sleeps until the next check. Then
continue_as_new starts a fresh run with the same ID and an
empty history, which receives only what it is given, here the SKU and
the outcomes so far.

The entity workflow's second job is to receive messages for the SKU
from the outside world. It has a stable, unique ID,
agent-SKU-123, for the SKU's life. Each request the agent
sends out, such as an approval request or a purchase order, carries the
SKU or the purchase order number so that the answer can be sent back
to agent-SKU-123. The manager approving or
the supplier replying doesn't need to know which replenishment is
running; the entity passes the message on to the current one.

the manager's approval, sent to SKU-123

  await client.get_workflow_handle("agent-SKU-123").signal(
      "deliver", args=["approval", {"approved": True}])

the entity's address: messages from outside

  class SkuAgent:
      ...
      @workflow.signal
      async def deliver(self, event: str, payload: dict):
          if self.active:
              await workflow.get_external_workflow_handle(self.active).signal(
                  Replenish.deliver, args=[event, payload])

the replenishment's inbox: messages from the entity

  @workflow.defn
  class Replenish:
      @workflow.run
      async def run(self, stock: StockLevel) -> ReplenishmentResult:
          state = State(stock.sku, stock.on_hand, stock.safety_stock)
          for turn in range(MAX_TURNS):
              ...
          return outcome(state)
  
      @workflow.signal
      def deliver(self, event: str, payload: dict):
          self.inbox[event].append(payload)

Every message for SKU-123 goes through SKU-123's entity workflow
to the running replenishment, which applies them one at a time.

In the code, Stock Replenisher is now two classes:
SkuAgent, the entity workflow, and Replenish,
the process workflow that runs the loop from earlier.

That brings us to the waiting.

Waiting for people and systems

As we've seen so far, long-horizon agents spend much of their time
waiting for humans or external systems. The durable execution runtime
provides primitives for waiting efficiently. While these agents wait,
the runtime ensures they don't hold a process in memory or use
compute.

This matters because, with the entity-workflow pattern we saw
earlier, every SKU will have its own workflow, and that workflow will
run as long as the SKU is stocked. A manufacturer with thousands of
parts will have thousands of agents. At any given time, almost all of
them will be idle or waiting. If a wait holds a process or memory, then
the cost increases in proportion to the number of SKUs and the length
of the waits, not in proportion to the work actually done.

workflow.wait_condition suspends the workflow until a
message of the right type is in the inbox, or the timeout expires. The
wait's timer is also logged, and so is the message that ends it. So a
wait survives a crash just as a step does.

Figure 6: The wait's timer and the message both go into the log

Waking up has a cost too. Rebuilding a workflow means replaying the
log, and that cost grows with the log's length. If wake-up is slow or
expensive, then having thousands of agents waiting is just as
infeasible as keeping thousands of processes open for weeks. In our
runs, waking a workflow took under three seconds and tens of
milliseconds of CPU.

If replay gets too slow, the next thing to try is a snapshot, so only the log after it has to be
replayed. This matters most for the entity, whose log would keep
growing as long as the SKU is stocked. On Temporal,
continue_as_new has the same effect: it carries the
entity's state into a fresh run with an empty history. On Cloudflare,
the entity is a Durable Object, which reads its current state from
storage and has no log to replay. A replenishment workflow doesn't need
a snapshot. Its log stays short because the run ends.

A wait ends in one of two ways: a message arrives, or the timeout
expires. If the supplier hasn't got back to us within seven days, then
wait_for_supplier_reply will mark the order as
overdue:

class WaitForSupplierReply(Wait):
    event, days = "supplier-reply", 7
    ...
    def apply(self, state, payload):
        if payload is None:
            state.supplier_overdue = True
        else:
            state.supplier_reply = SupplierReply(**payload).message

class ChaseSupplier(Call):
    ...
    def allowed(self, state):
        return (state.supplier_overdue and state.chases < MAX_CHASES
                and is_open(state))

That means chase_supplier is allowed, and the model
will see it as an option on its next turn.

In this run, the supplier goes quiet after the order is placed. Look
for the wait that ends with a timeout seven days later, then the chase
that follows it, and the reply a day after the chase.

Retrying failed calls, at least once

Calls to external systems can fail because of network issues or
because the receiving system is down. Durable execution runtimes
provide primitives to retry failed calls with configurable retry
policies, so they don't overload the receiving system.

RETRY = RetryPolicy(maximum_attempts=6,
                    initial_interval=timedelta(seconds=2),
                    backoff_coefficient=2)

@workflow.run
async def run(self, stock: StockLevel) -> ReplenishmentResult:
    ...
    result = await workflow.execute_activity(
        run_tool, args=[tool, state],
        activity_id=f"{tool.name}-{turn}", retry_policy=RETRY)
    ...

What the runtime can't do is make a retry safe. If the supplier
takes the order and the response is lost, no result is recorded, so
the runtime calls the supplier again. This is known as at-least-once semantics, as
opposed to exactly-once. This could result in duplicate
orders if not handled properly.

To get the effect of exactly once for a write, the receiver must
recognise repeats using an idempotency key, which the harness must
keep the same on every attempt and replay.

In the place_order tool, the PO number becomes the
idempotency key.

class PlaceOrder(Call):
    ...
    async def execute(self, state, context):
        ...
        key = state.po.po_number
        ack = await context.supplier.place_order(po, idempotency_key=key)
        return PlaceOrderResult(placed=True, price=price, ack=ack)

In the run below, the supplier's system is down on the first
attempt. The runtime retries with the same PO number as its
idempotency key. Look for the two attempts on place_order and the
single order at the supplier.