The Agent Loop — From LLM to Agent

The Core Distinction

Completion Engine

LLM

Receives tokens, predicts the next ones. Single inference call, no persistence, no action beyond text generation.

📝 Receive prompt

⚙️ Single inference

💬 Generate text response

🔚 Done — state discarded

System That Acts

Agent

Receives a goal, formulates a plan, executes via tools, observes results, and adapts. Persistent loop pursuing objectives.

👁 Observe — gather state & context

🧠 Plan — determine next action

🔧 Act — invoke tools, produce output

🔄 Loop — until goal met or escalation

The Agent Loop

Observe → Plan → Act

The defining structural pattern that transforms a stateless inference call into a process that pursues objectives across multiple steps.

Phase 1

Observe

Gather information about current state and environment. Context window constraints determine what the agent can attend to.

Read latest user message or tool result

Query knowledge bases for context

Check workflow state and progress

Retrieve relevant memory

Phase 2

Plan

Determine what to do next. Can be implicit (next-token prediction) or explicit (structured decomposition of complex tasks).

Decompose goal into sub-tasks

Evaluate candidate approaches

Select tools to invoke

Assess whether to escalate

Phase 3

Act

Execute the plan. This is where tool use becomes essential — without tools, the agent can only generate text about what it would do.

Invoke APIs and external systems

Execute code and computations

Produce structured outputs

Update state and memory

Repeat until goal met · stalled · or escalated

Four Ingredients

What Transforms an LLM into an Agent

Individually necessary, collectively sufficient. Remove any one and the system degrades.

Planning

Decompose goals into steps and revise as information arrives. Gives the agent directionality.

Without it

Reactive — responds but doesn't pursue

Tool Access

Interact with systems beyond the model's parameters: APIs, databases, code execution.

Without it

Theoretical — talks but can't act

Memory

Retain and retrieve information across interactions, beyond the context window's limits.

Without it

Amnesiac — forgets everything between calls

Autonomy

Calibrated degree of independence. Which decisions the agent makes alone vs. which require human approval.

Without it

Inert — needs approval for every action

Orchestration Patterns

Workflow

Deterministic sequences with explicit branching. Each step predefined, high auditability.

Flexibility Low — predefined paths

Auditability High — fully traceable

Best for Regulated processes

Agentic

A coordinator agent dynamically plans which agents to invoke, in what order, based on the task.

Flexibility High — adaptive routing

Auditability Medium — dynamic paths

Best for Complex, variable tasks

Choreography

No central coordinator. Independent agents react to events and each other's outputs.

Flexibility Very high — event-driven

Auditability Lower — distributed paths

Best for High-scale, loose coupling