Agent Design

RAG, Tools, or Long Context?

Three mechanisms for getting organisational knowledge into an agent, with three different failure modes. Answer the questions in priority order — the first two usually settle it — and watch the recommendation shift.

Q1How fresh must the answer be?
The highest-leverage question, and the one most often skipped.
Current system state — balance, order status, inventory decisive
Changes on a human timescale — policy, docs, contracts
Static or historical — does not change
Q2Does the answer need to be authoritative?
Acted upon, quoted to a customer as fact, or defensible in a dispute.
Yes — it must come from the system of record, auditably decisive
No — it informs a human who decides
Q3How large is the relevant corpus per request?
Ask what is relevant per request, not how much exists in total.
Fits comfortably in the window — one contract, one case file
Exceeds the window — selection is unavoidable
Q4Is the query pattern known or open-ended?
Enumerable question shapes versus "find anything relevant".
A small number of predictable shapes
Genuinely open-ended across a large corpus
Q5What are the access control requirements?
Frequently decisive, and frequently discovered late.
Different users see different subsets decisive
Uniform access for all consumers
Recommendation
Tools0
Retrieval0
Long context0

No answers yet

Work through the questions on the left. Most production systems end up composing more than one mechanism rather than picking a single winner.

Choose by which failure you can tolerate

Retrieval

Fails silently and plausibly

Wrong chunks, a missed document, or a superseded version produce a confident answer grounded in the wrong material. No error is raised. The chunk that says "effective until Q2 2025" reads identically to current policy unless something preserved that distinction.

Tools

Fails loudly and expensively

Errors, timeouts, and rate limits are visible — a genuine advantage — but they cost latency on every request, and the agent must handle the failure rather than merely receiving worse input.

Long context

Fails gradually and by omission

Material may be attended to unevenly, particularly mid-context. Nothing errors; quality degrades in ways that are hard to attribute. Then it fails hard at the boundary — one token over and something gets dropped.