Tools it never trained on

Suppose you teach a model its tools during training. Now count the tools in the world: every internal API, every database whose schema changed last Tuesday, every service launched this morning. Each one needs a training run.

It does not scale, and it never will. That is not an engineering problem to grind down — the model’s knowledge stops on a date and the tail of tools is endless and always moving.

So invert it. The model does not need to know the tools. From the last chapter, all it needs is their descriptions — and descriptions are text, which we already know how to deliver: put them in the prompt.

Ask at the moment of use

So the sequence becomes: the application asks what is available, gets back a list of descriptions and schemas, and puts them in front of the model. Discovery at runtime, rather than capability baked into weights.

For that to work across tools nobody coordinated on, everyone has to agree on the shape of the answer. That agreement is the Model Context Protocol: a client (your application) talks to servers (whoever holds the capability), over ordinary JSON-RPC, in three kinds of thing —

tools — functions the model can invoke;
resources — data it can read, like files or records;
prompts — templates a server offers for common jobs.

Nothing here is clever. It is a plug shape. Its value is entirely that everybody uses the same one, so a tool written once works with any client — and a list can change during a conversation: a server that gains a tool announces it, the client re-reads the list, and the next prompt simply contains one more description.

An agent is a tool

Here is where it gets slightly recursive, and pleasantly so.

A tool is a name, a description and a schema, hiding some capability behind them. Nothing in that requires the thing behind the name to be a function. Put an entire agent back there — its own model, its own loop, its own tools — and describe it. The caller cannot tell, and does not need to.

That is the honest version of “multi-agent”: a main loop that can call a specialist the way it calls anything else. It buys real things — a focused prompt, a separate context that does not pollute the caller’s, maybe a cheaper model for the grunt work.

It also costs real things. Every agent is another loop making its own calls, so the compounding in chapter seven applies to each of them and then to the whole. A single well-prompted loop beats an org chart of agents more often than the diagrams suggest.

Go deeper: what a tool list costs you

Descriptions live in the context window, and they are re-sent on every single call. Fifty tools at eighty tokens each is four thousand tokens of overhead per turn, before the task has said anything.

Worse, selection gets harder as the menu grows — the same discrimination problem from the last chapter, now with fifty near-neighbours instead of four. Hence connecting only the servers a task needs, which is also, conveniently, the strongest defence against the tool being turned against you.

The loop now has tools, and every one of them returns text into a prompt that has a hard ceiling. Something has to decide what gets thrown away.

Sources · 2

Full bibliography →