What should AI models actually do?

The word of the year so far has been "agentic," and for good reason. Instead of a model relying on having all of the information in the world stuffed into its parameters, it can now call out to retrieve information or perform an action. Despite that, the largest models have kept getting larger, as if the only way to get smarter is to memorize more.

I keep coming back to the opposite move. Use external services for data. Feed knowledge into context when you need it. Let the model spend its capacity on the part that is actually hard: deciding what to fetch, what to ignore, and how to put the pieces together.

Where would you draw the line between what the model needs to know and what it should retrieve?

I spent a long time working on DNS. A nameserver that tried to store the internet would be a joke. The whole system works because the thing answering the query is small, fast, and willing to go look something up. The intelligence is in the lookup path, the cache, the policy, and the refusal to pretend you already know.

Models are being built like the opposite of that. We bake encyclopedias into weights, then act surprised when the encyclopedia is stale, expensive to update, and confident about the wrong year. Retrieval exists. Tools exist. We still train as if the only trustworthy memory is the one we melted into the matrix.

I do not think the model should know your company's inventory, or yesterday's incident, or the current price of anything. I do not think it should know the contents of a PDF you uploaded last Tuesday by having trained on it in some vague sense. That is a storage problem with a retrieval API. The model should know how to ask, how to notice a conflict, and how to stop when the source is bad.

What does belong in the weights, then?

Judgment. Procedure. Taste about what a good answer looks like. The ability to write a plan, check it, and revise it. A feel for when a tool result is incomplete. Language, math-ish reasoning, the shape of software, the shape of an argument. The stuff that does not go stale every time someone ships a new version of a product.

That still leaves a messy middle. Some facts are so common that retrieving them is theater. Some procedures are so specific that you should not pretend the base model knows them. I would rather be explicit about the messy middle than pretend a 400-billion-parameter blob has solved it.

If you take this seriously, a lot of the current race looks like the wrong leaderboard. Bigger perplexity on a static crawl is not the same as being useful on Tuesday morning with a messy inbox, a private wiki, and a production system that can actually break. Usefulness looks more like:

A small enough core that you can run it where the data already lives.
A memory you can inspect, edit, and delete.
Tools with boring contracts, not magic.
An evaluation loop against the jobs you actually care about, not a general "vibe."

I work on products where the model is not the product. The product is a six-minute session, or a household widget, or a coach that has to remember what already failed. The moment the model has to be a world oracle, the product gets worse. The moment it has to be a reliable worker with a file cabinet, things start to click.

So, what should AI models actually do? Not hold the world. Hold the method for moving through it. Know when to look something up, when to act, and when to say they do not know. The rest is infrastructure, and we already know how to build infrastructure if we stop asking the model to be the database.