jmd.gg
← Writing

Memory is all you need.

Here is a thought experiment I keep running. An email arrives early in the morning. Subject line: “Congrats on your new role at Anthr-oogle-AI” (no relation). Somebody has decided I am the person who gets to point the money cannon. What do I point it at?

Not a new attention variant. Memory.

Chat products like ChatGPT now keep notes on you. For one user that feels like a parlor trick: the assistant remembers your dog’s name and that you like terse answers. Cute. At the scale of a company, the same pattern is the first version of business intelligence that does not make you open a dashboard and squint.

Most “enterprise AI” still works like a very expensive intern on day one. No memory of the last incident. No memory of the decision you already made. No memory of the customer who churned because someone promised a date and missed it. Every session starts from a system prompt and a prayer, and then we blame the model when it contradicts last week’s email.

A useful system writes things down.

Not a transcript dump. A memory. Structured enough to retrieve, explicit enough that contradictions surface, and editable enough that a human can say “that is wrong” without retraining a 70-billion-parameter creature. The write path matters as much as the read path. If the model cannot form a clean note, you will retrieve sludge. If it cannot forget, yesterday’s bug becomes a permanent fact.

I build consumer products where this is not abstract. A posture score is a memory of a body on a given day. A household pad is a memory of who already took out the trash. A coach that does not remember the serum that burned you last month is not a coach. It is a slot machine with better copy.

The companies that get this will stop asking the model to be clever in a vacuum and start asking it to be consistent with a file it can point to. “We tried azelaic acid in March. Redness went down. You stopped because of the texture.” That sentence is worth more than another point of MMLU.

There is a research version of this and a product version. The research version is long-term memory and continual learning: whether the weights themselves should change as you live with a model. I care about that. I also think we will get 80% of the value from a boring store, a decent retriever, and a model that is honest about what it wrote down. Continual learning can wait until the filing cabinet works.

The hard parts are not romantic:

  • What is worth writing down? If you write everything, you have logs, not memory.
  • What happens when two notes disagree?
  • Who is allowed to read a note? Memory is how products accidentally become surveillance.
  • How do you show the memory made things better, not just bigger?

I would fund those questions before I funded another general chatbot that smiles and forgets.

So, back to the email. Handed the money cannon for a day, I would point a surprising fraction of the lab at memory: the write path, the conflict path, the forget path, and the evals that prove a returning user has a better week than a new one. Pretraining can be bought. A system that knows your organization, and can be corrected when it is wrong, cannot.

Attention was never all you needed. It is the engine. Memory is the job.

Justin DaCosta builds training systems for Amazon Nova and ships iOS apps on the side.