Writing
Notes
Models, memory, and the boring infrastructure that makes them useful.
MoE is probably good enough.
I went looking for a cleaner kind of sparsity than mixture-of-experts and did not find one. For almost everything I ship, MoE is enough.
Memory is all you need.
Chat products grew a memory of you. Taken seriously at company scale, that pattern is how you get business intelligence that is actually useful.
What should AI models actually do?
Frontier models keep getting bigger so they can hold more of the world in their weights. I am not convinced that is the job.