Stanford researchers compress LLM memory into 16 tokens and beat full context on five benchmarks
Fallahpour and colleagues at Stanford report 44.6% on PersonaMem-v2 against 30.9% for the next best method, using 16 tokens instead of the full collection.
By Ryan Merket · Published
Primary source: arXiv
Why it matters
Memory is where personalization and agent systems currently pay full-context prices or accept retrieval misses. A trainable module that reads an entire collection and answers in 16 tokens would cut inference cost sharply if it transfers beyond Qwen models and benchmark collections. The authors' own results show the trade: learned memory connects weakly related facts better than RAG and loses exact names that RAG keeps verbatim, which points to hybrid systems rather than a replacement.
Adibvafa Fallahpour (@adibvafa) and six co-authors released Memorilla on November 5th, a research system that compresses everything a frozen large language model needs to remember into 16 query-dependent memory tokens. In a thread on X, Fallahpour described the approach as "memory as a modality for LLMs," with the tokens fed into the model in the same position image embeddings occupy in a multimodal system.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.