Microsoft releases EvoLib for LLM agents that learn without weight updates

Why it matters

EvoLib gives agent builders a way to reuse lessons across tasks when model weights are inaccessible, but its value depends on reliable self-evaluation and production testing. ([github.com](https://github.com/microsoft/EvoLib))

Microsoft Research released EvoLib on July 30th, a test-time learning framework designed to help large language models improve across tasks without changing their underlying weights. The announcement follows a paper first posted on May 14th and revised on July 14th, with Microsoft also publishing a technical explainer and the EvoLib code. (microsoft.com)

https://x.com/msftresearch/status/2082860019016438152?s=46

The seven paper authors are Weijia Xu, Alessandro Sordoni, Chandan Singh, Zelalem Gero, Michel Galley, Xingdi Yuan and Jianfeng Gao. First author Weijia Xu is a senior researcher at Microsoft Research Redmond who earned her computer science doctorate at the University of Maryland, College Park. Her work focuses on language models that can plan, act and interact under changing goals. Jianfeng Gao, a Microsoft technical fellow and corporate vice president, sets company-wide direction for AI models used across products including Bing, Azure AI, Microsoft 365 and Copilot. (microsoft.com)

Turning agent histories into reusable knowledge

EvoLib addresses a weakness in many agent memory systems: retaining a transcript, reasoning trace or previous action does not ensure that a model can extract a useful lesson from it. Large archives can also make retrieval harder as similar, outdated or narrowly applicable records accumulate.

The framework processes a model's inference trajectories into two forms of knowledge. "Modular skills" are reusable procedures, such as functions for coding or subtask workflows for an agent. "Reflective insights" are natural-language rules distilled from failed attempts. The library is shared across problem instances, allowing knowledge extracted during one task to influence later tasks. (microsoft.com)

EvoLib consolidates similar entries rather than allowing the library to expand as an unstructured log. It also assigns weights using Information Gain and Future Information Gain. The first score measures whether an abstraction helps on the current task. The second gives credit to abstractions that later produce other useful knowledge. Entries with greater measured value become more likely to appear in subsequent prompts. (github.com)

That architecture matters for developers using closed models through APIs. EvoLib does not require access to model parameters or gradient updates, so Microsoft says it can operate around black-box LLMs. Its self-supervised evaluation can use executable tests, majority voting or an LLM judge instead of labeled training data. The released implementation includes an Azure OpenAI chat wrapper, though the researchers describe the framework as compatible with black-box models generally. (github.com)

Microsoft reports gains across math, code and agent tasks

The researchers evaluated EvoLib on 93 Harvard-MIT Mathematics Tournament problems from 2025 and 2026, 148 BigCodeBench Hard tasks, 80 LiveCodeBench v6 Hard problems, 90 ScienceWorld tasks and 60 PDDL planning tasks. The experiments used GPT-4o for BigCodeBench and o4-mini for the other benchmark groups. (arxiv.org)

According to the paper, EvoLib improved over base sampling by 11% to 20% in the static benchmark settings. On HMMT, its reported accuracy reached 77.4%, compared with 57% for the base model and 74.2% for Best-of-N sampling. Its BigCodeBench pass rate was 40.8%, versus 29.7% for the base model and 37.2% for Best-of-N. EvoLib tied Recursive Self-Aggregation at a 70% pass rate on LiveCodeBench. (arxiv.org)

Microsoft also reports that EvoLib beat Dynamic Cheatsheet, the strongest test-time learning baseline in its comparison, by 5% to 10% across three benchmarks. The researchers measured cost using a weighted token count that treated output tokens as four times as expensive as input tokens, reflecting typical proprietary-model pricing. Under reduced compute budgets, EvoLib retained larger gains than the test-time scaling baselines in the coding tests. (arxiv.org)

Those results come from controlled benchmarks selected and run by the EvoLib authors. They do not establish that an agent will improve safely during a long production deployment. EvoLib depends on the model's own evaluation producing a useful signal. The authors say it works best when checking an answer is easier than generating one, as with code that can be run against tests. Open-ended answers may require a separate verifier, and the paper does not establish performance outside math, coding and the evaluated agent environments. (arxiv.org)

The GitHub release carries an MIT license and includes evaluation paths for math, code, ScienceWorld and PDDL tasks. Microsoft labels it research code and advises against commercial, real-world or high-risk deployment without further testing. The warning identifies the central operational risk: a model that misjudges its own work can store bad lessons, reapply them and compound the error over time. (github.com)