Microsoft trains Qwen3.8-27B on 187 tasks to improve agent repeatability

The Microsoft and Hugging Face experiment raised average success by 9.08 percentage points across 507 benchmark tasks, while the model solved at least one attempt on seven fewer tasks.

By · Published · Updated

Primary source: Hugging Face Newsroom

Why it matters

Executable checks connect agent training to the actual records business systems retain. Microsoft's results show stronger repeatability on its benchmark.

Microsoft trains agents against the database state they leave behind — Tuhin Kundu and collaborators used executable database checks to fine-tune Qwen3.8-27B, improving repeated success while slightly reducing task coverage.

On October 22nd, Microsoft and Hugging Face published an open-source training system that rewards an AI agent for the database state it leaves behind. In the release results, Microsoft applied scientist Tuhin Kundu and collaborators report that fine-tuning Qwen3.8-27B with executable checks raised average success by 9.08 percentage points across a 507-task evaluation.…

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

Reader comments

Conversation for this story loads after sign-in.