Epoch AI 审计了15个基准，发现9个存在缺陷
首批报告指出了用于比较前沿模型的 SWE-bench Verified、Humanity's Last Exam 和其他测试中的缺陷。
By RuntimeWire Staff · Published
Primary source: Epoch AI
为何重要
人工智能排行榜指导产品声明、部署决策和政策制定。Epoch AI 的审计显示，一些头条得分反映的是有缺陷的评分器和含糊不清的任务，而非模型的能力。
Epoch AI, the research institute co-founded and led by Jaime Sevilla (@Jsevillamol), launched Benchmark Reviews on September 17th and classified nine of its first 15 reviewed benchmark versions as Flawed. Four received Verified labels, while Epoch AI lacked enough information to judge the remaining two.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.