Julius Research releases DeepFinance and the best model scores 54.8% on 27-hour deal work
Julius Research tested 16 model configurations on 21 anonymized PE and IB tasks averaging 27.1 human hours each, and none scored above 54.8%.
By Ryan Merket · Published
Primary source: x.com
Why it matters
Finance firms are being sold agents for diligence, valuation and underwriting, and DeepFinance gives them a harder test than document Q&A: 21 anonymized deals graded on 84 criteria each, including whether assumptions are sourced and auditable. On Julius Research's own run, the best configuration met only 54.8% of those criteria, and the documented failures were in judgment - choosing inputs, challenging assumptions, acting on evidence - rather than arithmetic. That is the gap a buyer has to price before letting an agent near a live deal.
Rahul Sonwalkar (@0interestrates), founder and CEO of Julius AI, released DeepFinance on November 5th through Julius Research, the San Francisco applied AI research lab behind the Julius data product. In a thread on X, Sonwalkar called DeepFinance the first ultra-long horizon benchmark for high finance work, a claim Julius Research carries on the benchmark itself and that rests on its narrow focus: end-to-end private equity and investment banking tasks graded on workflow and judgment over 20 to 30 hours…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.