RSNA opens $77,000 challenge for AI that reads knee MRI and reports

Why it matters

RSNA is testing whether multimodal medical AI can generalize across hospitals and languages while remaining efficient enough for real clinical infrastructure.

The Radiological Society of North America has opened a $77,000 Kaggle competition for machine learning models that detect 12 clinically important abnormalities in knee MRI examinations.

The contest opened on July 30th and requires participants to work with both medical images and radiology report text. Final submissions are due October 22nd, following an October 15th entry and team-merger deadline. The prize pool includes awards for the highest-scoring entries and a separate efficiency track.

RSNA built the benchmark from more than 5,000 knee MRI exams supplied by 16 institutions worldwide. Each exam is paired with its original report, with the collection spanning nine languages. RSNA says this is the first of its annual AI challenges to use images and report text together during model development and evaluation.

A multimodal test with clinical variation

Knee MRI interpretation requires radiologists to assess several structures and findings within the same examination. The modality can reveal ligament and meniscal injuries, cartilage loss, bone marrow lesions, effusion, synovitis and cysts. These findings can coexist, making the challenge a multilabel classification problem rather than a search for one isolated lesion.

Pairing scans with reports also gives competitors two imperfect views of the same case. The images contain the underlying anatomy, while reports encode the observations and terminology of the radiologist who interpreted the study. Models will have to reconcile those inputs without learning shortcuts tied to a particular hospital, language or reporting style.

The geographic spread of the dataset raises the difficulty. MRI protocols, scanner hardware and clinical documentation vary across institutions. A model that performs well because it recognizes one site's acquisition settings or vocabulary may fail when evaluated on data from another health system. The 16-institution design gives RSNA a way to test some of that distribution shift within a single competition.

RSNA puts efficiency on the leaderboard

The efficiency prizes address another recurring problem in medical imaging competitions: the most accurate entry may depend on an ensemble that is too expensive or slow for routine use. MRI exams contain substantially more data than a single radiograph, and processing each case through several large models can increase inference time and hardware requirements.

By rewarding efficiency separately, RSNA is giving competitors an incentive to compress models, reduce preprocessing and limit ensemble size while preserving diagnostic performance. That tradeoff matters in clinics where imaging software must process studies alongside existing radiology systems and cannot assume access to large clusters of accelerators.

The competition still measures benchmark performance, not clinical readiness. RSNA's challenge format gives participants labeled training data and evaluates final models against a hidden portion of the dataset. A high private-leaderboard score shows that a model matched the challenge's expert-derived labels under its test conditions. It does not establish safety across every scanner, patient population or clinical workflow.

Earlier research shows both the potential and the remaining gap. A 2021 study published in Radiology: Artificial Intelligence tested deep learning models on knee MRI abnormalities involving cartilage, bone marrow, menisci and the anterior cruciate ligament. Reported sensitivity ranged from 70% to 88%, while specificity ranged from 85% to 89%. AI assistance improved agreement in 10 of 16 comparisons involving attending physicians and trainees.

RSNA's new competition expands that problem across more institutions, more abnormality classes and multilingual reports. The strongest result will need to survive variation in both pixels and clinical language, then do so within the computational limits emphasized by the efficiency awards.

Teams must submit final models by October 22nd. RSNA has scheduled its winner-requirements deadline for November 5th, ahead of recognition for top-performing teams at its annual meeting.