Ai2 opens MoE training stack after testing a 1.2T-parameter setup

Ai2 says its system can train larger sparse models with less overhead. Its trillion-parameter result measures system performance, not a completed model.

By · Published

A generic compute rack with grouped server trays and fiber links represents Ai2’s newly opened MoE training stack.

Ai2 says its system can train larger sparse models with less overhead. Its trillion-parameter result measures system performance, not a completed model.