vLLM lets compressed-tensors block-FP8 weights survive an engine restart

The merged change lets a restarted engine map processed CUTLASS block-FP8 weights through CUDA IPC, with exact greedy output checked on a B300.

By · Published

Black server racks with green and amber indicator lights are linked by a glowing golden stream of digital blocks.

The merged change lets a restarted engine map processed CUTLASS block-FP8 weights through CUDA IPC, with exact greedy output checked on a B300.