Hugging Face fixes TRL CISPO default that shrank gradients fivefold in its reproduction
Hugging Face's TRL v1.18.0 sets the CISPO default to 5.0 after an inherited 0.2 ceiling clipped on-policy weights near 1.
By RuntimeWire Staff · Published

Hugging Face's TRL v1.18.0 sets the CISPO default to 5.0 after an inherited 0.2 ceiling clipped on-policy weights near 1.