🤗 HuggingFaceSignificantQuentin Gallouédec
TRL v1.4: Chunked NLL Loss Achieves 34% VRAM Reduction for Supervised Fine-Tuning
TRL v1.4 release introduces chunked NLL loss for supervised fine-tuning, achieving significant VRAM reduction while maintaining loss quality and often improving training speed. Benchmark shows Qwen3-14B at 16k sequence l…