重點導讀
Mistral Large 4 於 8 個 NVIDIA GB200 節點上，以 LoRA RL 於 NVFP4 檢查點完成 75 步訓練，AIME 2024 avg@8 從約 64% 提升至 75%。
推薦理由
原文展示 75 步 LoRA RL 可將 Mistral Large 4 的 AIME 2024 avg@8 從 64% 提升至 75%，凸顯強化學習價值。
vLLM@vllm_project
·1 天前
🎉 Day-0 RL for @MistralAI’s Mistral Large 4 with @NovaSkyAI’s SkyRL and @vllm_project!
On 8 @nvidia GB200 nodes, LoRA RL on the NVFP4 checkpoint brings AIME 2024 avg@8 from ~64% to ~75% in 75 training steps.
vLLM generates rollouts, while FSDP2 or Megatron handles training. The recipe uses rank-32 LoRA, NVFP4 quantization-aware training, FP8 KV-cache simulation, routing replay (R3), and token-level truncated importance sampling.
The FSDP2 run maintains a mean absolute trainer–vLLM logprob gap of ~0.02–0.026. Megatron reaches similar accuracy with a comparable gap.
推文來源 · X (Twitter)在 X 參與討論