大型模型通常需要很多 GPU 記憶體,也要花不少時間訓練。
- Thinking Machines Lab 開源了 594B 引數的 Inkling 2 Small,採用 4-bit 前向運算訓練。
- 這讓預訓練時間縮短 8%、RL 時間縮短 20%,效率約損失 2%。執行模型時,NVFP4 至少需要 500GB GPU 記憶體,MXFP8 至少需要 800GB。
- 後者在評測上只高出約 1–2 個百分點。團隊也說,訓練從預訓練到最後一次 RL rollout 都維持相同數值設定,因此釋出的 4-bit 模型不需要另外校準。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
這次發布交代了 4-bit 訓練節省的時間與效率代價,也比較 NVFP4 和 MXFP8 的記憶體需求及評測差距。
Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes.
> now available for download on HuggingFace under an Apache 2.0 licens
> Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss.
> Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals.
> Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step.
> The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.