跳到正文
𝕏 Rohan Paul(x.com)·· 9 天前精選品質評分71
重點導讀

大型模型通常需要很多 GPU 記憶體,也要花不少時間訓練。

  • Thinking Machines Lab 開源了 594B 引數的 Inkling 2 Small,採用 4-bit 前向運算訓練。
  • 這讓預訓練時間縮短 8%、RL 時間縮短 20%,效率約損失 2%。執行模型時,NVFP4 至少需要 500GB GPU 記憶體,MXFP8 至少需要 800GB。
  • 後者在評測上只高出約 1–2 個百分點。團隊也說,訓練從預訓練到最後一次 RL rollout 都維持相同數值設定,因此釋出的 4-bit 模型不需要另外校準。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

推薦理由

這次發布交代了 4-bit 訓練節省的時間與效率代價,也比較 NVFP4 和 MXFP8 的記憶體需求及評測差距。

Rohan Paul
Rohan Paul@rohanpaul_ai
·9 天前
𝕏

Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes.
> now available for download on HuggingFace under an Apache 2.0 licens
> Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss.

> Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals.

> Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step.

> The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論