重點導讀
大型模型需要很多記憶體,但高頻寬記憶體(HBM)供應有限。
- Positron 的 Titan 伺服器改用較便宜、類似智慧型手機使用的 DRAM,單臺伺服器最多可容納 32 兆引數的模型。
- 推論雲端 Parasail 宣佈將大規模部署新一代 Titan 伺服器,起步規模為 5MW。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Rohan Paul@rohanpaul_ai
·3 天前
Memory capacity per watt is becoming one of the most useful ways to judge AI hardware.
Positron Titan runs multi-trillion-parameter models on commodity DRAM, and @parasail_io is the first inference cloud putting it online at scale.
Parasail, an inference cloud that already runs several chip types behind one API, just announced it will be the first to deploy Positron’s next-gen Titan servers at scale, starting with 5MW.
Titan keeps giant models in cheap smartphone-style memory instead of scarce high-bandwidth memory (HBM), so one server can hold a model of up to 32 trillion parameters.
推文來源 · X (Twitter)在 X 參與討論
來源: Rohan Paul (x.com)
在 X 查看推文