跳到正文
𝕏 PyTorch(x.com)·· 12 天前品質評分58
重點導讀

開源社群在 TPU v7 上為 Muse Spark Open 建立 megakernel,於 AgentX(1M context)達到每位使用者 433 tok/s、每顆晶片最高 25.9K tok/s。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

PyTorch
PyTorch@PyTorch
·12 天前
𝕏

Great to see the open source community pushing inference performance forward on PyTorch:

A megakernel for Muse Spark Open on TPU v7 got 433 tok/s per user and up to 25.9K tok/s per chip on AgentX (1M context). All 62 layers of the 816B MoE run as one Pallas program on a single TPU v7 host, with SGLang on the TorchTPU PyTorch backend.

- 1.94x faster per token than our previous path
- DFlash2 speculative decoding: 2.0x per token
- fp8 KV, HiCache and session-aware eviction for long agent sessions

Read the full technical deep dive here: https://t.co/xmxOjlrZtK

@radixark

推文來源 · X (Twitter)在 X 參與討論
來源: PyTorch (x.com)
在 X 查看推文

讀者討論

理性交流 · 即時討論