跳到正文
𝕏 Hao AI Lab(x.com)·· 23 天前品質評分48
重點導讀

JetSpec++ 將因果式平行草擬、隱藏狀態預測器與錨定 token 解碼器結合,讓下一個區塊能在目前區塊驗證前先行草擬,並將草擬與驗證管線化,隱藏幾乎所有草擬成本。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Hao AI Lab
Hao AI Lab@haoailab
·23 天前
𝕏

Introducing JetSpec++: we find causal parallel drafting paired with hidden states predictor and anchor token decoder can effectively draft the next future block before the current one is verified, and spending the freed time on self refinement.

This allows drafting to be pipelined with verification, hiding almost all of its drafting cost.

JetSpec++ reaches up to 1.54x end-to-end speedup over DSpark and 7.45x over autoregressive decoding while keeping lossless, at 1,361 TPS on Qwen3-8B and 367 TPS on Qwen3.8-27B in BF16.

With NVFP4 and kernel optimizations, JetSpec++ further reaches over 4,000 TPS on Qwen3-8B and over 900 TPS on Qwen3.8-27B, on two GB200 GPUs. ⚡️

Project page: https://t.co/tuhf2KPfsW

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論