JetSpec++ 將因果式平行草擬、隱藏狀態預測器與錨定 token 解碼器結合,讓下一個區塊能在目前區塊驗證前先行草擬,並將草擬與驗證管線化,隱藏幾乎所有草擬成本。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Introducing JetSpec++: we find causal parallel drafting paired with hidden states predictor and anchor token decoder can effectively draft the next future block before the current one is verified, and spending the freed time on self refinement.
This allows drafting to be pipelined with verification, hiding almost all of its drafting cost.
JetSpec++ reaches up to 1.54x end-to-end speedup over DSpark and 7.45x over autoregressive decoding while keeping lossless, at 1,361 TPS on Qwen3-8B and 367 TPS on Qwen3.8-27B in BF16.
With NVFP4 and kernel optimizations, JetSpec++ further reaches over 4,000 TPS on Qwen3-8B and over 900 TPS on Qwen3.8-27B, on two GB200 GPUs. ⚡️
Project page: https://t.co/tuhf2KPfsW