vLLM 宣佈在 vLLM 0.32.0 支援百度 ERNIE 5.5 Flash Base,並提供 FP8 與 BF16 推論部署方式。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
同一評測設定下,ERNIE 5.5 Flash Base 在兩項程式基準高於 DeepSeek-V4-Flash-Base,並提供引數規模與推論部署方式供比較。
🎉 Day-0 support for @Baidu_Inc’s ERNIE 5.5 Flash Base with vLLM!
280B total parameters, 14B active per token. Pre-trained on 33.5T tokens, with 128K training context and up to 1M tokens for fine-tuning and inference. Open Base weights for domain adaptation, instruction tuning and RL research.
Code highlights: 82.3% HumanEval+ and 40.1% LiveCodeBench pass@1 (both 1-shot), versus 69.5% and 31.9% for DeepSeek-V4-Flash-Base under the same evaluation setup.
Efficiency is built into the architecture: 37 compressed-attention layers + 6 MLA-DSA layers, Multi-Router Latent MoE, and ECHO — reusing the final layer as a speculative draft.
Serve FP8 or BF16 with Baidu’s vLLM-ERNIE plugin on vLLM 0.32.0. ECHO runs through vLLM’s MTP path; recipes include expert parallelism, KV offload and prefill/decode disaggregation, with runs verified on @NVIDIA GB200 and H100.
Weights are on @huggingface and @ModelScope2022.
Recipe: https://t.co/OhlAm9XYKB
FP8: https://t.co/pzNsf3LBb9
BF16: https://t.co/igWS26BQ62