跳到正文
𝕏 vLLM(x.com)·· 17 天前精選品質評分63
重點導讀

vLLM 宣佈在 vLLM 0.32.0 支援百度 ERNIE 5.5 Flash Base,並提供 FP8 與 BF16 推論部署方式。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

推薦理由

同一評測設定下,ERNIE 5.5 Flash Base 在兩項程式基準高於 DeepSeek-V4-Flash-Base,並提供引數規模與推論部署方式供比較。

vLLM
vLLM@vllm_project
·17 天前
𝕏

🎉 Day-0 support for @Baidu_Inc’s ERNIE 5.5 Flash Base with vLLM!

280B total parameters, 14B active per token. Pre-trained on 33.5T tokens, with 128K training context and up to 1M tokens for fine-tuning and inference. Open Base weights for domain adaptation, instruction tuning and RL research.

Code highlights: 82.3% HumanEval+ and 40.1% LiveCodeBench pass@1 (both 1-shot), versus 69.5% and 31.9% for DeepSeek-V4-Flash-Base under the same evaluation setup.

Efficiency is built into the architecture: 37 compressed-attention layers + 6 MLA-DSA layers, Multi-Router Latent MoE, and ECHO — reusing the final layer as a speculative draft.

Serve FP8 or BF16 with Baidu’s vLLM-ERNIE plugin on vLLM 0.32.0. ECHO runs through vLLM’s MTP path; recipes include expert parallelism, KV offload and prefill/decode disaggregation, with runs verified on @NVIDIA GB200 and H100.

Weights are on @huggingface and @ModelScope2022.

Recipe: https://t.co/OhlAm9XYKB
FP8: https://t.co/pzNsf3LBb9
BF16: https://t.co/igWS26BQ62

推文來源 · X (Twitter)在 X 參與討論
來源: vLLM (x.com)
在 X 查看推文

讀者討論

理性交流 · 即時討論