跳到正文
𝕏 vLLM(x.com)·· 10 天前品質評分46
重點導讀

vLLM 已提供 Inkling-2-Small 的 Day-0 支援,這款 500B MoE 模型可接收文字與圖片輸入,並產生文字回應。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

vLLM
vLLM@vllm_project
·10 天前
𝕏

🎉 Day-0 support for @thinkymachines’ Inkling-2-Small in vLLM!

A 500B MoE with text + image inputs and text output. Bring the image into the conversation: ask a question with visual context, and get a text response from the same model.

Under the hood: a new ViT image encoder, mixed NVFP4/MXFP8 MoE layers, and DSpark speculative decoding.

@inferact leads the vLLM integration and optimization, working with Thinking Machines Lab on model code and checkpoints. The work spans MoE and attention kernels, the vision encoder and image preprocessing, plus draft-model GEMMs for speculative decoding.

Thanks to the TML, Inferact and vLLM teams bringing the model to an open inference stack!

推文來源 · X (Twitter)在 X 參與討論
來源: vLLM (x.com)
在 X 查看推文

讀者討論

理性交流 · 即時討論