ERNIE 5.5 Flash Base 以 280B 引數與 1M-token 上下文，採用稀疏 MoE、FP8/BF16 儲存，並最佳化注意力與專家路由，達成同規模基礎模型 SOTA，為需高效推理的應用提供即插即用的架構；同時支援 8k 上下文的微調與 1M-token 推理，並可直接使用預訓練權重；其訓練系統與架構設計專為高效訓練與推理而生，適合大規模部署。
AI 伴隨問答
針對這篇報導有疑問？可直接點選快捷問題或向 AI 追問背景脈絡與技術細節：
ERNIE 5.5 Flash Base just dropped! Time to open that project folder. 🚀
Our efficient sparse MoE language model draws on data spanning multiple domains and languages, achieving SOTA on multiple benchmarks among similarly sized base models. Its architecture and training system were designed together for efficient training and inference, with advances in attention mechanisms, expert routing, and training objectives:
- 280B total parameters; 14B activated per token
- Pre-trained on 33.5T tokens
- Up to 1M-token context for fine-tuning and inference
- FP8 and BF16 weights
The weights come straight from pre-training, with no mid- or post-training. Compare methods from the same checkpoint, measure what each adds, and shape the model around your own tasks.