百度開放 ERNIE-5.5-Flash-Base 權重,提供尚未經中訓練與後訓練的基礎檢查點,供領域適配、指令微調與 RL 研究使用。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
模型保留預訓練後、尚未經中後訓練的檢查點,並提供基準分數與長上下文注意力設計,可供評估領域適配與指令微調的起點。
Baidu is opening ERNIE-5.5-Flash-Base straight out of pre-training: an open-weight foundation for domain adaptation, instruction tuning, and RL research.
🤖 https://t.co/SVrO40OurO
• 280B total parameters; 14B activated per token
• Pre-trained on 33.5T tokens, with training context up to 128K
• Up to 1M-token context for fine-tuning and inference
• FP8 and BF16 weights
🛠️ A starting point for your own training: The checkpoint precedes all mid- and post-training, leaving developers to choose their own adaptation methods and objectives.
📊 Strong base-model results: Scores 78.4 on MMLU-Pro, 82.3 on HumanEval+, and 40.1 on LiveCodeBench, ahead of the three compared base models on these benchmarks under a unified evaluation pipeline.
🧠 Compressed attention for long inputs: Combines 37 HCA layers, each caching one entry per 128 tokens, with six MLA-DSA layers for full-context access through sparse attention.
⚙️ A built-in speculative draft: ECHO reuses the backbone's final layer to predict future tokens, replacing a separate MTP module.
📦 Deployment: Transformers, SGLang, and vLLM-ERNIE paths are available. The Base model uses the completions endpoint.