跳到正文
𝕏 ModelScope(x.com)·· 17 天前精選品質評分79
重點導讀

百度開放 ERNIE-5.5-Flash-Base 權重,提供尚未經中訓練與後訓練的基礎檢查點,供領域適配、指令微調與 RL 研究使用。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

推薦理由

模型保留預訓練後、尚未經中後訓練的檢查點,並提供基準分數與長上下文注意力設計,可供評估領域適配與指令微調的起點。

ModelScope
ModelScope@ModelScope2022
·17 天前
𝕏

Baidu is opening ERNIE-5.5-Flash-Base straight out of pre-training: an open-weight foundation for domain adaptation, instruction tuning, and RL research.
🤖 https://t.co/SVrO40OurO

• 280B total parameters; 14B activated per token
• Pre-trained on 33.5T tokens, with training context up to 128K
• Up to 1M-token context for fine-tuning and inference
• FP8 and BF16 weights

🛠️ A starting point for your own training: The checkpoint precedes all mid- and post-training, leaving developers to choose their own adaptation methods and objectives.

📊 Strong base-model results: Scores 78.4 on MMLU-Pro, 82.3 on HumanEval+, and 40.1 on LiveCodeBench, ahead of the three compared base models on these benchmarks under a unified evaluation pipeline.

🧠 Compressed attention for long inputs: Combines 37 HCA layers, each caching one entry per 128 tokens, with six MLA-DSA layers for full-context access through sparse attention.

⚙️ A built-in speculative draft: ECHO reuses the backbone's final layer to predict future tokens, replacing a separate MTP module.

📦 Deployment: Transformers, SGLang, and vLLM-ERNIE paths are available. The Base model uses the completions endpoint.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論