跳到正文
𝕏 Philipp Schmid(x.com)·· 12 天前品質評分57
重點導讀

Philipp Schmid 與 Artificial Analysis 推出 MKA(Multimodal Knowledge Agent),用來評估 AI 代理處理視覺複雜檔案與知識工作的表現。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Philipp Schmid
Philipp Schmid@_philschmid
·12 天前
𝕏

Evaluations are the backbone of any frontier progress. Today we're launching MKA (Multimodal Knowledge Agent) with @ArtificialAnlys, a benchmark for agents doing knowledge work on visually complex files.

- 200 tasks across 10 verticals and 83 occupations, written by domain experts
- Each task takes a seasoned professional 3 to 8 hours
- ~70 yes/no rubric checks per task, weighted 75% factual accuracy and 25% visual design
- Every model runs in the same harness (Stirrup) with the same tools, sandbox and turn budget
- 200 tasks stay private with Artificial Analysis, so they can't leak into training data
- 10 public tasks on @huggingface 🤗

Gemini 4 Carbon is #2 on quality, 3 points behind Claude Fable 5.2, at $6.29 per task vs $31.97.

Blog: https://t.co/n6WZLdmc8D
Results: https://t.co/sib7sPjoO6

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論