Philipp Schmid 與 Artificial Analysis 推出 MKA(Multimodal Knowledge Agent),用來評估 AI 代理處理視覺複雜檔案與知識工作的表現。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Evaluations are the backbone of any frontier progress. Today we're launching MKA (Multimodal Knowledge Agent) with @ArtificialAnlys, a benchmark for agents doing knowledge work on visually complex files.
- 200 tasks across 10 verticals and 83 occupations, written by domain experts
- Each task takes a seasoned professional 3 to 8 hours
- ~70 yes/no rubric checks per task, weighted 75% factual accuracy and 25% visual design
- Every model runs in the same harness (Stirrup) with the same tools, sandbox and turn budget
- 200 tasks stay private with Artificial Analysis, so they can't leak into training data
- 10 public tasks on @huggingface 🤗
Gemini 4 Carbon is #2 on quality, 3 points behind Claude Fable 5.2, at $6.29 per task vs $31.97.
Blog: https://t.co/n6WZLdmc8D
Results: https://t.co/sib7sPjoO6