跳到正文
𝕏 Artificial Analysis(x.com)·· 11 天前品質評分53
重點導讀

Artificial Analysis 比較模型在 VIALS-AA 使用工具與單次作答時的表現,工具使用下各模型平均分數約從 25% 提升至 47%。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Artificial Analysis
Artificial Analysis@ArtificialAnlys
·11 天前
𝕏

Take away its tools, and Claude Fable 5.2 scores below Claude Haiku 5.5 with tools on VIALS-AA; across models tool use nearly doubles scores, from ~25% to ~47% on average

VIALS-AA is our implementation of @joinHandshakeAI's VIALS benchmark for visual reasoning in life sciences research. By default, models work as agents: they think, run code and view images in a sandbox. We reran VIALS-AA in a one-shot format: one request with the question and images, and no tools.

Lower cost models in an agent harness can beat single-shot large models on both cost and performance in visual reasoning and interpretation of scientific artifacts.

Claude Fable 5.2 (max) leads VIALS-AA at 67% with tools, at a cost of $4.20 per task; without tools it scores 44% at $0.73 per task. Claude Haiku 5.5 (max) scores 49% with tools, at $0.07 per task: higher than Fable without tools, at about a tenth of the cost.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論