跳到正文
𝕏 elvis(x.com)·· 6 小時前品質評分74
重點導讀

導語:Microsoft 的 ParetoSaddler 透過在搜尋結束時才加入成本偏好,顯著提升 Agent Harness 的質量與成本效益。

  • GAIA2:質量從 56.22% 提升至 62.89%,成本從 $0.78 降至 $0.45。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

elvis
elvis@omarsar0
·6 小時前
𝕏

Super interesting finding on optimizing agent harnesses.

When you optimize an agent harness for both quality and cost, apply your cost preference only at the end of the search, since applying it at every step throws away harnesses that lead to better ones.

That is the key insight behind Microsoft's ParetoSaddler.

At each step, an optimizer edits a parent harness. Both versions run several times, and a Bayesian bootstrap over those runs decides whether the candidate shows credible progress in quality or in cost. Promoted candidates stay available as parents, and the final deployment choice is made separately.

On GAIA2, quality goes from 56.22% to 62.89% while cost per task drops from $0.78 to $0.45. On ClawMark, quality goes from 70.86% to 81.35%, and cost drops from $1.17 to $1.01.

The Pareto-selected harnesses also beat quality-only optimization on both quality and cost.

If your harness search only tracks accuracy, add repeated runs and a cost axis before deciding which candidates to keep.

https://t.co/gF5XACoUv0

Chat with Paper: https://t.co/NSriK3hC2t

推文來源 · X (Twitter)在 X 參與討論
來源: elvis (x.com)
在 X 查看推文

讀者討論

理性交流 · 即時討論