跳到正文
𝕏 Rohan Paul(x.com)·· 3 天前品質評分59
重點導讀

AI 助理會把以前用過的檔案路徑、欄位名稱或指令存成技能,方便下次少做一些工作;但應用程式或 API 改變後,這些舊資料可能悄悄失效。

  • Amazon 的研究測試 Claude Opus 4.7 和 Haiku 4.5 在環境改變前後的表現,並用不呼叫模型的檢查方式找出過時內容,只修補有衝突的部分。在 Terminal-Bench 2 中,原本讓 Opus 多拿 4.7 分的技能,環境改變後反而比不用技能低 11.4 分。
  • 稅務表格測試中,修補後 Haiku 的欄位準確率從 30.5% 提高到 97.5%。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Rohan Paul
Rohan Paul@rohanpaul_ai
·3 天前
𝕏

New Amazon paper finds that saved agent skills can hurt once the environment changes, and checking and repairing them selectively restores their value.

Agent skills store past experience, like file paths, field names, or commands, so agents can skip work later. When an app or API changes, some of that stored detail goes stale without warning.

They tested Claude Opus 4.7 and Haiku 4.5 on 4 settings, including Terminal-Bench 2 and IRS tax forms, before and after a change. Their fix replays saved checks witho1ut model calls and repairs or rebuilds only the conflicting skill content.

On Terminal-Bench 2, a skill that had added 4.7 points for Opus cut its pass rate 11.4 points below no skill after the change. Maintenance lifted Haiku's tax-form field accuracy from 30.5% with the stale skill to 97.5%.

Check saved agent skills against the current environment before reuse, instead of waiting for tasks to fail.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論