AI 助理在一種任務中學會從自己的嘗試改進,不代表換到另一種任務也有效。
- 這份整理比較了視覺推理、圖片生成、應用程式與網頁操作、機器人等情境,並列出各種技巧需要什麼條件,例如多數決需要能互相比較的答案,程式檢查則需要能計算的結果。
- 整理把技巧在不同情境的適用情況分為可移植、有條件、受阻或多餘,但這些是推測,還沒有跨任務實驗驗證。
- 作者建議,換用技巧前先確認新任務能提供所需的檢查方式,再和使用相同資源的基準方法比較。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
This survey argues that self-improvement tricks for multimodal agents only carry over to tasks that have what the trick relies on, like checkable answers.
AI agent learn from its own attempts in 1 area, like questions about images, can fail in another, like robotics, where there is nothing to check.
Self-improving agents now span visual reasoning, image generation, app and web navigation, and robotics, but each method is tested in only 1 setting. Majority voting, for example, scores short visual answers but finds nothing to compare in robot action trajectories.
The author reviewed methods in the 4 settings and listed each trick's needs, like answers a vote can compare or results a program can check. Each trick then gets a verdict per setting: portable, conditional, blocked, or redundant.
In 1 cited audit of 500 questions, 24% of kept majority-vote labels were wrong, against 5.6% for program-computed answers. These verdicts are predictions, not results from transfer experiments.
Before moving a self-improvement trick to a new task, confirm that task offers the check the trick relies on, then test against a same-budget baseline.