AI 助理要完成很多步驟的任務時,可能先做比較容易的事,漏掉必須先完成的步驟。
- 這篇論文比較決策模型和一般大型語言模型在 ALFWorld 裡的表現,發現六個受測模型都曾先做較容易的子目標。
- 看完整段行動後,它們也會把功勞算給實際執行時跳過的步驟。研究者再用其中一個決策模型教 Qwen2.5-1.5B,成功率在 ALFWorld 從 72.8% 升到 82.9%,在 WebShop 從 56.8% 升到 71.4%。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Great paper on decision models for long-horizon agents.
For agent RL, you want the decision model to evaluate completed trajectories and leave action selection to the policy.
Decision models output a typed distribution, such as a probability for each option. As actors on ALFWorld, all six models tested (three decision models and three general LLMs) went for an easier subgoal before the one that had to come first.
Shown the full trajectory and asked whether each action helped, the same models gave credit to the steps they skipped while acting.
Used as a teacher for GRPO, one of the decision models raised a Qwen2.5-1.5B policy from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop, including on tasks it rarely completes as an actor.
Chat with Paper: https://t.co/iPv0b4FdMa