跳到正文
𝕏 Rohan Paul(x.com)·· 8 天前精選品質評分68
重點導讀

AI 助理做任務時,訓練方法常只看最後有沒有成功,很難知道中間哪一步做得好不好。

  • 浙江大學研究讓決策模型 JEV 在任務結束後回頭檢查每個動作,評估這些動作是否有幫助,再把評分用於訓練另一個 AI 助理。
  • JEV 自己完成 ALFWorld 模擬家事任務的成功率是 42.9%,但拿來評分後,Qwen2.5-1.5B 的成功率從 72.8% 提高到 82.9%。
  • 研究也指出,挑選評分模型時,可以看它評估任務紀錄的能力,不必只看它自己做任務的表現。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

推薦理由

研究比較模型自己執行任務的表現與事後評分能力,呈現兩者可能不同,並展示逐步評分如何影響訓練結果。

Rohan Paul
Rohan Paul@rohanpaul_ai
·8 天前
𝕏

AI model that is itself bad at a task can still train another AI to do it well.

New paper shows that a decision model called JEV, an AI model built to pick between options instead of writing text, completes only 42.9% of simulated household tasks on its own, yet by reviewing another AI’s moves afterward it raised that AI’s success from 72.8% to 82.9%.

New Zhejiang University paper shows that an AI model that is bad at a task can still train another AI to do it well, by reviewing each of its moves after the attempt ends.

Agent RL usually rewards only the final outcome, so every step gets the same credit. Scoring each step usually needs per-environment labels or an LLM judge at every action.

They tested decision models, which return a choice with probabilities instead of text, as actors and as step graders on ALFWorld and WebShop. As a grader, the model sees the whole run and scores whether each action helped, and those scores are added to GRPO training.

The decision model JEV completes only 42.9% of ALFWorld tasks when acting. As a grader, it lifts a Qwen2.5-1.5B policy to 82.9% success, up from 72.8% with GRPO outcome rewards alone.

Choose step graders by how well they judge recorded runs, not how well they act, and ask about each action, not the outcome.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論