JoMoHa 通過雙策略 RL 訓練,提升 harness 與任務解法協同,顯著提高 ALFWorld、WebShop 與 DBBench 的表現。
- 方法:訓練兩個策略——一寫 harness,另一解題,保持獨立歷史與價值估計以避免不相容。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Nice read from Salesforce AI Research and colleagues.
There is a lot of interest in self-improving harnesses, but the harness editor is usually a fixed frontier model.
JoMoHa trains two policies with RL from the same task outcomes. One writes the executable harness, and the other solves the task inside it.
Updating the model and the harness in turns caused trouble. Performance rose and then fell as the two became incompatible. Keeping separate histories and value estimates for each policy fixed that.
JoMoHa beat weight-only and harness-only training on ALFWorld, WebShop, and DBBench, averaging 70.3% and reaching 97.0% on ALFWorld. Training the harness writer also beat freezing it.
Chat with Paper: https://t.co/xrL5A1yO5S