重點導讀
Wayne Chi 說明 Interaction-as-a-Judge 如何讓電腦操作代理透過遊玩遊戲與使用應用程式,改善評估與後續生成。
- 代理操作應用程式並產生互動軌跡,再交由 LLM 評審或訓練過的獎勵模型評估。
- 演講涵蓋以 Code Arena 的真實對戰進行偏好評估,以及根據 Agent Arena 中真實使用者抱怨整理需求,進行正確性評估。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Arena.ai@arena
·10 天前
Arena research intern and PhD candidate at Carnegie Mellon University, Wayne Chi, explains Interaction-as-a-Judge: letting computer-use agents play games and use apps to better improve evaluation and downstream generation.
A computer-use agent navigates an application, producing interaction traces for evaluation by an LLM judge or a trained reward model. The talk covers preference evaluation using real Code Arena battles and correctness evaluation using requirements mined from real user complaints in Agent Arena.
Watch the full talk:
https://t.co/xlqOsUOT9e
推文來源 · X (Twitter)在 X 參與討論