重點導讀
Meta的研究提出 AutoRefine,利用既有訊號調整 LLM 程式碼評審的提示詞,讓評審結果在部署中的程式開發 AI 助理上維持校準。
- 調整訊號:包括測試是否通過,以及程式碼變更是否合併。
- 實測結果:評審與實際結果的一致度從 0.16 提升至 0.51。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
elvis@omarsar0
·12 天前
Great paper from Meta with practical tips for tuning LLM judges.
This one is on keeping LLM code judges calibrated and tested on a deployed coding agent.
LLM judges often decide whether an agent's code change is good. But their prompts go stale every time the agent changes.
AutoRefine fixes the prompt using signals teams already have, like whether tests passed and whether the change got merged.
On Meta's deployed agent, agreement between the judge and real outcomes rose from 0.16 to 0.51.
Chat with Paper: https://t.co/EkCBb5imTP
推文來源 · X (Twitter)在 X 參與討論