要做出好用的 RAG 系統,光靠人設計的方法是否足夠?
- AutoRAGBench 用九項開發任務測試 AI 助理:不提供起始程式碼或做法,讓它在六小時內分析資料、建立流程,再根據實驗結果調整。GPT-6.1 Sol 搭配 Codex CLI 平均得 56.1 分。
- 每項任務都採用已發表最佳方法組成的人類基準,平均為 44.4 分。若從人類設計的系統開始改良,AI 助理在七項任務中平均增加 16.6 分。論文也比較九種 AI 助理設定,並追蹤它們在六小時內如何逐步開發。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Can coding agents build better RAG systems than the best published human designs?
AutoRAGBench gives an agent nine RAG development tasks to measure this. There is no starter code and no suggested approach. The agent gets six hours and one H800 GPU to analyze the data, build a pipeline, and refine it from experiments.
GPT-6.1 Sol with Codex CLI averages 56.1. The composite human baseline, which takes the best published method on each task, averages 44.4.
When the agent starts from a human-designed system instead of from scratch, it adds 16.6 points on average across seven tasks.
The paper compares nine agent setups and also studies how each one's development process unfolds over the six hours.
Paper: https://t.co/lBaGGVw7KK
Chat with Paper: https://t.co/2AvIc2kCe1