程式開發 AI 助理常拿到寫得很完整的任務,但真正的團隊工作裡,需求可能散在同事手上。
- TokenWave AI 推出 SWE-CoWork 這項基準測試,讓 AI 助理在模擬職場中工作,並把 82% 的需求交由隊友掌握,測試它會不會向對的人詢問。結果顯示,表現最好的模型雖能通過約 63% 的個別檢查,完整完成的任務仍不到 101 項中的 8 項。
- 提供完整規格也讓 GPT-6 Astra 通過的檢查增加 16 個百分點,但它仍只完整完成 8 項任務中的 101 項。這項測試呈現了通過單項檢查和完成整個工作之間的差距。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Coding-agent benchmarks are starting to look like a new hire’s first week.
TokenWave AI has released SWE-CoWork, a new benchmark that tests whether AI coding agents can work like real team members.
Giving GPT-6 Astra the full spec up front lifted its passed checks by 16 points, yet it still fully finished only 8 of 101 tasks. SWE-CoWork puts coding agents in a simulated workplace modeled on real-world workflows, then links each failed check to a specific requirement.
Most coding benchmarks hand an AI agent a complete written task, but SWE-CoWork puts it in a simulated workplace where 82% of the requirements sit with teammates, so the agent only learns what to build by asking the right person. The best models pass about 63% of the individual checks yet fully finish fewer than 8% of 101 tasks.