跳到正文
𝕏 elvis(x.com)·· 16 天前品質評分59
重點導讀

斯坦福研究推出 Collaborative Intelligence Benchmark,測試 240 個任務評估 AI 代理在團隊協作中的表現。

  • 最高分:18 個模型中最佳為 82.2/100。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

elvis
elvis@omarsar0
·16 天前
𝕏

Great weekend read from Stanford on measuring AI agents as teammates.

(bookmark it)

If you build multi-agent systems or build with agent teams, this one is worth your time.

They try to measure one of agents' most important capabilities: collaboration.

They propose the Collaborative Intelligence Benchmark, which has 240 tasks across 8 teamwork skills. The tasks come from real records such as aviation accident reports, clinical cases, coding tasks, and negotiations.

The best of 18 models scores 82.2/100. Every model loses 20 to 47 points once a teammate creates a problem.

Frontier models almost always refuse unsafe shortcuts and report misconduct. Holding a position under pressure is much harder for them.

In a negotiation where the teammate pushes, adds guilt, and threatens to walk away, they reach the best deal for the team only 14% of the time. They ask about the teammate's needs in 99% of those runs, then accept the teammate's offer in 81%.

When a model takes over a session where a weaker model cheated, the pass rate drops from 97% to 81%. In 44 of 56 failed coding takeovers, the cheat is still in the final code, often after the model pointed it out.

Nearly half of all failures happen after the model has already recognized the problem. Models with almost the same capability score differ by up to 11 points as teammates.

160 of the tasks are public with code and data. The other 80 are held out for evaluation.

https://t.co/w5N9y9gRG2

Chat with Paper: https://t.co/0JxWKcSaO0

推文來源 · X (Twitter)在 X 參與討論
來源: elvis (x.com)
在 X 查看推文

讀者討論

理性交流 · 即時討論