跳到正文
𝕏 Rohan Paul(x.com)·· 9 天前品質評分56
重點導讀

AI 助理進入公司 Slack 時,難處不只是回答問題,還得判斷要不要插話、該查什麼資料,以及何時保持安靜。

  • Deepnote 推出 Agent Initiative Benchmark,讓各模型在模擬的公司 Slack 討論串裡做這些決定,並由兩個大型語言模型評分。GPT-6 Astra 得 8.97 分,GPT-6.1 Sol 得 8.65 分,每次任務費用是 $0.11。
  • 排名第三的 Claude Fable 5.2 每次任務費用是 $1.87。這項測試也會看 AI 助理是否找對服務、回覆並標記合適的人。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Rohan Paul
Rohan Paul@rohanpaul_ai
·9 天前
𝕏

This is a beautiful concept for a benchmark.

Deepnote (@deepnotehq), an agent workspace company, released the Agent Initiative Benchmark, a test that drops proactive agents into a company Slack and scores whether they know when to jump in, what to say, and when to stay quiet.

GPT-6 Astra led with 8.97 out of 10, and GPT-6.1 Sol followed at 8.65 for $0.11 per task, against $1.87 per task for third-placed Claude Fable 5.2.

It runs every model through the same loop:

> A thread starts in a simulated company’s Slack
> The agent decides whether to act at all
> It picks which of 11 services to check and investigates
> It posts and tags the right people, or stays silent
> Two LLM evaluators score the run 0 to 10 against a reference answer

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論