跳到正文
𝕏 elvis(x.com)·· 2 天前品質評分48
重點導讀

醫院工作常要查保險、申請事前授權、處理理賠和索取病歷,還得在不同系統間小心操作。

  • Hospital-bench 用模擬醫療系統測試 AI 助理,系統裡有 13 個應用程式和 131 種工具。
  • 20 個模型共完成 48 項任務。GPT-6.2 Astra 平均得分最高,為 62.4%。
  • Claude Opus 5.5 完整通過的任務最多,共 17 項。安全也是大問題:有模型洩漏受限病歷、錯改已付款的理賠,或做了超出許可權的事。GPT-6.2 Astra 和 Gemini 4 Neon 沒有任務因造成傷害而被判零分,但有些模型最多有 10 項。結果顯示,AI 助理要可靠又安全地完成跨系統工作,仍有很大挑戰。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

elvis
elvis@omarsar0
·2 天前
𝕏

Introducing Hospital-bench, a new agent benchmark from @zavoai.

It tests hospital operations work. Agents handle coverage checks, prior authorizations, claims, and records requests inside a simulated health system with 13 apps and 131 tools.

20 models ran 48 tasks. GPT-6.2 Astra led with a 62.4% mean score, while Claude Opus 5.5 fully passed the most tasks (17/48).

Safety was a major challenge. Models made errors, including disclosing restricted patient records, incorrectly modifying paid claims, and acting beyond their authority.

GPT-6.2 Astra and Gemini 4 Neon had no tasks zeroed due to harm, while some models had as many as 10.

The results show that completing complex, multi-system workflows reliably and safely remains a major challenge for AI agents.

推文來源 · X (Twitter)在 X 參與討論
來源: elvis (x.com)
在 X 查看推文

讀者討論

理性交流 · 即時討論