重點導讀
同一款聊天機器人就算完全沒改版,使用者也可能隨時間覺得它答得越來越差。
- 研究者分析 Chatbot Arena 上 1,260,967 次匿名投票,只比較固定版本的模型,發現使用者每月投下「兩個都很差」的票,平均增加 1.71 個百分點。
- 這類投票原本平均佔 17.1%。不過在 2024 年,較強模型在有明確勝負的投票中仍贏得差不多比例,因此模型排名大致維持不變。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
Rohan Paul@rohanpaul_ai
·9 天前
People rate the exact same AI chatbots more harshly each month, so a chatbot that never changes starts to look worse.
They studied 1,260,967 blind votes on Chatbot Arena, where people compare answers from 2 anonymous models. They kept only pairs of frozen models, such as dated snapshots and fixed open weights, so the models never changed.
On the same pair, "both are bad" votes rose 1.71 percentage points a month, on a 17.1% average. Yet in 2024 the stronger model won about the same share of decisive votes, so rankings held.
推文來源 · X (Twitter)在 X 參與討論
來源: Rohan Paul (x.com)
在 X 查看推文