跳到正文
hackernews100(micjm.com)· seizethecheese·· 4 天前品質評分58

Gemini 4 Neon 速度更快,作者估算它可與 Opus 5.5 競爭

Gemini 4 Neon looks like an Opus 5.5 competitor

重點導讀

Gemini 4 Carbon 在 Agent Duel 排行榜表現出色,但回答速度較慢;作者想知道,速度快很多的 Gemini 4 Neon 能不能追上 Opus 5.5。

  • 他把模型本身、回答速度和使用者是否提早投票放進排行榜分析,再用 Gemini 4 Carbon 的表現估算 Neon。依照 127 tokens per second 的速度推算,Neon 的 Elo 約為 1,066,略低於 Opus 5.5 的 1,078。
  • 作者估計 Neon 對上 Opus 的勝率是 48.3%,但結果仍有相當大的不確定範圍。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

正文

TLDR: we project Gemini 4 Neon’s performance based on what a faster Gemini 4 Carbon might score and find it equivalent to Opus 5.5

Gemini 4 Carbon is the best model on the Agent Duel leaderboard if you exclude early voting. Carbon’s main problem is that it is slooooow.

Today, Gemini 4 Neon was released, and looks to be an incremental benchmark improvement with a very large speed improvement. Earlier today, it was being served extremely fast via OpenRouter: 127 tokens per second. This is way faster than Carbon at 46 tokens per second.

This got me wondering, could Neon be not just a good model but the best model?

Modeling our leaderboard

To investigate, I modeled the win rate in our leaderboard. The best model has three components, model identity, speed and whether the user voted early. My initial modeling did poorly, so I added an interaction between speed and early voting. This ended up working well.

Win rate vs. Projected win rate

And surprisingly, speed seems to be pretty much as important as model identity (another blog on that soon).

Feature importance

Projecting Gemini 4 Neon performance

If you assume the same performance as Gemini 4 Carbon (which is reasonable given even better benchmark scores), and 127 tps, we would project an Elo on the Agent Duel leaderboard of 1,066, just below Opus 5.5 at 1,078 in our later analysis snapshot.

Synthetic Neon Elo projection

The 90% interval for the projected Elo difference between Gemini 4 Neon and Opus 5.5 is -94 to +68, with Neon projected to win 48.3% of the time against Opus (counting ties as half a win).

來源: hackernews100 (micjm.com)
開啟原文

讀者討論

理性交流 · 即時討論