跳到正文
𝕏 Rohan Paul(x.com)·· 13 天前品質評分54
重點導讀

NVIDIA 等團隊發現,無關文字主要會削弱模型運用問題作答的能力,而非直接推向錯誤答案。

  • - 可能機制:研究者在開放權重模型中找出一小組「intrusion heads」,會讀取無關文字並削弱答案。
  • 遮蔽這些注意力頭對上下文的注意力,可保留使用有用證據的注意力頭。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Rohan Paul
Rohan Paul@rohanpaul_ai
·13 天前
𝕏

New Nvidia + other labs paper shows that irrelevant text hurts mainly by weakening the model's use of the question, not by directly pushing against the right answer.

Prompts in RAG, tool use, and long-context apps often carry text that has nothing to do with the question. Even a line like "Computers are useful" lowers a model's support for the right answer.

They traced this inside open models and found intrusion heads, a small set of attention heads that read the unrelated text and weaken the answer. Masking those heads' attention to the context leaves the heads that use helpful evidence untouched.

Across 4 tasks on 7B-scale models, masking about 10 heads recovered 57% of the accuracy lost to distractors on average. An oracle fix that needs a matching clean run recovered 97.6%.

If you serve open-weight models with noisy retrieved context, a training-free mask on a few model-specific heads can win back much of the lost accuracy.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論