跳到正文
hackernews100(arxiv.org)· lwhsiao·· 2 天前品質評分69

研究讓 27B 語言模型靠推理修改自己的權重

Language Models Can Rewrite Their Own Weights

重點導讀

語言模型通常要再訓練一次,或在回答時多花運算,才能改善表現;這篇研究提出讓模型自己修改權重的方法。

  • 研究團隊讓一個開放權重的 27B 模型檢查答錯的題目、修改自己的權重,再用練習題確認修改是否有效,只保留有用的版本。研究指出,修改後的模型在沒看過的題目上準確率提高 6.6 個百分點,連續 8 次修改後表現也都上升。
  • 它還能修復一份受損的自身副本,讓副本在沒看過的影像題目上恢復到原本準確率的 80% 以上。這些結果來自研究中的測試,摘要也指出同一模型系列裡較小的模型無法做到這件事。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

正文

Authors:Jiajun Fan, Neil He, Changyi Yang, Andy Mao, Xi Zhu, Chaoran Cheng, Kaiqu Liang, Shuaike Shen, Zetian Li, Ruihan Guo, Ge Liu

View PDF HTML (experimental)

Abstract:A language model is trained once and frozen. Making it better takes another training run, or a prompt, a harness or extra computation at inference that is gone once the task ends. We show a third way: to our knowledge for the first time, a language model rewrites its own weights by its own reasoning and becomes better, with no gradients and no training. We call this self-rewriting: the model is its own optimizer. In earlier work, a frontier model edited a much smaller one; here an open 27B model, given its weights, its graded answers and the ability to run itself, diagnoses its own failure and rewrites the weights behind it. It surpasses itself: its own edit makes it 6.6 points more accurate on problems it never saw. It keeps getting better: it tests its edits on practice problems and submits only what works, each edit improving on the last, and over eight consecutive sessions its accuracy only rises. It repairs a damaged copy of itself, like a broken robot that fixes itself without being rebuilt: the copy solves almost nothing, yet with no retraining and without ever reading the original weights, the model brings it back to over 80% of the original accuracy on image tasks it never saw. And the ability emerges with scale: in one model family, the smaller models make no edit or break themselves, and only the largest improves and repairs itself. Unlike test-time scaling, self-rewriting writes its gain into the weights, at no cost per answer; unlike training, it runs only forward passes, as serving does. The wall between training and inference is a limit of our tools, not a property of intelligence. We believe self-rewriting is a necessary step toward recursive self-improvement: a model that, like a human, does not separate training from inference could keep upgrading itself for life, with no larger model, no harness or prompt engineering, and no pre- or post-training.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21885 [cs.AI]
  (or arXiv:2611.21885v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.21885

arXiv-issued DOI via DataCite

Submission history

From: Jiajun Fan [view email]
[v1] Mon, 16 Nov 2026 20:55:08 UTC (6,363 KB)
[v2] Wed, 18 Nov 2026 08:57:59 UTC (6,379 KB)

來源: hackernews100 (arxiv.org)
開啟原文

讀者討論

理性交流 · 即時討論