跳到正文
Hacker News (official RSS)(arxiv.org)·· 14 天前品質評分57

Next-Concept Modeling 以由粗到細的平行生成加速文字生成

Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

重點導讀

Next-Concept Modeling(NCM)提出兩階段、由粗到細的文字生成方法,逐層預測離散程式碼,並在多項文字生成基準中以更快速度達到相近或更好的結果。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

正文

View PDF HTML (experimental)

Abstract:Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant computational workload. However, DLMs degrade quickly with fewer sampling steps, process the full sequence at every step, and require many passes over the dataset to reach good quality. We introduce Next-Concept Modeling (NCM), a two-stage, coarse-to-fine framework: a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes, and a transformer generates one level at a time, predicting all codes within a level in parallel. At matched parameter scales, NCM achieves better generative perplexity than the strongest DLM baseline on LM1B while running $21\times$ faster. On OpenWebText, NCM is competitive with the strongest DLM while running $10\times$ faster; on XSum, it matches the strongest DLM's ROUGE-L while being $31\times$ faster. On the reasoning task of Sudoku, NCM outperforms autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations. On the reasoning task of Sudoku, NCM outperforms autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2610.14162 [cs.CL]
  (or arXiv:2610.14162v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2610.14162

arXiv-issued DOI via DataCite

Submission history

From: Hermann Kumbong [view email]
[v1] Fri, 9 Oct 2026 17:49:45 UTC (352 KB)

讀者討論

理性交流 · 即時討論