升級破壞現有程式碼:LLM 在舊版 PHP 上的研究
When Modernization Breaks Working Code: A Study of LLMs on Legacy PHP
重點導讀
LLM 在將舊版 WordPress 4.0 PHP 轉換為 PHP 8.3 時,常常破壞原有功能,研究顯示不變基準往往表現更佳。
- 測試規模:5 個 LLM、100 個 WordPress 4.0 PHP 檔案,使用 630 個隱藏行為檢查與 511 個 Rector 發現。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
正文
Abstract:Pasting a legacy file into an LLM and asking it to "modernize this" is an easy shortcut. But is the result better than leaving the code alone? In this paper, we study this single-file setting with five LLMs, each asked to modernize 100 WordPress 4.0 PHP files to PHP 8.3 while preserving behavior, with no project context or execution feedback. We evaluate every output with 630 hidden behavioral checks and 511 Rector-detected modernization findings. We compare against two baselines: the unchanged originals and the rule-based tool Rector. Often, the answer is no. Untouched, 83 files already pass every check; the best LLM leaves only 59. Each model breaks 32-51 working files while repairing at most 10, whereas Rector breaks 7 and repairs 6. Most of this damage is unnecessary. Only 37 findings (7 percent) require a change under PHP 8.3, yet in compatible files with no required change, the LLMs break 36-60 percent (Rector: 8 percent). The model with the highest modernization score is no more compatible than its closest competitors. Most regressed checks come from files that no longer load at all, often because of type declarations PHP itself rejects. These are errors that need no project context to avoid. Evaluations of LLM-based modernization should therefore measure against a no-change baseline and report modernization and behavioral preservation separately.
| Comments: | 6 pages, 5 figures, 5 tables. Preprint |
| Subjects: | Software Engineering (cs.SE) |
| Cite as: | arXiv:2610.40269 [cs.SE] |
| (or arXiv:2610.40269v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40269 arXiv-issued DOI via DataCite |
Submission history
From: Afridi Mahmud [view email]
[v1]
Thu, 29 Oct 2026 15:29:45 UTC (1,421 KB)
來源: Hacker News (official RSS) (arxiv.org)
開啟原文