CMU 的研究顯示，編碼代理能夠自動調校 LLM 服務堆疊，將 GPU 成本效益提升至 4.87 倍。
- 實驗物件：Claude Sonnet 5.5 建構 Gemma E4B 的服務系統。
AI 伴隨問答
針對這篇報導有疑問？可直接點選快捷問題或向 AI 追問背景脈絡與技術細節：
Impressive paper from CMU.
Soon, we will have coding agents that can tune the entire stack, from prompts to token serving.
This work experiments with coding agents to tune an LLM serving stack.
They get Claude Sonnet 5.5 to build a serving system for Gemma E4B that delivers 4.87x the value per GPU dollar of the vLLM reference.
ServerBench asks coding agents to design a complete, specialized serving system and decide how many concurrent sessions to run. The score is latency-qualified token value per dollar of GPU cost, measured by replaying real coding sessions with their dependencies intact. An independent check verifies the system works.
Across 11 coding-agent models and three deployments, 22 of 33 runs produced a verified system and 13 beat the strongest baseline.
The best designs raised capacity and chose the load to match it. Failed runs often shipped optimizations whose new code paths were never tested.
Chat with Paper: https://t.co/ODYAwhxGUJ