SketchAgentBench 評估 LLM 在 3D 空間推理上的表現

A benchmark for 3D coding agents

11 models · 25 reference photos · 275 runs

One photo. One prompt. In a browser three.js playground, agents code, render, and refine a 3D scene—testing spatial understanding and coding.

CompareAny two models, side by side _→_

1. (1)Results
2. (2)Models
3. (3)Method

Results

How well did each agent model what it saw? A vision model compares two runs of the same photo, blind to names and costs; the ranking below is the Elo fit of those votes.

1. Gemini 4 Argon has the highest floor: it never placed below #4 on any photo.
2. Gemini 4 Argon is the steadiest of the top six: its rank moves by 3 places across the photos it was ranked on.

Anchored at Claude Sonnet 5.5 = 1000; only gaps between models matter (100 points ≈ 64% head-to-head). Equal Elo shares a rank; ties are ordered by worst photo, then recorded cost.

Elo vs. costBlue rings highlight the best Elo–cost trade-offs (the Pareto frontier). Notes compare neighboring models on the frontier. Totals include only recorded data.Frontier model Other model: matched or beaten using no more money Curve and shading: visual guides, not predictions

Elo, cost, time and tokens for each model, cheapest first

1. Gemini 4 Argon leads at Elo 1544; GPT-6 Astra is 4 behind, inside the overlap of their 95% intervals.
2. Kimi K3.1 Pro has the highest Elo per dollar: 1185 for $1.50 in recorded API cost.
3. Gemini 4 Argon costs 7.4× as much as Kimi K3.1 Pro for 359 more Elo.

Best run for each photo

What agents built from each of the 25 reference photos. These are the highest-scoring final scenes; ties favor the lower recorded cost.

Every model. Every photo.

All 11 models tackling the same 25 spatial modeling tasks. Each cell is the model's rank on that photo, where 1 is best. ✕ marks a run that failed and produced no model to rank. Open a model to explore what it built.

Choose a reference photo

How the benchmark works

Same photos, same prompts, same step budget.

By the numbers

recorded API cost

total recorded runtime

recorded tokens

recorded model steps

successful runs

Vote Two runs of the same photo are shown to a vision model together, with model names and costs hidden. It compares six rendered views of each scene and picks A, B, or tie—1120 votes over 560 pairs.Elo Those votes become one Elo per model, anchored at Claude Sonnet 5.5 = 1000.

Build, inspect, refine. Agents work in an in-browser three.js and manifold.js playground that emulates Blender's core modeling loop—writing geometry code, checking rendered views, and refining 3D structure—to test spatial understanding and coding. Every run can be replayed and edited live in the playground.

Reading the Elo. Only the gap between two models matters; the sign just says who is ahead. A 100-point gap is a 64% win chance for the model ahead, 36% for the one behind; 200 points, 76% and 24%; 400 points, 91% and 9%. Claude Sonnet 5.5 is pinned at 1000 as a reference point.

Questions & answers

What the benchmark measures.

01 What do agents do in this benchmark?
Reconstruct a reference photo as a 3D scene using code. Agents write geometry, inspect rendered views, and revise the scene to better match the photo.

02 What capabilities does this reveal?
It probes spatial reasoning: inferring shape, scale, depth, and object placement from an image. It also probes whether an agent understands how a 3D scene projects into a 2D view, including perspective and occlusion.

03 Why does this matter beyond reconstruction?
These skills also matter when composing scenes from existing 3D assets: choosing their scale, orientation, and placement, and modeling simple missing objects directly. The benchmark probes foundations of that workflow, rather than asset-library use itself.

04 Why a browser three.js playground instead of Blender?
Heavy 3D suites like Blender conflate spatial reasoning with software-specific workflows, and agents easily get sidetracked tweaking shader nodes and textures—work that diffusion models handle far better than code. By running a minimal manifold.js geometry runtime and three.js renderer in the browser, the benchmark keeps materials simple and focuses strictly on spatial understanding, 3D structure, and coding. With zero local setup, you can replay any run or test your own agent right in the playground.

05 Is this image-to-3D generation?
Not in the diffusion or neural-reconstruction sense. Rather than generating a textured mesh in one pass, the agent writes explicit, editable code to construct clean geometry and part hierarchy through visual feedback.

ResultsModelsMethodFAQ