用 DOOM 遊戲構建神經網路：完整教學與實作指南

Neural networks that run on DOOM's game mechanics: voodoo dolls, conveyor belts, lava and doors.
Watch the films · Try it in Colab: draw a digit and DOOM reads it, nothing to install

A number is a crowd of voodoo dolls on a conveyor belt. An activation function is a staircase they have to climb. A matrix multiply is dolls running across lava lines, each line raising a room's light level. Attention is dolls squeezing under ceilings that another number has lowered. When the dolls stop, the light levels are read back out of the running game and compared bit for bit with an independent integer reference. One value off by one and the run fails.

_{The marine's brain is compiled into the level he is fighting in. The dots on the right are 1,345 voodoo dolls of him. They run his transformer and choose his next move; the level senses the demons and pulls the trigger. Watch the film.}

Walk a digit, DOOM reads it

The input is a 28×28 floor in the level. Every tile the marine walks on while holding FIRE inks itself, and the level writes that pixel into the brain. He presses READ. 1,355 voodoo dolls run an MNIST network, compare all 45 pairs of digit scores and count the wins. Only the winner's doll gets through, and it lights that digit on the wall. Python advances the clock, then reads back the 784 pixels, the hidden layer, every comparison and the answer, and checks all 248 values against NumPy.

doomformer draw               # opens a window: arrow keys to walk, hold Ctrl to draw, Space on the READ switch
doomformer draw --digit 7     # headless: the marine walks a 7 (Python presses his keys in place of a player)

          ▄▄▄▄▄▄▄▄▄▄▄▄▄▄
       ▄████████████████
       ████▀▀     ▀████▀
       ▀▀▀        ▄███▀
                  ████
                  ███
                  ███
                  ███
                 ████
                 ███▀
                 ████
                 ▀███

DOOM reads: 7  (right)   -- the winner's doll lit it on the 7-segment display
EXACT: all 248 values DOOM computed (hidden layer, 45 score comparisons and 10 win counts, each with its twin, and the one-hot answer) equal the NumPy integer reference computed from the 784 pixel ports DOOM holds (tolerance 0); one display firing
pixels: the 784 ports equal the level's canvas (the tiles the pen inked)
READ pass: 5,705 game tics = 163 s of real-time DOOM at 35 tics/s, 79 s wall | Python sent 58 commands, 29 readout scripts each followed by the ack script it waits for (0 waited more than one advance), and nothing else (0 rejected)

Every digit 0 to 9 in three walked variants: 30 read correctly, all 30 exact. In a window DOOM runs in real time, so a READ takes 2¾ minutes; headless it takes 76 to 99 s. How it works · film

Choose your skill level

pip install git+https://github.com/mottopanikeiku/doomformer   # Python 3.11+, Linux or macOS

The first run builds DOOM's script compiler, so you need git, cmake and a C compiler. Or skip all of it: the Colab notebook installs it, lets you draw a digit for DOOM to read, and runs the softmax demo.

Times are from a 16-core Linux machine running 8 DOOM games at once.

I'm too young to die: what you'll see

tiny integer MLP 4-6-5-3 compiled into one DOOM map: 60 voodoo dolls, 1222 sectors
input 1/5  x=[-71, -82, 64, 37] -> y=[11, 4, -15]  EXACT (3 outputs + 22 hidden readouts equal the integer reference; 488 game tics)
...
5/5 EXACT

Hey, not too rough: DOOM reads handwritten digits

784 pixels → 64 → 45 comparisons → 10 win counts → the answer, all in one DOOM map: 1,355 voodoo dolls and 158,550 sectors. Each ink pixel releases a doll that runs across the hidden layer's weight lines. Dolls then compare every pair of digit scores, and only the digit that wins all nine of its comparisons gets a doll through. DOOM picks the digit and Python checks it. In DOOM the network read 99 of the first 100 MNIST test digits correctly and 966 of 1,000 sampled from all 10,000, every one of them exact. On all 10,000 the same integer network gets 97.16 %.

$ doomformer mnist --digits 2 --seed 1

          ███
          ████                      ▄▄▄█████████▄▄
           ████                     ▀█████▀▀▀▀▀████
            ████                             ▄▄███▀
             ███▄                       ▄▄▄███████▄▄
             ████                       ████▀▀▀▀▀▀█████
              ████                                  ▀███
               ████                      ▄▄         ▄██▀
               ▀████                     █████▄▄██████▀
                ▀██                        ▀▀▀▀▀▀▀▀▀▀

DOOM reads: 1  right            DOOM reads: 3  right
test #46                        test #51

2/2 read correctly | 2/2 EXACT: every value DOOM computed (248 per digit: hidden layer, comparisons, win counts and
the one-hot answer) equals the NumPy integer reference (tolerance 0); Python only compared | 5,696 game tics per digit | 88 s

To read your own digit from a photo or drawing: doomformer mnist --image my7.png.

Hurt me plenty: a transformer names a level

A transformer trained on 136 official DOOM level names, split into 38 small maps that a pool of DOOM games runs. Python copies the integers between games and picks each letter from the logits DOOM computed.

DOOM wrote: 'GHEER'   (NOT in its training corpus: the model made it up)
  174 tile runs in 8 parallel DOOM games | 10,462 voodoo dolls in the tile maps |
  6,902 values read back and compared with the NumPy integer reference (tolerance 0) | 15 s
EXACT: every value DOOM exposed equals the integer reference, and the reference writes the same name

--seed 3, --seed 4 and so on give other names. --one-map puts a whole transformer into a single map (embedding, attention, MLP and logits with no Python between layers), at about 90 s per letter.

--softmax runs the same model trained with softmax attention. The dolls compute the exponentials (a staircase whose steps are spaced like a logarithm) and divide by their sum at run time (a count of comparisons). How that works is in docs/HOW_IT_WORKS.md.

DOOM wrote: 'FINAL INEALL'   (NOT in its training corpus: the model made it up)
  494 tile runs in 8 parallel DOOM games | 4,886 voodoo dolls in the tile maps |
  106,834 values read back and compared with the NumPy integer reference (tolerance 0) | 59 s
EXACT: every value DOOM exposed equals the integer reference, and the reference writes the same name

--one-map --softmax does it inside a single map with no Python between layers: 4,209 dolls, about 2 minutes per letter.

--model doomnames_modern adds RMSNorm and a gated (ReGLU) MLP to the softmax attention. The dolls also compute the sums of squares (as sums of odd numbers), the integer square root (a staircase at d·k²) and the division by it. It wrote MILL in 170 tile runs, all 62,000 values exact.

It writes on its own. --one-map --loop builds one map that writes a whole name with no Python in the loop. At every position the dolls compare all 30 logits, and the winner's doll moves the next position's token input to its letter. The level then rewinds and runs the next pass by itself, five passes for a 6-token context. Python only advances the clock and reads: every value of every pass is checked, and the final text has to equal greedy generation. It wrote DAN (5,875 dolls, about 3 minutes per pass), and DAGE with --softmax.

Ultra-Violence: ask DOOM a question

A 2-layer transformer trained on 189 facts about DOOM. Voodoo dolls across a few thousand DOOM game instances compute every value, and every value is checked. The first letter takes a few minutes and the rest arrive about every 20 seconds. Some answers, all exact in DOOM:

It knows its cheat codes. Ask it anything else and you get confident DOOM nonsense.

Nightmare!: the big brain

The 2-layer, 24,279-parameter transformer trained on 11,341 real DOOM level names, compiled into 106 maps with 56,390 voodoo dolls and 785,460 sectors. With these settings it writes DEMON CANYON, which is not among its 12,934 training names:

106 tile maps (56,390 voodoo dolls, 785,460 sectors) built in 6 s; running them in 8 DOOM game processes ...
DOOM wrote: 'DEMON CANYON'
  1,690 tile runs in 8 parallel DOOM games | 56,390 voodoo dolls in the tile maps |
  120,666 values read back and compared with the NumPy integer reference (tolerance 0) | 135 s
EXACT: every value DOOM exposed equals the integer reference, and the reference writes the same name

Close the doors in front of one neuron (L1.m unit 33) and the same brain writes DEMON DEATH.

How it works

A matrix multiply. Each doll is one unit of an activation, and each lava line it crosses adds a weight to a room's light level.	Attention. Dolls squeeze under ceiling gates that another live value has lowered; counting who gets through is a multiply.
Ablation is a door. Shoot the switch, the door slams, and one neuron's dolls never arrive.	DEMON CANYON, a level built from the name the DOOM brain wrote, played by a marine whose brain also runs on DOOM.

flowchart LR
  NET["your network<br/>(PyTorch / NumPy)"] --> INT["integers<br/>(quantised)"]
  INT --> MAP["a DOOM map<br/>lanes, gates, lava lines, doors"]
  MAP --> RUN["ViZDoom runs it<br/>voodoo dolls walk"]
  RUN --> READ["read back<br/>sector light levels"]
  READ --> CHECK{"== NumPy integer<br/>reference?"}
  CHECK -->|"every value"| EXACT["EXACT"]
  CHECK -->|"off by one"| ERR["MismatchError"]

Loading

DOOM turns every extra player-1 start into a voodoo doll: a copy of the player that rides conveyors and triggers every line it crosses. An activation of 5 is five dolls. A staircase lets a doll past each step only if the value clears that step's height, which gives ReLU, rounding and clamping in one shape. Sums accumulate in sector light levels, which turn out to be saturating int16 values.

Getting exact results meant measuring the engine. Two floor moves on one sector in the same tic: one is silently lost. A teleport line cancels lines crossed earlier in the same tic. Maps wider than about 34,000 units put things in the wrong room. The full list is in docs/HOW_IT_WORKS.md and docs/engine_limits.md.

Hall of fame

The bigger experiments, all exact. Their code is in demos/, with training in training/ and every trained model in models/. Numbers and caveats for each are in docs/DEMOS.md and docs/FLY.md.

	A transformer named a DOOM level. DEMON CANYON, written across 1,594 DOOM tile instances. film
	DOOM plays DOOM. A marine whose policy runs on 55 DOOM games walks the level DOOM named, which he has never seen: doors, two kills, exit. 102 decisions, all exact. film
	DOOM learns. Weights are ceiling heights, and learning is voodoo dolls raising and lowering them. A blank brain taught by an aimbot inside one game scores 4 kills in its first life, then 24, 26, 22 and 24. 931/931 decisions exact. film
	A fruit fly. A 255-neuron circuit from the FlyWire connectome steers the marine from inside one map: eat, refuse, groom, escape. 169/169 steps exact. film · code
	A whole transformer in one map. Embedding, attention, MLP and logits with no Python between layers: 3,548 dolls write SHIL. film

• His brain is the level. A level script senses the nearest demon, 1,345 voodoo dolls of the marine run his transformer and pick the action, and the level pulls the trigger. Python advances the clock and reads. 50/50 decisions exact. film · code
• Lobotomy. One 7-pellet shotgun blast to his DOOM brain turns him into a pacifist with 0 kills. This lesion was picked from a scan of 3,000 random blasts. film
• The big one. A 4-layer transformer writes UNHOLY STRONGHOLD over 10,166 DOOM tile instances and 1.9 million game tics. film
• The showreel.

Build your own

Compile your own network into a DOOM map and let its outputs open doors, flash lights or shove the marine.

import doomformer as df

net = df.MLP.from_torch(my_torch_mlp, calib_x, levels=8)   # or df.MLP(...) by hand, or df.MLP.from_numpy(...)
print(net.quantization_report(test_x))                     # how close the integer model is to your float model
with df.compile(net, actions={"y0": [df.act.door_open(tag=7)]}, world=my_room) as brain:
    y = brain(net.encode(x))                               # computed by voodoo dolls, verified exactly
    brain.record([net.encode(x)], "dolls.gif")             # film the dolls doing it

Spiking networks (df.SpikingNet), transformers (df.Transformer.load("doomnames")), conv nets (from_torch lowers Conv1d/2d exactly) and learners that update their weights inside the map (df.Perceptron) work the same way. df.with_argmax lets the dolls pick the winning output. df.Agent puts the senses and the muscles in the level as well. Models too big for one map spread over a pool of DOOM games with mode="tiles", and doomformer plan tells you which one you need.

The guide is docs/BUILD_YOUR_OWN.md, and there are twelve small runnable examples. Made something cursed? Show and tell.

More DOOM

One map is one game, and one game uses one core. df.run_many gives each input to the next idle game, checks every pass, and returns the outputs in order. The games run on your machine or on any Ray cluster you start yourself.

1,000 MNIST test digits in 14 DOOM games on one 16-core machine took 2 hours: 966 read correctly, 1,000 exact, and 5,696,000 game tics, which is 45 hours of real-time DOOM. Every digit's result is in models/mnist/doom_1000.json.

ys = df.run_many(net, xs, workers=8)                  # 8 DOOM games on this machine
ys = df.run_many(net, xs, backend="ray")              # every CPU of your Ray cluster

pip install "doomformer[ray]"
ray start --head                                      # then `ray start --address=HEAD:6379` on the other machines
doomformer mnist --all --ray --record mnist.json      # all 10,000 MNIST test digits, every result in one file

Budget one core and 1.25 GB of RAM per game for the MNIST map. Details in docs/scaling.md.

What's in the box

Cheat codes (FAQ)

IDCLIP Is this really DOOM?

It is ViZDoom 1.3.2, a ZDoom-based engine, with Freedoom's free assets, running headless. The maps use ZDoom's UDMF format and ACS level scripts. ACS moves the input gates, resets the dolls and copies results out. In the closed-loop levels it also senses (the nearest demon, the canvas tiles the marine walked on) and carries out the action the dolls chose. Every multiply-add and activation is dolls crossing lines, and in DRAWPAD, mnist and his brain is the level so is the final choice.

IDKFA What does Python do, then?

It compiles the maps, writes the inputs, waits, reads the results and checks them. In DRAWPAD and his brain is the level the level writes its own inputs. For models split over many maps, Python copies integers between games, and for text it picks each letter from the logits DOOM computed, except with demo --one-map --loop, where the dolls pick every letter and write it back themselves. Attention is ReLU-style integer attention, or softmax attention computed by the dolls (demo --softmax); doomnames_modern adds RMSNorm and a gated MLP. Training happens offline in PyTorch, except for the learners that update their own weights inside the map.

IDDQD How do you know DOOM computed it and not Python?

Every value is read back from the running game and compared with a NumPy integer reference that never sees the map, with tolerance 0. The tests also try to break it. They raise one weight line in a compiled map from 5 to 6, and the check fails by exactly +1 on exactly the inputs whose dolls cross that line. They make a readout echo a value Python wrote, build maps without the barriers the engine needs, and let a settle check trust ViZDoom's 4,096-object list. Each of these has to be caught.

IDSPISPOPD Is it fast?

No. One small MLP pass takes 726 ms in DOOM and 11 µs in NumPy, about 70,000× slower. Reading one handwritten digit takes 5,705 game tics: 76 to 99 s headless, 2¾ minutes of real-time DOOM. The demons are frozen while the marine thinks. Speed was never the goal.

IDBEHOLD Does the marine actually learn to play?

He imitates a scripted aimbot that sees exact demon positions, binned into 86 possible observations. That is supervised imitation. There is no reward-driven RL and no learning from pixels. What matters here is where the network runs.

IDDT Why?

People have run DOOM on calculators, pregnancy tests and lawnmowers. This goes the other way: DOOM is the computer.

Licensing

Code and docs: MIT. The shipped level-name lists (Doom Wiki, CC BY-SA 4.0; id Software's source, GPL-2.0) and the 100 sample MNIST digits (CC BY-SA 3.0) keep their licences; see THIRD_PARTY.md. Freedoom graphics are BSD-licensed. DOOM is a trademark of id Software / ZeniMax, and this project is not affiliated with them.