Learn · LLM inference

Inside a model, level by level

Updated 11 Nov 2026 · 1 min read

What the model computes, from tokens down to the loops a GPU runs. Every level is a diagram from another page, so this page and those pages always show the same picture.

Click a box with this badge to open what is inside it.
Map · 29 levels

The 36 layers have the same shape and run one after another. Prefill runs the stack once over the whole prompt; decode runs it once per new token and reuses the KV cache each layer wrote.

One forward pass of gpt-oss-120b
TRANSFORMER LAYER × 36Text"kubectl get pods"Tokenizero200k_harmony BPE →token idsEmbedding lookupid → vector of 2,880Final RMSNorm →unembedding2,880 → 201,088 logitsSamplersoftmax, temperature,top-pNext tokentoken idsresidual streamx + attentionx + MLP, after layer 36last position only
Tokens in, next-token probabilities out. Both halves of a layer open into the same view: the kernels a GPU runs for one layer.

How to read it

  • Click a box with the badge to open it below; click it again, or press Esc, to close it. The breadcrumb goes back up; Map lists every level.
  • Boxes that lead to the same thing open the same panel: attention and MoE both open the layer’s kernels; every GEMM opens one GEMM view.

Where each level comes from

Level Page
The forward pass Tokens and transformers
One layer as kernels Prefill and decode kernels
Attention and the KV cache Attention and the KV cache, KV cache management
FlashAttention FlashAttention
The router Mixture of experts
A GEMM GEMM tiling