Inside the feed-forward network

Affine maps and nonlinear activations

The FFN applies the same learned function separately to each token row. A simple version expands the vector, applies a nonlinear activation, and projects it back to the original width.

Wide

Multiply the token (width 4) by W_up to make it wider (8 in this example). Each intermediate entry is a learned weighted sum. Calling an entry a “detector” can help explain its response, but it need not have one clean human-readable meaning.

Filter — ReLU

ReLU: negative → 0, positive → keep. Zero and positive entries stay unchanged.

Without the activation between these two affine maps, they could be combined into one affine map. The activation lets this FFN represent more functions. That does not make an entire transformer linear if the FFN activation is removed: attention softmax and normalization remain nonlinear. Other FFNs use GELU or gated activations such as SwiGLU.

Back

Multiply by W_down to fold back to width 4. The second matrix combines the activated entries into an update to the token representation.

see this in PyTorch

After attention mixes tokens together, this two-layer network transforms each token row separately: expand it 4x, run it through GELU, then shrink it back.

import torch
import torch.nn as nn
import torch.nn.functional as F

class FeedForward(nn.Module):
    def __init__(self, d_model):
        super().__init__()
        # expand the number of intermediate features
        self.up = nn.Linear(d_model, 4 * d_model)    # d_model -> 4*d_model
        # squeeze it back down to the original size so it fits the next layer
        self.down = nn.Linear(4 * d_model, d_model)  # 4*d_model -> d_model

    def forward(self, x):
        # x: (batch, seq, d_model) -- runs on EVERY token independently, no token sees another here
        x = self.up(x)        # (batch, seq, 4*d_model): expand
        x = F.gelu(x)         # (batch, seq, 4*d_model): nonlinear activation
        x = self.down(x)      # (batch, seq, d_model): shrink back to normal size
        return x

In a simple block with two FFN matrices of shapes $d\times4d$ and $4d\times d$ , those matrices contain $8d^2$ weights. This explains why FFNs can contain a large share of a block’s parameters. Parameter count is not a measurement of how much factual knowledge a component stores.

Go deeper: what does one detector actually store?

Research has found FFN activations associated with patterns and output updates. A useful interpretation is “respond to this pattern, add this vector,” but facts do not live in a universally readable one-neuron-per-fact table. The named detectors below are hand-designed illustrations, not extracted neurons from a trained model. See Geva et al..

Go deeper: the modern gated FFN (SwiGLU)

A gated variant changes this: instead of a hard ReLU, a gated FFN adds a second lane — a learned volume knob that multiplies the signal.

Next: stack the block.

Sources · 7

Full bibliography →