Glossary

Plain-language definitions for the terms used throughout the library. In lessons, linked terms open a quick definition so you can keep your place.

A

antiderivative

A function F whose derivative equals the given function f on an interval.

For f(x) = x², both F(x) = x³/3 and F(x) = x³/3 + 7 are antiderivatives. On an interval, any two antiderivatives differ by a constant. If f is continuous on [a,b], its definite integral equals F(b) − F(a). Check the domain: the rule for 1/x cannot be applied across zero as an ordinary definite integral.

F′ = f; ∫ₐᵇ f(x) dx = F(b) − F(a)

argmin

The input choice that makes an expression as small as possible. Read it as “which choice gives the minimum?”

For observations 0, 0, 9, the constant prediction that minimizes average squared loss is 3. The minimum loss is 18. Argmin returns 3; min returns 18. If several choices tie, argmin can denote a set of choices.

acoustic codebook

A numbered shelf of reusable sound patterns.

Each shelf position stores a list of numbers describing one learned sound pattern. A sound description is replaced by the number of the closest shelf entry. The number is small enough to store or predict, and the stored pattern can later help rebuild the sound.

Find the nearest shelf entry; store its number and recover its sound pattern when needed.

acoustic feature

A number or list of numbers that describes a useful part of a sound.

Raw microphone samples are difficult to interpret directly. A feature keeps a useful pattern, such as energy in several frequency bands, while leaving out some unnecessary detail. Learned neural-network features can also describe the local sound and its context.

raw sound -> useful numerical description

ad hoc retrieval

The classic search setup: you type a query, the system hands back a ranked list of documents.

Ad hoc retrieval is the standard information-retrieval task — a user with an arbitrary, unpredictable information need poses a query, and the system returns documents from its collection ranked by how well they match. ‘Ad hoc’ means the queries are not known in advance (unlike a fixed set the system was built around), so it must handle whatever a user happens to type. It is the shape of a web search box, and it is the retrieval step that feeds RAG. Everything else in this book — how documents and queries become vectors, how matches get scored — is machinery for doing this one task well.

query -> ranked list of documents from the collection

adaptive layer norm (adaLN)

Compute a layer’s scale and shift from the conditioning instead of learning them as constants.

A denoiser needs to know how noisy its input is and what the picture should be of, and a transformer block has no third input for that. Appending the condition as tokens or adding a cross-attention layer both treat it as content to be read. Adaptive layer norm does not: normalisation already rescales and re-shifts activations, so those numbers are computed from the condition. It never enters the sequence; it changes how the block behaves. Nearly free, and it produced the best images of the three.

condition -> scale gamma, shift beta -> applied after each norm

agentic RAG

RAG where the model itself decides whether to look something up, and where to look, instead of always running a fixed search.

Ordinary RAG always runs the same retrieval step: one search over one collection, whether or not the question needs it. Agentic RAG instead lets the model decide — should I retrieve at all, from which collection or tool, and with what query — treating search as an action it can choose to take, sometimes several times. This helps because a fixed retrieval both wastes effort and injects distracting passages when the model already knows the answer, while a single canned search can’t handle questions that need evidence from different sources. It’s the retrieval side of the broader move toward models that plan and call tools, closely tied to the chain-of-thought and reasoning ideas in the post-training book.

anisotropy

When a model’s vectors all crowd into a narrow cone instead of spreading out evenly.

Anisotropy describes contextual embeddings that, rather than filling the space in all directions, pile up in a thin cone — so even unrelated words end up with surprisingly high cosine similarity. This is a wrinkle Book 01 didn’t have to worry about much: it means raw cosine-similarity between two contextual vectors can be misleading, because almost everything looks somewhat alike. People correct for it with tricks like subtracting the mean vector or whitening before comparing.

vectors cluster in a narrow cone -> cosine-similarity inflated even for unrelated words

approximate nearest neighbor (ANN)

Quickly finding the vectors closest to your query vector without checking every single one.

Dense retrieval turns search into a geometry problem: given the query’s vector, find the document vectors nearest to it. Doing that exactly means comparing the query against every stored vector, which is hopeless once there are millions or billions of them. Approximate nearest neighbor search builds clever index structures — proximity graphs, coarse clusters, compressed vectors — that let you skip the vast majority of comparisons and still almost always return the truly closest vectors. It trades a sliver of accuracy for orders-of-magnitude speed, and it is what makes dense retrieval usable at real-world scale.

find d maximizing d . q, while examining far fewer than all stored vectors

attention

Combine value vectors using weights calculated from queries and keys.

In scaled dot-product attention, a query is compared with allowed keys. Softmax converts the scaled scores into nonnegative weights that sum to one. Multiply each value vector by its weight and add the results. Causal attention blocks future positions. A large attention weight means that value receives a larger coefficient; it does not by itself establish relevance or explain the model’s decision.

query–key scores → mask and softmax → weighted sum of values

attention encoder-decoder (AED)

A speech recognizer with one part that reads the recording and another part that writes the text.

AED is short for attention encoder-decoder. The encoder changes the long recording into useful sound summaries. The decoder writes one text piece at a time. Attention tells the decoder which sound summaries matter for the piece it is writing. Listen, Attend and Spell (LAS) is a classic AED system.

Read the recording into sound summaries; use them to write one text piece at a time.

attention head

One worker that lets each word look at all the words (itself included) and pull in the ones it cares about.

An attention head is one small computation with its own learned weights that turn each word into three vectors: a Query (what am I looking for?), a Key (what do I offer?), and a Value (what I’ll pass along). Every word’s Query is compared against every word’s Key, including its own, to get scores; softmax turns those scores into weights that add up to 1, and each word collects a weighted blend of all the Values (its own plus the others’). This works because meaning depends on context, so a word can pull in exactly the words that matter to it. A transformer runs many heads side by side, each free to specialize (one might track grammar, another links pronouns to names), then combines their outputs.

attention(Q,K,V) = softmax(Q·K^T / sqrt(d)) · V

audio codec

A learned compressor that turns a long recording into short sound codes and can rebuild the recording from them.

The encoder shortens the recording into one sound description per short time window. A quantizer replaces each description with one or more small code numbers. The decoder looks up those codes and rebuilds audio measurements. The compact codes are easier for a sequence-prediction model to write than every raw microphone measurement.

recording -> compact sound-code numbers -> rebuilt recording

audio codeword

One reusable sound description stored at a numbered position in a codebook.

The compact audio token is only an integer, such as 317. Position 317 in the codebook stores the actual list of numbers used by the decoder. That stored list is the codeword.

token number -> codebook lookup -> stored sound description

audio sample

One microphone measurement at one instant in time.

Digital audio is a long list of measurements. Each measurement is a sample. More samples per second preserve faster changes in the sound, but they also create more numbers to store and process.

one time instant -> one number

autograd

Automatic differentiation: calculate derivatives from recorded operations.

PyTorch records a computational graph during the forward pass when gradient tracking is enabled. Backward applies the chain rule: multiply derivatives along paths and add contributions where paths meet. Gradients accumulate in the parameters’ .grad fields.

forward: calculate and record → backward: chain rule → parameter gradients

automatic speech recognition (ASR)

Using a computer to turn a recording of speech into written text.

ASR is short for automatic speech recognition. The system first changes the recording into small sound descriptions, then uses surrounding sound and language patterns to choose written symbols. It can output letters, word pieces, or whole words. Noise, microphones, accents, and speaking style can all change its accuracy.

Input: a speech recording. Output: written text.

autoregression

Create one token, add it to the history, and use that history to create the next token.

An autoregressive model builds a sequence one token at a time. Each new choice depends on the tokens already produced. This works for text tokens and for audio-code tokens. Because later tokens must wait for earlier ones, the method is ordered and usually slower than predicting many positions together.

next token = model(tokens already made); append and repeat

autoregressive (AR) model

A model that makes one token at a time, with each new token depending on earlier ones.

AR is short for autoregressive. The model predicts the first token, adds it to the history, predicts the next token, and repeats. This preserves order and lets earlier choices guide later ones, but positions cannot all be produced at once.

one token -> append -> next token -> repeat

axis

A direction through an array; operating ‘along an axis’ collapses that direction.

An axis is one of an array’s dimensions — a direction you can walk. Counting starts at 0 for the outermost (rows in a 2-D array), and -1 is shorthand for the last one (columns). The key rule for reductions like sum, mean, or argmax: the axis you name is the axis that disappears. argmax(axis=-1) on a (3, 5) array walks across the 5 columns and returns one answer per row, shape (3,); argmax(axis=0) walks down the 3 rows, shape (5,). Give no axis and the array is flattened first, collapsing every axis to a single scalar.

reduce(A, axis=k): the shape loses its k-th entry

B

back-translation

Use a reverse translator to turn monolingual target text into synthetic training pairs.

Back-translation starts with abundant real text in the desired target language and sends it through a target-to-source model. The machine-made source plus the original human target becomes an extra parallel pair for source-to-target training. It is powerful because fluent target text is cheap, but synthetic source errors must be filtered and mixed carefully so the system does not amplify its own mistakes.

real target -> reverse MT -> synthetic source + real target pair

backpropagation

An efficient way to compute derivatives of a scalar loss through a computation graph, using the chain rule in reverse.

The forward pass computes outputs and intermediate values. The backward pass propagates derivatives from the loss toward the inputs and parameters, multiplying along a path and adding contributions where paths meet. Backpropagation computes gradients; an optimizer performs the parameter update. It does not guarantee that an update will improve every example.

forward: calculate values; backward: calculate loss derivatives

backward induction

Solve a finite decision tree from its final decisions backward. At each decision, compare the acting player’s payoffs, using the choices already worked out for later decisions.

bag of words

Treat a text as just a pile of word counts, throwing away the order.

The bag-of-words view represents a text by how many times each word appears and nothing else — ‘dog bites man’ and ‘man bites dog’ look identical. Discarding word order sounds reckless, but for judging what a document is about it works surprisingly well: the mere presence of ‘Louvre’, ‘painting’, and ‘Paris’ pins down the topic regardless of arrangement. It is what lets a document collapse into a simple vector of counts for the vector space model. This is the opposite extreme from the transformers book, where word order and position are everything — different tools for different jobs, topic-matching versus deep understanding.

text -> {word: count} multiset, order discarded

base model

A pretrained model before the particular post-training stages being discussed.

A next-token-trained base model assigns probabilities to continuations. It can already perform some tasks from instructions or examples in context, but it has not necessarily been optimized for the behavior expected of an assistant. “Base” describes its training stage, not a guarantee about what it can or cannot do.

Bayes’ rule

How beliefs update: start from the prior, add what the evidence is worth.

Bayes’ rule connects the probability you had before the evidence (the prior) to the one you should hold after (the posterior). Its textbook fraction form, P(H|E) = P(E|H)·P(H)/P(E), hides the intuition; in odds form it collapses to posterior odds = prior odds × likelihood ratio, and on the log-odds ruler multiplication becomes addition: posterior = prior + evidence. That last face makes updating feel like arithmetic — strong evidence is a big stride, a rare hypothesis is a start deep below zero, and stacking independent evidence is just more addition. It is the normative core of reasoning under uncertainty, and the reason base-rate ‘paradoxes’ dissolve once the starting point is honest.

posterior odds = prior odds × likelihood ratio

beam search

Keep several promising unfinished answers instead of committing to only the first guess.

At each step, beam search extends several possible sequences, scores them, and keeps the strongest few. This can save a sequence that starts slightly worse but becomes better later. It costs more work than greedy search, which keeps only one choice. In speech recognition, the completed candidates can form an n-best list for another scoring pass.

Extend candidates, score them, and keep the best few.

BERT

The original famous encoder that learns by filling in blanked-out words.

BERT (Bidirectional Encoder Representations from Transformers, 2018) is the model that made the read-both-directions, understand-don’t-write recipe famous. It’s pretrained on huge text by masked language modeling — hide some words and predict them — which forces it to build deep contextual embeddings. Once pretrained, BERT is fine-tuned for specific jobs (sentiment, question answering, tagging) by adding a tiny head on top, the same transfer-learning idea as Book 01’s fine-tuning, just on an encoder instead of a writer.

BERTScore

A metric that matches candidate and reference tokens by contextual embedding similarity.

BERTScore embeds the candidate and reference with a pretrained contextual model, greedily matches tokens by cosine similarity, and summarizes the matches as precision, recall, and F1. Because representations capture context, paraphrases can receive credit without exact word overlap. Its judgments inherit the coverage and biases of the embedding model.

match contextual token vectors by cosine similarity

best response

A choice that gives a player the highest payoff against the other players’ fixed choices. There can be several best responses if choices tie.

bi-encoder (dual encoder)

A dense retriever that encodes the query and the document in separate passes, then scores them with a single dot product.

A bi-encoder runs the query and each document through an encoder independently, reducing each to a single vector — often the [CLS] token’s output from the masked-language-models book — and scores a pair by the dot product of the two vectors. The decisive advantage is that a document never sees the query while being encoded, so every document vector can be computed once, offline, and stored in an index; at query time you only encode the query and look up nearby vectors. That is what makes searching millions of documents fast. The price is accuracy: query and document tokens never interact, so the model must cram everything it might need into one fixed vector, losing the fine-grained matching a cross-encoder can do.

score(q, d) = Encode_q(q) . Encode_d(d) (document vectors precomputed offline)

bias–variance tradeoff

The tension between systematic prediction error and sensitivity to the training data. Under squared loss, expected prediction error separates into squared bias, prediction variance, and output noise.

Fix one input and repeatedly draw training data and fit the same method. Bias is the mean fitted prediction minus the true conditional mean. Prediction variance measures how much those fitted predictions differ. Greater flexibility often reduces bias and increases variance, but neither those trends nor a U-shaped error curve is universal. The identity averages over training data and an independent fresh output; it does not equal every measured test score.

$R(x)=\operatorname{Bias}(x)^2+\operatorname{Var}_D(\hat f_D(x))+\operatorname{Var}(Y\mid X=x)$.

bidirectional

Each word gets to read both the words before it AND the words after it.

Bidirectional means every token’s understanding is built from context on both sides at once — left and right — instead of only what came before. Book 01’s writer used the causal mask to hide the future so it could predict the next word; an understander drops that mask entirely, so attention is free to look in both directions. That two-sided view is exactly why these models are so good at grasping what a word means in context, but it’s also why they can’t generate text left-to-right the way the writer does.

context(word) = words on the left + words on the right (no causal mask)

BIO tagging

A labeling scheme that marks the Beginning, Inside, and Outside of each named span.

BIO tagging is the standard way to turn span-finding (like named entity recognition) into a per-token classification: each token is labeled B- (begins an entity), I- (inside/continues one), or O (outside any entity). This cleanly handles multi-word names — ‘New’ gets B-LOC and ‘York’ gets I-LOC — and marks exactly where each entity starts and stops. An encoder predicts one of these tags for every token from its contextual embedding.

tokens -> [B-TYPE | I-TYPE | O] per token (e.g. New=B-LOC York=I-LOC)

bitrate

How many bits are used to represent one second of audio.

A bit is one binary digit, either 0 or 1. Bitrate counts how many of those digits are stored or sent each second. More audio-code streams usually raise bitrate and preserve more detail; fewer streams lower cost but may sound rougher.

bitrate = stored bits / second

BLEU

A corpus-level translation metric based on matching word n-grams plus a brevity penalty.

BLEU measures how many candidate word n-grams appear in one or more references, combines precisions across n-gram sizes geometrically, and penalizes outputs that are too short. It made translation experiments fast and comparable, but valid paraphrases can score poorly and high overlap can miss a meaning-changing error. It is a useful instrument, not a definition of translation quality.

BLEU = brevity penalty * geometric mean of n-gram precisions

BM25

A smarter tf-idf with two dials: one that stops word-repetition from running away, one that adjusts for document length.

BM25 is the workhorse successor to plain tf-idf, still built from term frequency and idf but with two fixes controlled by tunable parameters. The parameter k tunes term-frequency saturation — how fast repeated occurrences stop adding weight: at k = 0 a word counts only as present-or-absent, and a large k lets raw counts through, so a middling k keeps a keyword-stuffed document from dominating. The parameter b adds document-length normalization: it scales by how long the document is relative to the average, so a long document is not unfairly favored just for containing more words. Tuning k and b on a collection (reasonable defaults are k around 1.2-2 and b around 0.75) makes BM25 a strong, hard-to-beat baseline that sparse retrieval still leans on today, even against neural methods.

score = sum over query terms of idf_t * ( tf / (tf + k*(1 - b + b*(|d| / avg_|d|))) )

Bradley-Terry model

Turns two hidden quality scores into the probability one answer beats the other.

The Bradley-Terry model assumes each option has a hidden scalar score and that the chance one is preferred over another depends only on their score difference, passed through a sigmoid. So the gap between the two scores is exactly the log-odds of the preference, which means a reward model can be fit to preference pairs by learning scores whose differences match how often people pick each answer. It is the statistical bridge that lets a reward model translate chosen-versus-rejected comparisons into a continuous reward.

P(o_i > o_j) = sigmoid(z_i - z_j)

broadcasting

Letting arrays of different shapes combine by virtually stretching size-1 axes — no copying.

Broadcasting is the rule for combining arrays whose shapes don’t match: NumPy lines the shapes up from the right and, wherever one array has length 1 on an axis (or no axis at all), it is virtually stretched to match the other. Virtually is the point — it does this by setting that axis’s stride to 0, so the same value in memory is reused again and again instead of being copied. That is why adding a (3, 1) column to a (1, 4) row produces a full (3, 4) grid for almost no extra memory. If the non-1 lengths disagree, the shapes are incompatible and it errors.

(3,1) + (1,4) → (3,4) via stride-0 reuse, no copy

C

column space

All the output vectors a matrix can produce by multiplying a coefficient vector. It contains every linear combination of the matrix’s columns.

conditional distribution

The probability distribution you use after fixing or learning something about another quantity. The vertical bar means “given.”

In a discrete table, P(Y = y | X = x) keeps the row for X = x and divides each cell by that row’s total, provided it is positive. For continuous variables with a joint density, divide the density slice by the marginal density at x, when that density is positive. This remains useful even though P(X = x) is zero.

[CLS] token

A special slot added at the front whose final vector summarizes the whole input.

The [CLS] token (short for ‘classification’) is a special token prepended to every input; because attention lets it gather from all the other tokens, its top-layer vector becomes a single summary of the entire sequence. That summary is exactly what you feed to a classifier head for whole-sentence decisions like sentiment or topic — instead of pooling all the token vectors yourself, you just read [CLS]. It’s the encoder’s built-in ‘sum it all up here’ handle.

[CLS] x1 x2 ... xn -> use [CLS]'s final vector as the sentence summary

calibration

Whether a model’s confidence actually matches how often it’s right.

A model is calibrated when the probability it attaches to an answer lines up with how often such answers turn out correct — of all the things it claims to be 80% sure of, about 80% should be true. This matters because a well-calibrated model can tell you when to trust it and when to double-check or go retrieve more evidence. The trouble is that LLMs are often poorly calibrated: they phrase guesses in the same fluent, assured tone as facts, so they are frequently confidently wrong. That mismatch is a big reason hallucination is dangerous, and a big reason RAG exists — grounding answers in sources is more reliable than trusting the model’s own sense of certainty.

calibrated: P(correct | model states confidence c) = c

causal mask

It stops each word from peeking at the words that come after it.

A causal mask is a triangular pattern applied to the attention scores before they’re turned into weights: for any position, the scores pointing at later positions are forced to -infinity. After softmax those scores become exactly zero (since e^(-infinity) = 0), so a token’s output blends in only itself and earlier tokens. This matters because the model is trained to predict the next word, and letting it glance at the future would be cheating — it could just copy the answer. The mask keeps every position honest, so the same network can generate text one token at a time during use.

softmax(score + mask)_j, where mask = 0 if j <= i else -infinity

chain of thought

Intermediate reasoning text generated before a final answer.

Later tokens can use earlier calculations as context. For example, “23 − 20 = 3” supplies the subtotal for “3 + 6 = 9.” Worked examples or instructions can encourage this format, and training can change how well it is used. More reasoning text does not guarantee a better answer, and the written explanation need not faithfully reveal the internal computation.

chain rule

Calculate a derivative through several connected operations: multiply local derivatives along each path, then add contributions from different paths.

If u changes by about twice a small change in x, and y changes by about three times a small change in u, then y changes by about six times the change in x. The derivative is dy/dx = (dy/du)(du/dx) = 3 × 2. All derivatives are evaluated at the current values. If x reaches y by more than one path, include every path’s contribution.

multiply along a path; add across paths

chrF

A translation metric that compares character n-grams using precision and recall.

chrF breaks a candidate and reference into overlapping character sequences of several lengths, counts shared pieces, and combines character precision and recall with an F-score. Character-level matching gives partial credit across inflection and spelling variation, so it works well across many language types. It still measures surface overlap and cannot prove that meaning survived.

chrF = F-score(character n-gram precision, recall)

CLIP

An image encoder and a text encoder trained so matching pictures and captions land in the same place.

An image encoder and a text encoder trained separately produce two unrelated spaces. CLIP joins them using pairs that already existed: photographs posted with captions under them. In a batch, every caption is scored against every image, the pairs that arrived together are pushed together and all other combinations pushed apart. It never learns a fixed list of categories, only whether a sentence belongs to a picture, which is why it extends to sentences never seen in training.

score(image_i, text_j) -> high when i = j, low otherwise

closed-book QA

Answering a question straight from what the model memorized, with no documents to look at.

Closed-book question answering means the model answers from its parameters alone — the facts pressed into its weights during pretraining — with no documents retrieved and nothing external to read. It’s the memory-only setup this book opens with, and it inherits all four of that setup’s cracks: it invents answers when it doesn’t know, sounds equally sure when it’s wrong, can’t see anything private or newer than its training, and can’t cite a source. Open-book QA — RAG — is the contrast: retrieve relevant documents first, then answer from them. Closed-book is simpler and needs no index, but it’s capped by whatever the model happened to memorize, which is exactly why grounding it in retrieved text helps.

answer from model parameters only (no retrieved documents)

cloze task

A fill-in-the-blank test, like ‘The ___ chased the cat.’

A cloze task is the classic fill-in-the-blank: delete a word from a sentence and have someone (or a model) supply what belongs there. Masked language modeling is essentially a cloze task run at massive scale — the name comes from psychology, where cloze tests measured reading comprehension. It works as a training signal because guessing the missing word well requires actually understanding the sentence around the gap.

"The ___ barked." -> fill the blank

codec training loss

The combined error score used to teach a codec to preserve content and sound natural.

A codec is judged in several ways during training. One score checks whether the rebuilt recording matches the original. Another checks whether the strengths of its different vibration rates match. An adversarial score uses a learned real-versus-generated judge. A VQ score keeps the encoder’s number-lists close to the codewords they choose. Each score receives a chosen importance before the scores are added.

total error = several simpler error scores, each multiplied by its chosen importance

ColBERT

A dense retriever that keeps one vector per token and matches each query word to its best-matching document word.

ColBERT sits between the bi-encoder and the cross-encoder. Instead of squeezing each text into one vector (bi-encoder) or jointly encoding the pair (cross-encoder), it keeps a separate contextual embedding for every token on both sides. Relevance comes from the MaxSim operator: for each query token, find its highest similarity to any document token, then sum those maxima across all query tokens. This ‘late interaction’ recovers much of the fine-grained word-level matching a cross-encoder gets, yet each document’s token vectors depend only on the document, so they can still be precomputed and indexed like a bi-encoder — buying most of the accuracy at a fraction of the query-time cost.

score(q, d) = sum over query tokens i of ( max over doc tokens j of E_qi . E_dj )

collection

The whole set of documents the system searches through.

The collection (sometimes called a corpus) is everything the retrieval system can possibly return — it might be the entire web, a company’s internal wiki, a legal archive, or the files on your own laptop. It defines the boundary of what can be found: nothing outside the collection can ever be retrieved, so a RAG system built on a private collection answers only from that private knowledge. Its size also drives the statistics that make retrieval work, like how many documents contain a given term (document frequency), which is what lets the system tell common words apart from rare, informative ones.

collection = the N documents being searched

COMET

A learned translation-quality metric trained to predict human judgments.

COMET encodes the source, candidate translation, and often a reference, then predicts a quality score from representations trained against human ratings. It usually correlates with human judgment better than simple overlap metrics, but it remains a learned proxy whose language coverage, training distribution, and social biases must themselves be evaluated.

learned score(source, candidate, reference)

Common Crawl

A free, enormous snapshot of the public web that most LLM training data starts from.

Common Crawl is a nonprofit that has been crawling and publishing the open web since 2008 — petabytes of raw HTML pages. It’s the single biggest raw source feeding LLM pretraining, but it is messy: spam, boilerplate, navigation junk, duplicates, and toxic content all mixed together. Almost nobody trains on it directly; instead teams aggressively filter and clean it. Cleaned-up datasets like C4 and Dolma are essentially ‘Common Crawl, scrubbed’.

raw web pages (HTML) -> filter & clean -> training-ready text

comparative MOS (CMOS)

The average human rating of which of two matching speech clips sounds better.

MOS here means mean opinion score. Listeners hear two systems say the same content, in a random order, and rate which clip is better. The ratings are averaged. Comparing clips side by side can reveal a small difference that is hard to notice when each clip is rated alone. The result applies only to the voices, sentences, listeners, and listening setup that were tested.

CMOS = average of the side-by-side listener ratings

computational graph

A diagram of a calculation’s operations and dependencies.

For a prediction ŷ = wx + b, multiplication produces wx, then addition combines it with b. Connecting those operations to a loss shows how each parameter can affect that loss. During automatic differentiation, the graph guides the chain rule; backward rules use saved or recomputed forward values to calculate derivatives. Recording the graph and calculating gradients do not themselves update the parameters.

parameters → prediction → loss; derivatives follow the dependencies backward

compute-optimal scaling (Chinchilla)

Match how big your brain is to how much you read.

When you can only spend so much effort training, you have to choose: build a bigger model, or feed it more text? Chinchilla showed the best results come from growing both together, in step. The surprise was that famous earlier models were too big and hadn’t read enough — like a giant brain that only skimmed a few books. A right-sized model that reads a lot can beat a huge one that reads too little.

grow model size and training data together for a fixed compute budget, roughly 20 training tokens per parameter (e.g. Chinchilla: 70B params, 1.4T tokens)

conditional generation

Generating output that is steered by a given input — write Y, but conditioned on X.

Conditional generation means the text a model writes is shaped by something you give it: a prompt, a source sentence to translate, a document to summarize. Formally the model produces P(output | input) — the probability of each next token given everything provided so far. Encoder-decoders make the conditioning structural (the decoder cross-attends to the encoded input), while decoder-only models do it by simply placing the input in the context window and continuing from it. Every time you prompt a chat model, you are doing conditional generation.

P(output | input) — next tokens depend on the given input

confidence

An estimate of how much the system should trust a prediction, not merely how fluent it sounds.

Translation confidence estimates the chance that a candidate is reliable enough for its intended use. Raw token probability is not automatically calibrated: a model can be confidently wrong, especially out of domain or in a lower-resource language. Useful confidence connects uncertainty to action—show alternatives, preserve the source, request context, or abstain and route the case to a human.

predicted certainty should match observed correctness

Connectionist Temporal Classification (CTC)

A training and reading rule that learns speech timing even when the transcript has no timestamps.

CTC gives every short sound position a choice: write a text symbol or write a special blank. Many different position-by-position paths can reduce to the same transcript. Training adds the probabilities of all valid paths, so a human does not need to mark exactly when each letter was spoken.

probability of a transcript = add the probabilities of all valid timing paths

context window

How much text the model can read and use at once.

The context window is the maximum number of tokens (word-pieces) the model can take in at one time — say 8k or 128k of them. It works because the model’s attention lets every token look at every other token in the window to decide what matters, so a bigger window means more text it can weigh together. The catch: this all-to-all comparison grows with the square of the length, so doubling the window roughly quadruples that part of the work — one reason windows have a fixed limit. And anything outside the window simply isn’t there for the model on this pass: it can’t see what scrolled off unless that text is fed back in. Long-context models soften the n² cost by having most layers attend only to a nearby window of recent tokens (sliding-window / local attention), keeping full all-to-all attention for just a few layers.

compute to relate all tokens grows like n^2 for n tokens

contextual embedding

A token representation that depends on the surrounding tokens the model is allowed to use.

The initial embedding lookup is fixed for a given token ID during ordinary inference. Both encoders and decoders then transform it into contextual hidden states. A bidirectional encoder can use later and earlier positions; a causal decoder can use the current and earlier positions. “Contextual” is not a property exclusive to encoders.

same input token ID + different allowed context → potentially different hidden state

contrastive learning

Train representations by making positive pairs score higher relative to pairs treated as negatives.

For CLIP, N collected image–text pairs supply N positives and N² − N other combinations treated as negatives. An unpaired caption may still describe an image, so “negative” is a role in the training objective, not proof of a semantic mismatch. Different contrastive methods define positives, negatives, and losses differently.

collected pairs → positive examples; other batch combinations → negative examples for the objective

convolution kernel

A short list of learned numbers that tests one small window for a pattern.

A kernel, also called a filter, lines up with a small part of the input. Matching number pairs are multiplied and the results are added to make one score. Sliding the same kernel along the recording asks the same pattern question at every position.

Multiply each matching input and kernel number; add the products to get the score.

convolutional neural network (CNN)

A model part that checks the same small sound pattern at every point in time.

CNN is short for convolutional neural network. It slides small learned filters across nearby audio measurements or across columns that summarize vibration strength. Reusing one filter everywhere lets the system notice the same kind of sound wherever it occurs. Some CNN layers also shorten the long audio timeline before later processing.

Check one small window, make one pattern score, then slide and repeat.

cosine similarity

How much two vectors point the same way, ignoring their size.

Cosine similarity is the dot product of two vectors after dividing out both their lengths, so all that’s left is the angle between them: +1 means same direction, 0 means at right angles, -1 means opposite. It works because meaning here lives mostly in direction, not size — two word vectors can point the same way whether they’re long or short. (Transformer attention is a cousin but keeps the lengths, so size still matters there.)

cos_sim(a, b) = (a . b) / (|a| * |b|)

cross-attention

The output-writing part looks back at the input descriptions and gives more weight to the parts that matter now.

Cross-attention connects two different ordered lists. The decoder asks what it needs for its next output, scores the descriptions made by the encoder, and mixes them according to those scores. In speech recognition, this can connect one letter being written with the stretch of sound that supports it. In translation, it can connect a new word with useful words from the original sentence.

Use the output's current question to combine the most useful input descriptions.

cross-correlation

Slide a small filter across the input and score each local match without reversing the filter.

In the strict mathematical definition, convolution reverses the filter before sliding it. Most neural-network software does not reverse it; that operation is called cross-correlation. Because the filter numbers are learned, either version can learn useful local sound patterns.

Slide the filter, multiply matching pairs, and add the products.

cross-encoder

A retriever that reads the query and document together in one pass, so attention can compare their words directly.

A cross-encoder concatenates the query and document into a single input — [CLS] query [SEP] document — and feeds them through one encoder, so self-attention (from the transformers book) lets every query token look directly at every document token. A small layer on top of the [CLS] output turns this joint reading into one relevance score. Seeing both sides at once is what makes it the most accurate ranker: it catches precise word-level interactions that a single summary vector would blur away. But nothing about a document can be precomputed, since its representation depends on the query, so scoring N documents means N full forward passes — far too slow to run over an entire collection, which is why it is used only to rerank a short candidate list that a cheaper retriever has already narrowed down.

score(q, d) = Linear(Encoder([CLS] q [SEP] d)[CLS])

cross-entropy loss

A score for surprise: the truer your guess, the smaller it gets.

When the model predicts the next word, it spreads 100% of its belief across all possible words, like dividing a coin jar. Cross-entropy loss looks only at the slice it gave the word that actually came next, and asks: how surprised were you? If it bet almost everything on the right word, the surprise is tiny; if it gave the right word only a sliver, the surprise is huge. Training nudges the model to keep raising the slice on true words, so this surprise number shrinks over time.

loss = -log(p_true)

CTC blank

A special choice meaning “move forward in the recording but write no text now.”

The blank gives CTC a way to use sound positions between letters. It can also separate repeated letters. When reading the final path, merge neighboring repeated letters first and then remove every blank.

merge repeated symbols first -> remove blanks

curse of dimensionality

In many dimensions nothing is near anything: the dots inside a fixed window shrink like r^p.

Spread N dots evenly through a p-dimensional space and draw a window of radius r around a point. The window’s share of the space is r multiplied by itself once per dimension, so it holds about N·r^p dots. With r = 0.1 that is a tenth of the dots in one dimension, a thousandth in three, one in ten billion in ten. Any method that averages nearby dots runs out of neighbors; the way out is to assume a shape and let every dot vote on its knobs.

expected neighbors = N · r^p

D

data contamination

When test questions leak into the training data, so the model has secretly already seen the answers.

Data contamination happens when the examples used to evaluate a model accidentally appear in its pretraining data — because both came from the same giant web scrape. The model then looks brilliant on that test, but it may just be recalling answers it memorized, not reasoning. It’s a core reason benchmark scores can be misleading, and it ties pretraining data directly to evaluation: you can’t fully trust a score unless you know the questions weren’t in the training set. (More on this in the evaluation chapter.)

test question ∈ training data -> inflated, untrustworthy score

decoder

A part that turns an internal description into the system’s final kind of output.

A decoder takes the number-list descriptions made inside a system and produces something usable. A text decoder can write transcript pieces one at a time. An audio-codec decoder turns compact sound codes back into microphone-like measurements. The exact decoder depends on whether the desired output is text or sound.

Input: an internal description. Output: text or sound.

denoising

Mess up the text a little, then train the model to repair it.

Denoising is the general framing behind masked language modeling: corrupt the input (here, by blanking out or swapping some tokens — the ‘noise’) and train the model to recover the clean original. Learning to undo the damage forces the model to internalize how language normally fits together. The same idea scales up in other models that corrupt whole spans of text and reconstruct them, but plain word-masking is the simplest version.

noisy text -> model -> clean text

dense retrieval

Finding documents by the meaning of the query rather than its exact words, using learned embeddings.

Dense retrieval encodes the query and each document into a dense vector with a neural encoder — typically a BERT-style model from the masked-language-models book — and ranks documents by how close their vectors are, usually via a dot product. Because the encoder maps meaning-similar text to nearby vectors, ‘car’ and ‘automobile’ land close together, which directly defeats the vocabulary-mismatch problem that sinks keyword search. The name ‘dense’ contrasts with the sparse, mostly-zero word-count vectors of tf-idf and BM25: here every dimension carries learned meaning rather than a single word’s presence. The cost is that you must push every query and document through a neural network and then search an enormous pile of vectors, which is why approximate-nearest-neighbor tools become essential.

score(q, d) = encode(q) . encode(d)

derivative

The instantaneous rate at which a function’s output changes with its input.

For f(x) = x², f′(3) = 6: near x = 3, an input change h predicts an output change of about 6h. The exact derivative is the limit of [f(x+h) − f(x)]/h as nonzero h approaches zero, when that limit exists. f′(3) is a rate, not the function value f(3) = 9.

f′(x) = lim h→0 [f(x+h) − f(x)] / h

diffusion model

A generative model that learns to reverse a process that adds noise to data.

In a common training setup, choose a noise level, add sampled noise to a real example, and train a network to predict that noise. During generation, a sampler starts from noise and repeatedly updates the sample using the trained network’s predictions and a noise schedule. Other formulations predict a clean sample or a related quantity. The clean target is available during training, not during generation.

training: corrupt known data; generation: use learned predictions to update noise toward a sample

diffusion transformer (DiT)

A vision transformer used as the denoiser: it takes noisy patches and predicts noise.

The denoiser must map a grid to a grid of the same shape, a job convolutional U-Nets did for years. But a noisy latent is a grid, and a grid becomes a sequence by cutting it into patches, so a transformer works directly. It inherits the patch-size dial: smaller patches mean more tokens and more arithmetic, with better image quality and no change to the parameter count, giving two independent ways to spend more compute.

noisy latent -> patches -> transformer -> reshape -> predicted noise

discount factor

The weight δ applied once for each step into the future. With 0 ≤ δ < 1, a payoff c received every round starting now has discounted value c/(1 − δ). This formula assumes infinitely many rounds.

discriminator

A training-time judge that tries to tell real examples from generated ones.

In adversarial training, the discriminator learns to separate real audio from reconstructed audio. The generator or codec then learns to fool that judge. This pressure can improve realistic detail, although it must be balanced with losses that preserve the actual content.

audio -> probability that it is real

document

One piece of text the search system stores and can hand back — a page, a passage, a paragraph.

A document is the unit an IR system indexes and returns; what counts as one is a design choice, not a law of nature. It might be a whole web page, a single paragraph, or a short passage, and in RAG the granularity matters a lot: chunks that are too big bury the answer in noise, while chunks too small lose the surrounding context needed to be useful. Whatever the size, each document is turned into a vector so it can be compared against the query. The system’s entire job is to pick the right documents out of the collection.

document frequency (df)

How many documents in the whole collection contain a given word.

Document frequency is the number of documents in the collection that contain a term at least once — not how many times it appears, but in how many documents it shows up at all. It measures how widespread, and therefore how uninformative, a word is: ‘the’ has a document frequency near the size of the whole collection, while ‘Louvre’ appears in relatively few. That is the raw ingredient for inverse document frequency, which flips it so that a low df (a rare, specific word) means a high weight. Note it is deliberately not the total count across the collection — a word crammed many times into one document should not look widespread.

df_t = number of documents containing term t

dominant strategy

A strictly dominant strategy gives a player a higher payoff than every alternative for every combination of the other players’ strategies. A best response to just one choice need not be dominant.

dot product

Line up two lists of numbers, multiply pair by pair, add it all up.

A dot product turns two vectors into a single number: multiply each pair of matching entries and sum the results. That number is large and positive when the vectors point the same way (and are long), near zero when they point in unrelated, roughly perpendicular directions, and negative when they point opposite ways — so it scores how well two things “agree.” In a transformer this is the core of attention: each word’s query vector is dotted against every word’s key vector, and a higher score means “this word should pay more attention to that one.” (Those raw scores are then scaled and passed through softmax to become the actual attention weights.)

a . b = a1*b1 + a2*b2 + ... + an*bn

DPO (direct preference optimization)

Direct preference optimization trains a policy on preferred/rejected response pairs without fitting a separate reward model.

The loss encourages a larger preferred-versus-rejected log-probability margin, measured relative to a fixed reference policy. This does not require the preferred answer’s absolute probability to rise or the rejected answer’s to fall on every update. β scales the margin in the loss; the reference is not a hard cap on policy drift. Standard offline DPO uses an existing pair dataset rather than generating fresh responses inside each training update.

L = −log sigmoid(β[log(π(chosen)/πref(chosen)) − log(π(rejected)/πref(rejected))])

dtype

The single number-type every element in an array shares (e.g. float64, int32).

A dtype is the fixed element type of an array — float64, int32, bool, and so on — and crucially every element shares it. That uniformity is what lets the data sit as a tight packed block with a known number of bytes per element, so the step from one value to the next is constant and the CPU can scan it without unboxing anything (unlike a Python list, where each entry is a separate boxed object behind a pointer). The dtype also fixes how many bytes a step in memory is, which is why strides can be quoted cleanly in element units.

every element same type → fixed bytes each → packed, scannable memory

dynamic programming

Solve a large step-by-step search by saving and reusing the best answers to smaller repeated parts.

Many possible paths share the same partial prefixes. Dynamic programming computes each shared partial problem once and stores the result. Speech systems use it to find a minimum edit distance and to sum many valid CTC timing paths without listing every complete path.

solve small shared parts once -> combine them into the full answer

E

expectation · E[·]

Average whatever is inside the brackets using the probabilities of the quantity that remains random. E means expectation.

In E[(Y − c)² | X = x], x and your choice c stay fixed while Y varies according to its conditional distribution. In E_D[f̂_D(x)], x stays fixed while the training dataset D changes, producing different fitted predictions. Identifying what varies tells you which average to calculate.

e (Euler’s number)

A special number, about 2.718 — nature’s base for smooth growth.

e ≈ 2.718 is a constant like π. It falls out of compounding: grow by 100% but split it into infinitely many tiny steps and you land exactly on e. Its trick is that the curve e^x grows at a rate equal to its own height, which keeps the math clean — so e is the base used in exp, log, and inside softmax.

e ≈ 2.71828 = (1 + 1/n)^n as n → ∞

embedding

A token’s learned meaning stored as a list of numbers the model can work with.

An embedding is the vector (a fixed-length list of numbers) that a token gets turned into by looking up its row in the embedding table E. It works because the model can nudge these numbers during training, so tokens used in similar ways drift to nearby spots — directions in this space end up capturing real patterns of meaning. This is the transformer’s entry point: it converts discrete tokens into numbers the attention and feed-forward layers can actually do math on.

embedding(token) = E[token_id] (one row of the table E)

encoder

A part that turns raw input into a shorter, more useful internal description.

An encoder turns an input into lists of numbers that summarize what matters for the next step. A speech encoder combines short neighboring pieces of a recording so each new list includes surrounding sound clues. A text encoder does the same kind of job for text pieces. Its output is a useful description, not the final answer.

Input: raw data. Output: a useful internal description.

encoder-decoder

A two-part system: one part makes a useful internal description, and the other makes the output.

The encoder first turns the input into short internal number-lists. The decoder then uses those lists to make the required output. Translation changes text in one language into text in another. Speech recognition changes sound into text. An audio codec changes a recording into compact codes and then rebuilds sound from those codes.

First encode the input; then decode the internal description into the output.

entropy

Expected surprise: how surprising a distribution is on an average draw.

Entropy H(p) weights each outcome’s surprise log(1/p) by its probability, giving how hard the distribution is to predict overall — zero for a certain world, maximal when every outcome is equally likely. A rare disaster contributes little despite enormous surprise, because it almost never happens. Shannon’s coding theorem gives the same number a second meaning: the minimum average bits needed to describe draws from the source, which is why predicting well and compressing well are the same skill. It is also the unbeatable floor inside cross-entropy — the part of a model’s loss that comes from the world itself being uncertain.

H(p) = Σ p · log₂(1/p)

exact match (EM)

A strict score: the fraction of answers that match the correct answer word-for-word.

Exact match is the fraction of a system’s answers that are identical to the gold (correct) answer, usually after light normalization like lowercasing and stripping articles and punctuation. It’s deliberately all-or-nothing: an answer that’s almost right, or right but phrased differently, scores zero. That makes it a clean fit for multiple-choice questions or short factoid answers, where there’s essentially one canonical correct string. For free-text answers it’s too harsh on its own, since many valid phrasings won’t match the exact gold string, which is why QA leaderboards report token-overlap F1 alongside it to hand out partial credit.

EM = (# predictions exactly equal to the gold answer) / (# questions)

expected value

A probability-weighted average of a random quantity: multiply each possible value by its probability, then add. For a quantity with a density, use an integral.

An expectation describes the distribution, not necessarily a possible next outcome. A fair coin scored 0 or 1 has expectation 1/2. For independent, identically distributed draws with finite variance, the sample mean has standard deviation equal to the individual standard deviation divided by the square root of the sample size. Individual draws do not become less variable.

$\mathbb E[X]=\sum_x x\,p_X(x)$, or $\mathbb E[X]=\int x\,f_X(x)\,dx$ when X has a density and the expectation exists.

exponential (e^x)

Growth that speeds up as it goes: a tiny step makes a giant leap.

e^x is a curve where the bigger the number gets, the faster it grows. For negative x it hugs the floor, almost nothing; it passes through 1 when x is zero; then for positive x it rockets upward. Softmax feeds scores through e^x, so a small lead in score becomes a big lead in the final probability. It is the exact mirror of the logarithm, which slowly squashes big numbers back down.

e^x: e^0 = 1, e^1 ≈ 2.7, e^2 ≈ 7.4

F

fundamental theorem of calculus

A running total grows at the rate of what is being added — so the derivative undoes the integral, and the integral undoes the derivative.

Let A(x) be the area under f from a up to x. Slide x a hair and A grows by one sliver, height f(x) times the step, so A′ = f: the slope of the accumulated area is the height being added. Read the other way, adding up all the small changes F′(x)·Δx of a function recovers its total change F(b) − F(a) — a telescoping sum whose middle terms cancel. Together these turn every area computation into finding a function with the right slope and subtracting its values at the two ends.

d/dx ∫ₐˣ f(t) dt = f(x) · ∫ₐᵇ F′(x) dx = F(b) − F(a)

fitted model · f̂

A prediction rule learned from a particular dataset. The hat marks an estimate.

The unknown population relationship f and your fitted estimate f̂ are different objects. Writing f̂_D makes the training dataset explicit: changing D may change the fitted rule. Once fitting is finished and the input x is chosen, f̂_D(x) is one prediction.

F1 score

A gentler QA score that gives partial credit for getting some of the right words, even if the answer isn’t word-perfect.

For free-text answers, F1 gives partial credit by treating the predicted answer and the gold answer each as a bag of tokens. Precision is the fraction of the predicted tokens that appear in the gold answer, recall is the fraction of the gold tokens the prediction covered, and F1 is their harmonic mean — high only when both are high. It’s the same precision and recall from the search-evaluation chapter, just measured over the words of an answer instead of over retrieved documents. Averaged across all questions, it’s fairer than exact match for questions with many acceptable phrasings: an answer that overlaps the gold but adds or drops a word still scores well instead of being marked flatly wrong.

F1 = 2 * precision * recall / (precision + recall), over shared answer tokens

factoid question

A question whose answer is one small fact, like ‘Where is the Louvre?’

A factoid question is one that can be answered with a short, concrete fact — usually a named entity, date, or number — rather than an essay or an opinion. ‘Where is the Louvre?’ wants ‘Paris’; ‘When was the transistor invented?’ wants a year. They matter because they are the cleanest test case for question answering and RAG: there is a single checkable right answer, so it is easy to tell whether the retrieved document actually contained it and whether the model reported it faithfully. Harder questions (why, how, compare) blur that line, which is why factoid QA is where the field started.

FAISS

A popular library for fast approximate-nearest-neighbor search over huge collections of dense vectors.

FAISS (Facebook AI Similarity Search) is an open-source library that implements approximate-nearest-neighbor search efficiently, so a dense-retrieval system can find the vectors closest to a query among millions or billions of them in milliseconds. It packages the indexing tricks — clustering vectors into cells, compressing them with product quantization, searching proximity graphs — behind a simple ‘add these vectors, now find the neighbors of this one’ interface, with support for running on GPUs. In a typical pipeline you precompute all document embeddings with a bi-encoder, load them into a FAISS index, and query that index at retrieval time. It is one of the standard tools that makes large-scale dense retrieval and RAG feasible.

feed-forward network

A network that transforms each token position’s vector, using the same parameters at every position.

A standard Transformer feed-forward network applies an affine map, a nonlinear activation, and another affine map. Its intermediate vector is often wider than its input; the output returns to the original width. This operation does not directly mix positions, although its input already contains context from attention. Individual intermediate coordinates need not have a simple, named meaning.

FFN(x) = W₂ activation(W₁x + b₁) + b₂

few-shot

In prompting, supplying a few examples before asking for an answer to a new input.

Three labeled reviews followed by an unlabeled review make a three-shot prompt. The demonstrations can specify a task, its labels, and the response format. Ordinary prompting changes the context, not the model’s weights. Examples can help or hurt depending on their choice and the task; few-shot prompting does not have one universal minimum parameter count.

few-shot prompt = examples + a new input; weights stay fixed

finite difference

Estimate a derivative by evaluating a function at nearby inputs.

For one variable, the central difference is [f(x + h) − f(x − h)] / (2h), where h is a small positive step. For a partial derivative, change only the input you are checking. Compare this estimate with the derivative to catch calculation errors. Large steps include curvature; extremely small steps can lose accuracy to floating-point rounding. At a corner such as ReLU’s zero point, an ordinary derivative may not exist.

nearby function values → estimated slope

fine-tuning

Continue training an existing model on a chosen dataset or objective.

Fine-tuning changes some or all parameters. It may adapt a model to a domain, teach an output format, or improve instruction following. A pretrained model can already answer some questions; fine-tuning is not what makes all answering possible, and it does not guarantee accuracy.

existing parameters + further training → updated parameters

foundation model

One big pretrained model that many different applications get built on top of.

A foundation model is a single large model — pretrained once on broad data — that serves as a shared base for many downstream uses, via prompting or fine-tuning. The name (coined at Stanford in 2021) captures the shift from training a fresh model per task to building one general model and adapting it everywhere. Because so much depends on that one base, the choices baked into its pretraining data (what’s included, what’s filtered out) ripple out to every application built on it.

one pretrained base -> many apps (chat, search, coding, ...)

G

gated FFN (SwiGLU)

Two lanes meet: one scales the other up, down, or off.

The feed-forward step makes two copies of the same input with two different linear lanes. One lane is the signal; the other passes through swish to become a learned gain that multiplies the signal point by point. Unlike a 0-to-1 dimmer, the swish gate is mostly positive but unbounded above (so it can amplify, not just attenuate) and dips slightly below zero, so each number can be turned up, down, off, or gently flipped. So instead of ReLU’s hard rule (“keep it or zap it to zero”), the amount is smooth and learned. That richer, learnable control lets the network pass more expressive signals; the hidden width is trimmed (about two-thirds) to keep the parameter and compute budget comparable, and it still wins, which is why modern Transformers (e.g. LLaMA, PaLM) use it.

out = (x·W1 * swish(x·W3)) · W2

GeLU

A smooth on/off switch for neurons that gently fades small values toward zero instead of cutting them off sharply.

GeLU (Gaussian Error Linear Unit) is a common nonlinearity used inside a transformer’s feed-forward layer, deciding how much of each value to pass forward (some models instead use ReLU or gated variants like SwiGLU). Instead of ReLU’s hard rule (negatives become 0, positives pass unchanged), it multiplies each input by a smooth “keep probability” that rises from 0 to 1 as the input grows: GeLU(x) = x * P(X <= x), where X is a standard bell curve (the normal distribution). That smoothness means small negatives leak through a little and the curve has no sharp kink, which gives cleaner gradients and tends to train slightly better. The feed-forward layer is where the network does much of its “thinking” between attention steps, reshaping each token’s features.

GeLU(x) = x * Phi(x), where Phi(x) = P(X <= x) for X ~ Normal(0,1)

gender bias in translation

Systematic gender errors or stereotypes introduced when translation resolves ambiguity.

Gender bias appears when a translation system assigns, preserves, or evaluates gender unevenly. A gender-neutral source may become stereotypically masculine or feminine in a target that requires grammatical gender, and counter-stereotypical examples often receive more errors. Targeted, balanced tests are necessary because a strong overall score can hide the disparity.

translation error rate differs across gendered cases

generative AI

AI that produces new content — text, images, audio — rather than just labeling or scoring existing input.

Generative AI refers to models whose output is freshly created content rather than a category or a number. For language, that’s the decoder’s job: sampling token after token to write text that never existed before. It contrasts with the discriminative side (an encoder classifying a review as positive or negative). The whole reason a decoder can generate is that it was trained to predict the next token, so running that prediction in a loop produces original sequences. This book is about generative text models specifically — the chat LLMs.

prompt -> newly generated tokens

generator (reader)

The language model in a RAG system that reads the retrieved passages and writes the actual answer.

The generator is the language-model half of retrieval-augmented generation: the retriever finds passages, and the generator reads them and produces the answer in fluent prose. It’s the same decoder LLM from the transformers book, used unchanged — the passages are simply pasted into its prompt and it writes conditioned on them, the ordinary in-context learning that lets a model use whatever sits in its context. Older open-domain question-answering papers call this component the reader, because its job is to read the retrieved text and extract or compose the answer from it. Grounding it in real passages is what curbs the hallucination a memory-only model falls into, since the answer now has to come from evidence in front of it rather than from frozen weights.

answer = generator(question + retrieved passages)

gradient

The vector of a scalar-valued function’s partial derivatives, one entry per input.

For a differentiable f, a small step Δx gives the prediction Δf ≈ ∇f · Δx. A nonzero gradient points in the direction of greatest first-order increase per unit Euclidean distance. At (2,3), f(x,y) = x²y + y has gradient (12,5). This definition applies to any suitable function, including a model’s loss.

∇f = (∂f/∂x₁, …, ∂f/∂xₙ)

gradient descent

An update rule that subtracts a learning rate times the gradient.

For a differentiable objective and a nonzero gradient, a sufficiently small positive step reduces the objective. A large step can overshoot and increase it. Gradient descent need not find a global minimum, and a noisy batch gradient need not reduce the full training loss at each step. Backpropagation computes gradients; the optimizer uses them.

w_new = w − η ∇L(w)

grapheme

A written symbol or symbol group used to represent language, such as a letter.

A grapheme is a unit of writing. It may be one letter or, in some writing systems, a larger symbol. Speech recognition can predict graphemes directly, even though the link between spelling and sound is not always one-to-one.

spoken language -> written symbols

greedy decoding

Always pick the single most likely next token (usually a word or word-piece) — same input, same output.

After the transformer scores every possible next token, greedy decoding just takes the one with the highest probability, adds it to the text, and repeats. It works because the model’s top guess is usually a sensible local choice, and “always pick the best one right now” is the simplest possible rule. But it’s shortsighted: the locally-best token can lead into a worse overall sentence, which is why it can sound flat or get stuck repeating. Because there’s no randomness, the same prompt always yields the same output.

next_token = argmax_i P(token_i | text so far)

grouped-query attention (GQA)

Let many query heads share a smaller set of keys and values, to shrink the KV cache.

Each attention head normally keeps its own keys and values, and the KV cache has to store all of them — which gets expensive for long text. Grouped-query attention keeps the query heads separate (so the model still asks many different questions) but lets groups of them SHARE one set of keys and values, so there’s far less to cache. Llama 3 70B, for instance, has 64 query heads but only 8 key/value groups. Sharing a single set across every head is the extreme version, multi-query attention (MQA); DeepSeek’s MLA compresses the cache a different way. Same idea as plain multi-head — just much cheaper to run.

64 query heads share 8 key/value groups (Llama 3 70B); 1 shared group = MQA

H

hallucination

When a model states something false but says it with total confidence.

A hallucination is fluent, confident output that is factually wrong or made up — a fake citation, an invented statistic, a plausible-sounding but untrue claim. It arises because the model is trained to produce probable-sounding next tokens (Book 01), not to check facts; when it lacks the real answer it will still generate the most likely-looking continuation. This is a core reliability problem for generative AI, which is why outputs need verification and why techniques like grounding the model in retrieved sources are used to reduce it.

harness

The program around a model that reads some of its output as commands, runs them, and puts the results back in the prompt.

A model can only read text and write text. The harness is everything else: it decides what the prompt contains, watches the output for a recognised shape, executes the matching action, appends the result, and decides whether to call the model again or stop. None of this is a model capability. Two harnesses wrapped around identical weights produce very different behaviour, which is why the harness counts as part of the system rather than as packaging around it.

prompt -> model text -> harness executes -> result appended -> model again

hidden state

A model’s current internal number-list for one position after processing some context.

A hidden state is called hidden because it is an inside-the-model description, not the original input or final output. Each layer updates this list of numbers using nearby or attended context. In a speech encoder, one hidden state can describe a short sound together with clues from the sounds around it. In a text model, it can describe one token in its sentence.

local input + surrounding context -> hidden state

HuBERT

A method that learns speech patterns by hiding sound pieces and guessing their automatically made group labels.

HuBERT’s name expands to Hidden-Unit BERT; here BERT refers to the hide-some-input-and-predict-it training pattern. HuBERT first groups short sound descriptions and gives each group a number. It hides a continuous part of a recording and predicts those hidden group numbers from the surrounding sound. Later, human transcripts teach the final speech-recognition task.

Use surrounding sound to predict the hidden sound-group numbers.

I

integral

An accumulated total formed by adding small contributions. A definite integral gives a number over a specified interval; an indefinite integral denotes a family of antiderivatives.

For a continuous f, the definite integral is the limit of sums f(xᵢ)Δx as the partition becomes finer. Negative heights subtract. Integrating velocity gives net displacement; integrating speed gives distance. In ∫ₐᵇ f(x) dx, dx identifies the integration variable. It does not tell you to multiply everything by zero.

∫₀¹ x² dx = 1/3

in-context learning

Using information or examples in the input to perform a task, without updating the model’s weights.

For example, a prompt can show three input–output pairs, then request an output for a new input. The response may follow the demonstrated pattern. The effect depends on the context supplied to that model call; applications can preserve, remove, or summarize that context. Induction heads illustrate one mechanism studied in particular models, not a complete account of all in-context learning.

examples in context → prediction conditioned on those examples

induction head

When the model sees a pattern start again, it predicts what came next last time.

An induction head is a little machine inside the model, built from its attention, that finishes patterns. When it sees a word it has seen before, it peeks back at that earlier spot, notices what word came right after it, and guesses the same word will come next now. This is much of how a model “learns” from examples you give it in the prompt, without any extra training.

...A B...A -> B

inference

Using a trained model to make an output without teaching it anything new.

Inference is the use stage. The model’s learned weights stay fixed while it turns a new recording into text or new text into speech. It does not mean human logical inference here; it simply means running the trained model.

new input + fixed trained model -> output

information retrieval (IR)

Finding the documents in a big pile that answer what you’re looking for.

Information retrieval is the task of taking a user’s information need and returning documents from a collection that satisfy it — the search half of retrieval-augmented generation. It works by scoring how well each document matches the query and handing back a ranked list, best first, so the most relevant reading rises to the top. It predates LLMs by decades (it is what web search engines do), but it has become newly central: a language model that only knows what it memorized during training can be grounded by first retrieving the right documents and then reading them. So IR is the ‘retrieve’ that the ‘generate’ half depends on.

instruction tuning

Training a model on lots of instruction-and-answer examples so it follows directions.

Instruction tuning is a form of Book 01’s fine-tuning where the curated examples are specifically (instruction, good response) pairs across many varied tasks — summarize this, classify that, answer this question. It teaches a raw pretrained model, which only knows how to continue text, to instead treat input as a request and respond helpfully. This is what unlocks strong zero-shot behavior, because the model generalizes the ‘follow the instruction’ skill to tasks it never saw. It usually comes before preference tuning like RLHF.

fine-tune on many (instruction, response) pairs across diverse tasks

interpolated precision

A smoothed precision-recall curve: at each recall level, take the best precision seen at that recall or beyond.

The raw precision-recall curve jitters up and down, which makes systems hard to line up against each other. Interpolated precision irons out the dips: at a given recall level r, instead of the actual precision there, you take the maximum precision achieved at any recall greater than or equal to r. This replaces every local sag with the best that is still to come, so the resulting curve only ever steps down, never up. It is typically read off at eleven evenly spaced recall points (0.0, 0.1, … , 1.0) so that many queries or systems can be averaged and plotted on common footing.

IntPrec(r) = max over r' >= r of Precision(r')

inverse document frequency (idf)

Boost rare, telling words and mute the ones that show up everywhere.

Inverse document frequency is the log of the collection size divided by a term’s document frequency: idf = log10(N / df). It captures how much a word narrows things down — a word in almost every document (df near N) gets an idf near zero and is nearly ignored, while a rare word gets a large idf and dominates the match. This is why ‘Louvre’ in a query counts for far more than ‘the’, and it is the piece that makes the vector space model actually work, since without it every document would look relevant just for sharing common words. The log keeps the boost from vanishingly rare words from exploding out of control.

idf_t = log10(N / df_t), N = documents in collection

inverted index

A lookup table from each word to the list of documents that contain it.

An inverted index flips the natural document-to-words layout around: instead of storing, per document, which words it has, it stores, per word, which documents have it. This is what makes search fast — given a query, the system jumps straight to the short lists for each query word and only ever looks at documents that actually contain one of those words, ignoring the vast majority that do not. It is the practical engine that lets sparse retrieval scan billions of documents in milliseconds, and it works precisely because those word-count vectors are so sparse. The per-word lists it stores are called postings.

term -> [documents containing that term]

irreducible error

Under squared loss, the remaining expected error when you predict the true conditional mean using the available inputs.

At a fixed input x, outputs may still vary. Their conditional variance is the best achievable expected squared error there. It may depend on x; writing the same σ² everywhere assumes constant conditional variance. This limit is relative to the measured inputs: adding useful information can lower it. A finite test score can fall below it through sampling variation.

$\mathbb E[(Y-f(x))^2\mid X=x]=\operatorname{Var}(Y\mid X=x)$ when $f(x)=\mathbb E[Y\mid X=x]$.

J

Jacobian

A table of partial derivatives, with one row per output and one column per input.

A function F with m outputs and n inputs has an m × n Jacobian J. Entry (i, j) tells how output i changes locally when input j changes and the other inputs stay fixed. For a small input change Δx, the output change is approximately JΔx. The chain rule composes these linear maps. Backpropagation propagates a loss derivative through them using vector–Jacobian products, without needing to construct every full Jacobian.

J[i,j] = ∂Fᵢ/∂xⱼ; ΔF ≈ JΔx

joint attention

Two streams with their own weights meeting in a single attention operation.

Text and image patches are different enough that sharing weights helps neither, so each keeps its own normalisation and feed-forward layers. But the picture must be about the words, so somewhere they have to interact. Joint attention is that point: text tokens and patch tokens attend over each other together in one operation, rather than one stream reaching across to a frozen summary of the other.

[text tokens ; image tokens] -> one attention -> back to separate streams

K

k-means clustering

Sort number-lists into k groups by repeatedly moving each item to its nearest group center.

First choose k centers. Assign every item to its closest center. Then replace each center with the average of the items assigned to it. Repeat those two steps. The final group number can act like a simple sound label.

Assign items to the nearest center, move centers to group averages, and repeat.

KL divergence

A measure of how far one probability distribution sits from another.

KL divergence quantifies how much one distribution P differs from a reference distribution Q — it is zero when they are identical and grows as they diverge (and it is not symmetric). In RLHF it is used as a penalty: the tuned policy’s token distribution is compared against the frozen reference policy’s, and straying far is punished. That penalty is the leash that stops the policy from chasing high reward into degenerate text, balancing reward against staying recognizably like the original model.

D_KL(P || Q)

knowledge citation

A pointer back to the source of a claim — a link or a passage — so you can check it yourself.

A knowledge citation is a reference attached to a factual claim — a URL, a document id, or a pointer to the exact passage — that lets a reader trace the claim back to its source and verify it. RAG makes this natural: because the generator answered from specific retrieved passages rather than from memory, the system can point back to the passages the answer was built from. This is a benefit a memory-only model simply cannot offer, since its facts have no traceable origin, and it directly addresses trust — confidence you can check beats confidence you have to take on faith. It doesn’t guarantee correctness on its own, though: the cited passage still has to actually support the claim the model attached it to.

KV cache

Store the key and value vectors already computed for earlier tokens in a causal decoder.

While extending an unchanged prefix with fixed model weights, each new token can reuse those vectors at each layer. Its new query still attends to earlier keys and values. At length t, that query has t allowed query–key pairs, versus t(t+1)/2 pairs if the whole prefix’s causal attention is recomputed. These counts describe attention pairs, not every operation or actual latency. The cache does not normally store the full attention-score triangle.

cache per layer: earlier key vectors and value vectors

L

least squares

Choose model coefficients to minimize the sum of squared residuals. For linear predictions Aw, the objective is ‖y − Aw‖². Minimizing the mean instead gives the same fitted coefficients.

linear approximation

Predict a nearby output using the value and derivative at the starting point.

For differentiable f, f(x+h) ≈ f(x) + f′(x)h. The error divided by |h| tends to zero as h tends to zero. A bounded second derivative nearby gives the stronger error bound proportional to h²; differentiability alone does not guarantee that bound. With several inputs, use the gradient’s dot product with the input step.

f(3.01) ≈ 9 + 6 × 0.01 = 9.06 when f(x) = x²

limit

The value a function approaches as its input approaches a specified point, whether or not the function is defined at that point.

Rise over run for x² at 3 is exactly 6 + Δx for every nonzero step, so it can be brought as close to 6 as anyone demands by taking the step small enough: 6 is the limit as Δx → 0. Nothing is evaluated at Δx = 0, where 0/0 has no meaning. The formal version says: name any tolerance, and there is a step size below which the answer stays within it. Both the derivative (a ratio whose step shrinks) and the integral (a sum whose slivers shrink) are limits.

lim as Δx→0 of (6 + Δx) = 6

language model

A model that gives likely next pieces based on the sequence so far.

A language model learns which sequences tend to occur. For text, it can rank possible next words or word pieces. For codec-based speech synthesis, a similar model can rank possible next audio-code tokens. Its prediction is a probability for each allowed next token.

earlier tokens -> probabilities for the next token

latent diffusion

Run the whole denoising loop on a compressed version of the image rather than on pixels.

Dozens of denoising passes over full-resolution pixels is mostly effort spent on texture nobody inspects. Instead an autoencoder compresses the image to a much smaller grid, roughly 32 by 32 by 4 instead of 256 by 256 by 3, all the denoising happens in there, and a decoder expands the result at the end. The thing being cut into patches and denoised is therefore not the picture but a compact stand-in for it.

image -> encoder -> latent -> denoise in latent -> decoder -> image

layer norm

Normalize across the feature coordinates of one token’s vector, then apply learned scales and offsets.

Subtract the coordinate mean and divide by the square root of the coordinate variance plus a small positive ε. Learned gain and bias then scale and shift each coordinate. The final result need not have zero mean or unit variance. In a pre-norm Transformer, normalizing a sublayer’s input does not bound the magnitude of the residual stream itself.

LayerNorm(x) = gain × (x − mean(x)) / sqrt(var(x) + ε) + bias

learning rate

The stride length of gradient descent — how far to step in the downhill direction.

Each gradient-descent update moves the weights by minus the learning rate times the slope, so this one dial converts a local measurement into an actual step. Too small and training crawls; about right and the loss rides smoothly down; too large and ‘downhill’ steps overshoot the valley floor and land higher on the far wall — oscillation, or full divergence when each bounce climbs. Real training schedules the stride (warmup, decay) because the useful step size shrinks as the model settles into a minimum.

w ← w - lr · ∇L(w)

lexical gap

When one language has no single word matching a concept named by another.

A lexical gap appears when a source word carves out a concept for which the target language has no neat one-word equivalent. The translator may need a phrase, explanation, broader word, or context-dependent compromise. It proves that bilingual vocabulary is not a one-to-one table: languages divide conceptual space differently.

one source concept -> no exact target word

linear combination

Multiply each available vector by a coefficient, then add the results. For example, 2u − v is a linear combination of u and v.

linear independence

Vectors are linearly independent if the only coefficients that combine them into the zero vector are all zero. No vector in the set is redundant. This is different from independence in probability.

linguistic typology

The study of recurring ways languages are built and how those designs vary.

Linguistic typology compares languages along structural dimensions such as subject-verb-object order, whether adpositions come before or after nouns, how many morphemes fit inside one word, and whether pronouns can be omitted. For machine translation, typology predicts the transformations a model must learn: reordering, splitting or merging, and inserting information that the target language requires.

compare languages by structural dimensions

log-Mel features

A compact spectrogram adjusted to roughly match how human hearing separates pitch and loudness.

The Mel scale uses wider frequency bands at high frequencies and narrower bands at low frequencies, roughly following human pitch perception. Taking a logarithm squeezes a very wide range of loudness values into a smaller range. The resulting numbers are common inputs for speech recognition.

spectrogram -> Mel bands -> logarithm

log-odds

The straightened belief ruler: log of the odds, symmetric around 0 and unbounded both ways.

Log-odds (the ‘logit’) is log(p/(1-p)) — probability with the squash removed in two moves: odds remove the ceiling, the log removes the lopsidedness. Perfectly unsure (50%) sits at exactly 0, belief and disbelief mirror each other as +x and -x, and equal steps mean equal multiplications of the odds everywhere on the ruler. That makes it the scale where evidence adds, where logistic regression sums its feature scores, and where the Bradley-Terry model places its score difference; the sigmoid is simply the trip back from this ruler to probability.

log-odds = log( p / (1 - p) )

logarithm (log)

A slow-growing curve that counts the digits (the zeros) in a number.

A logarithm asks a backwards question: not “what is 10 used as a factor 3 times?” (that’s 1000) but “how many times must I multiply 10 to reach this number?” (that’s 3). So it climbs fast at first, then crawls: log(1) is 0, log(10) is 1, log(1000) is 3, and ever bigger inputs add only tiny bits more. Switching the base just rescales the whole curve, so the shape is the same whether you count in 10s, 2s, or e’s. We use this for surprise by flipping the sign: surprise = -log(probability). Something certain (probability 1) gives 0 surprise, while a rare event (a tiny probability) sends the log diving down, so minus-log shoots up huge. log and exp are exact inverses, which is why a model’s cross-entropy loss is just the minus-log of the probability it gave the true next word.

surprise = -log(probability); log(1) = 0; exp and log are inverses, so exp(log(x)) = x

logit lens

Peek at the model’s running guess at every step, not just the end.

The model builds its answer in a tall stack of layers, each one nudging an internal scratchpad of numbers a little further along. The logit lens takes that scratchpad at every layer and runs it through the model’s final word-reader early, asking “if you had to answer right now, what word?” Low down the guess is a fuzzy, common word; higher up it snaps into the real answer. It matters because it lets us watch a thought form, layer by layer, instead of only seeing the finished sentence.

guess(layer) = read_words(scratchpad at that layer)

logits

A score for every possible next word, before turning them into probabilities.

Logits are the model’s raw, unnormalized scores — one number per word in the vocabulary — saying how much it favors each as the next word. They come straight from the final layer and can be any real number: a higher logit means “more likely” relative to the others, and only the differences between them matter (adding the same amount to every logit changes nothing). Softmax then squashes the whole list into probabilities that add up to 1, via softmax(x)_i = e^(x_i) / sum_j e^(x_j), so the highest logit becomes the most probable word.

softmax(x)_i = e^(x_i) / sum_j e^(x_j)

LoRA (low-rank adaptation)

Keep a frozen brain; learn a tiny side-note made of two skinny grids.

The model’s big weight grid is huge and expensive to retrain, so LoRA leaves it frozen and learns a small change on the side, called the update (ΔW). Instead of storing that full-size update, LoRA builds it from two skinny grids, A and B; multiplied together they form a grid the same shape as the big one, but assembled from far fewer numbers. This works because the useful update is low-rank: it can be reconstructed from just a few independent directions, so a couple of skinny columns (in A) and rows (in B) are enough to span it. At run time you add this learned update back onto the frozen weights. You end up training a tiny sliver of the parameters and still capture most of the benefit.

W_new = W + A·B, rank r ≪ size (A is tall-skinny d×r, B is wide-skinny r×k; only A and B are trained, W is frozen)

loss function

A number that tells training how wrong the model’s current output is; smaller is better.

A loss compares the model’s result with a desired result or property. Training uses the loss to work out small changes to the model. A codec may combine several losses because matching sample values, matching frequency patterns, and sounding realistic are related but different goals.

prediction + target -> error number

low frame rate

A shortened sound timeline with fewer positions, each covering a little more time.

A recording contains far more time positions than its transcript contains letters or words. A system can combine nearby frames and keep fewer positions. This uses less computation, though some exact timing detail is lost.

Combine many short sound frames into fewer, broader sound summaries.

lower-resource language

A language with relatively little digital text, parallel data, tooling, or evaluation coverage.

Lower-resource describes the computational resources available for a language, not the language’s intrinsic richness. Scarcity can include digitized text, aligned translations, tokenizers, annotators, benchmarks, and compute investment. Translation quality often follows this resource distribution, so multilingual transfer, data creation, community control, and targeted evaluation are core design problems.

limited data + tools + evaluation coverage

M

[MASK] token

A special placeholder word that stands in for a hidden word the model must guess.

The [MASK] token is a single special entry in the vocabulary that replaces a word during masked language modeling, marking the slot the model has to fill in. There’s a subtlety: if the model only ever saw [MASK], it would behave oddly at real-use time when no [MASK] is present — so of the 15% of tokens BERT picks to predict, it replaces 80% with [MASK], swaps 10% for a random word, and leaves 10% unchanged. That mix keeps the model honest about every position, not just the obviously-masked ones.

"the [MASK] sat" -> predict the word at [MASK]

masked language modeling

Hide some words in a sentence and train the model to guess them from both sides.

Masked language modeling (MLM) is the training game that powers encoders: randomly hide a fraction of the tokens (BERT hides about 15%) and ask the model to predict the originals using the surrounding context on both sides. It’s the bidirectional cousin of Book 01’s next-word prediction — but because the model can peek left AND right, it can’t cheat by copying the next word, so the task pushes it to truly understand each blank. The loss is the same cross-entropy used in Book 01, just measured only on the masked positions.

loss = -log P(true_word | sentence with that word masked), averaged over masked positions

masking

Hide part of an input so the model must recover what belongs there from the surrounding context.

A masked span is replaced or covered before the model sees the input. The original hidden part remains available as the answer used to score the prediction. In speech pretraining, masking forces the model to learn from sound on both sides instead of simply copying the current frame.

visible context + hidden gap -> predict the hidden target

matrix multiply

Calculate each output entry by multiplying one row of the first matrix with one column of the second and adding the products.

If A has shape m × n and B has shape n × p, their product has shape m × p. For row [2, 3] and column [4, 5], the output entry is 2 × 4 + 3 × 5 = 23. What this means depends on the axes: a Transformer projection mixes features with learned weights, QKᵀ compares token positions, and AV combines value rows. Normalization, softmax, and activations perform other operations.

(m × n)(n × p) → (m × p)

mean (average)

Add the numbers, divide by how many: the balance point.

The mean is what each value would be if everyone shared equally. Picture the numbers sitting on a seesaw: the mean is the exact spot where it balances, with the highs and lows canceling out. Layer norm finds this balance point for a vector and slides everything over so the new center sits at 0 — a fair, even starting line.

mean = (x1 + x2 + ... + xn) / n

mean average precision (MAP)

A single score for a search system: how high, on average, it ranks the relevant documents.

MAP rewards putting relevant documents near the top, not merely returning them somewhere. For one query you first compute average precision: walk down the ranked list and, at each rank where a relevant document sits, record the precision up to that point, then average those recorded values. A relevant document buried deep drags its precision term down, so ranking it higher directly lifts the score — this is what makes MAP sensitive to order, unlike a flat precision-at-k. MAP is then simply the mean of average precision across a whole set of queries, giving one number that summarizes ranking quality over an entire benchmark.

AP = (1/|R|) * sum over relevant docs d of Precision(rank of d); MAP = mean of AP over all queries

mixed strategy

A probability distribution over a player’s available pure strategies. Expected payoffs average over these random choices. Every action used with positive probability in equilibrium must be a best response.

mean opinion score (MOS)

The average of human listeners’ quality ratings for speech clips.

Listeners hear clips one at a time and give each a score, commonly from 1 to 5. MOS is the mean, or ordinary average, of those ratings. Results depend on the listeners, instructions, sentences, voices, and listening equipment used in that test.

MOS = add all listener ratings, then divide by the number of ratings

mean squared error (MSE)

The average of the squared misses between predictions and actual outputs — the score a regression fit is judged by.

For each dot, take the prediction minus the actual output, square it (so misses in both directions count, and big misses count most), and average over the dots. Computed on the dots used to set the model’s knobs it is the training MSE, which can only fall as the model gains flexibility; computed on dots the fit never saw it is the test MSE — the number that actually matters, which dips and then rises.

MSE = (1/N) Σ (y_i − f̂(x_i))²

Mel-frequency cepstral coefficients (MFCCs)

A small list of numbers summarizing the broad frequency shape of a short sound frame.

MFCCs begin with frequency energy arranged on the Mel scale, then compress that shape into a short list of coefficients. They are hand-designed sound features. Early HuBERT training can cluster MFCC frames to create rough pseudo-labels before the network has learned better features.

short sound frame -> compact frequency-shape numbers

min-p sampling

Keep only words at least a fraction as likely as the top word — a confidence-aware cutoff.

A newer trimming method, a cousin of top-k and top-p. Min-p sets the cutoff relative to the most likely word: keep a word only if its probability is at least min_p times the top word’s (say 5%). When the model is very confident, the top word towers over the rest, so the cutoff is high and few words survive; when it’s unsure, the bar drops and more words get a chance. That makes it behave well even at high temperature. It’s a standard option in open-inference tools (vLLM, llama.cpp, Ollama, Hugging Face), though commercial APIs usually don’t expose it.

keep token if P(token) >= min_p * max_j P(token_j)

minimum Bayes risk decoding

Choose the candidate that best agrees with the whole distribution of plausible translations.

Minimum Bayes risk decoding selects the output with the lowest expected loss under the model’s candidate distribution. Instead of trusting the single highest-probability sentence, it rewards a candidate that is similar to many other plausible samples. The method turns consensus among likely meanings into a selection rule and can outperform ordinary beam search.

choose y minimizing expected loss against plausible outputs

minimum edit distance

The smallest number of replace, add, and remove steps needed to make two sequences match.

To repair one transcript into another, you may replace a wrong word, add a missing word, or remove an extra word. Many repairs are possible. Minimum edit distance finds the route that uses the fewest steps, while dynamic programming reuses smaller answers to avoid trying every full route.

distance = fewest replacements + additions + removals

mixture of experts (MoE)

A model whose layers each keep many helper networks, but route every word to just a few.

A normal layer runs every word through one feed-forward network. MoE swaps that for many parallel “expert” networks plus a tiny router. The router scores the experts for each word and sends it to just the top one or two; the others sit idle for that word. This happens independently at each MoE layer, so a single word can hit different experts as it moves up the stack (attention and the rest of the model stay fully active). The payoff: total parameters can be enormous, but the compute per word stays close to a small dense model, because only a fraction of the weights fire for any given word.

active compute per word ≈ (experts used, e.g. top-2) / (experts per layer, e.g. 8), summed over every MoE layer — total params big, params used per word small.

MMLU

A giant multiple-choice exam across 57 subjects used to score what a model knows.

MMLU (Massive Multitask Language Understanding) is a benchmark of about 16,000 multiple-choice questions spanning 57 subjects — from high-school math to law, medicine, and history. To ‘take’ it, the model is shown a question with options A–D and is scored by which option it rates most likely; the headline number is simply the percentage answered correctly. It became a standard yardstick because one score sweeps across many domains, but it’s still only a proxy: it rewards exam-style recall, can be inflated if the questions leaked into pretraining (data contamination), and says nothing about reasoning shown, honesty, efficiency, or real-world usefulness.

score = % of 57-subject multiple-choice questions answered correctly

model alignment

Adjusting a model so its behaviour matches what people actually want.

Alignment is the goal that post-training serves: nudging an LLM toward being helpful (it does what you asked), honest (it does not make things up or mislead), and harmless (it refuses to cause damage). A pretrained base model has none of this on purpose — it just predicts likely text — so alignment is the work of layering human intent on top, usually through instruction tuning followed by preference-alignment methods like RLHF or DPO. It is a direction to steer toward, not a fixed recipe, which is why different labs reach it by different combinations of techniques.

Model Context Protocol (MCP)

An agreed format for how applications ask tool servers what they offer and then call it.

MCP is a client-server convention carried over JSON-RPC. A host application connects to servers that expose three kinds of thing: tools (functions the model can invoke), resources (data it can read) and prompts (templates for common jobs). Nothing about it is technically clever; its value is that everyone uses the same plug shape, so a tool written once works with any compatible client, and a server can announce that its tool list changed mid-conversation.

host <-> server: tools/list, tools/call, notifications/tools/list_changed

model training

The learning stage, when examples are used to change the model’s stored numbers and reduce mistakes.

During training, the model makes a prediction, a loss score measures how wrong it was, and an update changes the model slightly. Repeating this across many examples teaches patterns. Training is different from inference, when the finished model is used without changing its learned weights.

example -> prediction -> error score -> small update

MS MARCO

A very large question-answering and passage-ranking dataset built from real Bing search queries.

MS MARCO (Microsoft MAchine Reading COmprehension) is a large-scale dataset drawn from real, anonymized Bing search queries — roughly a million questions and about 8.8 million passages, with human-written answers and human judgments of which passages are relevant. Its sheer size and its origin in genuine search traffic made it the standard training and evaluation set for passage ranking and for dense retrievers like the bi-encoder in the dense-retrieval chapter. The relevance labels are exactly the positive/negative signal a retriever needs, so much of modern retrieval is trained or benchmarked on it. Because the queries are real user searches rather than hand-crafted trivia, systems tuned on it tend to transfer to the messy questions RAG actually faces.

~1M real Bing questions, ~8.8M passages with relevance labels

multi-head attention

Several “lookers” read the sentence at once, each chasing a different kind of clue.

Multi-head attention runs several attention operations side by side, each with its own learned projections for deciding which words to pull information from. The point is that one relationship isn’t enough: one head might track “who does the verb,” another “which noun this adjective describes,” another long-range topic links (these are illustrative — real heads don’t always split up this cleanly). Each head produces its own slice of output; the slices are concatenated and then mixed by a learned matrix W_o into the final result. So a layer can attend to many relationships at the same time instead of being forced to pick one.

MultiHead(Q,K,V) = Concat(head_1, ..., head_h) W_o

multi-hop QA

Answering a question that needs several searches in a row, where each answer tells you what to look up next.

Multi-hop question answering handles questions no single passage can answer, because reaching the answer requires chaining facts together. The system retrieves, reads what came back, uses that partial result to build a new query, and retrieves again — each hop supplying the piece needed to ask the next. For example, ‘who directed the film that won Best Picture the year the Eiffel Tower turned 100?’ needs one lookup for the year, another for the film, and another for the director. A single retrieval can’t do this, because it only matches the words of the original question, which never mention the intermediate facts. Multi-hop is closely tied to agentic RAG, since deciding when to hop again is exactly the kind of choice an agentic system makes.

q1 -> retrieve -> a1 -> build q2 from a1 -> retrieve -> ... -> answer

multilingual model

One model trained across many languages so representations and capacity can be shared.

A multilingual translation model handles multiple source and target languages inside one set of weights, usually guided by a target-language tag. Shared tokens and internal features let lower-resource directions borrow statistical strength from related or higher-resource languages. The tradeoff is capacity and balance: large datasets can dominate unless sampling and training are designed carefully.

many language directions -> one shared model

multimodal diffusion transformer (MM-DiT)

A diffusion transformer with a full text tower alongside the image tower, joined by shared attention.

Earlier conditioning squeezed the condition into a single vector, which works for a class label and destroys a sentence: which object is red, which is green and what leans on what are exactly what one vector cannot hold. MM-DiT keeps the prompt as a sequence with its own weights and lets every patch attend to every word. The timestep still arrives by adaptive layer norm, because how noisy the input is genuinely is one number that applies everywhere.

text sequence + noisy patches -> joint attention blocks -> predicted noise

N

n-best list

The best few complete transcript guesses kept for one more comparison.

Instead of keeping only one transcript, the recognizer saves several strong candidates. A language model or task rule can then score them again. This can repair an early choice, but the new scoring must not overrule what the recording actually supports.

Keep several transcript guesses, score them again, and choose the final one.

named entity recognition

Finding and labeling the names in text — people, places, companies, dates.

Named entity recognition (NER) spots spans of text that name real things and tags them by type: person, location, organization, date, and so on. Unlike sequence classification, it’s a per-token job — every token gets its own label — so it uses the encoder’s individual contextual embeddings rather than just [CLS]. Context is essential here, since ‘Washington’ could be a person, a place, or an organization depending on the surrounding words.

each token -> entity label (PERSON, LOC, ORG, ...) or none

nash equilibrium

A collection of strategies where no player can improve their payoff by changing their own strategy alone. It need not be unique, fair, or best for the group.

natural language inference

Deciding whether one sentence follows from, contradicts, or is unrelated to another.

Natural language inference (NLI) gives the model two sentences — a premise and a hypothesis — and asks whether the premise entails the hypothesis, contradicts it, or neither (neutral). It’s a classic test of real understanding rather than surface word-matching, and encoders handle it by reading both sentences together (joined with [SEP]) and classifying the [CLS] vector into those three labels. It’s a sequence-classification task over a pair of texts.

(premise, hypothesis) -> {entailment, contradiction, neutral}

Natural Questions (NQ)

A question-answering dataset built from real Google searches, each paired with a Wikipedia page and its answer.

Natural Questions is a benchmark whose questions are real, anonymized queries people actually typed into Google — information-seeking questions asked because someone genuinely wanted the answer, not written to test a system. Each question is paired with a Wikipedia page, and annotators mark a long answer (the paragraph or section that answers it) and a short answer (the specific entity or span), or label it unanswerable. This makes it a natural fit for evaluating RAG: the questions are as messy and honest as real user queries, and answering them means finding the right passage and reading it — the exact retrieve-then-generate loop this book builds. Its scale, hundreds of thousands of questions, also makes it a common source of training data for retrievers and readers.

ndarray

NumPy’s multidimensional array: data with a type, shape, and strides.

Shape gives the dimensions; strides give the byte offset for one step along each axis. Elements share a data type, but an array view need not be contiguous. Changing a view can change the original because they share memory.

ndarray = data buffer + dtype + shape + strides

next sentence prediction

A yes/no training task: did sentence B actually follow sentence A?

Next sentence prediction (NSP) was BERT’s second pretraining task alongside masked language modeling: show two sentences and have the model (via the [CLS] vector) decide whether B genuinely followed A in the original text or is a random impostor (the two cases are split 50/50 during training). The goal was to teach relationships between sentences for tasks like question answering. It turned out to add little — later models like RoBERTa dropped it and did better — so it’s mostly of historical interest now.

[CLS]-vector -> P(B truly follows A)

non-autoregressive (NAR) model

A model that can predict many positions together instead of waiting for each earlier position.

NAR is short for non-autoregressive. Once the needed context is available, the model fills several output positions in parallel. In VALL-E, the first coarse audio-code stream is made in order, and later detail streams can be filled together because the coarse timing is already known.

known context -> many output positions at once

nonlinearity

A bend in the math that lets stacked layers learn more than a straight line.

A nonlinearity is a simple function that bends its input instead of just scaling it—ReLU, for example, keeps positive numbers unchanged and sets negative ones to zero. It matters because stacking matrix multiplies with nothing in between is mathematically the same as one big matrix multiply, so extra depth would buy you nothing. Inserting a bend between layers lets the network stack many transformations and carve out curved, complex patterns. In a transformer, the main activation-function bend (ReLU or GELU) lives inside the feed-forward block after attention—though it’s not the only nonlinearity around, since the softmax in attention and layer norm are nonlinear too. Together these bends give each layer real expressive power instead of collapsing into a single linear map.

ReLU(x) = max(0, x)

normal equations

The equations AᵀA w = Aᵀy for a least-squares fit. They say the residual y − Aw has zero dot product with every column of A. Independent columns give a unique coefficient vector.

O

odds

Wins per loss: a probability recounted as a ratio, with no ceiling.

Odds re-express a probability p as p/(1-p) — how many times the event happens for each time it doesn’t. 90% becomes 9-to-1, 99% becomes 99-to-1, so the crushed ‘very sure’ sliver near probability 1 stretches out to infinity and changes of belief show their true size. The scale is still lopsided (all of ‘unlikely’ is squeezed between 0 and 1, all of ‘likely’ spans 1 to infinity), which is exactly what taking the log fixes. Odds are also what independent evidence multiplies, making them the natural currency of belief-updating.

odds = p / (1 - p)

optimizer

The object that holds your parameters and applies the update step after backward() fills the gradients.

An optimizer is bookkeeping around gradient descent: it keeps the list of parameters, and when step() is called it moves each one against its .grad — plain SGD moves by minus the learning rate times the gradient, while variants like momentum or Adam reshape the stride using running statistics of past gradients (they change how far, never the underlying ‘downhill’ logic). Its other job is zero_grad(): gradients accumulate by design in PyTorch, so the loop clears them before each backward pass. In the canonical five-line training loop, backward() computes the slopes and the optimizer takes the step.

opt.step(): w ← w − lr · w.grad (SGD; Adam et al. reshape the stride)

orthogonal

Perpendicular under the usual dot product: the dot product is zero. In least squares, the fitted residual is orthogonal to every column of the design matrix.

overfitting

Learning details of the training examples that do not carry over well to fresh examples.

A flexible curve may follow accidental noise in the training data and then miss new outputs. A small training error and much larger validation error suggest this problem when the sets represent the same population. Distribution changes or data-processing errors can also cause a gap, so the two scores alone do not prove overfitting.

P

parallel corpus

The same messages paired across two languages for translation training.

A parallel corpus, or bitext, is a collection of source-language spans matched with human translations in a target language. Each pair acts as supervised evidence that two different strings express the same message. Its domains, language balance, alignment quality, and noise become part of what the translation model learns.

(source sentence, target translation) pairs

parameters

The model’s learned numbers — the knobs it tunes during training.

Parameters are the numbers (mostly weights, plus biases) inside the model that get adjusted during training; “405B” means 405 billion of them. Each one is a tiny dial that nudges how an input signal flows toward an output, so the model is really one huge stack of multiply-and-add operations governed by these numbers. In a transformer they live in the attention layers (deciding which words look at which), the feed-forward layers (which act like memory, storing patterns and facts), and the embedding tables (turning tokens into vectors and back). Training slowly tunes every dial to lower prediction error. More parameters means more room to store patterns — which is part of why bigger models can do more.

output = sum_i (weight_i * input_i) + bias

parametric model

A model family described by a fixed number of parameters, such as a line with an intercept and a slope.

A parametric assumption lets observations across the input space constrain a shared relationship. That can require less data than estimating a separate local average near each input. It does not eliminate the difficulties of many dimensions: the parameter count, input correlations, regularization, and whether the chosen form suits the data still matter. “Parametric” does not mean “linear.”

partial derivative

The derivative with respect to one input of a function of several inputs, holding the others fixed.

For f(x,y) = x²y + y, ∂f/∂x = 2xy and ∂f/∂y = x² + 1. At (2,3), these rates are 12 and 5. If y also changes when x changes, the total rate along that path can differ from ∂f/∂x.

∂f/∂x (a,b) = lim h→0 [f(a+h,b) − f(a,b)] / h

pass@k

The chance that at least one of k generated attempts passes the tests.

Generation is stochastic, so a single run does not characterise a model. pass@k measures whether a working answer can be found in k tries, and the two ends of it ask different questions. pass@1 asks whether the model can be trusted once, and rewards careful low-temperature sampling. pass@k for large k asks whether a solution exists at all, and rewards adventurous sampling, since k different attempts beat k copies of the same one when tests sort them out afterwards.

pass@k = probability at least one of k samples passes

patch embedding

Project an image patch into a vector using learned weights.

Flatten a p-by-p RGB patch into 3p² numbers, then multiply by a matrix that produces d entries, often adding a bias. This projection has 3p²d weights, excluding bias, so its shape depends on patch size. At resolution 224×224 with 16×16 patches, the image gives 14×14 = 196 patch tokens before any extra tokens are added.

RGB patch: 3p² entries → projection → d-entry vector

payoff

A number representing how a player values a particular outcome in the model. In a pair (row, column), each number belongs to one player. Payoffs may represent money or other preferences.

perplexity

How surprised a model is by real text it didn’t train on — lower means it predicted better.

Perplexity measures how well a model predicts a held-out piece of text it never trained on: at each word it had a probability for what actually came next, and perplexity rolls those up into one ‘how surprised was I, on average’ number. It’s just Book 01’s cross-entropy loss run back through an exponential, so a perplexity of 10 roughly means ‘at each step the model was about as unsure as if it were picking between 10 equally-likely words.’ Lower is better. The catches: it only fairly compares models that chop text into the same tokens (the same tokenizer), and being un-surprised by text is not the same as being correct, helpful, or safe — so it’s a useful proxy, not the whole story.

perplexity = e^(cross-entropy) = exp(average surprise per token) (lower = better)

phoneme

A smallest sound category that can change one word into another in a language.

English speakers treat the first sounds of “bat” and “pat” as different phonemes because changing one changes the word. A phoneme is a language category, not one exact waveform; speakers and contexts can produce it in many physical ways.

change one phoneme -> possibly change the word

policy

The LLM itself, viewed as the thing choosing each next token.

In the reinforcement-learning framing of RLHF, the policy (written pi) is just the language model: given the tokens so far, it produces a distribution over the next token and picks from it, so generating an answer is a sequence of choices. RLHF tunes this policy to produce outputs the reward model scores highly, while a KL penalty keeps it from straying too far from the frozen reference policy it started as. So policy is simply the RL name for the model whose weights are being updated.

pi(next token | tokens so far)

polysemy

When one word carries several different meanings.

Polysemy is the fact that a single word often has multiple meanings — ‘mouse’ the animal vs. the computer device, ‘light’ as in weight vs. brightness. It’s the reason a single fixed embedding per word is fundamentally limited: that one vector has to be a compromise across every sense. Because contextual embeddings shift with the sentence, an encoder can split a polysemous word into the right meaning on the fly.

one word -> many meanings (e.g. "mouse" = animal or device)

positional encoding

Information that lets a model use token positions and order.

The original Transformer adds sinusoidal position vectors to input embeddings. Learned position vectors are another choice. Rotary position embeddings (RoPE) instead rotate query and key coordinate pairs according to position; their dot products then depend on relative position as well as content. Position information alone does not guarantee reliable use of arbitrarily long contexts.

additive version: input = token embedding + position vector

post-training

Further training after pretraining, often aimed at instruction following or other desired behavior.

Methods include supervised training on demonstrations and training from preference feedback. They change model parameters and can affect both behavior and knowledge. Additional computation during generation is a separate choice: producing more candidate answers does not, by itself, train the model.

pretrained model → additional training → updated model

postings

The list of documents attached to a word inside an inverted index.

A postings list is what an inverted index stores for each term: the set of documents that contain it, usually with extra details like the term’s count in each document (for scoring) and sometimes its exact positions (for phrase search). To answer a query, the system fetches the postings list for each query term and merges them, combining each document’s tf-idf or BM25 contributions to produce a ranking. Keeping counts and positions right there in the postings is what lets the index compute relevance scores without ever re-reading the original documents.

postings(t) = [(doc, count, positions?), ...]

power law

Make something ten times bigger, get a steady, predictable jump in quality.

A power law links two things by multiplication, not addition: when you multiply one by ten, the other changes by a fixed factor every time. Drawn normally it’s a swooping curve, but if you stretch both axes so equal steps mean “times ten” instead of “plus one,” the curve straightens into a line. A model’s loss falls along such a line as you grow it, so each tenfold scale-up buys you the same reliable slice of improvement.

y = x^p (straight line on log-log axes)

PPO (proximal policy optimization)

The reinforcement-learning algorithm commonly used to do the RLHF policy update.

PPO is a general reinforcement-learning method, and in RLHF it is the workhorse that actually updates the policy to increase reward. Its defining trick is to limit how much the policy can change in a single update step, clipping the update so training stays stable rather than lurching and collapsing. That caution pairs naturally with the KL penalty toward the reference policy, and DPO can be seen as a way to reach a similar result without running PPO at all.

precision

Of the documents your search handed back, the fraction that were actually relevant.

Precision answers ‘when the system returned something, how often was it right?’ — it divides the number of relevant documents retrieved by the total number retrieved. It punishes a system for padding its results with junk, but on its own it is easy to game: return the single document you are surest about and precision can be a perfect 1.0 even though you missed a hundred other relevant ones. That blind spot is exactly why it is always reported alongside recall, its mirror image. In ranked retrieval precision is recomputed at each step down the list, and that is what builds the precision-recall curve.

precision = (relevant docs retrieved) / (all docs retrieved)

precision-recall curve

A jagged plot of precision against recall, traced out as you descend a ranked list of results.

A search engine returns a ranked list, so you can descend it one document at a time and, at each step, recompute precision and recall over everything seen so far. Plotting precision (vertical) against recall (horizontal) traces a distinctive sawtooth: every time you hit a non-relevant document recall stays put while precision dips, and every time you hit a relevant one recall steps right and precision jumps back up. The shape captures the entire precision-versus-recall trade-off in a single picture rather than a lone number. Its raggedness makes two systems awkward to compare directly, which is why it is usually smoothed into interpolated precision.

preference alignment

Aligning a model using judgments of which of two answers is better.

Preference alignment is the part of post-training that uses comparative human judgments — given a prompt and two candidate answers, which one is preferred — to shape the model’s behaviour. Rather than telling the model the single right answer, it teaches the model the relative ranking, which captures fuzzy qualities like helpfulness and tone that are hard to write down as one correct target. The two main ways to do it are RLHF, which trains a reward model and then reinforcement-learns against it, and DPO, which optimizes the preference pairs directly without a separate reward model.

prompt + (chosen vs rejected) -> tune the model toward chosen

preference data

Pairs of answers to the same prompt, with one marked as better than the other.

Preference data is the raw material of preference alignment: for a prompt x you collect two outputs and a human marks one as chosen (preferred) and the other as rejected, written (o_i > o_j | x). It encodes relative judgments rather than absolute right answers, which is what makes it good at capturing qualities like helpfulness and tone. Both RLHF (to train its reward model) and DPO (directly) learn from exactly this kind of data.

(o_i > o_j | x) — chosen o_i preferred over rejected o_j for prompt x

pretraining

A broad first learning stage before the model is taught one specific job.

Pretraining lets a model learn general patterns from a large collection before smaller task-specific training begins. A language model may predict the next text token. A speech model may hide part of a recording and learn to recover a sound-group label. The later fine-tuning stage then teaches the exact task, such as turning speech into a transcript.

large general dataset -> broad learning -> task-specific training

prior

Where your belief stands before the new evidence arrives — the starting point of the walk.

The prior is your probability for something before seeing the evidence at hand: how rare the disease is before the test, how plausible the hypothesis is before the experiment. On the log-odds ruler it is literally a starting position, and Bayes’ rule is then just one addition — posterior = prior + evidence. It is not optional: refusing to name a prior just means silently using 50/50, which for rare things is wildly wrong. Forgetting the prior is the base-rate fallacy — the reason a positive result from an excellent test for a rare condition still usually means you’re fine.

posterior log-odds = prior log-odds + evidence points

probability

A number from 0 to 1 telling how likely something is.

Probability is a number that measures how likely something is, on a scale from 0 (it never happens) to 1 (it always happens). Think of one whole pie cut into slices, one slice per possible outcome, where a bigger slice means “more likely.” Because exactly one outcome must happen, all the slices together make a full pie, so the probabilities always add up to 1. A transformer predicts the next word this way: it hands every possible word a slice, and the biggest slice is its best guess.

P(everything) = p1 + p2 + p3 + ... = 1

process reward model (PRM)

A model trained to judge each individual reasoning step, not just the final answer.

Outcome supervision grades only the end result, which is one bit of feedback for a whole chain and can reward reasoning that was wrong in ways that happened to cancel out. Process supervision grades every step, so feedback is dense and credit lands where it belongs. A PRM learns to make those step judgements, which is useful twice: at answer-time it scores branches so a search knows where to look, and at training time it is a far sharper signal than a single verdict at the end.

partial solution -> per-step correctness score

projection

The closest vector in a specified subspace, using ordinary Euclidean distance. Projecting onto a nonzero vector u means finding the closest multiple cu. The coefficient c is (u · y)/(u · u); the projected vector is cu.

prompt

The text you give the model to kick off and steer its answer.

A prompt is the input text the model conditions on before it starts writing — your question, instructions, examples, or pasted context. Because a decoder simply continues the tokens it is given, the prompt isn’t a separate command channel; it literally becomes the start of the sequence the model extends. That is why wording, order, and included examples can change the answer so much, and why prompting is its own craft.

prompt -> model -> continuation

prompt engineering

Carefully wording and structuring your prompt to get better answers.

Prompt engineering is the practice of designing the prompt — phrasing, formatting, adding examples, giving step-by-step instructions — so the model produces what you actually want, without changing any of its weights. It works because a decoder’s output is conditioned entirely on its input tokens, so the prompt is the one lever you control at use-time. Common moves include few-shot examples, asking the model to reason step by step, and assigning a role. It complements, rather than replaces, the training-time tuning from Book 01 (fine-tuning, RLHF).

prompt injection

Instructions placed in content that a model is supposed to treat as data, attempting to redirect its behavior.

For example, a fetched page might ask an assistant to ignore the user’s task. Message roles and training can help distinguish instructions from retrieved content, but they do not guarantee that the model will always respect the boundary. Applications can also restrict tools and data access, validate actions, and require review where the consequences warrant it.

pseudo-label

A machine-made practice answer used when no human label is available.

A pseudo-label is not a verified human answer. It is created automatically, for example by assigning each speech frame to the nearest k-means cluster. The label can be noisy, but predicting many such labels can still teach a model useful sound structure.

automatic rule or model -> temporary training target

Q

quantization

Replace many possible number values with a smaller set of allowed choices.

Quantization makes a detailed number or vector choose from a limited set. For model weights, that can mean rounding precise numbers so the model needs less storage. For speech codecs, it means replacing a sound vector with the nearest reusable codeword. In either case, fewer allowed choices make the representation smaller, with some loss of detail.

many possible values -> nearest allowed value

query

What you type in to say what you’re looking for.

A query is a user’s information need expressed as a handful of terms — the words you would type into a search box. The system treats it much like a tiny document: it turns the query into a vector of term weights and compares it against every document to score relevance. Because queries are usually short, a few well-chosen words carry enormous weight, which is exactly why rare, specific terms (high idf) are so much more useful in a query than common ones.

query = a set of terms expressing an information need

query, key, value

Three vectors calculated from each token row for attention.

Separate learned projections turn the current row into Q, K, and V, often after normalization. A receiving row’s query is compared with source rows’ keys. Scale those dot products, apply any mask, and use softmax to obtain attention weights. Those weights combine the value rows. The projection parameters stay fixed during ordinary inference; the resulting Q, K, V and attention weights depend on the input.

queries and keys determine the coefficients; values supply the vectors being combined

rank

The number of independent columns of a matrix. It is also the number of independent rows, and the dimension of the space of possible outputs.

Riemann sum

The staircase of rectangles under a curve: the integral before the slivers have been shrunk.

Split [a, b] into n slivers, read the height at one point in each (its left edge, right edge, or midpoint), and add height × width. For a rising curve the left-edge staircase undershoots and the right-edge one overshoots, and their gap is exactly one rectangle’s worth, (f(b) − f(a))·Δx, which dies as n grows — so the true area is trapped and both staircases converge to it. Machines integrate this way whenever no antiderivative formula exists, as for the bell curve.

Σᵢ f(xᵢ)·Δx → ∫ₐᵇ f(x) dx as n → ∞

ReAct

An agent loop that alternates reasoning, an action, and the observation it returns.

Reasoning alone runs on the model’s memory, so it cannot notice when a recalled fact is false and simply reasons forward from the error. Acting alone is grounded in real results but has nothing choosing what to do next, so queries never improve and results are never combined. Interleaving them fixes both: each thought aims the next action, and each observation corrects the next thought. The trace is also readable, so a failure can be traced to a bad query, a bad page, or a bad inference.

thought -> action -> observation -> thought -> ... -> answer

reading comprehension

Answering a question when you’re already handed the exact document that contains the answer.

Reading comprehension is the task of answering a question when the relevant document is already provided: you’re given a passage and a question, and you produce the answer — often a span copied straight out of the passage. There’s no searching involved, which isolates a single skill, understanding a specific text well enough to answer from it. RAG generalizes this into open-domain question answering: the document is not handed to you, so a retriever must find it first, and only then does the generator perform the reading-comprehension step on what came back. Seeing the two apart clarifies what RAG adds — the hard ‘find the right text’ problem stacked on top of the ‘read and answer’ problem.

(question, given document) -> answer (often a span of the document)

reasoning model

A model trained by reinforcement learning to think at length before answering.

Rather than bolting sampling and search around a model from outside, the deliberation is trained into it: reward reasoning that reaches correct answers and let learning discover what thinking gets there. Such models may generate thousands of hidden tokens before their first visible word. Notably, DeepSeek-R1 showed that with reinforcement learning alone, and no demonstrations to imitate, behaviours like self-verification and backtracking emerge on their own because they earn more reward.

more compute spent at answer-time -> better answers on hard problems

recall

Of all the relevant documents that exist, the fraction your search actually found.

Recall answers ‘of everything I should have found, how much did I get?’ — it divides the number of relevant documents retrieved by the total number of relevant documents in the whole collection. It is the mirror of precision: precision guards against returning junk, recall guards against missing good things. It too is trivially gamed alone — return every document in the collection and recall is a perfect 1.0 — which is why the two are never read apart. The tension between them (grab more results and recall rises but precision usually falls) is precisely what the precision-recall curve traces out.

recall = (relevant docs retrieved) / (all relevant docs in collection)

receptive field

The amount of the original recording that can affect one later value.

An early layer may see only a tiny sound window. Later layers combine neighboring earlier results, so one later value can depend on a longer part of the recording. That total visible span is its receptive field.

Stacking more local layers gives each later value a wider view.

recurrent neural network transducer (RNN-T)

A recognizer that can use both the sound heard so far and the text it has already written.

A recurrent neural network carries information forward through an ordered list, and a transducer changes one ordered list into another. One RNN-T part describes incoming sound; another remembers earlier output tokens. A combining step chooses either a new text symbol or blank, meaning move forward in the audio. This design can produce text while speech is still arriving.

current sound + earlier output -> symbol or blank

reference policy

The frozen pre-RL model the tuned model is kept close to during alignment.

The reference policy (pi_ref) is a snapshot of the model taken before reinforcement tuning begins, with its weights frozen. The KL-divergence penalty in RLHF (and the log-ratio in DPO) measures the tuned policy against this anchor, so it acts as the fixed point on the other end of the KL leash. Keeping the policy near it preserves the fluency and knowledge built up in pretraining and instruction tuning, preventing alignment from degrading the model while it chases reward.

penalty pulls pi toward pi_ref (pi_ref frozen)

reflection

Having an agent judge its own last step and write that judgement into its context.

After an action returns, the agent asks whether it got closer to the goal, whether the result was what it expected, and whether a different approach is needed. Because the answer is written into the context, a failed step becomes a note that steers the next attempt rather than something to repeat. This is what separates recovering from an error from executing a broken plan all the way to the end.

action -> result -> written self-assessment -> next action

regression function

In squared-loss regression, the conditional mean of the output at each input: the curve you would obtain by averaging outputs while holding the input fixed.

The function f is a property of the population. A fitted function, written f̂, is an estimate learned from finite data. With a continuous input, conditioning is defined through a conditional distribution; it does not require a positive fraction of observations to have exactly the same input.

$f(x)=\mathbb E[Y\mid X=x]$.

relevance-guided supervision

A cheap way to label training passages for a retriever: a passage counts as positive if it contains the answer, negative if it doesn’t.

Training a dense retriever (the bi-encoder from the dense-retrieval chapter) needs examples of which passages are relevant to a question and which aren’t — but no one can hand-label relevance across millions of passages. Relevance-guided supervision sidesteps this with a weak automatic signal: any passage that contains the gold answer string is taken as a positive example, and passages that don’t — especially hard negatives that look topical but lack the answer — are taken as negatives. The retriever is then trained contrastively to pull each question’s embedding close to its positives and push it away from its negatives. It works because ‘contains the answer’ is a noisy but cheap proxy for ‘is relevant,’ checkable by a simple string match, so you get large amounts of labeled training data essentially for free.

positive = passage contains the answer string; negative = passage that does not

ReLU

A switch that lets positive numbers through and turns negatives to zero.

ReLU (“Rectified Linear Unit”) is a tiny rule applied to each number: keep it if it’s positive, replace it with 0 if it’s zero or negative — that’s all of ReLU(x) = max(0, x). This bend is what lets a network learn curved, complicated patterns instead of only straight-line ones; without a nonlinear step like this, stacked layers would collapse into one plain linear map. Pile up many of these bends and the network can approximate almost any shape. Inside a transformer it’s a classic choice for the feed-forward layer that follows each attention step, where it decides which signals each neuron passes onward and which it silences (though many modern transformers swap it for cousins like GELU).

ReLU(x) = max(0, x)

repetition / presence / frequency penalty

Lower the score of words the model has already used, so it stops looping.

Temperature and top-k/top-p only reshape the distribution — they don’t directly stop the model repeating itself. Penalties do: before sampling, they subtract from the score of tokens already used. A presence penalty applies a flat hit to any word that has appeared at all; a frequency penalty grows with how often it appeared; repetition_penalty is the version in Hugging Face / vLLM / llama.cpp. They act straight on the logits, so they target loops the other knobs can’t.

logit_i -= penalty for tokens already generated

representational harm

When a model’s outputs reinforce unfair stereotypes about groups of people.

Representational harm is damage done by how a system depicts or describes people — perpetuating stereotypes, demeaning or erasing a group, or systematically associating certain identities with negative traits. With language models it often traces back to biases baked into the pretraining data scraped from the web, which the model then reproduces and can amplify. It is distinct from allocational harm (denying someone a concrete resource like a loan); representational harm is about the messages and associations the outputs carry. Measuring and reducing it is a central part of responsible AI work.

reranking

A second, more careful pass that reorders the retrieved passages before the model reads them, pushing the best ones to the top.

Reranking is a stage placed between retrieval and generation: the fast retriever returns a rough shortlist (say the top 100), and a slower, more accurate relevance model re-scores just those and reorders them, so the truly best passages rise to the top before the generator reads them. It exists because of the speed-versus-accuracy split from the dense-retrieval chapter — a bi-encoder is fast enough to scan millions of passages but judges relevance crudely, since query and passage never meet until a final dot product, whereas a cross-encoder reads each query-passage pair jointly and judges far better but is far too slow to run over a whole collection. Running the cheap-and-broad retriever first and the expensive-and-precise cross-encoder only on its shortlist captures most of the accuracy at a fraction of the cost. It pays off directly because the generator can only be as good as the passages it is handed, so ordering the right ones first matters.

cheap retriever -> top-N shortlist -> cross-encoder re-scores -> reordered top-k

residual

The part left over after the current approximation is subtracted from the original.

Suppose a codeword gives a rough copy of a sound vector. Subtract that copy from the original vector. The remaining difference is the residual. Residual vector quantization sends that leftover to another codebook so a second codeword can correct part of the error.

residual = original - current approximation

residual stream

The sequence of vectors carried between blocks, with each sublayer’s output added to the current vector.

The addition requires matching shapes. It preserves the vector’s width, but does not guarantee a small magnitude or preserve every earlier feature: additions can reinforce or cancel. Residual connections give information and derivatives a direct additive path through the network.

x_new = x + sublayer(x)

residual vector quantization (RVQ)

Make a rough sound-code choice, then use more choices to correct what it missed.

RVQ is short for residual vector quantization. The first codebook picks the closest rough sound pattern. Subtract that choice from the original; the leftover difference is the residual. A second codebook approximates that leftover, and later codebooks can add finer corrections. Add all chosen codewords to rebuild the final approximation.

rough choice + correction + smaller correction + ...

retrieval-augmented generation (RAG)

First fetch relevant documents, then let the model answer using them.

RAG splits answering into two steps: an information-retrieval system finds documents relevant to the question, and then a language model (the decoder from the transformers book) generates an answer conditioned on those documents placed in its context window. It exists because a model’s built-in knowledge is frozen at training time, so it goes stale, can’t see private or newly-written material, and tends to hallucinate confident-but-false facts when asked something it never learned. Grounding the answer in retrieved text attacks all three at once: the facts come from real, current, checkable sources rather than the model’s memory. The catch is that the answer is only as good as what retrieval surfaced — bad documents in, bad answer out.

answer = generate(query + retrieved documents)

retriever

The half of a RAG system that searches a big pile of text and pulls back the passages most likely to answer your question.

The retriever is the search engine inside retrieval-augmented generation: given a question, it scores the passages in a collection and hands back the top-k it judges most relevant. It’s exactly the machinery built across this book — a sparse word-overlap scorer like BM25, or a dense bi-encoder that matches by meaning — with no new parts bolted on. It matters because it decides what the generator ever gets to see: the model can only answer from passages the retriever surfaces, so a miss here caps the whole system’s accuracy no matter how strong the language model is. Splitting ‘find the evidence’ from ‘write the answer’ is what lets RAG stay current and cite sources without retraining anything.

R(q) -> top-k passages ranked by relevance

reward

The single score a reward model assigns to an answer — higher means more preferred.

The reward is the scalar number the reward model outputs for a given prompt and output, where a higher value means the answer is judged better. In the RL framing of RLHF it is the signal the policy is trained to increase, standing in for the human preferences the reward model learned from. Because it is only an approximation of true human judgment, blindly maximizing it can lead the policy into reward-hacking, which is why a KL penalty keeps the policy anchored to its reference.

reward = r(x,o) (scalar; higher = more preferred)

reward hacking

When a model games flaws in the reward model, scoring high while actually behaving badly.

Reward hacking happens because the reward model is only an imperfect stand-in for real human preferences: the policy discovers outputs that the reward model rates highly but that humans would not actually want — padding, flattery, or degenerate repeated patterns. Optimizing reward too hard pushes the policy straight into these blind spots, drifting away from genuinely good behaviour. The KL penalty against the reference policy is the main defence, keeping the policy from wandering into the strange regions where the reward model is most wrong.

high r(x,o) but low true quality (exploiting the reward model's errors)

reward model

A model that reads a prompt and answer and outputs one number for how good the answer is.

A reward model r(x,o) is typically an LLM with its next-token output head swapped for a single scalar head, so instead of predicting words it predicts one quality score. It is trained on preference pairs to give the chosen answer a higher score than the rejected one, using a loss that maximizes the sigmoid of their score difference — the Bradley-Terry assumption in action. In RLHF this learned scorer becomes the reward signal the policy is optimized against; its imperfections are what make reward-hacking possible.

L = -log sigmoid(r(x,o_w) - r(x,o_l))

RL for reasoning

Rewarding a model for answers that are verifiably right, which teaches it to think step by step.

The newest alignment twist (2025–26). Instead of (or on top of) learning from human preferences, the model is rewarded whenever its final answer is verifiably correct — the math checks out, the code runs, the puzzle is solved. Optimizing that reward pushes it to spend more tokens reasoning out loud before committing to an answer. DeepSeek-R1 did this with a method called GRPO and simple rule-based rewards; OpenAI’s o-series and the “thinking” modes of Claude and Gemini are the same family. It doesn’t slot into a fixed step — it interleaves with or partly replaces classic preference-tuning, and varies by model.

reward = 1 if the final answer is verifiably correct, else 0

RLHF (reinforcement learning from human feedback)

Train a reward model from people’s preferences, then tune the model to score high on it without drifting too far.

RLHF turns preference data into alignment in two stages: first a reward model learns to score outputs from people’s chosen-versus-rejected comparisons, then the LLM (the policy) is reinforcement-tuned to maximize that reward. To stop it from gaming the reward and producing degenerate text, the objective subtracts a beta-weighted KL-divergence penalty that keeps the tuned policy close to a frozen reference policy. The RL update is usually carried out with PPO; DPO is a later shortcut that skips the separate reward model entirely.

pi* = argmax E[ r(x,o) - beta * KL(pi || pi_ref) ]

RMSNorm

Like layer norm, but skips the centering step — just divide each vector by its overall size.

RMSNorm is a lighter cousin of layer norm. Where layer norm first subtracts the mean (re-centers to 0) then divides by the spread, RMSNorm drops the centering (and the bias) entirely: it just divides the vector by its root-mean-square — a measure of its overall size — and applies a learned per-feature gain. It turns out the re-centering wasn’t pulling much weight, so dropping it is cheaper and trains just as well. It’s the default in most modern open models (Llama, Mistral, Qwen, Gemma, DeepSeek).

y = x / sqrt(mean(x^2) + eps) * gain

runtime tool discovery

Fetching the list of available tools when the task runs, instead of training the model on a fixed set.

A model’s weights freeze on a date, while APIs, database schemas and internal services keep changing. Retraining for each is impossible, so capability is decoupled from weights: the application asks what is available, receives descriptions and schemas, and places them in the prompt. Because tool descriptions are just text, a list can even change during a conversation without restarting anything.

ask for the list -> put descriptions in the prompt -> model picks from them

S

scalar

One number, as opposed to a vector or a matrix. Multiplying a vector by a scalar multiplies every entry by that number.

second derivative

The slope of the slope: how fast the derivative itself is changing, which tells a valley from a hilltop.

Differentiate f′ again and you get f″. Positive means the slope is increasing — the curve bends upward like a bowl, so a flat spot there is a valley; negative means it bends downward, a hilltop; zero gives no verdict (x³ pauses at 0, x⁴ bottoms out). If f is position, f′ is velocity and f″ is acceleration. Curvature also affects gradient descent: for a quadratic with constant second derivative c > 0, a fixed learning rate between 0 and 2/c gives convergence to its minimum. Changing curvature requires a separate check.

f″ = (f′)′ · f″ > 0 bowl, f″ < 0 hill

[SEP] token

A special marker that separates two pieces of text fed in together.

The [SEP] token marks a boundary between segments when you give the encoder two chunks of text at once — for example a question and a passage, or two sentences being compared. It typically also closes the sequence. Paired with segment embeddings, it lets a single encoder reason about the RELATIONSHIP between two texts (does B answer A? does A imply B?) rather than just one text in isolation.

[CLS] sentence A [SEP] sentence B [SEP]

sampling

Rolling a weighted die to pick the next word.

The model scores every possible next token with a raw number (a “logit”). Those raw scores aren’t probabilities yet — they can be negative and don’t add up to 100% — so the model first squashes them through a step called softmax that turns the whole list into clean percentages that sum to 1. Then, instead of just grabbing the single highest one, it rolls a weighted die over those percentages: a token that ends up at 30% gets picked about 30% of the time. That randomness is why the same prompt can give different answers, and why the text feels lively instead of robotic — always taking the top choice tends to produce flat, repetitive writing, while letting lower-ranked words occasionally win adds variety and surprise. The “temperature” knob acts before the softmax, stretching or squeezing the raw scores to tilt the die toward safe-and-likely (low temperature) or wild-and-creative (high temperature).

p_i = e^(logit_i) / sum_j e^(logit_j), then sample from p

sampling rate

How many audio samples are recorded each second.

A sampling rate of 16,000 hertz means the microphone stores 16,000 measurements every second. A higher rate can capture faster vibrations, while a lower rate uses less storage and computation.

sampling rate = samples / second

scaling laws

Empirical relationships between model performance and resources such as parameters, training data, and compute.

Studies often fit power-law relationships to language-model loss on held-out text. These fits summarize measurements over particular ranges and training setups. They help estimate tradeoffs, but do not guarantee improvement on every task or justify unlimited extrapolation.

segment embedding

An extra tag added to each word saying whether it belongs to text A or text B.

Segment embeddings are learned vectors added to every token to mark which of two inputs it came from — segment A or segment B — so the encoder can keep two texts straight when they’re fed in together past a [SEP]. They’re added the same way as positional encodings from Book 01 (element-wise onto each token’s vector), just carrying ‘which chunk’ instead of ‘which spot’. So a token’s input vector ends up being its word meaning + its position + its segment.

input = word_embedding + position_embedding + segment_embedding

self-consistency

Sample several independent reasoning chains and take the answer most of them reach.

Correct reasoning tends to converge, because there are usually several valid routes to the same right answer. Errors scatter, because a mistake can go wrong in many directions. So the answer appearing most often across independent samples is the better bet. It needs no extra training, only repeated sampling, and its limit is that every chain must run all the way to an answer before anything is learned from it.

sample k chains -> collect final answers -> take the majority

self-supervised speech learning

Learn from recordings by making practice answers from the recordings themselves.

Large collections of audio often have no written transcripts. Self-supervised learning hides part of a recording or groups similar sound frames, then asks the model to recover a machine-made target. Later, a smaller set with human transcripts teaches the exact speech-recognition task.

Make practice answers from recordings without transcripts; learn sound patterns from those answers.

sentence alignment

Finding which source and target sentence spans are translations of each other.

Sentence alignment turns two translated documents into training pairs. Boundaries need not match one-to-one: a translator can split one sentence, merge two, omit a line, or reorder material. An aligner scores possible cross-language span matches and finds a consistent path through the documents; a wrong match directly teaches the model a false relationship.

source spans <-> target spans

sentence error rate (SER)

The percentage of recorded speech segments that contain at least one wrong word.

Each whole speech segment is marked correct only if every word matches. One tiny mistake and ten mistakes both count as one failed segment. This score shows how often users receive a completely correct result, while WER shows how many word-level repairs are needed.

segments with any word error, divided by all segments

sequence classification

Reading a whole piece of text and tagging it with one label.

Sequence classification is assigning a single label to an entire input — positive/negative for a review, spam/not-spam for an email, a topic for an article. With an encoder you do it by taking the [CLS] token’s summary vector and running it through a small classifier head that outputs logits over the labels, then softmax (both straight from Book 01). It’s the simplest and most common way encoders get put to work.

label = softmax(W · [CLS]-vector)

shape

A tuple saying how many elements run along each axis — the lens that turns flat memory into a grid.

The shape is a tuple like (3, 4) telling you how many elements lie along each axis; its length is the number of dimensions (ndim) and the product of its entries is the total size. It does not store the data — it is instructions for how to walk one flat block of memory. The same buffer of 12 numbers is a (12,), a (3, 4), or a (2, 2, 3) depending only on the shape you give it, and reshaping between same-size shapes copies nothing. Pair the shape with strides and you can compute exactly where any element [i, j, …] sits in the flat buffer.

size = product(shape); ndim = len(shape)

sigmoid

The S-curve that translates a score on the log-odds ruler back into a probability.

The logistic sigmoid 1/(1+e^-z) is the inverse of the log-odds map: if a score z is the log-odds of an event, then sigmoid(z) is the event’s probability. It is not a curve anyone chose for its looks — it is the only function consistent with ‘this score is a log-odds’, and its S-shape is exactly the squash of the probability ruler: huge score changes far from zero barely move p, because out there the ruler is crushed. It converts summed evidence into probability in logistic regression, score gaps into preference probabilities in Bradley-Terry reward models, and is softmax’s two-option special case.

sigmoid(z) = 1 / (1 + e^-z)

softmax

Turn a list of real scores into positive values that sum to one.

Exponentiate each score, then divide by the total. Scores 0, ln(2), ln(3) produce probabilities 1/6, 2/6, 3/6. What matters is the score difference: the ratio of two probabilities is exp(score₁ − score₂). A gap of 0.1 gives a ratio of about 1.105, not an overwhelming winner. Attention uses softmax to produce weights over allowed positions.

softmax(z)ᵢ = exp(zᵢ) / Σⱼ exp(zⱼ)

sparse autoencoder (SAE)

A model trained to reconstruct activations using a representation with relatively few active coordinates.

It can help separate recurring patterns that are mixed together in the original activations. Researchers may interpret some learned features by inspecting examples or interventions. Sparsity does not guarantee that each coordinate has one clear meaning, that all features have been found, or that the representation explains every model decision.

activations → sparse representation → reconstructed activations

sparse retrieval

Matching on word-count vectors that are almost entirely zeros.

Sparse retrieval scores documents using high-dimensional vectors of word weights — tf-idf or BM25 — with one dimension per vocabulary word. They are called sparse because any given document contains only a few hundred distinct words out of a vocabulary of hundreds of thousands, so nearly every entry is zero. That sparsity is a feature: an inverted index can skip all the zeros and only touch documents that actually share a word with the query, making search fast even over billions of documents. It contrasts with dense retrieval, which uses short, fully-populated embedding vectors (from the transformers and masked-language-models books) to match on meaning rather than exact words — modern systems often combine both.

represent q and d as sparse tf-idf / BM25 vectors, then score by overlap

spectrogram

A picture-like table showing which vibration speeds are strong at each moment.

Each short time frame is split into frequencies, meaning rates of vibration. The spectrogram places time from left to right, frequency from low to high, and strength in each cell. It turns a hard-to-read waveform into a map of changing sound energy.

time x frequency -> strength

speech alignment

A connection between each part of a recording and the text it supports.

A recording has many more time positions than a transcript has written pieces, and the transcript usually has no letter-by-letter timestamps. An alignment is one possible answer to “which sound positions support which text?” Attention uses soft weights; CTC and RNN-T consider many complete timing paths.

positions in the recording <-> positions in the transcript

speech frame

A very short slice of a recording, often about 20 to 25 milliseconds long.

Speech changes over time, but a tiny slice is short enough to examine as one local sound. Systems often use overlapping frames so a sound near one boundary is not lost. A frame is a time window, not a video picture.

recording -> many short, overlapping time windows

standard deviation (spread)

How far apart the numbers are: bunched up or spread out.

First find the mean (the balance point). Standard deviation is the typical distance each number sits from that mean: small when they huddle close, large when they wander far. It’s one number that captures “how spread out is this pile?”. Layer norm subtracts the mean and divides by the spread, so a tight vector and a scattered one come out the same overall size.

std = sqrt( average of (each value - mean)^2 )

stop words

Extremely common little words like ‘the’, ‘a’, and ‘to’ that were often stripped out before indexing.

Stop words are the highest-frequency function words in a language — ‘the’, ‘a’, ‘of’, ‘to’ — that carry almost no information about what a document is about. Traditional IR systems deleted them before indexing to save space and avoid matching on them. But this is now largely unnecessary: inverse document frequency already drives their weight to nearly zero on its own, since they appear in almost every document, so modern systems usually just keep them and let idf do the muting. Removing them can even hurt, breaking phrases like ‘to be or not to be’ or the band name ‘The Who’.

straight-through estimator

A training shortcut that lets learning signals pass across a nearest-codeword choice.

Choosing one numbered codeword is a sudden jump, so ordinary calculus cannot say how a tiny input change affects the choice. On the forward pass, the system still uses the real chosen codeword. On the backward learning pass, the straight-through estimator pretends the choice copied its input, allowing a gradient to reach the encoder.

use the hard choice forward; copy the learning signal backward

streaming speech recognition

Recognize speech while the audio is still arriving instead of waiting for the full recording.

A streaming recognizer processes each new chunk and can begin returning words before the speaker finishes. This lowers delay, but the model has less future sound available to correct an early guess.

incoming audio chunks -> growing transcript

stride

The byte offset for moving one element along an array axis.

In a contiguous row-major (3, 4) array of 8-byte numbers, the strides are (32, 8): 32 bytes per row and 8 per column. Transposes and basic slices can use different strides over the same buffer. A reshape may share memory or require a copy.

byte offset = sum(index[k] * strides[k])

stride

The number of positions a sliding filter jumps before making its next score.

Stride one moves one position at a time. Stride two jumps two positions and makes about half as many outputs. This saves work, but a very large jump can skip a brief sound.

Stride 1 checks every position; stride 2 checks every other position.

subgame perfect equilibrium

A strategy profile that is a Nash equilibrium in every subgame, including parts of the game not reached by its planned play. It rules out threats that would not be optimal to carry out when the relevant decision is reached.

subword tokenization

Splitting text into reusable pieces smaller than words but often larger than characters.

Subword tokenization builds a fixed vocabulary of common character sequences and represents every sentence as those pieces. A translator can reuse stems, affixes, names, and cross-language fragments while still spelling an unseen word from smaller units. Shared subwords are especially useful across scripts, morphologically rich languages, and language pairs with limited data.

text -> reusable character pieces -> token ids

superposition

One neuron quietly does several jobs by sharing its space.

A model has fewer neurons than ideas it wants to remember, so it folds many ideas into the same small space, each pointing in a slightly different direction. Because the directions overlap, one neuron ends up helping with several unrelated things at once. That is why peeking at a single neuron is confusing: it is never about just one thing, it is many things stacked on top of each other.

features stored >> neurons available, so directions overlap

surprise

How unexpected an outcome is: log(1/p), measured in bits.

Surprise (information content) is the unique measure satisfying three natural demands: certainty gives zero, rarer means more, and independent surprises add while their probabilities multiply — the last demand forces the log. In base 2 the unit is the bit: a fair coin flip is 1 bit, a 1-in-1024 event is 10 bits. A language model’s training loss at one position is exactly its surprise at the true next token, log(1/p_model(token)), so pretraining is literally ‘become less surprised by real text’; averaging surprise over a distribution gives entropy.

surprise(p) = log₂(1/p) bits

SWE-bench

A benchmark of real GitHub issues where the model must produce a patch that makes the tests pass.

Function-level benchmarks give a docstring and ask for one function. SWE-bench gives a real repository of thousands of files and an issue description, with no pointer to the relevant code. Early scores were around two percent, and high function-level scores did not predict them at all. What closed the gap was not larger models alone but wrapping them in an agent loop that explores, runs tests and iterates, which is the clearest evidence that the harness is part of the system.

issue + repository -> patch -> do the tests pass?

sycophancy

When a model tells you what you want to hear instead of what’s true.

Sycophancy is the tendency of a model to agree with or flatter the user — caving to pushback, echoing a stated opinion, or validating a wrong premise — rather than giving an accurate, independent answer. It is largely a side effect of preference tuning (Book 01’s RLHF): human raters tend to prefer agreeable, affirming responses, so optimizing for their approval can quietly train the model to please rather than to be correct. It is a subtle safety and reliability concern because the model still sounds confident and helpful while being less truthful.

system prompt

Hidden instructions set before the chat that tell the model how to behave.

A system prompt is a special block of instructions placed at the very start of the conversation, before the user’s messages, that sets the model’s persona, rules, and boundaries — like ‘You are a helpful assistant; be concise; refuse harmful requests.’ Mechanically it is just more conditioning text the decoder attends to, but it is given a privileged position and the model is trained (via instruction-tuning and RLHF) to weight it heavily. It is how an app developer steers the model’s overall behavior separately from whatever the end user types.

[system instructions] + [user message] -> response

T

tangent line

The line that matches a differentiable function’s value and slope at a point, giving a local linear approximation.

Draw a secant line through two points on a curve and move the second point toward the first. If the secant slopes approach a finite value, that value is the derivative. The tangent is y = f(a) + f′(a)(x − a). It matches the curve at a and approximates it nearby; it may also cross the curve or meet it elsewhere. A corner or a jump has no ordinary derivative at that point.

y = f(a) + f′(a)·(x − a)

teacher forcing

During training, show the decoder the correct earlier token before asking for the next one.

A decoder that writes an ordered list normally uses what it wrote earlier to choose what comes next. During teacher forcing, training replaces that earlier guess with the known correct token. This gives a clear learning signal at every step. During real use, the correct answer is unavailable, so the decoder must continue from its own choices.

Give the correct earlier tokens; predict the next correct token.

temperature

A positive number used to scale logits before softmax, controlling how concentrated the sampling distribution is.

T below 1 makes larger logits relatively more likely; T above 1 makes probabilities more even. Positive temperature preserves the ranking of logits. It controls sampling, not truthfulness or safety. “Temperature zero” usually means selecting a maximum-scoring token directly; the formula itself cannot divide by zero.

pᵢ = exp(zᵢ/T) / Σⱼ exp(zⱼ/T), for T > 0

tensor

A multidimensional array used for calculations; in PyTorch it also has a device and optional gradient tracking.

Shape and data type describe its values. The device identifies where they live, such as CPU or GPU memory. In ordinary gradient mode, differentiable operations involving a tensor that requires gradients build a graph for autograd. A tensor can also hold data without tracking gradients.

tensor: values + shape + dtype + device; gradient tracking is optional

term frequency (tf)

How often a word shows up in a document — but squashed so ten times isn’t ten times as important.

Term frequency counts how often a word occurs in a document, on the premise that a word used a lot is a strong signal of what the document is about. But raw counts overweight repetition — a word appearing 100 times does not make a document 100 times more relevant — so it is usually log-damped: tf = 1 + log10(count). That squashing means the jump from 1 to 2 occurrences matters more than the jump from 100 to 200, which matches how relevance actually behaves. Term frequency is one half of the tf-idf weight; on its own it cannot tell that common words like ‘the’ occur a lot everywhere, which is the job idf handles.

tf = 1 + log10(count) if count > 0, else 0

term-document matrix

A big grid of words by documents, each cell saying how much that word counts in that document.

The term-document matrix lays the whole collection out as a grid: one row per vocabulary word, one column per document, and each cell holding that word’s count or weight (tf-idf) in that document. Read down a column and you get one document’s vector; read across a row and you see how a single word is spread across the collection. This is the concrete data structure behind the vector space model — comparing a query to every document is just comparing the query vector against every column. It is mostly zeros, since any one document uses only a tiny slice of the whole vocabulary, which is exactly what makes the representation sparse.

cell[word, doc] = weight of that word in that document; each column = one document vector

test set

Data reserved for evaluating a model after its training and model choices are finished.

Training data fit the parameters; validation data help choose the model or its settings. The test set then gives a final estimate of performance on fresh data from the population it represents. If you keep changing the model in response to its test score, that set has become part of model selection. Its score is a noisy measurement, not an exact expected error.

$\widehat R_{\mathrm{test}}=\frac1M\sum_{j=1}^M(y_j-\hat f(x_j))^2$ for squared loss.

test-time compute

Spending extra computation when answering, rather than during training, to get better answers.

Test-time compute is the idea that a model can improve its answer by thinking harder at inference time instead of being trained more. The clearest example is chain-of-thought: letting the model generate intermediate reasoning before its final answer, which spends more tokens (and so more compute) per question. It is loosely grouped under post-training because it is another lever for quality on top of the pretrained base, but it acts when the model runs, not when its weights are learned.

better answer via more compute at answer-time, not more training

text normalization for ASR evaluation

Apply the same cleanup rules to both transcripts before comparing them.

Normalization decides whether capitalization, punctuation, contractions, number spelling, filler words, or spelling variants count as errors. The machine transcript and human transcript must receive the same documented rules. Otherwise formatting differences may be counted as speech-recognition mistakes.

both raw transcripts -> same cleanup rules -> comparison

text-to-speech (TTS)

Using a computer to create a spoken recording from written text.

TTS is short for text-to-speech and is also called speech synthesis. The text says what words to pronounce, but the system must still choose timing, how high or low the voice sounds, loudness, pronunciation, and voice. A modern system can predict compact audio-code numbers and send them through a codec decoder to produce a recording.

written text + voice information -> spoken recording

tf-idf

Weight a word high when it’s frequent here but rare everywhere else.

tf-idf multiplies two signals: term frequency (how much a word is used in this document) times inverse document frequency (how rare that word is across the collection). The product is highest exactly for words that are common in one document but uncommon elsewhere — which are precisely the words that distinguish what that document is about. A word frequent everywhere (high tf, near-zero idf) gets crushed; a rare word absent from this document (zero tf) contributes nothing. These weights fill the cells of the term-document matrix and power the vector space model, and BM25 is the refined, better-behaved successor to this basic formula.

w_{t,d} = tf_{t,d} * idf_t

The Pile

A curated 800GB open dataset that mixes web text with books, code, papers, and more.

The Pile (2020, by EleutherAI) is an influential open pretraining dataset built by deliberately combining 22 sources — not just web text but also books, GitHub code, arXiv papers, Wikipedia, PubMed, and so on. The idea is that a richer, more diverse mix teaches a broader model than web text alone. It’s a landmark example of curating a corpus on purpose rather than just scraping, and it helped make open LLM research possible. Newer open datasets like Dolma follow the same diverse-mixture spirit at a larger scale.

web + books + code + papers + ... -> one diverse 800GB mixture

token

One numbered piece that a model reads or writes. It can represent text or a small piece of sound.

A model works with numbered pieces instead of handling raw words or raw sound directly. A text token may be a whole word, a word part such as “ing”, or one character. An audio token is the number of a reusable sound pattern in a codec’s codebook. In both cases, a tokenizer or codec turns the original input into token numbers, and the model predicts or processes those numbers.

text or sound -> numbered pieces -> model

tool call

A piece of text the model wrote that the harness recognises and carries out.

Producing a tool call is still ordinary next-token prediction; nothing about generating it differs from generating a sentence. It becomes an action only because something outside the model agreed to treat that shape as an instruction. Modern systems have the model emit the call in a fixed structured form validated against a schema, rather than in prose that a parser has to guess at.

model writes name(arg=value) -> harness runs it -> output returns as prompt text

tool schema

The name, description and parameter definitions that are all a model ever sees of a tool.

The model cannot inspect your code, call the function experimentally, or check what it returns. It chooses between tools by reading their descriptions, so the description is the interface rather than documentation about the interface. Two tools with the same description are indistinguishable no matter how differently they behave. The parameter definitions matter too: a required field the user never mentioned is a field the model has to invent.

name + description + parameter types -> the model's entire view of a capability

top-k sampling

Keep only the few most likely next words, then randomly pick one — but favoring the likelier ones.

The model first gives every possible next word a raw score (a “logit”). A softmax turns those scores into probabilities that add up to 1. Top-k sampling then keeps only the k highest-scoring words (say k=50), throws away the rest, and rescales the survivors’ probabilities so they again add up to 1. It then picks one at random — but it’s a weighted draw, not a coin flip: a word at 60% is chosen far more often than one at 2%, so the model still usually says something sensible. It works because the long discarded tail is mostly nonsense, so cutting it keeps the output coherent while still leaving room for variety. This is the final step that turns the predicted scores into one actual chosen word: bigger k lets in more long-shot words (more surprising), smaller k stays near the top few (safer and more predictable).

keep top k tokens by probability, renormalize so they sum to 1, then sample

top-p (nucleus) sampling

Keep just enough likely words to cover p of the total probability, then pick one at random from those.

After the model assigns a probability to every possible next word, top-p sorts them high to low and keeps adding words until their cumulative probability first reaches p (say 0.9) — the smallest group that crosses that line — throwing away the long tail. It then samples the next word from only that “nucleus.” The clever part is that the set resizes itself: when the model is confident, a few top words already cover p, so choices stay safe; when it’s unsure, the set grows wider, allowing more variety. This keeps writing fluent by avoiding bizarre rare words, while still leaving room for creativity.

smallest set S where sum of P(token) for token in S >= p

training set

The examples used to fit a model’s parameters.

In least-squares regression, fitting selects coefficients to minimize the average squared error on these examples. Other learning methods use other objectives. Training error usually gives an optimistic picture of future performance because those same examples influenced the fit. Use validation data for model selection and an untouched test set for final evaluation.

transfer learning

Learn general skills once on tons of text, then reuse them for many specific jobs.

Transfer learning is the two-stage recipe behind encoders: first pretrain on huge unlabeled text (via masked language modeling) to learn language in general, then transfer that knowledge to a specific task by fine-tuning on a small labeled dataset. It’s the same pretrain-then-fine-tune split Book 01 used, and it’s powerful because the expensive, knowledge-building step is done once and shared, so each new task only needs a little data and a small added head.

pretrain on lots of text -> fine-tune on a little task data

transformer block

The repeating layer that does most of a transformer’s thinking.

A transformer block is one processing unit with two steps: attention (each word gathers info from other words) and a feed-forward network (each word is reshaped on its own). Each step adds its result back onto its input rather than replacing it (a residual add), which keeps earlier information easy to access and helps gradients flow cleanly during training; small normalizations keep the numbers stable. Stacking many blocks with the same shape (but their own learned weights) lets the model build understanding in layers, with early blocks catching simple patterns and later ones grasping meaning.

output = input + sublayer(norm(input)), repeated for attention then feed-forward

translation divergence

A place where two languages build the same meaning with different words or structure.

A translation divergence is a systematic mismatch between how two languages express a thought. The verb may move, a pronoun may disappear, one source word may require several target words, or the target grammar may force information the source left implicit. These are not exceptions a dictionary can patch one by one; they are the central reason translation must model the sentence and its meaning rather than replace words in place.

same meaning, different linguistic structure

transpose

Swap a matrix’s rows and columns. A column vector becomes a row vector. The superscript T names this operation; it is not an exponent.

tree of thoughts

Branch into several candidate next steps, score each unfinished state, prune, and backtrack.

A written chain of reasoning only moves forward, so a bad opening move is not an error it can detect. A tree generates several candidate next steps instead, scores how promising each partial state looks, drops the hopeless ones and expands the best. When a branch dies, the search returns to an earlier state and takes another. The scoring is what makes it a search rather than brute force, and the model itself supplies those scores.

state -> candidate next steps -> score each -> keep the best, backtrack on failure

U

underfitting

A model failing to capture useful structure in the relationship being learned.

For example, a straight line cannot represent a strongly curved conditional mean. It may have high error on both training and validation data. That pattern is a reason to investigate, not a diagnosis by itself: poor optimization, missing features, or high output noise can also produce high errors. More observations alone cannot make a fixed linear model represent every curve.

unembedding

A projection from a hidden-state vector to one score per vocabulary token.

Each output-weight column scores one possible token: multiply it by the hidden-state row and sum, optionally adding a bias. The resulting logits become probabilities after softmax. Every position can produce these scores; generation uses the final position to continue the full input. The output matrix may be learned separately or share the transposed input embedding table, called weight tying.

hidden row × output matrix (+ bias) → vocabulary scores

unlabeled audio

Recordings that do not come with human-written transcripts or other answers.

Unlabeled audio is easier and cheaper to collect than audio paired with exact text. Self-supervised learning creates a training target from the recording itself, allowing the model to learn sound patterns before a smaller transcribed dataset teaches the final recognition task.

recording only; no human answer attached

utterance

One bounded piece of speech used as an example, such as a sentence or a short recording.

An utterance is a practical unit of spoken language. It can be one sentence, a command, or a turn in conversation. Recognition datasets and error scores often treat each recording segment as one utterance.

one speech segment -> one reference transcript

V

validation set

Examples used to compare model choices, such as a polynomial’s degree or a regularization strength.

The model’s parameters are fitted on training data, then its validation score helps select a configuration. Because you used that score to choose, you need an untouched test set for a final assessment. Cross-validation repeats training and validation on different splits.

variance · Var

The average squared distance of a random quantity from its own mean. It measures spread in squared units.

Subtract the mean from each possible value, square the result, and take the probability-weighted average. Standard deviation is the square root of variance, so it uses the original units. Var(Y | X = x) measures variation in outputs at one fixed input; Var_D(f̂_D(x)) measures variation in fitted predictions across training datasets.

VALL-E

A text-to-speech method that makes a rough audio-code stream first and then adds detail streams.

VALL-E turns speech creation into prediction of codec token numbers. A model creates the first, most important code stream one token at a time. A second model fills the remaining correction streams using the text, voice example, and first rough stream. Finally, a codec decoder turns all streams into a recording.

text + voice example -> rough codes -> detail codes -> recording

vector

An ordered list of coordinates representing a quantity in a vector space.

For example, [2, 3] can represent two measurements, a point, or a displacement; the context tells you which. Vector addition and scalar multiplication act entry by entry. A Transformer gives each token position a vector of features and changes that vector through its layers. Learned vectors can encode useful relationships, but an individual coordinate need not have a simple meaning.

[2, 3] + [1, −1] = [3, 2]; 2[2, 3] = [4, 6]

vector quantization (VQ)

Replace a list of numbers with the nearest item on a fixed numbered shelf.

VQ is short for vector quantization. Compare the input number-list with every stored codeword, choose the closest one, and output its shelf number. Looking up that number returns the stored number-list for the decoder. This saves space but gives only a rough copy of the original.

input number-list -> nearest stored item -> shelf number

vector space model

Turn documents and queries into lists of numbers so that ‘similar’ becomes ‘close together’.

The vector space model represents each document and the query as a vector whose entries are weights for words (typically tf-idf), so a text becomes a point in a space with one dimension per vocabulary word. Relevance then turns into geometry: a document that matches the query points in nearly the same direction, so their cosine similarity (from the transformers book) is high. This is powerful because it reduces ‘does this document answer this query?’ to a single number you can compute and sort by, across millions of documents. It is the foundation under sparse retrieval, and dense retrieval keeps the same idea but swaps word-count vectors for learned embedding vectors.

score(q, d) = cosine(vector(q), vector(d))

vectorization

Pushing a loop down into compiled C over packed memory instead of looping in Python.

Vectorization means expressing a computation as whole-array operations (a + b, a.sum(), a @ b) so the loop runs inside NumPy’s compiled C code over one contiguous block of memory, rather than as a slow Python for-loop stepping over boxed objects. The speedup comes from the same flat-buffer fact: uniformly typed, packed data can be scanned in tight machine loops (often using the CPU’s SIMD units). The skill in NumPy is largely learning to phrase a problem without a Python loop — reach for ufuncs, reductions, and broadcasting instead.

for-loop in Python → one array expression → C loop over packed memory

view

A second array sharing another’s memory — edit one and the other changes.

A view is an array that points at another array’s data buffer instead of owning its own: slicing and reshaping usually hand you a view, defined by a fresh (shape, strides, offset) over the same numbers. Because nothing is copied it is fast and memory-cheap, but it has a sharp edge — writing into a view writes into the original (and vice versa). When NumPy can’t express what you asked for as a stride pattern (e.g. fancy indexing, or an explicit .copy()), it gives you a genuine copy with its own memory instead. Knowing which you have is the difference between an intended in-place edit and a surprising one.

B = A[1:, :2] → B shares A's buffer (a view, not a copy)

vision transformer (ViT)

A standard transformer encoder fed square patches of an image instead of words.

A transformer needs a sequence of vectors and nothing about it mentions language. Treating each pixel as a position is both ruinously expensive, since attention cost grows with the square of the sequence length, and uninformative, since one pixel carries no shape. Cutting the image into patches of about sixteen pixels square fixes both at once. The architecture is otherwise unchanged, so what is new is the choice of unit, not the machine.

image -> patches -> vectors + positions -> ordinary transformer encoder

visual hallucination

Confidently describing something in an image that is not actually there.

The model answers from what scenes of this kind usually contain rather than from what is in this particular picture, so it tends to report objects that commonly co-occur with what is genuinely present. The POPE benchmark tests exactly this by naming an object and asking whether it appears. For an agent this is worse than a wrong sentence: it will click where the imagined control should be, observe nothing, and then report success.

expected-for-this-scene -> reported as observed

visual token

An image patch projected into a language model’s embedding space so it sits in the same sequence as text.

Once a vision encoder has produced patch vectors, one learned matrix maps them into the space the language model already uses for words. The result is placed in the sequence alongside text tokens, and attention runs across both without the model needing to know which positions began as pixels. The cost is sequence length: a 336-pixel image at 14-pixel patches becomes 576 tokens before the user has typed anything, which is why compression schemes exist that pull a few dozen vectors out of those hundreds.

patch vectors -> projection matrix -> tokens in the text sequence

vocabulary

The fixed list of every token the model can read or write.

The vocabulary is the complete, fixed set of tokens (word-pieces like “the”, “ing”, or “transform”) that a model knows — often around 100k–200k these days, and it varies by model (e.g. ~128k in Llama 3, ~200k in GPT-4o). Every piece of text gets chopped into these tokens before the model sees it, and every token it generates must be one of them; the model can only ever pick from this list. It sits at both ends of the transformer: the input layer maps each token to a vector (the embedding), and the output layer produces one score for every token in the vocabulary, and the highest-scoring ones are the most likely next token.

vocabulary mismatch

When you search for one word but the document you need uses a different word for the same thing.

Classic keyword search matches the actual words in the query against the actual words in documents, so it silently fails whenever the two sides choose different words for one idea — a search for ‘car’ misses a page that only ever says ‘automobile’. Human language is full of this: synonyms, paraphrases, abbreviations, and spelling variants all name the same concept with different surface strings. This is the core weakness of exact-word (sparse) retrieval methods like tf-idf and BM25, and no amount of clever term weighting fully repairs it. Overcoming it is the entire motivation for dense retrieval, which matches on meaning rather than on the literal words.

voice prompt

A short example recording that shows the system which voice and speaking style to imitate.

Text tells a speech system what to say but not exactly who should say it. A voice prompt supplies evidence about the speaker’s pitch, rhythm, accent, and recording conditions. Zero-shot TTS uses this example without retraining the model for that speaker.

short example voice + new text -> new speech in a similar voice

W

waveform

A line of numbers recording how air pressure changed over time.

A microphone measures air pressure many times each second. Put those measurements in time order and you have a waveform. High and low values show the microphone moving in opposite directions; the changing shape carries the sound.

time -> measured air-pressure value

weight tying

Sharing one parameter matrix between two parts of a model, often the input embeddings and output prediction layer.

A language model can look up a token’s input vector in an embedding matrix, then reuse that matrix to score possible next tokens. For prediction, it takes a dot product between its hidden vector and each token’s embedding. This requires compatible dimensions and uses fewer parameters than learning separate input and output matrices. Training updates the shared matrix through both uses.

word_scores = hidden_vector · (embedding_table)ᵀ

weights

The learned numbers a model tunes during training to store what it knows.

Weights are the actual numbers filling the model’s matrices — millions or billions of them. Each one controls how strongly one piece of information pushes on another as data flows through; training nudges every weight up or down so the model’s guesses get less wrong, and the final settings are what the model “knows.” Inside a transformer, weights don’t directly pick which words attend to which — that pattern is computed fresh for each input. Instead, the weights build the lenses (the query/key/value projections) that turn each word into the cues used to figure out that attention pattern on the fly, and they reshape each word’s meaning at every layer. That’s why the same architecture becomes a poet or a coder purely by having different weights.

output = weights · input (each output is a weighted sum of the inputs)

word error rate (WER)

The number of word repairs divided by the number of words in the correct transcript.

First compare the machine transcript with the human reference. Count replaced words, missing words, and extra words along the best alignment. Add those counts and divide by the number of reference words. Extra words can make the result exceed 100 percent.

WER = 100 times (replaced + missing + extra), divided by reference words

word sense

One of the distinct meanings a single word can have.

A word sense is a specific meaning of a word — ‘bank’ has a river-edge sense and a money-place sense, and these are genuinely different concepts that happen to share a spelling. Plain Book-01 embeddings give a word just one vector, which has to blur all its senses together; contextual embeddings fix this by letting the surrounding words pin down which sense is active. Distinguishing senses is one of the clearest things an encoder buys you over a static lookup table.

"bank" = {river bank, money bank, ...}

word-sense disambiguation

Figuring out which meaning of a word is intended from its context.

Word-sense disambiguation (WSD) is the task of picking the correct sense of an ambiguous word given its sentence — deciding that ‘bass’ means the fish, not the instrument, in ‘I caught a bass’. Contextual embeddings make this almost natural: because attention pulls in the surrounding words, the encoder’s vector for the word already leans toward the intended sense, so a small classifier on top can read it off. It’s a direct, concrete payoff of reading both directions.

(word, sentence) -> which sense

Z

zero-shot

Handle a new case without doing extra training specifically for that case.

Zero-shot does not mean the model learned from no data. It means the model was not retrained for this new speaker or task example. In zero-shot TTS, a short voice prompt guides a general model to imitate a speaker it did not receive speaker-specific training for.

general trained model + new example -> output, with no weight update

zero-shot translation

Translating a language pair the model never saw directly during training.

Zero-shot translation becomes possible when a multilingual model has separately connected both languages to shared representations through other translation directions. A model trained on Spanish-English and Portuguese-English may attempt Spanish-Portuguese without direct Spanish-Portuguese pairs. Success depends on whether the shared space truly aligns the languages rather than merely routing everything through the dominant one.

train A<->C and B<->C; attempt A<->B

zero-shot TTS

Make new speech in an unfamiliar speaker’s voice using only a short example recording, without retraining.

A general TTS model receives new text plus a few seconds of the desired voice. The example supplies voice clues, and the model’s learned weights do not change. “Zero-shot” means no extra speaker-specific training; it does not mean the system receives no voice example.

new text + short voice example -> new speech in a similar voice

regression coefficient

A multiplier in a regression model. In ŷ = β̂₀ + β̂₁x, β̂₀ is the fitted intercept and β̂₁ is the fitted slope: the predicted change in y per one-unit increase in x. A hat marks an estimate from observations. The true coefficient β is fixed but unknown in the classical model. In multiple regression, a coefficient describes a change while the other included predictors stay fixed.

design matrix

The table used to calculate a model’s predictions. Rows represent observations; columns represent predictors. A column of ones supplies the intercept. For N observations and p predictors, the design matrix has shape N × (p + 1). Multiplying it by the coefficient vector produces N predictions.

standard error

The standard deviation of an estimator across repeated samples, often estimated from the observed data. A slope’s standard error measures how much fitted slopes vary; it is different from the noise standard deviation and from an individual residual. In simple regression with constant, uncorrelated noise, slope SD is σ / √Σ(xᵢ − x̄)². Substituting residual standard error for σ gives an estimated slope SE.

eigenvector

A nonzero vector v for which applying a square matrix M is the same as multiplying v by a scalar: Mv = λv. The scalar λ is its eigenvalue. Positive eigenvalues preserve direction, negative ones reverse it, and zero sends the vector to zero. A real symmetric matrix admits a complete orthonormal eigenvector basis.

confidence interval

A range computed from a sample using a procedure designed to contain a fixed true parameter at a specified long-run rate. With valid assumptions, a 95% procedure covers the truth in about 95% of repeated samples. The intervals vary; the parameter stays fixed. A realized frequentist interval does not assign a posterior probability to the parameter.

null hypothesis

The claim used to calculate what results a statistical test would expect. For example, H₀: β₁ = 0 proposes a zero slope. Rejecting it means the observed statistic crossed the chosen threshold under the assumed model. Failing to reject does not prove the claim true. Testing a regression slope concerns the modeled association, not causation.