NumPy, from first principles

A NumPy array combines a data buffer with its element type, shape, and strides. Shape describes the dimensions; strides locate successive elements along each axis. We’ll use six numbers to see how these pieces work together.

A line of numbers, not a list

Why a flat, packed line at all? Because the obvious alternative — a Python list — doesn’t keep your numbers together. It keeps pointers to number-objects scattered across memory, each separately boxed with its own type tag. To add them up, the computer chases a pointer to a random address for every single value.

An array throws that away: same type, packed end to end, so element i is one fixed step away. Reading is a straight walk.

The same six numbers, two layouts. Press “sum” — the list takes one random pointer-jump per value (its numbers live scattered elsewhere); the array makes zero, sweeping one packed block. Same answer, very different cost.

Try it

hit sum the values and watch the two “random jumps” counters — 6 versus 0. That gap is where NumPy’s speed comes from.

That packed line is the thing every idea below is built on.

Dimensionality is just how you walk the line

A “2-D array” isn’t laid out as a square in memory — it’s the same flat line, read in rows of a chosen width. Change the shape and the same twelve numbers become a row, a 3×4 grid, or a 2×2×3 cube. Nothing moves.

In this example, each reshape shares the same 12 values. Hover a cell to find its location in the original buffer.

Try it

hover the cells. Every grid cell lights up the same spot in the flat line — proof the grid is a lens, not a copy.

The number of dimensions is simply how many numbers are in the shape tuple. Strides are the other half of the instruction: how far to step in the flat line to move one along each axis. Together, shape and strides describe how to locate an element.

Where does element [i, j] actually live?

For a contiguous row-major (3, 4) array, moving down one row advances four elements; moving across a column advances one. Element [i, j] sits at flat position i*4 + j. NumPy stores strides in bytes. A transpose or basic slice shares the buffer; a reshape can share it when the layout permits, but may need a copy. See NumPy’s guide to copies and views.

Axes: name one, lose one

An axis is a direction to walk the array. In a reduction such as sum, mean, or argmax, the named axis is removed from the output shape by default. With keepdims=True, it remains as a dimension of length 1.

Same grid of scores. axis=-1 scans across each row (one answer per review); axis=0 scans down each column (one per class); no axis flattens everything to a single number. Watch the output shape lose the axis you named.

Try it

flip between the three modes. The score grid never changes — only which direction you collapse, and so what the answer means.

This is why axis=-1 turns up everywhere in deep learning: a batch of rows, collapse the last axis, get one answer per row. It’s the same argmax-over-the-final-axis you meet at the end of a transformer’s softmax.

Broadcasting: stretch, don’t copy

Two arrays with different shapes still combine — NumPy lines their shapes up from the right and stretches any length-1 axis to fit. The trick: it doesn’t copy the stretched values, it sets that axis’s stride to zero, so the same number in memory is simply read again and again.

A (3,1) column plus a (1,3) row fills a (3,3) grid. The gray copies were never written to memory — each is the same source value, re-read through a stride of 0.

Try it

hover a result cell — its two sources light up. A (3,1) column and a (1,3) row become a full (3,3) grid, without first copying the inputs into full grids.

That’s all broadcasting is: stretching is just re-reading.

Views: a slice shares the memory

Slice an array and you don’t get new numbers — you get a view: a fresh (offset, shape, strides) pointing at the same line. So writing into the slice writes into the original. Ask for .copy() and you get a genuinely new line instead.

B = A[2:6] is a window onto A — the amber cells. Write B[0] = 99 and A[2] changes too; they are one cell with two labels. Switch to .copy() and B becomes a separate line.

Try it

with B = A[2:6], press B[0] = 99 — A[2] changes too, because they’re the same cell. Flip to .copy() and the link is cut.

Vectorize: send the loop to C

Now the payoff. Because the line is packed and single-typed, you almost never write a Python loop over it. You write the whole-array expression a + b, and the loop runs inside NumPy, in compiled C, straight across that packed memory.

Same sum c = a + b, two ways. The Python loop pays interpreter overhead on every element; the array op runs the loop once, in C, over the packed buffer. The bar counts Python-level operations, not milliseconds.

Try it

drag N. The Python loop’s step count climbs with the array; the vectorized version stays a single call — its loop ran in C, over the same flat line from the very first section.

Reading an array operation

For an unfamiliar operation, check the input shapes, the axis it acts on, and the output shape. If it produces another array, check whether that array shares memory with the original. These questions explain the behavior of the examples above.