What is a matrix?
A matrix is a rectangular grid of numbers arranged in rows and columns. A matrix with \(m\) rows and \(n\) columns is called an \(m \times n\) matrix:
$$A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \end{bmatrix} \quad \text{is a } 2 \times 3 \text{ matrix}$$
That definition sounds bureaucratic until you see what matrices do. A matrix is a machine for transforming space. Multiply a vector by a matrix and the vector rotates, stretches, shears, or projects. The animation above is a single 2×2 matrix continuously transforming a grid — that is what every layer of a neural network does to its input, millions of dimensions at a time.