Seven operations form the entire vocabulary of matrix arithmetic. Master these and everything else in linear algebra is built from them.
1. Addition and subtraction
Two matrices of the same dimensions are added by adding corresponding entries. Subtraction works the same way.
$$\begin{bmatrix} 1 & 3 \\ 2 & 0 \end{bmatrix} + \begin{bmatrix} 4 & -1 \\ 5 & 2 \end{bmatrix} = \begin{bmatrix} 5 & 2 \\ 7 & 2 \end{bmatrix}$$
Addition is commutative (\(A + B = B + A\)) and associative. Nothing surprising here — the surprises start with multiplication.
2. Scalar multiplication
Multiplying a matrix by a single number (a scalar) multiplies every entry by that number:
$$3 \begin{bmatrix} 2 & -1 \\ 0 & 4 \end{bmatrix} = \begin{bmatrix} 6 & -3 \\ 0 & 12 \end{bmatrix}$$
3. Matrix multiplication
The most important operation in this entire site. The entry in row \(i\), column \(j\) of the product \(AB\) is the dot product of row \(i\) of \(A\) with column \(j\) of \(B\):
$$(AB)_{ij} = \sum_{k} a_{ik}\, b_{kj}$$
Two rules govern everything:
Shape rule. \(A\) must have as many columns as \(B\) has rows. An \(m \times n\) matrix times an \(n \times p\) matrix gives an \(m \times p\) matrix. The inner dimensions must match; the outer dimensions survive.
Order matters. In general \(AB \neq BA\). Matrix multiplication is not commutative — applying a rotation then a stretch is not the same as stretching then rotating.
$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = \begin{bmatrix} 1{\cdot}5 + 2{\cdot}7 & 1{\cdot}6 + 2{\cdot}8 \\ 3{\cdot}5 + 4{\cdot}7 & 3{\cdot}6 + 4{\cdot}8 \end{bmatrix} = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}$$
4. Transpose
The transpose \(A^T\) flips a matrix over its diagonal: rows become columns. An \(m \times n\) matrix becomes \(n \times m\).
$$A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \qquad A^T = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}$$
Key identity: \((AB)^T = B^T A^T\) — transposing a product reverses the order. The transpose shows up constantly in AI: the attention mechanism in every transformer model computes \(QK^T\), a query matrix times a transposed key matrix.
5. The identity matrix
The identity matrix \(I\) has ones on the diagonal and zeros everywhere else. It is the matrix equivalent of the number 1: multiplying by it changes nothing, so \(AI = IA = A\).
$$I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$$
6. The inverse
The inverse \(A^{-1}\) is the matrix that undoes \(A\): \(A^{-1}A = AA^{-1} = I\). Only square matrices can have inverses, and not all of them do. For a 2×2 matrix there is a closed formula:
$$A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \quad\Rightarrow\quad A^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$$
The quantity \(ad - bc\) in the denominator is the determinant. If it is zero, the inverse does not exist — the matrix has collapsed space into a lower dimension and the collapse cannot be undone.
7. The determinant
The determinant \(\det(A)\) is a single number that measures how much a matrix scales area (in 2D) or volume (in 3D). For 2×2: \(\det(A) = ad - bc\).
| Determinant | Geometric meaning |
|---|
| det = 2 | The transformation doubles every area |
| det = 1 | Areas are preserved (e.g. pure rotation) |
| det = 0 | Space is flattened — the matrix is singular, no inverse |
| det < 0 | Space is flipped (orientation reversed) then scaled |
Operation reference
| Operation | Notation | Requirement | Result shape |
|---|
| Addition | A + B | Same shape | Same shape |
| Scalar multiple | kA | Any matrix | Same shape |
| Multiplication | AB | cols(A) = rows(B) | rows(A) × cols(B) |
| Transpose | Aᵀ | Any matrix | n × m from m × n |
| Inverse | A⁻¹ | Square, det ≠ 0 | Same shape |
| Determinant | det(A) | Square | A single number |