Visualising invertibility and diagonalisability of matrices
Given any mathematical structure, we are often interested in classifying them. For real matrices, we can classify them using invertibility, (real) diagonalisability, Jordan normal form etc.
We will try to visually understand invertibility, diagonalisability, why JNF is ill-conditioned and how the QR decomposition works by considering the induced action of matrices on the real projective line.
Particularly, we will see why the Jordan normal form is numerically ill-conditioned and why QR decomposition always exists.
Setup
Given some matrix \(\begin{bmatrix} a & b \\ c & d \end{bmatrix}\) on \(\mathbb{R}^2\), we wish to understand how acts on the plane. Instead of considering specific points, let’s consider how the matrix acts on the line \(y=(\tan \theta) x\) for some \(\theta\) in \([0, \pi)\). As the matrix’s action is linear, we know that the image would be some other line \(y=(\tan \phi) x\) for some \(\phi\), chosen to be in \([0, \pi)\). There’s an issue if the matrix maps the line to the origin, in which case \(\phi\) is undefined. Whenever \(\phi\) is defined, we would express \(\phi\) as a function of \(\theta\), \(\phi(\theta)\).
Example: The identity matrix maps all lines to itself, as such \(\phi(\theta) = \theta\). Similarly for any \(\lambda \neq 0\), the matrix \(\begin{bmatrix} \lambda & 0 \\ 0 & \lambda \end{bmatrix}\) has \(\phi(\theta) = \theta\).
Example: The anticlockwise rotation matrix \(\begin{bmatrix} \cos \psi & -\sin \psi \\ \sin \psi & \cos \psi\end{bmatrix}\) has \(\phi(\theta) \equiv \theta + \psi \, (\text{mod} \, \pi)\)
We could visualize this further. Below, we use the red point to denote the point \((a,c)\) and the blue point to denote \((b,d)\). These two points completely determine the matrix \(\begin{bmatrix} a & b \\ c & d \end{bmatrix}\). The green curve is the plot of \(\phi\) against \(\theta\). The dotted black line is the plot of \(f(\theta) = \theta\).
Here are a few claims, the proofs of which are left to the reader. Most of them correspond to standard results about eigenvectors.
Claim: \(\phi(\theta)\) is strictly increasing if and only if the deteriminant of the matrix is positive. \(\phi(\theta)\) is strictly decreasing if and only if the deteriminant of the matrix is negative.
Claim: The matrix is diagonalizable if and only if \(\phi(\theta)\) has two or more fixed points.
Claim: If the matrix is invertible and has only one eigenvalue (e.g. a horizontal shear), then \(\phi(\theta)\) has one and only one fixed point. In fact, at the fixed point, \(\phi(\theta)\) must be tangential to the black dotted line. This suggests that we can perturb \(\phi(\theta)\) into a function that intersects with the black dotted line two or zero times (we can perturb the matrix into a diagonalisable or a non-diagonalisable matrix). This suggests that the Jordan Normal Form is sensitive to pertubations.
Claim: Matrices are upper triangular if and only if \(\phi(0)\) is \(0\) or is undefined.
The above claim motivates why we could always decompose a (invertible) matrix into a product of an (rotation) orthonormal and upper triangular matrix — Given a monotone \(\psi(\theta)\), we could always horizontally translate \(\psi\) such that \(\psi(0) = 0\). This is called the QR decomposition.
Claim: Matrices are lower triangular if and only if \(\phi(\pi/2)\) is \(\pi/2\) or is undefined.
Going Further
To make visualisation easier, we have added a yellow line that passes through the origin and intersects with the yellow point. The purple line is the image of the yellow line under the matrix. The hollowed yellow point tells you the angle \(\theta\) of the yellow line.
We could also extend these ideas to understanding linear dynamical systems.