The Characteristic Equation

Let \(A\) be an \(n \times n\) matrix. In the previous section we learned that the eigenvectors and eigenvalues of \(A\) are vectors \(\vec {x}\) and scalars \(\lambda \) that satisfy the equation

\begin{align}\label {def:eigen} A \vec {x} = \lambda \vec {x}\end{align}

We listed a few reasons why we are interested in finding eigenvalues and eigenvectors, but we did not give any process for finding them. In this section we will focus on a process which can be used for small matrices. For larger matrices, the best methods we have are iterative methods, and we will explore some of these in The Power Method and the Dominant Eigenvalue.

For an \(n \times n\) matrix, we will see that the eigenvalues are the roots of a polynomial called the characteristic polynomial. So finding eigenvalues is equivalent to solving a polynomial equation of degree \(n\). Finding the corresponding eigenvectors turns out to be a matter of computing the null space of a matrix, as the following exploration demonstrates.

If a vector \(\vec {x}\) is an eigenvector satisfying Equation (def:eigen), then clearly it also satisfies \(A\vec {x}-\lambda \vec {x} =\) 0 \(\vec {0}\).

It seems natural at this point to try to factor. We would love to “factor out” \(\vec {x}\). Here is the procedure:

\begin{align*} A\vec {x}-\lambda \vec {x} &= \vec {0} \\ A\vec {x}-\lambda I\vec {x} &= \vec {0} \\ (A-\lambda I)\vec {x} &= \vec {0} \end{align*}

This shows that any eigenvector \(\vec {x}\) of \(A\) is in the row spacecolumn spacenull space of the related matrix, \(A-\lambda I\).

Since eigenvectors are non-zero vectors, this means that \(A\) will have eigenvectors if and only if the null space of \(A-\lambda I\) is nontrivial. The only way that \(\mbox {null}(A-\lambda I)\) can be nontrivial is if \(\mbox {rank}(A-\lambda I)\) \(=\)\(<\)\(>\) \(n\).

If the rank of an \(n \times n\) matrix is less than \(n\), then the matrix is singular. Since \((A-\lambda I)\) must be singular for any eigenvalue \(\lambda \), we see that \(\lambda \) is an eigenvalue of \(A\) if and only if

\begin{equation}\label {eqn:chareqn} \mbox {det}(A-\lambda I) = \answer {0} \end{equation}

In theory, Exploration exp:slowdown offers us a way to find eigenvalues. To find the eigenvalues of \(A\), one can solve Equation (eqn:chareqn) for \(\lambda \).

Eigenvalues

It is not always possible to completely factor the characteristic polynomial using only real numbers. However, a fundamental fact from algebra is that every degree \(n\) polynomial has \(n\) roots (counting multiplicity) provided that we allow complex numbers. This is why sometimes eigenvalues and their corresponding eigenvectors involve complex numbers. The next example illustrates this point.

Let \(T=\begin{bmatrix} 1 & 2 & 3\\ 0 & 5 & 6\\ 0 & 0 & 9\end{bmatrix}\). Compute the eigenvalues of this matrix. (List your answers in an increasing order.)
\[\lambda _1=\answer {1},\quad \lambda _2=\answer {5},\quad \text {and}\quad \lambda _3=\answer {9}\]

What do you observe about the eigenvalues?

The eigenvalues are the diagonal entries of the matrix.

What property of the matrix makes this “coincidence" possible?

\(T\) is a triangular matrix.

The matrix in Exploration Problem is a triangular matrix, and the property you observed holds in general.

See Practice Problem .

One final note about eigenvalues. We began this section with the sentence, "In theory, then, to find the eigenvalues of \(A\), one can solve Equation (1) for \(\lambda \)." In general, one does not attempt to compute eigenvalues by solving the characteristic equation of a matrix, as there is no simple way to solve this polynomial equation for \(n>4\). Instead, one can often approximate the eigenvalues using iterative methods. We will explore some of these techniques in The Power Method and the Dominant Eigenvalue.

Eigenvectors

Once we have computed an eigenvalue \(\lambda \) of an \(n \times n\) matrix \(A\), the next step is to compute the associated eigenvectors. In other words, we seek vectors \(\vec {x}\) such that \(A\vec {x}=\lambda \vec {x}\), or equivalently,

\begin{align}\label {eqn:nullspace} (A-\lambda I) \vec {x}=\vec {0} \end{align}

As we have seen in the previous section, a (non-zero) scalar multiple of an eigenvector is an eigenvector. So for any given eigenvalue \(\lambda \) there are infinitely many eigenvectors associated with it. In fact, the eigenvectors associated with \(\lambda \), together with the zero vector, form a subspace of \(\RR ^n\).

See Practice Problems and ??.

This motivates the following definition.

Given an eigenvalue \(\lambda \), there is an associated eigenspace \(\mathcal {S}_{\lambda }\). Our goal is to find a basis of \(\mathcal {S}_{\lambda }\). Then any eigenvector \(\vec {x}\) associated with \(\lambda \) will be a linear combination of the vectors in that basis. In seeking a basis for \(\mathcal {S}_{\lambda }\), we are trying to find a basis for the set of vectors that satisfy \((A-\lambda I) \vec {x}=\vec {0}\), which means we seek a basis for \(\mbox {null}(A-\lambda I)\).

Let’s return to the examples we did in the first part of this section.

We conclude this section by establishing the significance of a matrix having an eigenvalue of zero.

A square matrix \(A\) is singular if and only if \(\det {A}=0\). But \(\det {A}=0\) if and only if \(\det {A-0I}=0\), which is true if and only if zero is an eigenvalue of \(A\).

Practice Problems

Compute the eigenvalues of \(A\), and find the corresponding eigenspaces.
\[A=\begin{bmatrix}4&1\\8&-3\end{bmatrix}\]
Answer: (List the eigenvalues in an increasing order.)
\[\lambda _1=\answer {-4},\quad \lambda _2=\answer {5}\]

A basis for \(\mathcal {S}_{\lambda _1}\) is \(\left \{\begin{bmatrix}\answer {-1/8}\\1\end{bmatrix}\right \}\). A basis for \(\mathcal {S}_{\lambda _2}\) is \(\left \{\begin{bmatrix}\answer {1}\\1\end{bmatrix}\right \}\).

Compute the eigenvalues of \(A\), and find the corresponding eigenspaces.
\[A=\begin{bmatrix}1&-2\\2&1\end{bmatrix}\]
Answer:
\[\lambda _1=\answer {1}+\answer {2}i,\quad \lambda _2=\answer {1}-\answer {2}i\]

A basis for \(\mathcal {S}_{\lambda _1}\) is \(\left \{\begin{bmatrix}i\\\answer {1}\end{bmatrix}\right \}\). A basis for \(\mathcal {S}_{\lambda _2}\) is \(\left \{\begin{bmatrix}-i\\\answer {1}\end{bmatrix}\right \}\).

Let \(T=\begin{bmatrix} 1 & 2 & 3\\ 0 & 5 & 6\\ 0 & 0 & 9\end{bmatrix}\). Compute a basis for each of the eigenspaces of this matrix, \(\mathcal {S}_1\), \(\mathcal {S}_5\), and \(\mathcal {S}_9\).
Let
\[A=\begin{bmatrix} 9 & 2 & 8\\ 2 & -6 & -2\\ -8 & 2 & -5\end{bmatrix}\]
Compute the eigenvalues of \(A\).
One of the eigenvalues of \(A\) is -3.

Answer:

(List your answers in an increasing order.)

\[\lambda _1 = \answer {-3},\quad \lambda _2 = \answer {-1},\quad \lambda _3 = \answer {2}\]

Compute a basis for each of the eigenspaces of \(A\), \(\mathcal {S}_{\lambda _1}\), \(\mathcal {S}_{\lambda _2}\), and \(\mathcal {S}_{\lambda _3}\).

Answer: A basis for \(\mathcal {S}_{\lambda _1}\) is \(\left \{\begin{bmatrix}1\\\answer {2}\\\answer {-2}\end{bmatrix}\right \}\), a basis for \(\mathcal {S}_{\lambda _2}\) is \(\left \{\begin{bmatrix}-2\\\answer {-2}\\\answer {3}\end{bmatrix}\right \}\),

and a basis for \(\mathcal {S}_{\lambda _3}\) is \(\left \{\begin{bmatrix}2\\\answer {1}\\\answer {-2}\end{bmatrix}\right \}\).

Complete Example 28 by showing that a basis for \(\mathcal {S}_{1-i}\) is given by \(\left \{\begin{bmatrix}0\\i\\1\end{bmatrix}\right \}\), where \(\mathcal {S}_{1-i}\) is the eigenspace associated with the eigenvalue \(\lambda =1-i\) of the matrix \(D=\begin{bmatrix} 0&0&0\\ 0 &1&1\\ 0 & -1&1\end{bmatrix}\).
Prove Theorem 14.
What do we know about the determinant of a triangular matrix?
Recall that a vertical stretch/compression of the plane is a linear transformation whose standard matrix is
\[M_v=\begin{bmatrix}1&0\\0&k\end{bmatrix}\]
Find the eigenvalues of \(M_v\). Find a basis for the eigenspace corresponding to each eigenvalue.

Answer: A basis for \(\mathcal {S}_1\) is \(\left \{\begin{bmatrix}1\\\answer {0}\end{bmatrix}\right \}\) and a basis for \(\mathcal {S}_k\) is \(\left \{\begin{bmatrix}\answer {0}\\1\end{bmatrix}\right \}\)

Sketch several vectors in each eigenspace and use geometry to explain why the eigenvectors you sketched make sense.

Recall that a horizontal shear of the plane is a linear transformation whose standard matrix is
\[M_{hs}=\begin{bmatrix}1&k\\0&1\end{bmatrix}\]
Find the eigenvalue of \(M_{hs}\).

Answer: \(\lambda =\answer {1}\)

Find a basis for the eigenspace corresponding to \(\lambda \).

Answer: A basis for \(\mathcal {S}\) is \(\left \{\begin{bmatrix}1\\\answer {0}\end{bmatrix}\right \}\)

What is the algebraic multiplicity of \(\lambda \)? \(\answer {2}\)

What is the geometric multiplicity of \(\lambda \)? \(\answer {1}\)

Recall that a counterclockwise rotation of the plane through angle \(\theta \) is a linear transformation whose standard matrix is
\[M_{\theta }=\begin{bmatrix}\cos \theta& -\sin \theta \\\sin \theta& \cos \theta \end{bmatrix}\]
Verify that the eigenvalues of \(M_{\theta }\) are
\[\lambda =\cos \theta \pm \sqrt {\cos ^2\theta -1}\]
Use algebra, then use geometry to explain why \(\lambda \) is a real number if and only if \(\theta \) is a multiple of \(\pi \).

Suppose \(\theta =\pi \). Then \(\lambda =\answer {-1}\) has algebraic multiplicity \(\answer {2}\). Which of the following describes the corresponding eigenspace?

All vectors in \(\RR ^2\). All vectors along the \(x\)-axis. All vectors along the \(y\)-axis. All vectors along the line \(y=x\).
Recall that a reflection of the plane about the line \(y=mx\) is a linear transformation whose standard matrix is
\[M_{y=mx}=\frac {1}{1+m^2}\begin{bmatrix} 1-m^2 & 2m \\ 2m & m^2-1 \end{bmatrix}\]
Verify that the eigenvalues of \(M_{y=mx}\) are \(1\) and \(-1\).

Find a basis for eigenspaces \(\mathcal {S}_{1}\) and \(\mathcal {S}_{-1}\). (For simplicity, assume that \(m\neq 0\).)

Answer: A basis for \(\mathcal {S}_{1}\) is \(\left \{\begin{bmatrix}1\\\answer {m}\end{bmatrix}\right \}\) and a basis for \(\mathcal {S}_{-1}\) is \(\left \{\begin{bmatrix}m\\\answer {-1}\end{bmatrix}\right \}\)

Choose the best description of \(\mathcal {S}_{1}\).

All vectors in \(\RR ^2\). All vectors with “slope" \(m\). All vectors with “slope" \(1/m\). All vectors with “slope" \(-1/m\).

Choose the best description of \(\mathcal {S}_{-1}\).

All vectors along the line \(y=mx\). All vectors parallel to the \(x\)-axis. All vectors parallel to the \(y\)-axis. All vectors perpendicular to the line \(y=mx\).

Use geometry to explain why the eigenspaces you found make sense.

Prove that the eigenvectors associated with an eigenvalue \(\lambda \) of an \(n \times n\) matrix \(A\), together with the zero vector, form a subspace of \(\RR ^n\).

Exercise Source

Practice Problem is adopted from Problem 7.1.11 of Ken Kuttler’s A First Course in Linear Algebra. (CC-BY)

Ken Kuttler, A First Course in Linear Algebra, Lyryx 2017, Open Edition, p. 361.