Je bent je ingevulde velden bij deze pagina aan het verwijderen. Ben je zeker dat je dit wilt doen?
You are erasing your filled-in fields on this page. Are you sure that is what you want?
Nieuwe Versie BeschikbaarNew Version Available
Er is een update van deze pagina. Als je update naar de meest recente versie, verlies je mogelijk je huidige antwoorden voor deze pagina. Hoe wil je verdergaan ?
There is an updated version of this page. If you update to the most recent version, then your current progress on this page will be erased. Regardless, your record of completion will remain. How would you like to proceed?
Recall that an \(n \times n\) matrix \(A\) is diagonalizable if and only if it has \(n\) linearly independent eigenvectors. (see Diagonalizable Matrices
and Multiplicity) Moreover, the matrix \(P\) with these eigenvectors as columns is a diagonalizing matrix for \(A\), that is
\begin{equation*} P^{-1}AP \mbox { is diagonal.} \end{equation*}
As we have
seen, the nice bases of \(\RR ^n\) are the orthogonal ones, so a natural question is: which \(n \times n\) matrices have \(n\) orthogonal eigenvectors, so
that columns of \(P\) form an orthogonal basis for \(\RR ^n\)? These turn out to be precisely the symmetric matrices (matrices for which \(A=A^T\)),
and this is the main result of this section.
Orthogonal Matrices
A collection of non-zero, pairwise orthogonal vectors in \(\RR ^n\) is called an orthogonal set of vectors. An orthogonal set of vectors is
called orthonormal if \(\norm {\vec {q}} = 1\) for each vector \(\vec {q}\) in the set. A set of orthogonal vectors \(\{\vec {v}_{1}, \vec {v}_{2}, \dots , \vec {v}_{k}\}\) can be “normalized”, i.e. converted into an
orthonormal set \(\left \{ \frac {1}{\norm {\vec {v}_{1}}}\vec {v}_{1}, \frac {1}{\norm {\vec {v}_{2}}}\vec {v}_{2}, \dots , \frac {1}{\norm {\vec {v}_{k}}}\vec {v}_{k} \right \}\). In particular, if a matrix \(A\) has \(n\) orthogonal eigenvectors, they can (by normalizing) be taken to be orthonormal.
The corresponding diagonalizing matrix (we will use \(Q\) instead of \(P\)) has orthonormal columns, and such matrices are very easy
to invert.
The following conditions are equivalent for an \(n \times n\) matrix \(Q\).
1.
\(Q\) is invertible and \(Q^{-1} = Q^{T}\).
2.
The rows of \(Q\) are orthonormal.
3.
The columns of \(Q\) are orthonormal.
First note that condition 1 is equivalent to \(Q^{T}Q = I\). Let \(\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\) denote the columns of \(Q\). Then \(\vec {q}_{i}^{T}\) is the \(i\)th row of \(Q^{T}\), so the \((i, j)\)-entry of \(Q^{T}Q\) is \(\vec {q}_{i} \dotp \vec {q}_{j}\). Thus \(Q^{T}Q = I\)
means that \(\vec {q}_{i} \dotp \vec {q}_{j} = 0\) if \(i \neq j\) and \(\vec {q}_{i} \dotp \vec {q}_{j} = 1\) if \(i = j\). Hence condition 1 is equivalent to 3. The proof of the equivalence of 1 and 2 is similar.
Orthogonal Matrices An \(n \times n\) matrix \(Q\) is called an orthogonal matrix if it satisfies one (and hence all) of the conditions in
Theorem 1.
The rotation matrix \(\begin{bmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{bmatrix}\) is orthogonal for any angle \(\theta \).
See Practice Problem ??.
In view of 2 and 3 of Theorem 1, orthonormal matrix might be a better name. But orthogonal matrix is standard.
It is not enough that the rows of a matrix \(A\) are merely orthogonal for \(A\) to be an orthogonal matrix. Here is an
example.
Let \(A=\begin{bmatrix} 2&1&1\\-1&1&1\\0&-1&1 \end{bmatrix}\)
1.
Check that matrix \(A\) has rows that are orthogonal.
2.
Check that matrix \(A\) has columns that are NOT orthogonal.
3.
Check that matrix \(A\) has rows that are NOT orthonormal.
4.
Create a matrix \(Q\) by normalizing each of the rows of \(A\).
5.
Check that \(Q\) is an orthogonal matrix.
Click the arrow to see the answer.
You should get \(Q = \begin{bmatrix} \frac {2}{\sqrt {6}} & \frac {1}{\sqrt {6}} & \frac {1}{\sqrt {6}} \\ \frac {-1}{\sqrt {3}} & \frac {1}{\sqrt {3}} & \frac {1}{\sqrt {3}} \\ 0 & \frac {-1}{\sqrt {2}} & \frac {1}{\sqrt {2}} \end{bmatrix}\), and one can check that this is orthogonal in a number of ways.
This exploration can certainly be done by hand (although it takes some time), but it also makes for a very nice Octave
exercise.
%Exploration from Section 9.4 Orthogonal Matrices and Symmetric Matrices
A=[2 1 1; -1 1 1; 0 -1 1]
%Check that matrix A has rows that are orthogonal.
A(1,:)*transpose(A(2,:))
A(2,:)*transpose(A(3,:))
A(1,:)*transpose(A(3,:))
%Check that matrix A has columns that are NOT orthogonal.
transpose(A(:,1))*A(:,2)
%(This is 1 of 3 calculations to do.)
%Check that matrix A in the Octave window has rows that are NOT orthonormal.
%(See the results from the first question.)
%Create a matrix Q by normalizing each of the rows of A.
q1=A(1,:)/norm(A(1,:));
q2=A(2,:)/norm(A(2,:));
q3=A(3,:)/norm(A(3,:));
Q = [q1;q2;q3]
%Check that Q is an orthogonal matrix.
Q*transpose(Q)
%(You may get numbers close to zero in some places you expect
%to get zero due to rounding error)
We studied the idea of closure when we studied Subspaces of \(\RR ^n\). The next theorem tells us that orthogonal matrices are closed
under matrix multiplication.
1.
If \(P\) and \(Q\) are orthogonal matrices, then \(PQ\) is also orthogonal. (We say that the set of orthogonal matices is closed
under matrix multiplication.)
2.
If \(P\) is an orthogonal matrix, then so is \(P^{-1} = P^{T}\).
For 1, \(P\) and \(Q\) are invertible, so \(PQ\) is also invertible and
\begin{equation*} (P^{-1})^{-1} = P = (P^{T})^{T} = (P^{-1})^{T} \end{equation*}
shows that \(P^{-1}\) is orthogonal.
Symmetric Matrices
We now shift our focus from orthogonal matrices to another important class of \(n \times n\) matrices called symmetric matrices. A
symmetric matrix is a matrix which is equal to its transpose. We saw a few examples of such matrices in Transpose of a
Matrix.
When we began our study of eigenvalues and eigenvectors, we saw numerous examples of matrices with entries that were
real numbers with eigenvalues that were complex numbers. It can be shown that symmetric matrices only have real
eigenvalues. We also learned that some matrices are diagonalizable while other matrices are not. It turns
out that every symmetric matrix is diagonalizable. In fact, we can say more, but first we need the following
definition.
An \(n \times n\) matrix \(A\) is said to be orthogonally diagonalizable if an orthogonal matrix \(Q\) can be found such that \(Q^{-1}AQ = Q^{T}AQ\) is diagonal.
We have learned earlier that when we diagonalize a matrix \(A\), we write \(P^{-1}AP=D\) for some matrix \(P\) where \(D\) is diagonal, and the diagonal
entries are the eigenvalues of \(A\). We have also learned that the columns of the matrix \(P\) are the corresponding eigenvectors of \(A\).
So when a matrix is orthogonally diagonalizable, we are able to accomplish the diagonalization using a matrix \(Q\) consisting of \(n\)
eigenvectors that form an orthonormal basis for \(\RR ^n\). The following remarkable theorem shows that the matrices that have this
property are precisely the symmetric matrices.
Real Spectral Theorem Let \(A\) be an \(n \times n\) matrix. Then \(A\) is symmetric if and only if \(A\) is orthogonally diagonalizable.
If \(A\) is orthogonally diagonalizable, then it is an easy exercise to prove that it is symmetric. You are asked to do this in Practice
Problem 27.
To prove the “only if” part of this theorem, we assume \(A\) is symmetric, and we need to show it is orthogonally diagonalizable.
We proceed by induction on \(n\), the size of the symmetric matrix. If \(n = 1\), \(A\) is already diagonal. If \(n > 1\), assume that we know the “only if”
statement holds for \((n - 1) \times (n - 1)\) symmetric matrices. Let \(\lambda _{1}\) be an eigenvalue of \(A\), and let \(A\vec {x}_{1} = \lambda _{1}\vec {x}_{1}\), where \(\norm {\vec {x}_{1}} = 1\). Next, set \(\vec {q}_{1}=\vec {x}_{1}\), and use the
Gram-Schmidt algorithm to find an orthonormal basis \(\{\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\}\) for \(\RR ^n\). Let \(Q_{1} = \begin{bmatrix} | & | & & | \\ \vec {q}_1 & \vec {q}_2 & \cdots & \vec {q}_n \\ | & | & & | \end{bmatrix}\), so that \(Q_{1}\) is an orthogonal matrix. We have
where the block \(B\) has dimensions \(1 \times (n-1)\), and the block under \(\lambda _1\) is a \((n-1) \times 1\) zero matrix, because of the orthogonality of the basis
vectors.
Next, using the fact that \(A\) is symmetric, we notice that
so \(Q_{1}^TAQ_{1}\) is symmetric. It follows that \(B\) is also a zero matrix and that \(A_{1}\) is symmetric. Since \(A_{1}\) is an \((n - 1) \times (n - 1)\) symmetric matrix, we may apply the
inductive hypothesis, so there exists an \((n - 1) \times (n - 1)\) orthogonal matrix \(Q\) such that \(Q^{T}A_{1}Q = D_{1}\) is diagonal. We observe that \(Q_{2} = \begin{bmatrix} 1 & 0\\ 0 & Q \end{bmatrix}\) is orthogonal, and we
compute:
is diagonal. Because \(Q_{1}Q_{2}\) is orthogonal by Theorem 71, this completes the proof.
Because the eigenvalues of a real symmetric matrix are real, Theorem 10 is called the Real Spectral Theorem, and the set of
distinct eigenvalues is called the spectrum of the matrix. A similar result holds for matrices with complex entries (Theorem
??
).
Find an orthogonal matrix \(Q\) such that \(Q^{-1}AQ\) is diagonal, where \(A = \begin{bmatrix} 1 & 0 & -1 \\ 0 & 1 & 2 \\ -1 & 2 & 5 \end{bmatrix}\).
respectively.
Moreover, by what at first appears to be remarkably good luck, these eigenvectors are orthogonal. We have \(\norm {\vec {x}_{1}}^{2} = 6\), \(\norm {\vec {x}_{2}}^{2} = 5\), and \(\norm {\vec {x}_{3}}^{2} = 30\), so
Actually, the fact that the eigenvectors in Example 12 are orthogonal is no coincidence. These vectors certainly must be
linearly independent (they correspond to distinct eigenvalues). We will see that the fact that the matrix is symmetric
implies that the eigenvectors are orthogonal. To prove this we need the following useful fact about symmetric
matrices.
Hence \((\lambda - \mu )(\vec {x} \dotp \vec {y}) = 0\), and so \(\vec {x} \dotp \vec {y} = 0\) because \(\lambda \neq \mu \).
Now the procedure for diagonalizing a symmetric \(n \times n\) matrix is clear. Find the distinct eigenvalues and find orthonormal bases for
each eigenspace (the Gram-Schmidt algorithm may be needed when there is a repeated eigenvalue). Then the set of all
these basis vectors is orthonormal (by Theorem 17) and contains \(n\) vectors. Here is an example.
Hence the distinct eigenvalues are \(0\) and \(9\) are of algebraic multiplicity \(1\) and \(2\), respectively. The
geometric multiplicities must be the same, for \(A\) is diagonalizable, being symmetric. It follows that \(\mbox {dim}(\mathcal {S}_0) = 1\) and \(\mbox {dim}(\mathcal {S}_9) = 2\). Gaussian elimination
gives
The eigenvectors in \(\mathcal {S}_{9}\) are both orthogonal to \(\vec {x}_{1}\) as Theorem 17 guarantees, but not to each other. However, an orthogonal
basis can be found using projections. (See Gram-Schmidt Orthogonalization)
is
an orthogonal matrix such that \(Q^{-1}AQ\) is diagonal.
It is worth noting that other, more convenient, diagonalizing matrices \(Q\) exist. For example, \(\vec {y}_{2} = \begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix}\) and \(\vec {y}_{3} = \begin{bmatrix} -2 \\ 2 \\ 1 \end{bmatrix}\) lie in \(\mathcal {S}_{9}(A)\) and they are orthogonal.
Moreover, they both have norm \(3\) (as does \(\vec {x}_{1}\)), so
is a nicer orthogonal matrix with the property that \(\hat {Q}^{-1}A\hat {Q}\) is diagonal.
Let \(A\) be an \(n \times n\) matrix. \(A\) has an orthonormal set of \(n\) eigenvectors if and only if \(A\) is orthogonally diagonalizable.
Let \(\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\) be orthonormal eigenvectors of \(A\) with corresponding eigenvalues \(\lambda _1, \lambda _2, \ldots , \lambda _n\) . We must show \(A\) is orthogonally diagonalizable. Let \(Q = \begin{bmatrix} | & | & & | \\ \vec {q}_1 & \vec {q}_2 & \cdots & \vec {q}_n \\ | & | & & | \end{bmatrix}\) so
that \(Q\) is orthogonal. We have
where \(D\) is the diagonal matrix with diagonal entries \(\lambda _1, \lambda _2, \ldots , \lambda _n\). But then \(Q^TAQ=D\), proving this half of the theorem.
For the converse, if \(A\) is orthogonally diagonalizable, then by Theorem 10 it is symmetric. But then Theorem 17 tells us that
eigenvectors corresponding to distinct eigenvalues are orthogonal. Because \(A\) is (orthogonally) diagonalizable,
we know the geometric multiplicity of each eigenvalue is equal to its algebraic multiplicity. This implies that
we can use Gram-Schmidt on each eigenspace of dimension \(> 1\) to get a full set of \(n\) orthogonal eigenvectors.
If we are willing to replace “diagonal” by “upper triangular” in the real spectral theorem, we can weaken the requirement that \(A\)
is symmetric to insisting only that \(A\) has real eigenvalues.
Schur Triangularization Theorem If \(A\) is an \(n \times n\) matrix with \(n\) real eigenvalues, an orthogonal matrix \(Q\) exists such that \(Q^{T}AQ\) is upper
triangular.
There is also a lower triangular version of this theorem.
See Practice Problem ??
The eigenvalues of an upper triangular matrix are displayed along the main diagonal. Because \(A\) and \(Q^{T}AQ\) have the same
determinant and trace whenever \(Q\) is orthogonal (for they are similar matrices), Theorem 23 gives:
If \(A\) is an \(n \times n\) matrix with real eigenvalues \(\lambda _{1}, \lambda _{2}, \dots , \lambda _{n}\) (possibly not all distinct), then \(\det A = \lambda _{1}\lambda _{2} \dots \lambda _{n}\) and \(\mbox {tr} A = \lambda _{1} + \lambda _{2} + \dots + \lambda _{n}\).
This corollary remains true even if the eigenvalues
are not real.
Practice Problems
Normalize the columns to make the following matrix orthogonal. \(A = \begin{bmatrix} 1 & -4 \\ 2 & 2 \end{bmatrix}\)
