Essential Vocabulary

Here is a link to a list of these terms on Quizlet

[Picture]

Fundamental subspaces of a matrix

\(\mbox {null}(A)\) is the orthogonal complement of \(\mbox {row}(A)\), and \(\mbox {null}(A^T)\) is the orthogonal complement of \(\mbox {col}(A)\).

[Picture]

Gram-Schmidt process

An iterative process which constructs an orthogonal basis for a subspace. The idea is to build the orthogonal set one vector at a time, by taking a vector not in the span of the vectors in the current iteration of the set, and subtracting its orthogonal projection onto each of those vectors.

[Picture]

Orthogonal Basis

A set of orthogonal vectors that spans a subspace. (Any orthogonal set of vectors must be linearly independent.)

[Picture]

Orthogonal complement of a subspace

If \(W\) is a subspace, we define the orthogonal complement \(W^\perp \) as the set of all vectors orthogonal to every vector in \(W\), i.e.,
\[ W^\perp = \{\vec {x} \in \RR ^n \mid \vec {x} \dotp \vec {y} = 0 \mbox { for all } \vec {y} \in W\} \]

[Picture]

Orthogonal Decomposition Theorem

Let \(W\) be a subspace of \(\RR ^n\) and let \(\vec {x} \in \RR ^n\). Then there exist unique vectors \(\vec {w} \in W\) and \(\vec {w}^\perp \in W^\perp \) such that \(\vec {x} = \vec {w} + \vec {w}^\perp \).

[Picture]

Orthogonal matrices

An \(n \times n\) matrix \(Q\) is called an orthogonal matrix if its columns form an orthonormal set. This will happen if and only if its rows form an orthonormal set. Note also that \(Q\) is an orthogonal matrix if and only if it is an invertible matrix such that \(Q^{-1}=Q^{T}\).

[Picture]

Orthogonal projection onto a subspace

Let \(W\) be a subspace of \(\RR ^n\) with orthogonal basis \(\{\vec {f}_{1}, \vec {f}_{2}, \dots , \vec {f}_{m}\}\). If \(\vec {x}\) is in \(\RR ^n\), the vector
\[ \vec {w}=\mbox {proj}_W(\vec {x}) = \mbox {proj}_{\vec {f}_1}\vec {x} + \mbox {proj}_{\vec {f}_2}\vec {x} + \dots + \mbox {proj}_{\vec {f}_m}\vec {x} \]
is called the orthogonal projection of \(\vec {x}\) onto \(W\).

[Picture]

Orthogonal set of vectors

Let \(\{ \vec {v}_1, \vec {v}_2, \cdots , \vec {v}_k \}\) be a set of nonzero vectors in \(\RR ^n\). Then this set is called an orthogonal set if \(\vec {v}_i \dotp \vec {v}_j = 0\) for all \(i \neq j\). Moreover, if \(\norm {\vec {v}_i}=1\) for \(i=1,\ldots ,m\) (i.e. each vector in the set is a unit vector), we say the set of vectors is an orthonormal set.

[Picture]

Orthogonally diagonalizable matrix

An \(n \times n\) matrix \(A\) is said to be orthogonally diagonalizable if an orthogonal matrix \(Q\) can be found such that \(Q^{-1}AQ = Q^{T}AQ\) is diagonal.

[Picture]

Orthonormal basis

A set of orthonormal vectors that spans a subspace. (Any orthogonal set of vectors must be linearly independent by Theorem orthbasis.)

[Picture]

Orthonormal set of vectors

Let \(\{ \vec {v}_1, \vec {v}_2, \cdots , \vec {v}_k \}\) be a set of nonzero vectors in \(\RR ^n\). Then this set is called an orthogonal set if \(\vec {v}_i \dotp \vec {v}_j = 0\) for all \(i \neq j\). Moreover, if \(\norm {\vec {v}_i}=1\) for \(i=1,\ldots ,m\) (i.e. each vector in the set is a unit vector), we say the set of vectors is an orthonormal set.

[Picture]

Properties of orthogonal matrices

If \(Q\) is an orthogonal matrix, then...
1.
\(Q^{-1} = Q^T\) is orthogonal,
2.
\(\mbox {det} Q = \pm 1\),
3.
if \(\lambda \) is an eigenvalue of \(Q\), then \(|\lambda |=1\),
4.
the product of \(Q\) with any other orthogonal matrix will be an orthogonal matrix (i.e. orthogonal matrices are closed under matrix multiplication),

and

5.
\(Q\) is a length-preserving and angle-preserving linear transformation.

[Picture]

QR factorization

Let \(A\) be an \(m \times n\) matrix with independent columns. A QR-factorization of \(A\) expresses it as \(A = QR\) where \(Q\) is \(m \times n\) with orthonormal columns and \(R\) is an invertible and upper triangular matrix with positive diagonal entries.

[Picture]

Rank-Nullity Theorem for matrices (th:matrixranknullity)

Let \(A\) be an \(m\times n\) matrix. Then
\[\mbox {rank}(A)+\mbox {nullity}(A)=n.\]
This implies that the dimension of a subspace of \(\RR ^n\) plus the dimension of its orthogonal complement equals \(n\).

[Picture]

Spectral Theorem

If \(A\) is a real \(n \times n\) matrix, then \(A\) is symmetric if an only if \(A\) is orthogonally diagonalizable.

[Picture]