Orthogonal Projections

Given a line \(l\) and a vector \(\vec {v}\) emanating from a point on \(l\), it is sometimes convenient to express \(\vec {v}\) as the sum of a vector \(\vec {v}_{\parallel }\), parallel to \(l\), and a vector \(\vec {v}_{\perp }\), perpendicular to \(l\). If you have taken a physics course, you may have seen a force vector decomposed into the sum of two components: one parallel and one perpendicular to the direction of motion.

[Picture]

Suppose \(\vec {d}\) is a direction vector for \(l\). Then \(\vec {v}_ {\parallel }=k\vec {d}\) for some scalar \(k\). Our goal is to find \(k\).

\begin{align*}\vec {v}\dotp \vec {d}&=(\vec {v}_{\parallel }+\vec {v}_{\perp })\dotp \vec {d}\\ &=(k\vec {d}+\vec {v}_{\perp })\dotp \vec {d}\\ &=k\vec {d}\dotp \vec {d}+\vec {v}_{\perp }\dotp \vec {d}\\ &=k\norm {\vec {d}}^2+0\\ &=k\norm {\vec {d}}^2 \end{align*}

We conclude that

\[k=\frac {\vec {v}\dotp \vec {d}}{\norm {\vec {d}}^2}\]

and

\[\vec {v}_{\parallel }=k\vec {d}=\left (\frac {\vec {v}\dotp \vec {d}}{\norm {\vec {d}}^2}\right )\vec {d}\]

The vector \(\vec {v}_{\parallel }=\left (\frac {\vec {v}\dotp \vec {d}}{\norm {\vec {d}}^2}\right )\vec {d}\) is called the projection of \(\vec {v}\) onto \(\vec {d}\). In our discussion, \(\vec {d}\) is a direction vector for line \(l\). So, we can also say that \(\vec {v}_{\parallel }\) is the projection of \(\vec {v}\) onto \(l\).

To find \(\vec {v}_{\perp }\), observe that \(\vec {v}_{\perp }=\vec {v}-\vec {v}_{\parallel }\).

A Visual Guide to Creating an Orthogonal Basis

Given an arbitrary basis \(\{\vec {v}_1, \vec {v}_2\}\) of \(\RR ^2\), let’s start building our orthogonal basis, \(\{\vec {f}_1, \vec {f}_2\}\), by setting \(\vec {f}_1 = \vec {v}_1\). To find the next element of our orthogonal basis, consider the orthogonal projection of \(\vec {v}_2\) onto \(\vec {f}_1\). (See the figure below.)

[Picture] [Picture]

Next, let \(\vec {f}_2=\vec {v}_2-\mbox {proj}_{\vec {f}_1}\vec {v}_2\). Observe that \(\vec {f}_2\) is orthogonal to \(\vec {f}_1\) (See Theorem ?? of Orthogonality and Projections). This gives us an orthogonal collection \(\mathcal {B}=\{\vec {f}_1,\vec {f}_2\}\). It is intuitively clear that \(\vec {f}_1\) and \(\vec {f}_2\) are linearly independent. Therefore \(\mathcal {B}\) is an orthogonal basis of \(\RR ^2\).

The following exploration illustrates this process dynamically.

Choose an arbitrary basis \(\{\vec {v}_1, \vec {v}_2\}\) of \(\RR ^2\) by dragging the tips of vectors \(\vec {v}_1\) and \(\vec {v}_2\) to desired positions. Use the navigation bar at the bottom of the interactive window to go through the steps of constructing an orthogonal basis of \(\RR ^2\).

We can apply this process to any two-dimensional subset of \(\RR ^n\). The following exploration will guide you through the process of constructing an orthogonal basis for a plane spanned by two arbitrary vectors in \(\RR ^3\).

Let \(W =\mbox {span}\left ({\bf v}_1,{\bf v}_2\right )\). \(W\) is a plane through the origin in \(\RR ^3\). Use the navigation bar at the bottom of the interactive window to go through the steps of constructing an orthogonal basis for \(W\). RIGHT-CLICK and DRAG to rotate the image for a better view.

In the next exploration, we take the process of constructing an orthogonal basis to the edge of the visual realm and construct an orthogonal basis for \(\RR ^3\).

In the GeoGebra interactive below \(\{\vec {v}_1, \vec {v}_2, \vec {v}_3\}\) is a basis of \(\RR ^3\). Use check boxes to go through the steps for constructing an orthogonal basis starting with the given basis. RIGHT-CLICK and DRAG to rotate the image for a better view.

What we saw above are low-dimension examples of what is known as Gram Schmidt Orthogonalization.

Practice Problems

Find \(\mbox {proj}_{\vec {d}}\vec {v}\) if \(\vec {d}=\begin{bmatrix}-1\\3\end{bmatrix}\) and \(\vec {v}=\begin{bmatrix}1\\4\end{bmatrix}\).
\[\mbox {proj}_{\vec {d}}\vec {v}=\begin{bmatrix}\answer {-1.1}\\\answer {3.3}\end{bmatrix}\]
Find \(\mbox {proj}_{\vec {d}}\vec {v}\) if \(\vec {d}=\begin{bmatrix}0\\2\\1\end{bmatrix}\) and \(\vec {v}=\begin{bmatrix}-1\\-4\\2\end{bmatrix}\).
\[\mbox {proj}_{\vec {d}}\vec {v}=\begin{bmatrix}\answer {0}\\\answer {-2.4}\\\answer {-1.2}\end{bmatrix}\]
Show that \(\mbox {proj}_{\vec {d}}\vec {v}\) does not depend on the length of \(\vec {d}\) by proving that \(\mbox {proj}_{\vec {d}}\vec {v}=\mbox {proj}_{k\vec {d}}\vec {v}\) for \(k\neq 0\). What does this result mean geometrically? Illustrate your response with a diagram.
Find the projection of vector \(\vec {v}\) onto line \(l\). (If entering answers in decimal form, round to the nearest one hundredth.)

[Picture]

Answer:

\[\begin{bmatrix}\answer [tolerance=0.005]{-95/26}\\\answer [tolerance=0.005]{19/26}\end{bmatrix}\]
Convert the given basis \(\mathcal {B}\) of \(\RR ^2\) to an orthogonal basis by following the procedure described in this section with \(\vec {v}_1=\begin{bmatrix}2\\ 1\end{bmatrix}\).
\[\mathcal {B} = \left \{ \begin{bmatrix}2\\ 1\end{bmatrix}, \begin{bmatrix}1\\ -1\end{bmatrix}\right \}\]
\[\left \{\begin{bmatrix}2\\1\end{bmatrix},\begin{bmatrix}\answer {3/5}\\\answer {-6/5}\end{bmatrix}\right \}\]