Linear Independence

In the last section we considered properties a set of vectors should possess in order to be used to define a “good” coordinate system for a plane. Let \(S\) be a set of vectors in the plane. We have already established that in order to use vectors in \(S\) to define the axes of a “good” coordinate system, vectors in \(S\) must span the plane. This would guarantee that every vector in the plane can be written as a linear combination of elements of \(S\).

In addition, we observed that we want each vector in the plane to be represented as a unique linear combination of elements of \(S\). This gives us motivation to consider the question of uniqueness. We will now step away from coordinate systems in the plane and turn to this question in a more general setting.

Let \(\vec {v}_1, \vec {v}_2,\dots ,\,\vec {v}_k\) be vectors of \(\RR ^n\). Let \(V=\text {span}(\vec {v}_1, \vec {v}_2,\dots ,\,\vec {v}_k)\). Then every vector in \(V\) can be written as a linear combination of \(\vec {v}_1, \vec {v}_2,\dots ,\,\vec {v}_k\) in at least one way. Our interest here is in spanning sets where each vector in \(V\) has exactly one representation as a linear combination of \(\vec {v}_1, \vec {v}_2,\dots ,\,\vec {v}_k\).

Suppose that two linear combinations are equal

\[a_1\vec {v}_1+a_2\vec {v}_2+\dots +a_k\vec {v}_k=b_1\vec {v}_1+b_2\vec {v}_2+\dots +b_k\vec {v}_k\]

We are looking for a condition on the set \(\{\vec {v}_1, \vec {v}_2,\dots ,\,\vec {v}_k\}\) that guarantees that this representation is unique. This amounts to showing that \(a_i=b_i\) for each \(i\). Taking all terms to the left side gives us

\[(a_1-b_1)\vec {v}_1+(a_2-b_2)\vec {v}_2+\dots +(a_k-b_k)\vec {v}_k=\vec {0}\]

So the condition that guarantees uniqueness is that all coefficients \((a_i-b_i)\) must be zero. This motivates the following definition.

We now formalize Remark remark:LinIndEquiv as a theorem.

th:lindeplincombofother_a \(\implies \) th:lindeplincombofother_b If \(\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\) are linearly dependent, then

\begin{equation*}c_1\vec {v}_1+c_2\vec {v}_2+\ldots +c_j\vec {v}_j+\ldots +c_k\vec {v}_k=\vec {0}\end{equation*}

has a non-trivial solution. In other words at least one of the constants, say \(c_j\), does not equal zero. This allows us to solve for \(\vec {v}_j\):

\begin{align*} -c_j\vec {v}_j&=c_1\vec {v}_1+c_2\vec {v}_2+\ldots +c_k\vec {v}_k \\ \\ \vec {v}_j&=-\frac {c_1}{c_j}\vec {v}_1-\frac {c_2}{c_j}\vec {v}_2-\ldots -\frac {c_k}{c_j}\vec {v}_k \end{align*}

(Do you see why it was important to have one of the constants nonzero?) This shows that \(\vec {v}_j\) may be expressed as a linear combination of the other vectors.

th:lindeplincombofother_b \(\implies \) th:lindeplincombofother_c First, suppose \(\vec {v}_j\) is a linear combination of \(\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\). We will show that \(\vec {v}_j\) is redundant by showing that

\[\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )=\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right ).\]
To show equality of the two spans we will pick a vector in the left span and show that it is also an element of the span on the right. Then, we will pick a vector in the right span and show that it is also an element of the span on the left, and we will conclude that the sets are equal.

Observe that if \(\vec {w}\) is in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right )\), then it has to be in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )\). (Why?)

Now suppose \(\vec {w}\) is in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )\). We need to show that \(\vec {w}\) is also in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right )\).

By assumption, we can write \(\vec {v}_j\) as

\begin{equation}\label {eq:vj} \vec {v}_j=a_1\vec {v}_1+a_2\vec {v}_2+\dots +a_{j-1}\vec {v}_{j-1}+a_{j+1}\vec {v}_{j+1}+\dots +a_k\vec {v}_k. \end{equation}

Since \(\vec {w}\) is in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )\), we have

\[\vec {w}=b_1\vec {v}_1+b_2\vec {v}_2+\dots +b_j\vec {v}_j+\dots +b_k\vec {v}_k.\]
Substituting the expression in (eq:vj) for \(\vec {v}_j\) and simplifying, we obtain the following
\begin{eqnarray*}\vec {w}=(b_1+b_ja_1)\vec {v}_1+(b_2+b_ja_2)\vec {v}_2&+&\dots \\ &+&(b_{j-1}+b_ja_{j-1})\vec {v}_{j-1}\\ &+&(b_{j+1}+b_ja_{j+1})\vec {v}_{j+1}\\ &+&\dots \\ &+&(b_k+b_ja_k)\vec {v}_k.\end{eqnarray*}

This shows that \(\vec {w}\) is in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right )\). We now have

\[\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )=\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right ),\]
which shows that \(\vec {v}_j\) is redundant.

th:lindeplincombofother_c \(\implies \) th:lindeplincombofother_a Suppose that \(\vec {v}_j\) is redundant, so that

\[\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )=\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right ).\]
Consider a vector \(\vec {w}\) in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )\)
\begin{equation}\label {eq:w1} \vec {w}=a_1\vec {v}_1+a_2\vec {v}_2+\dots +a_j\vec {v}_j+\dots +a_k\vec {v}_k \end{equation}
Since the span contains ALL possible linear combinations of \(\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\), we may choose \(\vec {w}\) such that \(a_j\neq 0\).

By assumption, \(\vec {w}\) is also in \(\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right )\). Therefore, we can express \(\vec {w}\) as a linear combination

\begin{equation}\label {eq:w2} \vec {w}=b_1\vec {v}_1+b_2\vec {v}_2+\dots +b_{j-1}\vec {v}_{j-1}+b_{j+1}\vec {v}_{j+1}+\dots +b_k\vec {v}_k. \end{equation}

We complete the proof by showing there exists a non-trivial solution to

\begin{equation}\label {eq:LinIndepDefRepeated} c_1\vec {v}_1+c_2\vec {v}_2+\ldots +c_j\vec {v}_j+\ldots +c_k\vec {v}_k=\vec {0}.\end{equation}
Subtracting expression (eq:w2) from (eq:w1) we obtain
\begin{eqnarray*} \vec {0}=\vec {w}-\vec {w}=(a_1-b_1)\vec {v}_1&+&\dots \\ \nonumber &+&(a_{j-1}-b_{j-1})\vec {v}_{j-1}+a_j\vec {v}_j+(a_{j+1}-b_{j+1})\vec {v}_{j+1}\\ \nonumber &+&\dots \\ \nonumber &+&(a_k-b_k)\vec {v}_k \end{eqnarray*}

Recall that we ensured that \(a_j\neq 0\). This implies that we have a non-trivial solution to Equation eq:LinIndepDefRepeated.

These three parts of the proof show that if one of the conditions is true, all three must be true. It is a logical consequence that if one of the three conditions is false, all three must be false.

Geometry of Linearly Dependent and Linearly Independent Vectors

Theorem 6 gives us a convenient ways of looking at linear dependence/independence geometrically. When looking at two or more vectors, we ask, “can one of the vectors be written as a linear combination of the others?" We can also ask, “is one of the vectors redundant?” If the answer to either of these questions is “YES", then the vectors are linearly dependent.

A Set of Two Vectors

Two vectors are linearly dependent if and only if one is a scalar multiple of the other. Two nonzero linearly dependent vectors may look like this:

[Picture]

or like this:

[Picture]

Two linearly independent vectors will look like this:

[Picture]

A Set of Three Vectors

Given a set of three nonzero vectors, we have the following possibilities:

  • (Linearly Dependent Vectors) The three vectors are scalar multiples of each other.

    [Picture]

  • (Linearly Dependent Vectors) Two of the vectors are scalar multiples of each other.

    [Picture]

  • (Linearly Dependent Vectors) One vector can be viewed as the diagonal of a parallelogram determined by scalar multiples of the other two vectors. All three vectors lie in the same plane.

    [Picture]

  • (Linearly Independent Vectors) A set of three vectors is linearly independent if the vectors do not lie in the same plane. For example, vectors \(\vec {i}\), \(\vec {j}\) and \(\vec {k}\) are linearly independent.

Practice Problems

Are the vectors below linearly independent?
\[\begin{bmatrix}-1\\0\end{bmatrix}, \begin{bmatrix}2\\3\end{bmatrix},\begin{bmatrix}4\\-1\end{bmatrix}\]
Yes No
If we rewrite
\[c_1\begin{bmatrix}-1\\0\end{bmatrix}+c_2 \begin{bmatrix}2\\3\end{bmatrix}+c_3\begin{bmatrix}4\\-1\end{bmatrix}=\begin{bmatrix}0\\0\end{bmatrix}\]
as a system of linear equations, there will be more unknowns than equations.
Are the vectors below linearly independent?
\[\begin{bmatrix}1\\0\\5\end{bmatrix}, \begin{bmatrix}2\\2\\3\end{bmatrix},\begin{bmatrix}-1\\0\\1\end{bmatrix}\]
Yes No
If we let \(A\) be the matrix whose columns are these vectors, then \(\mbox {rref}(A)\) should tell us what we want to know.
Are the vectors below linearly independent?
\[\begin{bmatrix}3\\0\\5\end{bmatrix}, \begin{bmatrix}2\\0\\2\end{bmatrix},\begin{bmatrix}-1\\0\\-5\end{bmatrix}\]
Yes No
If we let \(A\) be the matrix whose columns are these vectors, then \(\mbox {rref}(A)\) should tell us what we want to know.
Are the vectors below linearly independent?
\[\begin{bmatrix}3\\1\\4\\1\end{bmatrix}, \begin{bmatrix}-2\\1\\1\\1\end{bmatrix}\]
Yes No
In a set of two vectors, the only way one could be redundant is if they are scalar multiples of each other.
(True or False?) Any set containing the zero vector is linearly dependent.
TRUE FALSE
Can the zero vector be removed from the set without changing the span?
(True or False?) A set containing five vectors in \(\RR ^2\) is linearly dependent.
TRUE FALSE
If we rewrite Equation 1 for five vectors in \(\RR ^2\) as a system of equations, how many equations and unknowns will it have? What does this imply about the number of solutions?
Given that
\[0\vec {v}_1+ 0\vec {v}_2+ 0\vec {v}_3=\vec {0}\]
what (if anything) can we conclude about linear independence of vectors \(\vec {v}_1, \vec {v}_2, \vec {v}_3\)?
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination
Given that
\[3\vec {v}_1+ 4\vec {v}_2- \vec {v}_3=\vec {0}\]
what (if anything) can we conclude about linear independence of vectors \(\vec {v}_1, \vec {v}_2, \vec {v}_3\)?
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination
Given that
\[2\vec {v}_1+ 0\vec {v}_2+ 0\vec {v}_3=\vec {0}\]
what (if anything) can we conclude about linear independence of vectors \(\vec {v}_1, \vec {v}_2, \vec {v}_3\)?
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination
Are the vectors in the diagram linearly dependent or independent?
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination

[Picture]

Are the vectors in the diagram linearly dependent or independent?
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination

[Picture]

Suppose \(\{\vec {v}_{1}, \dots , \vec {v}_{m}\}\) is a linearly independent set in \(\RR ^n\), and that \(\vec {v}_{m+1}\) is not in \(\mbox {span}\left (\vec {v}_{1}, \dots , \vec {v}_{m}\right )\). Prove that \(\{\vec {v}_{1}, \dots , \vec {v}_{m}, \vec {v}_{m+1}\}\) is also linearly independent.
Suppose \(\{{\vec {u}},{\vec {v}}\}\) is a linearly independent set of vectors. Prove that the set \(\{\vec {u} -\vec {v}, \vec {u}+2\vec {v}\}\) is also linearly independent.
Suppose \(\{{\vec {u}},{\vec {v}}\}\) is a linearly independent set of vectors in \(\RR ^3\). Is the following set dependent or independent \(\{\vec {u} -\vec {v}, \vec {u}+2\vec {v}, \vec {u}+\vec {v}\}\)? Prove your claim.

Text Source

A portion of this section was adapted from Section 5.2 of Keith Nicholson’s Linear Algebra with Applications. (CC-BY-NC-SA)

W. Keith Nicholson, Linear Algebra with Applications, Lyryx 2018, Open Edition, p. 271.