Matrix Multiplication

We will introduce matrix multiplication by first considering the special case of a matrix-vector product. In other words, the first matrix is \(m \times n\) and the second matrix is \(n \times 1\) for some positive integers \(m,n\).

Matrix-Vector Multiplication

We can now make a couple of observations about the matrix-vector product. The first observation is part of the definition, but it is still worth pointing out.

Let’s find another matrix-vector product.

Matrix-Matrix Multiplication

Matrix-matrix multiplication is simply an extension of the idea of matrix-vector multiplication. In order for the product definition to work, matrix dimensions must be compatible. Let \(A\) be an \(m\times n\) matrix, and let \(B\) be an \(n\times p\) matrix, then the product \(AB\) will be an \(m\times p\) matrix.

\[\begin{array}{ccccc} A& & B &=& AB\\ (m\times n) & &(n\times p) & &(m\times p) \end{array}\]

Just like with vector products, the inner dimensions must be the same, while the outer dimensions, \(m\) and \(p\), give us the dimensions of the product.

So the \((i,j)\)-entry of \(AB\) is the dot product of the \(i^{th}\) row of \(A\) and the \(j^{th}\) column of \(B\).

In terms of components, if the \(i^{th}\) row of \(A\) is

\[\begin{bmatrix}a_{i1}& a_{i2} &\ldots &a_{in}\end{bmatrix}\]

and the \(j^{th}\) column of \(B\) is

\[\begin{bmatrix}b_{1j}\\b_{2j}\\\vdots \\b_{nj}\end{bmatrix}\]

then the \((i,j)\)-entry of \(AB\) is given by

\begin{equation}\label {eq:ijentrymatrixproduct}a_{i1}b_{1j}+a_{i2}b_{2j}+\dots +a_{in}b_{nj}=\sum _{k=1}^na_{ik}b_{kj} \end{equation}

Properties of Matrix Multiplication

Let
\[A=\begin{bmatrix}1&2\\3&4\end{bmatrix}\quad \text {and}\quad B=\begin{bmatrix}5&6\\7&8\end{bmatrix}\]
Observe that both \(AB\) and \(BA\) are defined, and both products are \(2\times 2\) matrices. Let’s compute the two products
\[AB=\begin{bmatrix}19&22\\43&50\end{bmatrix}\quad \text {and}\quad BA=\begin{bmatrix}23&34\\31&46\end{bmatrix}\]
Clearly \(AB\neq BA\). We say that \(A\) and \(B\) do not commute.

While it is possible to find specific matrices that commute, matrix multiplication is not commutative in general.

One example of an \(n\times n\) square matrix that commutes with all \(n\times n\) matrices is the matrix \(I_n\) defined by

\[I_n=\begin{bmatrix}1&0&\ldots &0\\0&1&\ldots &0\\\vdots &\vdots &\ddots &\vdots \\0 &0 &\ldots & 1\end{bmatrix}\]

\(I_n\) has 1’s along the main diagonal and 0’s everywhere else. It is often useful to think of \(I_n\) as a matrix whose \(j^{th}\) column (and \(j^{th}\) row) is \(\vec {e}_j\), the \(j^{th}\) standard unit vector of \(\RR ^n\). When the dimensions of \(I_n\) are clear from the context, or irrelevant, we will omit the subscript \(n\) and simply refer to this matrix as \(I\).

You can easily convince yourself that \(I\) commutes with all square matrices of appropriate dimensions. Let

\[A=\begin{bmatrix}a&b&c\\d&e&f\\g&h&i\end{bmatrix}\quad \text {and}\quad I=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\]

Verify that \(AI=A\) and \(IA=A\).

Because \(I\) acts like the multiplicative identity \(1\) in regular multiplication, \(I\) (or \(I_n\)) is called the identity matrix.

Next we list several important properties of matrix multiplication. These properties hold only when matrix sizes are such that the products are defined.

We prove 1 using the expression in (1) for the \((i,j)\)-entry of a matrix product. (The proof of 2 is similar.) The \((i,j)\)-entry of \(A(B+C)\) is given by

\[ \sum _{k}a_{ik}( b_{kj}+c_{kj})=\sum _{k}a_{ik}b_{kj}+\sum _{k}a_{ik}c_{kj} \]
We recognize the right hand side as the \((i,j)\)-entry of \(AB+AC\). Thus \(A(B+C) =AB+AC\).

For 4,

\[c\left (\sum _{k}a_{ik}b_{kj}\right ) = \sum _{k}\left (c a_{ik}\right )b_{kj} = \sum _{k}a_{ik} \left (b_{kj} c\right ).\]
For 5, the \((i,j)\)-entry of the product \(AI\) is given by the dot product of the \(i^{th}\) row of \(A\) with the standard unit vector \(\vec {e}_j\). Clearly, this dot product is \(a_{ij}\). Because the \((i,j)\)-entry of the product \(AI\) is equal to the \((i,j)\)-entry \(A\), we conclude that \(AI=A\). The proof that \(IA=A\) is similar.

Note that we skipped the proof of 3, which is quite cumbersome using sigma notation. We will easily tackle this proof later in the course when we cover Composition of Linear Transformations.

Practice Problems

Explain why the following product is not defined.
\[\begin{bmatrix}a&b&c\\d&e&f\\g&h&i\end{bmatrix}\begin{bmatrix}1\\2\\3\\4\end{bmatrix}\]
Express the given product as a linear combination of the columns of the matrix.
\[\begin{bmatrix}1&2&3&4\\5&6&7&8\end{bmatrix}\begin{bmatrix}-2\\3\\-5\\7\end{bmatrix}=\answer {-2}\begin{bmatrix}1\\5\end{bmatrix}+\answer {3}\begin{bmatrix}2\\6\end{bmatrix}+\answer {-5}\begin{bmatrix}3\\7\end{bmatrix}+\answer {7}\begin{bmatrix}4\\8\end{bmatrix}\]
Find the dimensions of the product.
\[\begin{bmatrix}1&2&3\end{bmatrix}\begin{bmatrix}4\\5\\6\end{bmatrix}\]
Dimensions of product: \(\answer {1}\times \answer {1}\)
Find the dimensions of the product.
\[\begin{bmatrix}1&2\\3&4\\5&6\end{bmatrix}\begin{bmatrix}1&2&3&4\\5&6&7&8\end{bmatrix}\]
Dimensions of product: \(\answer {3}\times \answer {4}\)
Find the product.
\[\begin{bmatrix}1&3&-2&1\\-2&1&0&4\end{bmatrix}\begin{bmatrix}4\\-2\\1\\1\end{bmatrix}=\begin{bmatrix}\answer {-3}\\\answer {-6}\end{bmatrix}\]
Find the product.
\[\begin{bmatrix}1&2&3\end{bmatrix}\begin{bmatrix}-1&2&-3&1\\1&1&-1&-2\\0&1&2&1\end{bmatrix}=\begin{bmatrix}\answer {1}&\answer {7}&\answer {1}&\answer {0}\end{bmatrix}\]
Find the product.
\[\begin{bmatrix}1&2\\-1&0\\2&3\\-4&-1\end{bmatrix}\begin{bmatrix}-1&1\\2&-3\end{bmatrix}=\begin{bmatrix}\answer {3}&\answer {-5}\\\answer {1}&\answer {-1}\\\answer {4}&\answer {-7}\\\answer {2}&\answer {-1}\end{bmatrix} \]

Text Source

The section on Properties of Matrix Multiplication is an adaptation of Section 2.1 of Ken Kuttler’s A First Course in Linear Algebra. (CC-BY)

Ken Kuttler, A First Course in Linear Algebra, Lyryx 2017, Open Edition, p. 67-68.