In Section AugMatrices, we used a special class of matrices, the augmented matrices, to assist us in solving systems of linear equations. In this section, we study matrices as mathematical objects of their own accord, temporarily divorced from systems of linear equations. To do so conveniently requires some more notation. When we write \(A = \left [ a_{ij} \right ]_{m \times n}\), we mean \(A\) is an \(m\) by \(n\) matrix and \(a_{ij}\) is the entry found in the \(i\)th row and \(j\)th column. Schematically, we have

\[ \begin{array}{ccccc} & & \text {$j$ counts columns from left to right } & & \\ & & \xrightarrow {\hspace {1.25in}} & & \\ A & = & \left . \left [ \begin{array}{rrrr} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{array} \right ] \right \downarrow & \text {$i$ counts rows from top to bottom} \end{array} \]

With this new notation we can define what it means for two matrices to be equal.

Essentially, two matrices are equal if they are the same size and they have the same numbers in the same spots. For example, the two \(2 \times 3\) matrices below are, despite appearances, equal.

\[ \left [ \begin{array}{rrr} 0 & -2 & 9 \\ 25 & 117 & -3 \\ \end{array} \right ] = \left [ \begin{array}{rrr} \ln (1) & \sqrt [3]{-8} & e^{2\ln (3)} \\ 125^{2/3} & 3^{2} \cdot 13 & \log (0.001) \end{array} \right ]\]

Now that we have an agreed upon understanding of what it means for two matrices to equal each other, we may begin defining arithmetic operations on matrices. Our first operation is addition.

As an example, consider the sum below.

\[ \left [ \begin{array}{rr}2 & 3 \\ 4 & -1 \\ 0 & -7 \\ \end{array} \right ] + \left [ \begin{array}{rr} -1 & 4 \\ -5 & -3 \\ 8 & 1 \\ \end{array} \right ] = \left [ \begin{array}{rr} 2 + (-1) & 3+4 \\ 4+(-5) & (-1)+(-3) \\ 0+8 & (-7)+ 1 \\ \end{array} \right ] = \left [ \begin{array}{rr} 1 & 7 \\ -1 & -4 \\ 8 & -6 \\ \end{array} \right ] \]

It is worth the reader’s time to think what would have happened had we reversed the order of the summands above. As we would expect, we arrive at the same answer. In general, \(A+B = B+A\) for matrices \(A\) and \(B\), provided they are the same size so that the sum is defined in the first place. This is the commutative property of matrix addition. To see why this is true in general, we appeal to the definition of matrix addition. Given \(A =\left [a_{ij}\right ]_{m \times n}\) and \(B =\left [b_{ij}\right ]_{m \times n}\),

\[A + B = \left [a_{ij}\right ]_{m \times n} + \left [b_{ij}\right ]_{m \times n} = \left [ a_{ij} + b_{ij} \right ]_{m \times n} = \left [ b_{ij} + a_{ij} \right ]_{m \times n} = \left [b_{ij}\right ]_{m \times n} + \left [a_{ij}\right ]_{m \times n} =B+A\]

where the second equality is the definition of \(A+B\), the third equality holds by the commutative law of real number addition, and the fourth equality is the definition of \(B+A\). In other words, matrix addition is commutative because real number addition is.

A similar argument shows the associative property of matrix addition also holds, inherited in turn from the associative law of real number addition. Specifically, for matrices \(A\), \(B\), and \(C\) of the same size, \((A+B)+C = A+(B+C)\). In other words, when adding more than two matrices, it doesn’t matter how they are grouped. This means that we can write \(A+B+C\) without parentheses and there is no ambiguity as to what this means. These properties and more are summarized in the following theorem.

The identity property is easily verified by resorting to the definition of matrix addition; just as the number \(0\) is the additive identity for real numbers, the matrix comprised of all \(0\)’s does the same job for matrices.

To establish the inverse property, we note that per the definition of matrix addition,

\[A + B = \left [a_{ij}\right ]_{m \times n} + \left [-a_{ij}\right ]_{m \times n} = \left [a_{ij} -a_{ij} \right ]_{m \times n} = \left [ 0 \right ]_{m \times n} = 0_{m \times n}.\]

The fact that \(B+A =0_{m \times n}\) as well comes from the commutative property of matrix addition.

More about the additive inverse is true. If a matrix \(C = \left [c_{ij}\right ]_{m \times n}\) satisfies \(A + C = 0_{m \times n}\), then once again by the definition of matrix addition, we must have \(a_{ij} + c_{ij} = 0\) , or \(c_{ij} = -a_{ij}\) for all \(i\) and \(j\). This shows the matrix \(C\) must be the matrix \(B\) as described in Theorem matrixadditionprops which shows the additive inverse of a matrix is unique. In general, we denote the additive inverse of a matrix \(A\) using the (suggestive) symbol \(-A\).

With the concept of additive inverse well in hand, we may now discuss what is meant by subtracting matrices. You may remember from arithmetic that \(a - b = a+(-b)\); that is, subtraction is defined as ‘adding the opposite (inverse).’ We extend this concept to matrices. For two matrices \(A\) and \(B\) of the same size, we define \(A-B = A + (-B)\). At the level of entries, this amounts to

\[A-B = A + (-B) = \left [a_{ij}\right ]_{m \times n} + \left [-b_{ij}\right ]_{m \times n} = \left [a_{ij} + \left (-b_{ij}\right ) \right ]_{m \times n} = \left [a_{ij} - b_{ij} \right ]_{m \times n}\]

Thus to subtract two matrices of equal size, we subtract their corresponding entries. Surprised?

Our next task is to define what it means to multiply a matrix by a real number. Thinking back to arithmetic, you may recall that multiplication, at least by a natural number, can be thought of as ‘rapid addition.’ For example, \(2+2+2 = 3 \cdot 2\). We know from algebra that \(3x = x + x + x\), so it seems natural that given a matrix \(A\), we define \(3A = A + A + A\). If \(A =\left [a_{ij}\right ]_{m \times n}\), we have

\[3A = A + A + A = \left [a_{ij}\right ]_{m \times n} + \left [a_{ij}\right ]_{m \times n} + \left [a_{ij}\right ]_{m \times n} = \left [a_{ij} + a_{ij} + a_{ij} \right ]_{m \times n} = \left [ 3a_{ij}\right ]_{m \times n} \]

In other words, multiplying the matrix in this fashion by \(3\) is the same as multiplying each entry by \(3\). This leads us to the following definition.

The word ‘scalar’ means ‘scaling factor’ as we explain below. Every point \(P(x,y)\) in the plane can be represented by its position matrix, \(P\):

\[ (x,y) \leftrightarrow P = \left [ \begin{array}{r} x \\ y \\ \end{array} \right ] \]

Suppose we take the point \((-2,1)\) and multiply its position matrix by \(3\). We have

\[ 3P = 3 \left [ \begin{array}{r} -2 \\ 1 \\ \end{array} \right ] = \left [ \begin{array}{r} 3(-2) \\ 3(1) \\ \end{array} \right ] = \left [ \begin{array}{r} -6 \\ 3 \\ \end{array} \right ].\]

This new matrix corresponds to the point \((-6,3)\) which is the result of scaling both the horizontal and vertical directions by a factor of \(3\).

As did matrix addition, scalar multiplication inherits many properties from real number arithmetic. Below we summarize these properties.

As with the other results in this section, Theorem matrixscalarmultprops can be proved using the definitions of scalar multiplication and matrix addition. For example, to prove that \(k(A+B) = kA + kB\) for a scalar \(k\) and \(m \times n\) matrices \(A\) and \(B\), we start by adding \(A\) and \(B\), then multiplying by \(k\) and seeing how that compares with the sum of \(kA\) and \(kB\).

\[ k(A+B) = k \left (\left [a_{ij}\right ]_{m \times n} + \left [b_{ij}\right ]_{m \times n}\right ) = k \left [a_{ij} + b_{ij} \right ]_{m \times n} = \left [k \left (a_{ij}+b_{ij}\right )\right ]_{m \times n} = \left [ka_{ij} + kb_{ij}\right ]_{m \times n}\]

As for \(kA + kB\), we have

\[ kA + kB = k\left [a_{ij}\right ]_{m \times n}+k\left [b_{ij}\right ]_{m \times n} = \left [ka_{ij}\right ]_{m \times n}+\left [kb_{ij}\right ]_{m \times n} = \left [ka_{ij} + kb_{ij}\right ]_{m \times n} \, \, \checkmark \]

which establishes the property. The remaining proofs are similar and are left to the reader.

The properties in Theorems matrixadditionprops and matrixscalarmultprops establish an algebraic system that lets us treat matrices and scalars more or less as we would real numbers and variables. In the following example, we challenge the reader to justify each and every step of the calculations using either properties of matrix arithmetic.

While the solution to the previous example is written in excruciating detail, in practice many of the steps above are omitted. The reader is encouraged to solve the equation in Example matrixaddscalarex as they would any other linear equation, for example: \(3a-(2+5a)=-4+\frac {1}{3}(9)\).

We now turn our attention to matrix multiplication - that is, multiplying a matrix by another matrix. Based on the ‘no surprises’ trend so far in the section, you may expect that in order to multiply two matrices, they must be of the same size and you find the product by multiplying the corresponding entries. While this kind of product is used in other areas of mathematics, we define matrix multiplication to serve us in solving systems of linear equations.

To that end, we begin by defining the product of a row and a column. We motivate the general definition with an example. Consider the two matrices \(A\) and \(B\) below.

\[ \begin{array}{cc} A = \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ] & B = \left [\begin{array}{rrrr} 3 & \hphantom {-}1 & 2 & -8 \\ 4 & 8 & -5 & 9 \\ 5 & 0 & -2 & -12 \\ \end{array} \right ] \end{array}\]

Let \(R1\) denote the first row of \(A\) and \(C1\) denote the first column of \(B\). To find the ‘product’ of \(R1\) with \(C1\), denoted \(R1 \cdot C1\), we first find the product of the first entry in \(R1\) and the first entry in \(C1\). Next, we add to that the product of the second entry in \(R1\) and the second entry in \(C1\), and so on until we reach the last entry in \(R1\) and the last entry in \(C1\).

Using entry notation, \(R1 \cdot C1 = a_{11}b_{11} + a_{12}b_{21}+a_{13}b_{31} = (2)(3) + (0)(4) + (-1)(5) = 6 + 0 + (-5) = 1\). We can visualize this schematically as follows

\[ \left [\begin{array}{rrr} \fbox {2} & \hphantom {-}\fbox {0} & \fbox {-1} \\ -10 & 3 & 5 \\ \end{array} \right ] \left [\begin{array}{rrrr} \fbox {3} & \hphantom {-}1 & 2 & -8 \\ \fbox {4} & 8 & -5 & 9 \\ \fbox {5} & 0 & -2 & -12 \\ \end{array} \right ] \]
\[ \begin{array}{ccccc} \underbrace {\begin{array}{rl} \stackrel {\xrightarrow {\hspace {.75in}}}{\begin{array}{ccc} \fbox {2} & \hphantom {-}0 & -1 \end{array}} & \left . \begin{array}{c} \fbox {3} \\ 4 \\ 5 \\ \end{array} \right \downarrow \\ \end{array}} & & \underbrace {\begin{array}{rl} \stackrel {\xrightarrow {\hspace {.75in}}}{\begin{array}{ccc} 2 & \hphantom {-}\fbox {0} & -1 \end{array}} & \left . \begin{array}{c} 3 \\ \fbox {4} \\ 5 \\ \end{array} \right \downarrow \\\end{array}} & & \underbrace {\begin{array}{rl} \stackrel {\xrightarrow {\hspace {.75in}}}{\begin{array}{ccc} 2 & \hphantom {-}0 & \fbox {$-1$} \end{array}} & \left . \begin{array}{c} 3 \\ 4 \\ \fbox {5} \\ \end{array} \right \downarrow \\ \end{array}} \\ a_{11}b_{11} & + & a_{12}b_{21} & + & a_{13}b_{31} \\ (2)(3) & + &(0)(4)& + & (-1)(5) \\ \end{array}\]

To find \(R2 \cdot C3\) where \(R2\) denotes the second row of \(A\) and \(C3\) denotes the third column of \(B\), we proceed similarly. We start with finding the product of the first entry of \(R2\) with the first entry in \(C3\) then add to it the product of the second entry in \(R2\) with the second entry in \(C3\), and so forth. Using entry notation, we have \(R2 \cdot C3 = a_{21}b_{13} + a_{22}b_{23} + a_{23}b_{33} = (-10)(2) + (3)(-5) + (5)(-2) = -45\). Schematically,

\[ \left [\begin{array}{rrr} 2 & 0 & -1 \\ \fbox {-10} & \hphantom {-}\fbox {3} & \fbox {5} \\ \end{array} \right ] \left [\begin{array}{rrrr} 3 & \hphantom {-}1 & \fbox {2} & -8 \\ 4 & 8 & \fbox {-5} & 9 \\ 5 & 0 & \fbox {-2} & -12 \\ \end{array} \right ] \]
\[ \begin{array}{ccccc} \underbrace {\begin{array}{rl} \stackrel {\xrightarrow {\hspace {.75in}}}{\begin{array}{ccc} \fbox {$-10$} & 3 & 5 \end{array}} & \left . \begin{array}{c} \hphantom {-}\fbox {2} \\ -5 \\ -2 \\ \end{array} \right \downarrow \\ \end{array}} & & \underbrace {\begin{array}{rl} \stackrel {\xrightarrow {\hspace {.75in}}}{\begin{array}{ccc} -10 & \fbox {3} & 5 \end{array}} & \left . \begin{array}{c} \hphantom {-}2 \\ \fbox {$-5$} \\ -2 \\ \end{array} \right \downarrow \\\end{array}} & & \underbrace {\begin{array}{rl} \stackrel {\xrightarrow {\hspace {.75in}}}{\begin{array}{ccc} -10 & 3 & \fbox {$5$} \end{array}} & \left . \begin{array}{c} \hphantom {-}2 \\ -5 \\ \fbox {$-2$} \\ \end{array} \right \downarrow \\ \end{array}} \\ a_{21}b_{13}= (-10)(2) = -20 & + & a_{22}b_{23} = (3)(-5) = -15 & + & a_{23}b_{33} = (5)(-2) = -10 \\ \end{array}\]

Generalizing this process, we have the following definition.

Note that in order to multiply a row by a column, the number of entries in the row must match the number of entries in the column. We are now in the position to define matrix multiplication.

There are a number of subtleties in Definition matrixproduct which warrant closer inspection. First and foremost, Definition matrixproduct tells us that the \(ij\)-entry of a matrix product \(AB\) is the \(i\)th row of \(A\) times the \(j\)th column of \(B\). In order for this to be defined, the number of entries in the rows of \(A\) must match the number of entries in the columns of \(B\). This means that the number of columns of \(A\) must match the number of rows of \(B\). In other words, to multiply \(A\) times \(B\), the second dimension of \(A\) must match the first dimension of \(B\), which is why in Definition matrixproduct, \(A_{m \times \underline {n}}\) is being multiplied by a matrix \(B_{\underline {n} \times r}\).

Furthermore, the product matrix \(AB\) has as many rows as \(A\) and as many columns of \(B\). As a result, when multiplying a matrix \(A_{\underline {m} \times n}\) by a matrix \(B_{n \times \underline {r}}\), the result is the matrix \(AB_{\underline {m} \times \underline {r}}\).

Returning to our example matrices below, we see that \(A\) is a \(2 \times \underline {3}\) matrix and \(B\) is a \(\underline {3} \times 4\) matrix. This means that the product matrix \(AB\) is defined and will be a \(2 \times 4\) matrix.

\[ \begin{array}{cc} A = \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ] & B = \left [\begin{array}{rrrr} 3 & \hphantom {-}1 & 2 & -8 \\ 4 & 8 & -5 & 9 \\ 5 & 0 & -2 & -12 \\ \end{array} \right ] \end{array}\]

Using \(Ri\) to denote the \(i\)th row of \(A\) and \(Cj\) to denote the \(j\)th column of \(B\), we form \(AB\) per to Definition matrixproduct:

\[ \begin{array}{rclcl} AB & = & \left [\begin{array}{rrrr} R1 \cdot C1 & R1 \cdot C2 & R1 \cdot C3 & R1 \cdot C4 \\ R2 \cdot C1 & R2 \cdot C2 & R2 \cdot C3 & R2 \cdot C4 \\ \end{array} \right ] & = & \left [\begin{array}{rrrr} 1 & \hphantom {-}2 & 6 & -4 \\ 7 & 14 & -45 & 47 \\ \end{array} \right ] \\ \end{array} \]

Note that the product \(BA\) is not defined, since \(B\) is a \(3 \times \underline {4}\) matrix while \(A\) is a \(\underline {2} \times 3\) matrix; \(B\) has more columns than \(A\) has rows, and so it is not possible to multiply a row of \(B\) by a column of \(A\).

Even when the dimensions of \(A\) and \(B\) are compatible such that \(AB\) and \(BA\) are both defined, the product \(AB\) and \(BA\) aren’t necessarily equal. In other words, \(AB\) may not equal \(BA\) which means matrix multiplication is not, in general, commutative. That being said, several other real number properties are inherited by matrix multiplication, as illustrated in our next theorem.

The one property in Theorem matrixmultprops which begs further investigation is, without doubt, the multiplicative identity. The entries in a matrix where \(i=j\) comprise what is called the main diagonal of the matrix. The identity matrix has \(1\)’s along its main diagonal and \(0\)’s everywhere else. A few examples of the matrix \(I_{k}\) mentioned in Theorem matrixmultprops are given below. The reader is encouraged to see how they match the definition of the identity matrix presented there.

\[ \begin{array}{ccccc} [1] & \left [ \begin{array}{rr} 1 & 0 \\ 0 & 1 \\ \end{array} \right ] & \left [ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right ] & \left [ \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right ] \\ I_{1} & I_{2} & I_{3} & I_4 \\ \end{array} \]

The identity matrix is an example of what is called a square matrix as it has the same number of rows as columns. Note that to in order to verify that the identity matrix acts as a multiplicative identity, some care must be taken depending on the order of the multiplication. For example, take the matrix \(2 \times 3\) matrix \(A\):

\[A = \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ]\]

In order for the product \(I_{k}A\) to be defined, \(k = 2\); similarly, for \(AI_{k}\) to be defined, \(k = 3\). We leave it to the reader to show \(I_{2}A = A\) and \(AI_{3} = A\). In other words,

\[\begin{array}{rcl} \left [ \begin{array}{rr} 1 & 0 \\ 0 & 1 \\ \end{array} \right ] \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ] & = & \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ] \\ \end{array}\]

and

\[\begin{array}{rcl} \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ]\left [ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right ] & = & \left [\begin{array}{rrr} 2 & \hphantom {-}0 & -1 \\ -10 & 3 & 5 \\ \end{array} \right ] \\ \end{array}\]

While the proofs of the properties in Theorem matrixmultprops are computational in nature, the notation becomes quite involved very quickly, so they are left to a course in Linear Algebra. The following example provides some practice with matrix multiplication and its properties. As usual, some valuable lessons are to be learned.

Example matrixmultex illustrates some interesting features of matrix multiplication. First note that in the first problem, neither \(A\) nor \(B\) is the zero matrix, yet the product \(AB\) is the zero matrix. Hence, the the zero product property enjoyed by real numbers and scalar multiplication does not hold for matrix multiplication.

The second and third problems introduce us to polynomials involving matrices. The reader is encouraged to step back and compare our expansion of the matrix product \(\left (M - 2I_4\right )\left (M + 3I_4\right )\) in third probem with the product \((x-2)(x+3)\) from real number algebra. The exercises explore this kind of parallel further.

As we mentioned earlier, a point \(P(x,y)\) in the \(xy\)-plane can be represented as a \(2 \times 1\) position matrix. We now show that matrix multiplication can be used to rotate these points, and hence graphs of equations.

When we started this section, we mentioned that we would temporarily consider matrices as their own entities, but that the algebra developed here would ultimately allow us to solve systems of linear equations. To that end, consider the system

\[\left \{ \begin{array}{rcl} 3x - y + z & = & 8 \\ x + 2y - z & = & 4 \\ 2x+ 3y - 4z & = & 10 \\ \end{array} \right .\]

In Section AugMatrices, we encoded this system into the augmented matrix

\[\left [ \begin{array}{rrr|r} 3 & -1 & 1 & 8 \\ 1 & 2 & -1 & 4 \\ 2 & 3 & -4 & 10 \\ \end{array} \right ]\]

Recall that the entries to the left of the vertical line come from the coefficients of the variables in the system, while those on the right comprise the associated constants. For that reason, we may form the coefficient matrix \(A\), the unknowns matrix \(X\) and the constant matrix \(B\) as below

\[ \begin{array}{ccc} A = \left [ \begin{array}{rrr} 3 & -1 & 1 \\ 1 & 2 & -1 \\ 2 & 3 & -4 \\ \end{array} \right ] & X = \left [ \begin{array}{r} x \\ y \\ z \\ \end{array} \right ] & B = \left [ \begin{array}{r} 8 \\ 4 \\ 10 \\ \end{array} \right ] \end{array} \]

We now consider the matrix equation \(AX = B\).

\[ \begin{array}{rcl} AX & = & B \\ \left [ \begin{array}{rrr} 3 & -1 & 1 \\ 1 & 2 & -1 \\ 2 & 3 & -4 \\ \end{array} \right ] \left [ \begin{array}{r} x \\ y \\ z \\ \end{array} \right ] & = & \left [ \begin{array}{r} 8 \\ 4 \\ 10 \\ \end{array} \right ] \\ \left [ \begin{array}{rrr} 3x -y +z \\ x + 2y -z \\ 2x + 3y -4 z \\ \end{array} \right ] & = & \left [ \begin{array}{r} 8 \\ 4 \\ 10 \\ \end{array} \right ] \\ \end{array}\]

We see that finding a solution \((x,y,z)\) to the original system corresponds to finding a solution \(X\) for the matrix equation \(AX = B\). If we think about solving the real number equation \(ax = b\), we would simply ‘divide’ both sides by \(a\). Is it possible to ‘divide’ both sides of the matrix equation \(AX = B\) by the matrix \(A\)? This is the central topic of Section MatMethods.