Curve Fitting

We know that two points determine a line. Do you know how many points determine a quadratic function of the form \(f(x)=ax^2+bx+c\)? Given any number of points in the plane, is it always possible to find a polynomial function whose graph contains every one of the given points? To address these questions we will start with an alternative way of finding an equation of a line.

Consider two points \(A(-3, 2)\) and \(B(1,-2)\). We will find a function \(f\) whose graph is a line that passes through these points. We know that \(f(x)=ax+b\) for some constants \(a\) and \(b\). Because the graph of \(f\) passes through \(A\) and \(B\), we must have the following:
\[\begin{matrix} f(-3)&=&a(-3)&+&b&=&2\\ f(1)&=&a(1)&+&b&=&-2 \end{matrix}\]
To solve for \(a\) and \(b\), we need to solve the following matrix equation:
\[\begin{bmatrix}\answer {-3} & \answer {1}\\\answer {1} & \answer {1}\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}=\begin{bmatrix}\answer {2}\\\answer {-2}\end{bmatrix}\]
Solving the equation, we find that \(a=-1\) and \(b=-1\). This gives us:
\[f(x)=-x-1\]

The GeoGebra interactive below shows two points \(A\) and \(B\), together with the matrix equation that produces function coefficients for the function whose graph passes through \(A\) and \(B\). Drag the points around the plane to see how the matrix equation changes.

From a purely formal standpoint, we observe that the matrix equation has the form:

\[\begin{matrix}\text {point }A\rightarrow \\\text {point }B\rightarrow \end{matrix}\begin{bmatrix}x\text {-coordinate} & 1\\x\text {-coordinate} & 1\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}=\begin{bmatrix}y\text {-coordinate}\\y\text {-coordinate}\end{bmatrix}\]
where each row corresponds to one point.

Now we are ready to move to quadratic, and higher degree polynomial functions.

Linear function in Exploration exp:curveFitLine had two unknown coefficients that we needed to find in order to determine the function. Two points gave us a system of two equations and two unknowns.

A quadratic polynomial function, whose graph is a parabola, is given by:

\[f(x)=ax^2+bx+c\]

Three unknown coefficients will require three points to determine them.

We will find a quadratic function of the form \(f(x)=ax^2+bx+c\) whose graph passes through \(A(-2,2)\), \(B(0, -1)\) and \(C(1, 5)\). To do this, we need to find coefficients \(a\), \(b\) and \(c\) such that
\[\begin{matrix}f(-2)&=&a(-2)^2&+&b(-2)&+&c&=&2\\f(0)&=&a(0)^2&+&b(0)&+&c&=&-1\\f(1)&=&a(1)^2&+&b(1)&+&c&=&5\end{matrix}\]

The following GeoGebra interactive shows points \(A\), \(B\), and \(C\), together with the matrix equation, and its solution.

Drag the points around the plane to observe changes in the coefficient matrix. Think geometrically to find locations of \(A\), \(B\) and \(C\) such that

  • \(a=b=0\); \(c\neq 0\).
  • \(a=0\); \(b, c\neq 0\).

Observe the structure of the matrix equation.

\[\begin{matrix}\text {point }A\rightarrow \\\text {point }B\rightarrow \\\text {point }C\rightarrow \end{matrix}\begin{bmatrix}(x\text {-coordinate})^2&x\text {-coordinate} & 1\\(x\text {-coordinate})^2&x\text {-coordinate} & 1\\(x\text {-coordinate})^2&x\text {-coordinate} & 1\end{bmatrix}\begin{bmatrix}a\\b\\c\end{bmatrix}=\begin{bmatrix}y\text {-coordinate}\\y\text {-coordinate}\\y\text {-coordinate}\end{bmatrix}\]

In general (provided that no one point lies directly above another), given \(n\) points, we can always find an \((n-1)\)-degree polynomial function whose graph contains every one of the given points. To find such a polynomial function, given by \(f(x)=a_{n-1}x^{n-1}+\dots +a_1x+a_0\), we need to solve a system of \(n\) equations with \(n\) unknowns which translates into the following matrix equation.

\begin{equation}\label {eq:matEq}\begin{bmatrix}\vdots &\vdots &\vdots &\vdots \\(x\text {-coordinate})^{n-1}&\dots &x\text {-coordinate} &1\\\vdots &\vdots &\vdots &\vdots \end{bmatrix}\begin{bmatrix}a_{n-1}\\\vdots \\a_1\\a_0\end{bmatrix}=\begin{bmatrix}\vdots \\y\text {-coordinate}\\\vdots \end{bmatrix}\end{equation}

In Practice Problems prob:systemProblems1 and prob:systemProblems2 you will show that the matrix equation in (eq:matEq) has a unique solution if and only if no two of the given points share an \(x\)-coordinate.

Using Technology

Throughout this section we have omitted the tedious process of solving matrix equations. It is useful to practice solving smaller matrix equations by hand, but for larger matrices, we can use technology. Below is the Octave code that can be used to find the solution to Exploration . You will be able to modify this code to solve some of the practice problems. 

    % Define the coefficient matrix
    A=[4 -2 1;
    0 0 1;
    1 1 1];

    % Define vector b
    b=[2;-1;5];

    % Find the solution
    x=inv(A)*b

Link to code.

For more information on using Octave, please refer to the Octave Tutorial section of this text.

On the Dangers of Overfitting

It is exciting to know that we can fit a function to a set of data points, but before we get carried away fitting a 299-degree polynomial function to 300 points, let’s consider the following situation.

In the GeoGebra interactive below, you can see that points \(A\) - \(F\) form a somewhat linear pattern. A linear model can be used to describe these points. Click on the “Display linear model" check-box to see the trend line. (You will learn how to find such models in Least-Squares Approximation). You can see that even though the line does not pass through any of the given points, it fits the overall pattern of the points and can be used to estimate the \(y\)-coordinates of other points whose \(x\)-coordinates fall within the limits of the scatter plot.

It might be tempting to think that we can find a better model by finding a \(5^{th}\)-degree polynomial function whose graph contains every one of the six points. Click on the “Display 5th degree poly model" check-box to see the alternative model. Can this model be successfully used to make predictions?

Try moving individual points around to see how their placement affects the line and the curve.

Any modeling process which insists on fitting the existing data points exactly, at the risk of failing to predict future observations, is referred to as overfitting. While sometimes it is beneficial to have a curve that passes through specific points, more often it is the trend, not the individual instances, that we try to capture. We will return to this topic in Least-Squares Approximation.

Practice Problems

Problems -

Find a polynomial function of an appropriate degree that passes through the given points.

\[(-3,0), (2, -2), (-1,1)\]
\[f(x)=\answer {-0.3x^2-0.7x+0.6}\]
Plot the graph of \(f\) in the Desmos window below.

\[(0,4), (-2, 2), (-1,1), (0.5, 1)\]
\[g(x)=\answer {-3.2x^3-7.6x^2-1.4x+4}\]

Plot the graph of \(g\) in the Desmos window below.

Two GeoGebra screenshots are shown below:

[Picture] [Picture]

In the first screenshot, points \(A\) and \(B\) coincide. In the second screenshot, point \(A\) is located directly above point \(B\). In both cases, GeoGebra failed to produce a linear function whose graph passes through \(A\) and \(B\).

Based on what you know about functions and geometry, explain why the process fails for these two examples. How do your observations correspond to what happens from an algebraic standpoint?

Both systems are inconsistent. The first system is inconsistent, the second has infinitely many solutions. Both systems have infinitely many solutions. The first system has infinitely many solutions, the second system is inconsistent.
Prove that equation (1) has a unique solution if and only if no two given points share an \(x\)-coordinate.
Show that the rows of the matrix are linearly independent if and only if no two given points share an \(x\)-coordinate.
Under what circumstances is a solution not unique? Under what circumstances does a solution not exist?