Exploring Linear Models

We’re going to search for relationships within a dataset about all the states in the US. But first, let’s take a moment to (1) develop confidence in our ability to use functions for working with tables and making displays, and (2) build familiarity with a new dataset that we are going to spend a lot of time with.

In math, x = 4 will define a variable x to be the value 4.

This works in Pyret, too. But in Pyret, values can be more than just numbers!

Before we dig deeper into the State Demographics data, let’s think back to the animals at the shelter in order to introduce some new data science concepts.

Before we can draw a scatter plot, we have to make an important decision: which variable do we think of as the cause - called the explanatory variable - and which is the effect (response variable)?

In this case, which do we suspect is the cause and which is the effect: age or time-to-adoption?

We suspect that age affects the adoption time, so we’ll use age as our explanatory variable and weeks as our response variable. Now our question can be phrased algebraically: is there a function weeks(age) that fits the data?

It’s customary to use the horizontal axis for our explanatory variable and the vertical axis for the response variable. Each row in the dataset will be represented by a point on the scatter plot with age for x and weeks for y.

Students identify and make use of correlations in scatter plots. They learn to characterize their form as being linear, curved, or showing no clear pattern. They learn that linear patterns have direction, and they learn how to report strength (as well as direction) with a number called the "correlation."

Scatter plots let us visualize the relationship between two quantitative columns. If no relationship exists, the points in the scatter plot just appear as a shapeless cloud. But if there is a relationship, the points will form some kind of pattern. When we build scatter plots, we are searching for patterns between two quantitative variables.

These patterns can be described by three terms: form, direction and strength.

Now that you’ve dug into the role that form, direction and strength play in assessing a relationship between two quantitative variables, it’s time to put those concepts to work!

Building on prior knowledge of linear functions, students learn to find the line of best fit to model the relationship in a scatter plot that looks linear. This yields a predictor function that tells what y-value to expect for a given x-value. Students also learn how to quantify how closely a model fits a dataset, using residuals and S as a measure of how well their models fit the data.

Before we learn to fit linear models to scatter plots, let’s review. What do you remember about linear functions?

Linear relationships grow by fixed amounts, meaning that the difference between two y-values will always be the same over identical horizontal intervals. In the table shown to the right, you can see arrows pointing out the "jumps" between y-values for intervals of 1. Each jump is the same size.

Students are about to be asked to write the Slope-Intercept form of the line, given two points in our states dataset. If your students haven’t done much work with calculating slope and y-intercept from pairs of points recently, we recommend prepping them for success by having them complete Defining a Linear Function from Two Points.

When we see patterns in data, we can use those patterns to make predictions. We can even draw a line to show all the possible predictions at once! This line is our model of what we think is the underlying relationship.

Now our question can be phrased algebraically! We’re looking for a function that will model the relationship between college enrollment and income, so we want to know if there are values of m and b that will let this function fit the data well:

median-income(pct-college) = m × pct-college + b

Students confront the notion of "model fitness". How do we measure how well a model fits? How do we determine which of two models is best? First they’ll consider two models for a simple dataset and brainstorm how we could measure which fits better. Then they’ll test out their linear models using a new pyret function called fit-model, which draws the residuals and computes the Standard Deviation of the Residuals (S).

In the previous section, we came up with a linear model for the relationship between pct-college-or-higher and median-income, but it definitely wasn’t the best model.

How do we even measure how good a model is?

Pyret has a special function called fit-model that graphs whatever function we give it on top of a scatter plot of the dataset!

When you graph your model in Pyret, you can see that:

The difference between any real y and predicted y is called the residual, and it measures how far off that one point in the model is from the actual data.

There are many different tools to calculate the fitness of a model. You may have heard of R, R², etc…​

Statisticians and Data Scientists are careful to use the right tool for the job!

S is a measure of fitness, which refers to the Standard Deviation of the Residuals.

A model built from Alaska and Alabama predicts that a 1 percent increase in college degrees is associated with a $5613.67 increase in median household income.

With an S-value of 36165, we know that there’s enough error in the model to predict median incomes that are off by $36,165! That’s enough to double the median income of a state or cut it in half!

Compared to the size of the incomes in this dataset, an S value of $36,165 is pretty terrible. This model should not be trusted!

Students are introduced to a new pyret function called lr-plot, which uses linear regression to fit the best possible linear model to the data.

We’ve learned how to measure how well linear models fit the data and to decide which linear model does a better job of predicting values, but how do we find the best possible linear model?

In Statistics, an algorithm called linear regression is used to derive the slope and y-intercept of the best possible model by taking every datapoint into account. Linear regression consumes a dataset and produces a function representing the best linear model.

We could keep guessing and picking two points over and over, and never know if we found the best linear model. Linear regression automatically finds the best-possible model, for any dataset. This is pretty amazing!

Pyret’s lr-plot function finds the best model, and graphs it on top of a scatter plot, and tells us the slope and y-intercept.

Our model is built from data about all the existing states, which have college attendance rates between 18.3% (West Virginia) and 52.4% (Washington, DC). Suppose two new states were to join the union, one with a 30% college attendance rate and the other with a 90% attendance rate.

Is our model more reliable for one of these states than another? Why or why not?

Students are reminded of the three forms of linear models available to us, discuss when and why we might choose one form over another, and practice translating between them.

When trying to fit a piece into a puzzle, sometimes we rotate the piece to see it from a different angle. When fitting a model to a dataset, we might prefer to look at the linear relationship from different angles as well!

So far we have only discussed vertical shifts, but it is also possible to shift a line or curve horizontally.

The Slope-Intercept form of the line we’ve been using tells us about the slope (m) and the vertical shift. It is also possible to shift a line or curve horizontally, and for some of the non-linear models we will be exploring in this course, identifying the horizontal shift will be important. To prepare ourselves for that thinking, let’s look at how horizontal shifts would fit into our linear model.

Note: When the horizontal shift is zero, we can safely remove (h) from the equation. That’s exactly what we’ve been doing with our Slope-Intercept form.

We will mostly be using Slope-Intercept form of the line in this course, because it’s the simplest form that is defined in terms of the response variable, making it most compatible with the programming environment

But, depending on the information we have available to us - or who we’re writing this model for - we might want to use other forms of linear models. Fortunately, we can always translate any model into another!

You may already be familiar with the different forms of linear models available to us:

(1) Slope-Intercept Form makes it really easy to read the slope and y-intercept.

(2) Point-Slope Form makes it easy to find the equation of the line given a single point and slope.

(3) Standard Form makes it easy to find the x- and y-intercepts of the line.

Why we might choose to use one form over another?

While it’s easier to write one linear form or the other based on the information available to us, and might be easier for someone else to extract the information they’re looking for based on the model we supply them with, we can easily translate back and forth between linear forms!

If you needed to draw the graph of a linear model, which form would you like to start from? Why?