Exploring Linear Models

We’re going to search for relationships within a dataset about all the states in the US. But first, let’s take a moment to (1) develop confidence in our ability to use functions for working with tables and making displays, and (2) build familiarity with a new dataset that we are going to spend a lot of time with.

In math, 𝑥 = 4 will define a variable 𝑥 to be the value 4.

This works in Pyret, too. But in Pyret, values can be more than just numbers!

Before we dig deeper into the State Demographics data, let’s think back to the animals at the shelter in order to introduce some new data science concepts.

Pie and bar charts help us see the frequency of values in a single categorical column.

There are other displays, like histograms and box plots, that help us explore the distribution of values in a single quantitative column.

But what we really want is a display that will help us search for a relationship between two quantitative columns, and that’s exactly what scatter plots do!

Before we can draw a scatter plot, we have to make an important decision: which variable do we think of as the cause - called the explanatory variable - and which is the effect (response variable)?

In this case, which do we suspect is the cause and which is the effect: age or time-to-adoption?

We suspect that age affects the adoption time, so we’ll use age as our explanatory variable and weeks as our response variable.

It’s customary to use the horizontal axis for our explanatory variable and the vertical axis for the response variable. Each row in the dataset will be represented by a point on the scatter plot with age for 𝑥 and weeks for 𝑦.

Students identify and make use of correlations in scatter plots. They learn to characterize their form as being linear, curved, or showing no clear pattern. They learn that linear patterns have direction, and they learn how to report strength (as well as direction) with a number called the "correlation."

Scatter plots let us visualize the relationship between two quantitative columns. If no relationship exists, the points in the scatter plot just appear as a shapeless cloud. But if there is a relationship, the points will form some kind of pattern. When we build scatter plots, we are searching for patterns between two quantitative variables.

These patterns can be described by three terms: form, direction and strength.

Now that you’ve dug into the role that form, direction and strength play in assessing a relationship between two quantitative variables, it’s time to put those concepts to work!

Building on prior knowledge of linear functions, students learn to find the line of best fit to model the relationship in a scatter plot that looks linear. This yields a predictor function that tells what y-value to expect for a given x-value. Students also learn how to quantify how closely a model fits a dataset, using residuals and S as a measure of how well their models fit the data.

Before we learn to fit linear models to scatter plots, let’s review. What do you remember about linear functions?

Linear relationships grow by fixed amounts, meaning that the difference between two y-values will always be the same over identical horizontal intervals. In the table shown to the right, you can see arrows pointing out the "jumps" between y-values for intervals of 1. Each jump is the same size.

Students are about to be asked to write the Slope-Intercept form of the line, given two points in our states dataset. If your students haven’t done much work with calculating slope and y-intercept from pairs of points recently, we recommend prepping them for success by having them complete Defining a Linear Function from Two Points.

When we see patterns in data, we can use those patterns to make predictions based on that data. We can even draw a line to show all the possible predictions at once! These predictions represent our "best guess" at the underlying relationship in the data, as we try to model that relationship using math.

Let’s find a line to model the relationship between the percent of the population with college degrees and median income.

Students confront the notion of "model fitness". How do we measure how well a model fits? How do we determine which of two models is best? First they’ll consider two models for a simple dataset and brainstorm how we could measure which fits better. Then they’ll test out their linear models using a new pyret function called fit-model, which draws the residuals and computes the Standard Deviation of the Residuals (𝑆).

In the previous section, we came up with a linear model for the relationship between pct-college-or-higher and median-income, but it definitely wasn’t the best model.

How do we even measure how good a model is?

Pyret has a special function called fit-model that graphs whatever function we give it on top of a scatter plot of the dataset!

When you graph your model in Pyret, you can see that:

The difference between any real 𝑦 and predicted 𝑦 is called the residual, and it measures how far off that one point in the model is from the actual data.

There are many different tools to calculate the fitness of a model.

For our purposes, we’ll use the value of S to tell us how well or poorly our model fits.

The statistical term S refers to the Standard Deviation of the Residuals, which is a measure of how far away all of data points are from a model.

A model built from Alaska and Alabama predicts that a 1 percent increase in college degrees is associated with a $5613.67 increase in median household income.

With an S-value of 36165, we know that there’s enough error in the model to predict median incomes that are off by $36,165! That’s enough to double the median income of a state or cut it in half!

Compared to the size of the incomes in this dataset, an S value of $36,165 is pretty terrible. This model should not be trusted!

Students are introduced to a new pyret function called lr-plot, which uses linear regression to fit the best possible linear model to the data.

We’ve learned how to measure how well linear models fit the data and to decide which linear model does a better job of predicting values, but how do we find the best possible linear model?

In Statistics, an algorithm called linear regression is used to derive the slope and y-intercept of the best possible model by taking every datapoint into account.

Pyret has a function called lr-plot that will do just that.

Students are reminded of the three forms of linear models available to us, discuss when and why we might choose one form over another, and practice translating between them.

When trying to fit a piece into a puzzle, sometimes we rotate the piece to see it from a different angle. When fitting a model to a dataset, we might prefer to look at the linear relationship from different angles as well!

So far, we’ve focused on models using the Slope-Intercept form of the line. That’s because it’s the form that is defined in terms of the response variable, making it most compatible with the programming environment.

But depending on the information we have available to us - or who we’re writing this model for - we might want to use other forms of linear models. Fortunately, we can always translate any model into another!

You may already be familiar with the different forms of linear models available to us:

(1) Slope-Intercept Form makes it really easy to read the slope and y-intercept.

(2) Point-Slope Form makes it easy to find the equation of the line given a single point and slope.

(3) Standard Form makes it easy to find the x- and y-intercepts of the line.

Why we might choose to use one form over another?

While it’s easier to write one linear form or the other based on the information available to us, and might be easier for someone else to extract the information they’re looking for based on the model we supply them with, we can easily translate back and forth between linear forms!

If you needed to draw the graph of a linear model, which form would you like to start from? Why?