Linear Regression

Students are introduced to the concept of linear regression, and learn how to interpret the $\displaystyle r$-value. Essentially, they’re asked to think about how to quantify the strength of a correlation, and given the basic idea behind regression. For teachers who have the need and the bandwidth to go deeper, this is a good opportunity to teach the algorithm behind linear regression.

Have student write down what you think on Page 75, then quickly survey the class to see what they think.

🖼Show image To be more formal about this, we are asking whether being young or being small is more strongly correlated with being adopted faster. In the previous lesson, we looked at scatter plots as a way to visualize possible correlations between two variables in our dataset. What did we find?

Whenever there’s a possible linear relationship, Data Scientists try to draw the line of best fit, which cuts through the data cloud and can be used to make predictions. This line can be graphed on top of the scatter plot as a function, called the predictor.

Have students open their “Animals Dataset” files. (If you do not have this file, or if something has happened to it, they can always make a new copy.)

After looking for correlations in the point cloud, we are left with two questions:

Data scientists use a statistical method called linear regression to search for certain kinds of relationships in a dataset. When we draw our regression line on a scatter plot, we can imagine a rubber bands stretching vertically between the line itself and each point in the plot — every point pulls the line a little “up” or “down”. Linear regression is the math behind the line of best fit.

Give students some time to experiment, then share back observations. Can they come up with rules or suggestions for how to minimize error?

Students are introduced to the lr-plot function in Pyret, which performs a linear regression and plots the result.

Pyret includes a powerful display, which draws a scatterplot, draws the line of best fit, and even displays the equation for that line:

🖼Show image lr-plot is a function that takes a Table and the names of 3 columns:

Our goal is to figure out how strongly or weakly the variable on our x-axis explains the change in the variable on our y-axis, the x-variable is sometimes referred to as the explanatory variable and the y-variable is referred to as the response or “outcome” variable.

Have students create an lr-plot for our animals-table, using "names" for the labels, "age" for the x-axis and "weeks" for the y-axis.

The resulting scatterplot looks like those we’ve seen before, but it has a few important additions. First, we can see the line of best fit drawn on top. We can also see the equation for that line (in red), in the form $\displaystyle y = mx + b$. In this plot, we can see that the slope of the line is 0.714, which means that on average, each extra year of age results in an extra 0.714 weeks of waiting to be adopted. By plugging in an animal’s age for x, we can make a prediction about how many weeks it will take to be adopted.

A predictor only makes sense within the range of the data that was used to generate it. For example, if we extend our line out to where it hits the y-axis, it appears to predict that “unborn animals are adopted instantly”! Statistical models are just proxies for the real world, drawn from a limited sample of data: they might make a useful prediction in the range of that data, but once we try to extrapolate beyond that data we quickly get into trouble!

Students learn how to interpret $\displaystyle r$-values, which tell us the strength and direction of a correlation.

The correlation r is a number that tells us the direction and strength of a linear relationship between two quantitative variables. In other words, it tells us if the best-fitting line slopes up or down, and how tightly clustered or loosely scattered the points are around that line. If the number is positive, it means that the y-values tend to go up as the x-values go up. If it’s negative, it means the y-values go down as the x-values go up. The strength of a correlation is the distance from zero: an $\displaystyle r$-value of zero means there is no correlation at all, and stronger correlations will be closer to −1 or 1.

An r-value of ±0.65 or more is typically considered a strong correlation, and anything between ±0.35 and ±0.65 is “moderately correlated”. Anything less than ±0.35 may be considered weak. However, these cutoffs are not an exact science! Different types of data may be “noisier” than others, and in some fields an r-value of ±0.50 might be considered impressively strong!

How well can you interpret the results of a linear regression analysis? Can you write your own?

When looking at a regression for age v. adoption time for just the cats, we saw that the slope of the predictor function was +0.23, meaning that for every year older a cats is, we expect a +0.23-week increase in the time taken to adopt that cat. The $\displaystyle r$-value was 0.566, confirming that the correlation is positive and indicating moderate strength.

Have students read their text aloud, to get comfortable with the phrasing.

It’s worth revisiting this point again. It’s easy to be seduced by large r-values, but Data Scientists know that correlation can be accidental! Here are some real-life correlations that have absolutely no causal relationship:

All of these correlations come from the Spurious Correlations website. If time allows, have your students explore the site to see more!

Students repeat the previous activity, this time applying it to their own dataset and interpreting their own results. Note: this activity can be done briefly as a homework assignment, but we recommend giving students an additional class period to work on this.

Now that you’ve gotten some practice performing linear regression on the animals dataset, it’s time to apply that knowledge to your own data!

Have students share their findings with the class. Get excited about the connections they are making and the conclusions they are drawing! Encourage students to make suggestions to one another about further analysis.

🖼Show image

You’ve learned how linear regression can be used to fit a line to a linear cloud, and how to determine the direction and strength of that relationship. The word “linear” is important here. In the image on the right, there’s clearly a pattern, but it doesn’t look like a straight line! There are many other kinds of statistical models out there, but all of them work the same way: use a particular kind of mathematical function (linear or otherwise), to figure out how to get the “best fit” for a cloud of data.