Exploring Linear Models

Students explore the State Demographics dataset and, building on a discussion of the displays they previously made using the animals dataset, recognize the unique opportunity scatter plots offer for exploring relationships between columns.

We’re going to search for relationships within a dataset about all the states in the US. But first, let’s take a moment to (1) develop confidence in our ability to use functions for working with tables and making displays, and (2) build familiarity with a new dataset that we are going to spend a lot of time with.

In math, x = 4 will define a variable x to be the value 4. Any time we use x after that, we can substitute in the value of 4. This works in Pyret, too. But in Pyret, values can be more than just numbers! In this file, the variables alabama and alaska are defined as rows from the table.

Debrief the rest of the page with students. Then, initiate a conversation about the various column names, ensuring that students understand all of the terminology. Later in the lesson, students will examine relationships between income and education. We recommend posing the questions below to ensure that they are ready to do so.

Before we dig deeper into State Demographics Starter File, let’s think back to the animals at the shelter in order to introduce some new data science concepts.

Pie and bar charts help us see the frequency of values in a single categorical column. There are other displays, like histograms and box plots, that help us explore the distribution of values in a single quantitative column.

But what we really want is a display that will help us search for a relationship between two quantitative columns, and that’s exactly what scatter plots do!

Before we can draw a scatter plot, we have to make an important decision: which variable do we think of as the cause - called the explanatory variable - and which is the effect (response variable)?

In this case, which do we suspect is the cause and which is the effect: age or time-to-adoption?

We suspect that age affects the adoption time, so we’ll use age as our explanatory variable and weeks as our response variable.

It’s customary to use the horizontal axis for our explanatory variable and the vertical axis for the response variable. Each row in the dataset will be represented by a point on the scatter plot with age for 𝑥 and weeks for 𝑦.

Encourage students to first think about which columns might be related, and then create the scatter plot to search for this relationship, rather than making scatter plots for random pairs of columns. The dataset is designed so that students will quickly begin searching for relationships between varying levels of education and income, and there are linear relationships in each of these.

Note: Students will acquire the formal vocabulary that data scientists use to assess relationships in the next section of this lesson, which is all about identifying form, direction, and strength.

Students identify and make use of correlations in scatter plots. They learn to characterize their form as being linear, curved, or showing no clear pattern. They learn that linear patterns have direction, and they learn how to report strength (as well as direction) with a number called the "correlation."

Scatter plots let us visualize the relationship between two quantitative columns. If no relationship exists, the points in the scatter plot just appear as a shapeless cloud. But if there is a relationship, the points will form some kind of pattern. When we build scatter plots, we are searching for patterns between two quantitative variables.

These patterns can be described by three terms: form, direction, and strength.

Optional: Have students turn to Linear, Non-linear, or Bust? and decide whether each of the scatter plots could be modeled by a linear relationship, a nonlinear relationship, or that there doesn’t appear to be a pattern.

If the relationship clusters around a straight line, we can talk about direction.

Positive: The line slopes up as we look from left-to-right. Positive relationships are by far the most common because of natural tendencies for variables to increase in tandem. For example, “the older the animal, the more it tends to weigh”.

Negative: The line slopes down as we look from left-to-right. For example, “the older a child gets, the fewer new words he or she learns each day.”

Not every shape has a direction! For example, a curve can start out sloping upwards, but then peak and slope downwards.

How well does knowing the x-value allow us to predict what the y-value will be?

A relationship is strong if knowing the x-value of a data point gives us a very good idea of what its y-value will be (knowing a student’s age gives us a very good idea of what grade they’re in). A strong linear relationship means that the points in the scatter plot are all clustered tightly around an invisible line.

A relationship is weak if x tells us little about y (a student’s age doesn’t tell us much about their number of siblings). A weak linear relationship means that the cloud of points is scattered very loosely around the line.

If a relationship is linear, we can report both its direction and its strength with a single number between -1 and +1 called the correlation. The way correlation reports direction is simple: it’s greater than zero if the relationship is positive, and less than zero if it’s negative. As for strength, correlation is closer to 1 in absolute value for strong relationships and closer to 0 for weak relationships; moderate relationships would have a correlation closer to 0.5 in absolute value.

Now that you’ve dug into the role that form, direction and strength play in assessing a relationship between two quantitative variables, it’s time to put those concepts to work!

Review student answers, and have students explain their thinking for this activity. For students who are struggling, hearing what their peers are looking for is especially helpful at this stage.

Review student answers. Some of the answers are not so clear-cut, and students may disagree about what constitutes a "strong" vs. "weak" correlation. We’ve tried to choose scatter plots that clearly fall into one category or the other, but without diving into the algorithm for linear regression students may find this exercise somewhat subjective…​ and that’s ok!

Building on prior knowledge of linear functions, students learn to find the line of best fit to model the relationship in a scatter plot that looks linear. This yields a predictor function that tells what y-value to expect for a given x-value. Students also learn to use R-squared as a measure of how well their linear models fit the data.

Before we learn to fit linear models to scatter plots, let’s review. What do you remember about linear functions?

Linear relationships grow by fixed amounts, meaning that the difference between two y-values will always be the same over identical horizontal intervals. In the table shown to the right, you can see arrows pointing out the "jumps" between y-values for intervals of 1. Each jump is the same size. If the rate of change is constant, the relationship is linear.

Optional: Students are about to be asked to write the Slope-Intercept form of the line, given two points in our states dataset. If your students haven’t done much work with calculating slope and y-intercept from pairs of points recently, we recommend prepping them for success by having them complete Defining a Linear Function from Two Points.

This scatter plot appears to show a positive, linear relationship: states with higher percentages of college graduates tend to have higher median household incomes.

When we see patterns in data, we can use those patterns to make predictions based on that data. We can even draw a line to show all the possible predictions at once! These predictions represent our "best guess" at the underlying relationship in the data, as we try to model that relationship using math.

These models are just functions being graphed on top of the scatter plot, with the goal of minimizing the squared distances between the line and all the points on the plot. For relationships that are apparently linear, the "predictor function" is a linear model of the form y=mx+b. For historical reasons, this line of best fit is sometimes called the regression line.

When we make a model, we want it to be the closest possible approximation of all the points. If we used another line instead of the "line of best fit," it wouldn’t be as close to all the points as a group, and wouldn’t do as good a job at predicting y-values from x-values.

Let’s find the best fit we can make for this dataset!

Optional: If your students could use more support for finding the equation of the line between two points, direct them to the scaffolded version of Build a Model from Samples: College Degrees v. Income (Scaffolded) instead.

Confirm that students were able to successfully compute slope and y-intercept, define and test f(x) in Pyret, and evaluate the predictive value of f(x).

Pyret includes a function called fit-model. Find its Contract on the Contracts Page. _If you’re working with a printed workbook, the contracts pages are included in the back. Like scatter-plot, it consumes columns for our labels, our 𝑥s and our 𝑦s. However, it also consumes a function! It produces a scatter plot, with the function graphed on top of it.

R² describes the percentage of the variation in the y-variable that is explained by the x-variable in our model. In other words, an R² value of 0.20 could mean that “20% of the variation in median household income is explained by the percentage of college degrees in a state, according to our linear model”. Better models will explain a higher percentage of that variation.

If the model is perfectly linear, the R² value will be 1.00, meaning the 𝑦-values can be perfectly predicted by the 𝑥-values. Of course in the real world, the only "perfect" relationships are things like "height in inches v. height in centimeters". That relationship is perfectly linear…​but we don’t need to use modeling to figure that out! The R² value for no correlation at all is zero. If we just drew a horizontal predictor line in the middle of the data, it would mean that we expect a median income to equal whatever the average is but with no connection whatsoever to the percentage of people who finish college.

But sometimes models make predictions that are even worse than useless - they trend in the wrong direction altogether. Did you see any models with a negative R² value?

But how do we find the best model? In Statistics, an algorithm called linear regression is used to derive the slope and y-intercept of the best possible model by taking every datapoint into account. Pyret has a function that will do just that, called lr-plot.

Sometimes the slope or y-intercept of a linear model have too many digits to be displayed clearly. When this happens, Pyret will convert them to scientific notation. While students have encountered scientific notation before, they may not recognize 8.23𝑒5 as 8.23 × 10⁵. You should make sure they understand how to translate this notation into numbers before proceeding.

When we interpret a model, we try to make sense of the slope, the axes, the 𝑅² value, and the real data behind them. In this example, a model built from Alaska and Alabama predicts that a 1 percent increase in college degrees is associated with a $5613 increase in median household income. Based on the 𝑅² value of -15.63, this is a pretty terrible model and shouldn’t be trusted.

Students are reminded of the three forms of linear models available to us, discuss when and why we might choose one form over another, and practice translating between them.

When trying to fit a piece into a puzzle, sometimes we rotate the piece to see it from a different angle. When fitting a model to a dataset, we might prefer to look at the linear relationship from different angles as well!

So far, we’ve focused on models using the Slope-Intercept form of the line. That’s because it’s the form that is defined in terms of the response variable, making it most compatible with the programming environment. Depending on who we’re communicating with and what information we have available to us, we might opt to use other forms of linear models, but we can always translate any model into another!

You may already be familiar with the different forms of linear models available to us:

Why we might choose to use one form over another?

Pose the questions below to assess student understanding of when and why we might choose one form over another.

While it’s easier to write one linear form or the other based on the information available to us, and might be easier for someone else to extract the information they’re looking for based on the model we supply them with, we can easily translate back and forth between linear forms!

If you needed to draw the graph of a linear model, which form would you like to start from? Why?