Course Outline

segmentGetting Started (Don't Skip This Part)

segmentStatistics and Data Science: A Modeling Approach

segmentPART I: EXPLORING VARIATION

segmentChapter 1  Welcome to Statistics: A Modeling Approach

segmentChapter 2  Understanding Data

segmentChapter 3  Examining Distributions

segmentChapter 4  Explaining Variation

segmentPART II: MODELING VARIATION

segmentChapter 5  A Simple Model

segmentChapter 6  Quantifying Error

segmentChapter 7  Adding an Explanatory Variable to the Model

segmentChapter 8  Models with a Quantitative Explanatory Variable

segmentPART III: EVALUATING MODELS

segmentChapter 9  The Logic of Inference

segmentChapter 10  Model Comparison with F

segmentChapter 11  Parameter Estimation and Confidence Intervals

segmentPART IV: MULTIVARIATE MODELS

segmentChapter 12  Introduction to Multivariate Models

segmentChapter 13  Multivariate Model Comparisons

segmentFinishing Up (Don't Skip This Part!)

segmentResources
list College / Advanced Statistics and Data Science (ABCD)
Chapter 9: The Logic of Inference
9.1 The Problem of Inference
In previous chapters, you learned how to specify and fit statistical models to data, and to use GLM notation to represent those models (e.g., \(Y_{i}=b_{0}+b_{1}X_{i}+e_{i}\)). Such models are truly the best fitting model of the data, but data don’t always accurately represent the Data Generating Process (DGP).
What we really care about is the best model of the DGP (e.g., \(Y_{i}=\beta_{0}+\beta_{1}X_{i}+\epsilon_{i}\)). The complex model is always the best model of the data, but is it a better model of the DGP than the empty model? What are the true values of \(\beta_{0}\) and \(\beta_{1}\)?
Unfortunately, we can’t directly calculate the parameters in the DGP. We can estimate the parameters with \(b_0\) and \(b_1\), but we don’t know how accurate our estimates are. In the next few chapters, we will discuss how to infer what’s true about the DGP based on models we have estimated from a sample of data.
How to close the gap between our data and the DGP is often referred to as the problem of statistical inference. We’ve explored this problem of inference informally before. We know that the same DGP can produce a lot of different samples. Similarly, it’s hard to look at a sample and know exactly what DGP it came from. In these chapters, we explore solutions to this problem, outlining the logic of statistical inference, and what we have to gain from it.
A New Concept: Sampling Distribution
Key to solving the problem of inference will be a new and important concept that enables us to see how different samples from the same DGP could vary and how much the parameter estimates calculated from many different samples could vary. You can think of these many parameter estimates as a new kind of distribution called a sampling distribution.
Up to this point in the book, we have explored two kinds of distributions: the distribution of data (the sample), and the distribution of DGP (also called the population). The sampling distribution is the third leg of what we call the Distribution Triad – the distribution of an estimate across many possible samples, of equal size, from a given DGP.
Samples and populations are made up of objects you can measure (for example, thumb lengths or heights of students). Sampling distributions, in contrast, are made up of parameter estimates that you could calculate based on different samples of data from the same DGP (for example, a bunch of means or a whole distribution of \(b_1\)s). In this chapter we will focus on the sampling distribution of \(b_{1}\), the estimate of \(\beta_{1}\).
Rejecting the Empty Model: The Basic Idea
When we see a difference between two groups in the data we may be tempted to conclude that there is also a difference between the two groups in the DGP. That is, when \(b_1\) is some number that is not zero, we might be fooled into thinking that \(\beta_1\) is also not zero. The problem with that thinking is that even an empty model of the DGP, in which \(\beta_1=0\), can produce samples that have a difference.
The basic idea, which will unfold in this chapter, will require you to use your hypothetical thinking skills. You must ask: if the empty model is true, how likely would you be to observe the sample \(b_{1}\) that we found in our data? We’ll do this by using R to create a DGP in which \(\beta_1=0\), and letting it produce multiple samples of data. We’ll take a look at the \(b_1\)s calculated based on these multiple samples of simulated data and examine whether our real data look similar to the simulated data or not.