Statistics for Research

By the end of this guide you will be able to look at data from your own project, choose the right statistical test, run it in Python, and write up the result the way a scientist or a science fair judge expects. Your project doesn't need to involve machine learning. Lab experiments, surveys, and field measurements all use the same ideas. You need algebra and a little Python. Everything else is taught here.

What you'll be able to do when you finish

• Explain the difference between a sample and a population, and why two samples from the same population give different answers.
• Build and correctly interpret confidence intervals for means, proportions, and anything else (with the bootstrap).
• Run and interpret t-tests, ANOVA, chi-square tests, correlation, regression, and their nonparametric alternatives in SciPy.
• Say what a p-value does and does not tell you, in a way that would satisfy a statistician.
• Report effect sizes with confidence intervals, plan a sample size with a power analysis, and handle many tests without fooling yourself.
• Write results sentences in APA style, a reporting format that many journals and science fair judges expect.

How this guide fits with the others

Chapter 3 of Machine Learning Foundations already covers the mean, median, standard deviation, variance, z-scores, Pearson correlation, Bayes' rule, and the normal curve. This guide reviews those briefly and points you there for the formulas. The focus here is inference: using a sample to draw careful conclusions about a larger group. If you haven't written Python before, work through the first chapters of Python for Machine Learning first.

How each chapter is built

Each chapter starts with the idea in plain words, then shows the formula, then runs the code on real or clearly labeled made-up data. Every printed result in this guide came from actually running the code. Each chapter ends with a short Check yourself list. If you can answer those without looking back, move on.

Running the code

Use Google Colab. It's free, runs in your browser, and comes with the scientific Python libraries already installed. Make a new notebook, paste a code block into a cell, and press Shift + Enter. Code blocks within a chapter build on each other, so run them in order in the same notebook. If an import ever fails, run !pip install followed by the missing package's name in a cell. Some functions in this guide need SciPy 1.16 or newer. Check with import scipy; print(scipy.__version__).

Most examples use fixed random seeds, written np.random.default_rng(42), so your numbers should match the printed output exactly. If they differ in the last decimal place, a library version is the likely reason.

A suggested pace

Go at whatever pace works for you. If your project is due soon, your mentor can help you jump straight to the chapter your analysis needs, then come back for the rest.

Every measurement you take is a little bit different from the last one, and every group you study is only part of the group you care about. Statistics is the set of tools for drawing honest conclusions anyway. This chapter introduces the core problem the rest of the guide solves.

Populations and samples

The population is the entire group you want to make a claim about. The sample is the part of it you actually measure. A number that describes the population, like the true average, is a parameter. A number you calculate from your sample is a statistic. You almost never know the parameter, so you use a statistic to estimate it.

The last row matters. Even when there's no obvious "group," repeated measurements are a sample from all the measurements you could have made.

Different samples, different answers

Here's the central problem. Suppose a district has 50,000 students and their true average sleep is 7.0 hours. Five researchers each survey 25 random students. They won't get 7.0, and they won't agree with each other.

import numpy as np

rng = np.random.default_rng(42)

# A pretend population: nightly sleep (hours) for all 50,000 students in a district
population = rng.normal(loc=7.0, scale=1.2, size=50_000)
print(f"Population mean: {population.mean():.3f} hours")

# Five different researchers each survey 25 random students
for i in range(5):
    sample = rng.choice(population, size=25, replace=False)
    print(f"Sample {i + 1}: mean = {sample.mean():.2f} hours")

Population mean: 6.999 hours
Sample 1: mean = 6.64 hours
Sample 2: mean = 6.73 hours
Sample 3: mean = 6.56 hours
Sample 4: mean = 7.13 hours
Sample 5: mean = 7.22 hours

The population is made up, so we know the truth. In real research you get one sample and never see the population. Your sample mean of 6.56 or 7.22 is all you have, and you need to know how far it might be from the truth.

This sample-to-sample wobble is called sampling variability. It isn't a mistake. It happens even when every measurement is perfect, just because of which individuals ended up in the sample. Bigger samples wobble less:

# What if each researcher surveyed 400 students instead?
for size in [25, 100, 400]:
    means = [rng.choice(population, size=size, replace=False).mean() for _ in range(1000)]
    print(f"n = {size:>3}: sample means ranged from {min(means):.2f} to {max(means):.2f}")

n =  25: sample means ranged from 6.24 to 7.79
n = 100: sample means ranged from 6.63 to 7.39
n = 400: sample means ranged from 6.83 to 7.20

With 25 students, a sample mean could land anywhere from about 6.2 to 7.8 hours. With 400, it stays between about 6.8 and 7.2. Chapter 3 turns this observation into a formula.

What inference means

Statistical inference is reasoning from a sample to a population while being honest about sampling variability. It comes in two main forms:

• Estimation: "The average is probably between 6.8 and 7.3 hours." That's a confidence interval (Chapter 4).
• Testing: "Is the difference between these two groups bigger than sampling variability alone would produce?" That's a hypothesis test (Chapter 5).

Both depend on the sample being collected in a way that doesn't systematically favor some individuals. A random sample gives every member of the population a known chance of being chosen. If you survey only your friends, or only students in AP classes, no formula can fix the bias: your sample will differ from the population in a consistent direction, and a bigger sample just gives you a more precise wrong answer.

Where randomness comes from

Statistics needs randomness somewhere in your design. There are three common sources:

1. Random sampling from a population (surveys, field studies). It lets you generalize to that population.
2. Random assignment of subjects to treatment groups (experiments). It lets you claim the treatment caused a difference, because the groups differ only by chance and by the treatment.
3. Measurement error in repeated trials (physics and chemistry labs). Each trial differs slightly from the last for reasons you can't control.

Know which one your project has. A survey with random sampling but no random assignment can show that two things are related, but not that one causes the other (Chapter 10).