A headline announces that a new drug produces a statistically significant reduction in blood pressure. The study enrolled 50,000 participants. The p-value is smaller than 0.001. It sounds impressive—until you learn the average reduction was 0.4 millimeters of mercury, roughly the difference between measuring your pressure sitting versus standing.

This scenario plays out constantly in science journalism, policy debates, and clinical decisions. We've built an entire culture around the p-value, treating it as a verdict on whether something is real. But statistical significance answers only one narrow question: could this pattern have arisen by chance? It says nothing about whether the pattern actually matters.

Effect sizes fill that gap. They measure how much rather than just whether. And once you learn to look for them, you'll notice how often confident scientific claims quietly sidestep the question of magnitude—and how differently the world looks when magnitude is put back at the center.

A p-value tells you the probability of observing data at least as extreme as yours, assuming the null hypothesis is true. That's a technical answer to a technical question. It does not tell you whether the effect is large, useful, or worth acting on.

The confusion arises because significance depends heavily on sample size. Give me enough participants, and I can produce a statistically significant result for almost any real but trivial difference. A 2018 meta-analysis of Facebook use and well-being found statistically significant negative correlations across hundreds of thousands of subjects—yet the effect size, roughly r = 0.05, meant social media explained about 0.25% of the variance in happiness.

Compare that to medical trials. The famous physicians' aspirin study of the 1980s enrolled over 22,000 doctors and found a highly significant reduction in heart attacks. But the absolute risk dropped from about 1.7% to 1.0% over five years. Significant? Yes. Life-changing for any individual? That depends on how you weight a 0.7 percentage point shift.

The lesson isn't that p-values are useless. It's that they answer a preliminary question—is there likely something here?—not the decision-relevant one: how much is here, and does it matter for what I care about?

Takeaway

A p-value tells you whether an effect probably exists. It tells you nothing about whether the effect is large enough to matter. Always ask the second question after answering the first.

Effect sizes come in several standardized flavors, each suited to different data. Cohen's d expresses the difference between two group means in units of standard deviation. A d of 0.2 is conventionally small, 0.5 medium, and 0.8 large—though these labels are rules of thumb, not laws.

For relationships between continuous variables, the correlation coefficient r ranges from -1 to +1. Squaring it gives the proportion of variance explained. An r of 0.3 sounds respectable until you realize it accounts for only 9% of the variation—leaving 91% unexplained by your predictor.

In categorical outcomes, odds ratios and risk ratios quantify how much one group's odds or probability differ from another's. An odds ratio of 2.0 means the event is twice as likely in one group. But be careful: odds ratios can look dramatic even when absolute risks remain tiny. Doubling a 1-in-10,000 risk produces an alarming headline and a negligible personal concern.

Each metric answers the same underlying question in a different currency: how far apart are these groups, or how tightly do these variables move together? Learning to read them is like learning to read prices in a foreign currency—the numbers only mean something once you know the exchange rate.

Takeaway

Effect sizes are the exchange rate between statistical output and real-world meaning. Without them, you're reading prices without knowing the currency.

An effect size in isolation is still just a number. To decide whether it matters, you need reference points—benchmarks from your field, comparisons to known interventions, or estimates of what change would be practically consequential.

In education research, an effect of d = 0.4 was once considered the average yearly learning gain, but more recent syntheses suggest it varies dramatically by age and outcome. In medicine, a small effect on a serious outcome like mortality can dwarf a large effect on something inconsequential. Context beats convention every time.

One useful strategy is to translate effects into concrete units. Instead of reporting d = 0.3, say the treatment group scored, on average, four points higher on a 100-point test. Instead of an odds ratio of 1.5, report that the risk rose from 4% to 6%. These translations force honesty about magnitude and make claims accessible to non-specialists.

Another strategy is comparison. How does this intervention's effect compare to established alternatives? To lifestyle changes? To doing nothing? An educational program with d = 0.15 might be underwhelming until you learn that most interventions in the same domain hover near d = 0.05, and this one costs a fraction of the others.

Takeaway

Numbers become meaningful only through comparison. Always ask: compared to what, and by what standard of consequence?

The next time you encounter a scientific claim, resist the pull of the p-value. Ask instead: how large is the effect, in what units, and compared to what? These three questions cut through most of the noise in popular science reporting.

Effect sizes won't rescue bad studies or replace careful design. But they restore proportion to conversations that too often mistake detectability for importance.

Statistical significance is a screening question. Magnitude is where the real thinking begins—and where scientific findings translate into decisions worth making.