Boundaries and Drawbacks of Correlational Studies, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 5.6 Free with a free account

Boundaries and Drawbacks of Correlational Studies

Research Methods in Psychology · Chapter 5: Non-Experimental and Correlational ResearchSection 5.6 · lesson 34 of 9410 min · free with a free account

← Interpreting and Drawing Insights from Correlation DataNext lesson: Getting Started with Qualitative Research →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

Knowing what correlational research cannot do, and why.

  • Identify the six major limitations of correlational research
  • Trace range restriction, non-linearity and outlier effects
  • Avoid the ecological and atomistic fallacies
  • Choose strategies that strengthen a correlational design

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,369 words). Read along, or search it.

Non-Experimental and Correlational Research

Boundaries and Drawbacks of Correlational Studies

This chapter has argued that correlational research is powerful, and often indispensable. This final section argues the other half of the case. Knowing the limits of a tool is part of using it well. And correlational data carry six recurring limits, each with its own failure mode. By the end you will identify all six of them by name. You will trace what range restriction, non-linearity and outliers do to a coefficient. You will separate individual-level from group-level relationships, and avoid the ecological fallacy. You will apply the strategies that reduce these problems. And you will recognize when a correlational design simply cannot answer the question. Many of the loudest errors in social science begin exactly there.

Knowing The Limits

What Correlational Research Cannot Do

Start with the six limits as a set, because they compound each other. First, a correlation alone cannot establish causation. Second, confounding variables can produce spurious associations. Third, range restriction in a selected sample distorts the coefficient. Fourth, standard linear coefficients miss non-linear relationships. Fifth, outliers can inflate a correlation, or hide one. And sixth, relationships can be read at the wrong level of analysis. None of these is fatal to the method. All of them are silent, which is the actual danger. The analysis still runs. The output still looks respectable. Many high-profile errors in social-science communication come from ignoring one of the six.

The Famous Limit

No Causation from Correlation Alone

The first limitation is the one everybody has heard. A correlation alone cannot establish that one variable causes another. Three things can produce an association that looks causal but is not. Reverse causation, where the arrow runs the other way. Third variables, where something else is driving both. And plain chance. Now notice something important about this. It is not a flaw of correlational research. It is a structural consequence of measuring rather than manipulating. So the appropriate response is not to abandon correlational designs. It is to be honest about what they can and cannot conclude.

Confounding Variables

The Most Common Way Findings Mislead

Confounding is the most common reason correlational findings can mislead. A confound is any variable that systematically varies with both the predictor and the outcome. It creates an apparent relationship between them, where none may exist. You can reduce that risk, and researchers routinely do. Measure the confounds you think are likely, and control for them statistically. Partial correlation, multiple regression, and other multivariate techniques all do this. But here is the part that never fully goes away. Statistical control can address only the confounds the researcher thought to measure. The one you did not think of is not in your model. Unmeasured confounds remain a permanent threat to correlational interpretation. Permanent is the right word. No amount of statistical sophistication removes it.

Confounds In Practice

Four Correlations With Obvious Culprits

Four examples make the pattern concrete. The number of firefighters at a blaze correlates with the damage caused. More firefighters, more damage. The confound is the severity of the fire. Bigger fires require more firefighters. Coffee consumption has correlated with heart disease. The confound was smoking. Heavier coffee drinkers were historically heavier smokers. Hospital admission correlates with mortality. The confound is severity of illness. Sicker patients are admitted more readily. Use of fitness apps correlates with better health. The confound is pre-existing motivation among the people who choose to use them. Every one of these looks causal at first glance. That is exactly the problem.

Range Restriction

When the Sample Narrows the Truth

The third limitation is quieter, and it catches good researchers. Range restriction happens when your variables span a narrower range than the population does. The result is a coefficient that underestimates the true population correlation. Sometimes dramatically. Picture a study of SAT scores and first-year university grades. Now run that study at a highly selective university. Every admitted student already has a high SAT score. So the range of values in your sample is narrow. The correlation will appear weaker than it would in a broader population. One that included applicants with lower scores. Ask whether your sample restricts the range of an important variable, and interpret accordingly.

Missing The Curve

What a Straight Line Cannot See

The fourth limitation is one you have already met. Pearson's r captures only the linear component of a relationship. When variables are related in a curved pattern, the linear correlation can be near zero. A U-shape, or an inverted U. And the variables can still be strongly related. Three examples turn up again and again. Physiological arousal and task performance, known as the Yerkes-Dodson law. Age and various cognitive abilities, which climb and then fall across the life-span. And income and life satisfaction, at very high income levels. So a coefficient near zero is not proof that there is no relationship. It is proof that there is no straight-line relationship. Those are different claims.

Look First

Anscombe's Quartet and the One-Number Summary

Which leads to the single most practical rule in this section. Always look at the scatter-plot. A correlation coefficient is a one-number summary. A scatter-plot is a thousand-word picture. Two datasets with identical Pearson's r values can look completely different. Anscombe's quartet is the famous demonstration of exactly this point. Four datasets, matching coefficients, and four markedly different pictures. Computing a correlation without first inspecting the plot is a very common mistake. It is also an entirely avoidable one, which is what makes it worth naming. Look first. Compute second.

Outlier Influence

How One Extreme Value Distorts the Coefficient

Fifth, outliers. An outlier is an extreme value that lies far from the rest of the data. A single one can disproportionately influence Pearson's r. It can inflate a correlation, making it look stronger than the bulk of the data warrants. Or it can attenuate one, concealing a relationship that holds across most participants. Now the part people get wrong. Outliers are not always errors, and should not automatically be removed. They may represent genuine but unusual cases that deserve attention in their own right. Responsible practice is to identify them, and examine their characteristics. Consider their possible causes, then report analyses both with and without them. That way readers can judge how robust your conclusion actually is. Robust methods, such as Spearman's rho, are less sensitive to extreme values.

Level Of Analysis

The Ecological Fallacy

The sixth limitation is about levels, and it is the most interesting of the six. Correlations computed at one level of analysis do not necessarily hold at another. The ecological fallacy is assuming that group-level patterns apply at the individual level. The famous example concerns literacy in the early twentieth century United States. Across states, foreign-born populations were associated with higher literacy rates. Read carelessly, that suggests foreign-born individuals were more literate. At the individual level, the opposite was true. Foreign-born individuals were on average less likely to be literate than native-born ones. The state-level correlation reflected where immigrants settled. They tended to settle in states with higher general literacy rates. It was not a property of the immigrants themselves. The reverse error has a name too. Assuming individual relationships hold at group level is the atomistic fallacy. Both are avoided by matching the level of analysis to the research question.

Strengthening Strategies

How to Shore Up a Correlational Study

None of these limitations can be eliminated. Several strategies substantially reduce their impact. Multivariate analysis with measured confounds addresses confounding. Longitudinal designs that establish temporal precedence address reverse causation. Recruiting from broader populations addresses range restriction. Always plotting and inspecting the data addresses non-linearity and outliers. Robust correlation methods address outliers and non-normality. Matching the level of analysis to the research question addresses the two fallacies. And two more strategies work against everything on that list. Replication across populations and methods. And triangulation with experimental or qualitative evidence. Notice that most of these cost planning rather than money. They are decisions, made early.

Another Method

When a Correlation Cannot Answer the Question

One last piece of judgment, and it is the mature one. Correlational research is the right tool for many questions, but not for all. Reach for another method when a strong causal claim is required. Or when a randomized intervention is feasible and ethical. Or when the goal is to understand mechanisms rather than associations. In those cases, experimental or mixed-method designs may be more appropriate. Methodological maturity includes recognizing when a correlational study cannot deliver what is needed. And then choosing a different approach instead. Together with the survey, observational and archival approaches, this forms an essential toolkit. It is the non-experimental half of social research. Chapter six turns from quantitative non-experimental methods to qualitative inquiry. The methods that capture meaning, context, and lived experience.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Boundaries and Drawbacks of Correlational Studies

What is range restriction in correlation?
Range restriction occurs when a sample covers a narrower range of scores than the population, which usually makes the correlation underestimate the true relationship. For example, at a highly selective university nearly every student has a high SAT score, so the correlation between SAT scores and first-year grades looks weaker than it would among all applicants. Recruiting from broader populations helps reduce the problem.
What is the ecological fallacy?
The ecological fallacy is assuming that a relationship found at the group level holds for individuals. In W. S. Robinson's classic 1950 analysis of early twentieth-century US data, states with more foreign-born residents had higher literacy rates, yet foreign-born individuals were on average less likely to be literate. Immigrants tended to settle in high-literacy states. The reverse error, assuming individual patterns hold for groups, is the atomistic fallacy.
How do outliers affect a correlation coefficient?
A single outlier can inflate a correlation, making it look stronger than most of the data warrants, or attenuate it, hiding a relationship that holds for most participants. Outliers are not always errors and should not be deleted automatically. Responsible practice is to examine them, consider their causes, and report results with and without them. Rank-based methods such as Spearman's rho are less sensitive to extreme values.
Why should you look at a scatterplot before calculating a correlation?
A scatterplot reveals patterns that a single coefficient can hide. Anscombe's quartet shows four datasets with matching correlations that look completely different when plotted. A curved relationship, such as the inverted U linking arousal and performance in the Yerkes-Dodson law, can give a Pearson's r near zero even when the variables are strongly related. Plotting first also exposes outliers and other problems.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →