Linking Population, Sample, and Generalization, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 8.2 Free with a free account

Linking Population, Sample, and Generalization

Research Methods in Psychology · Chapter 8: Sampling and GeneralizationSection 8.2 · lesson 46 of 9410 min · free with a free account

← The Core Principles Behind SamplingNext lesson: Probability vs. Non-Probability Sampling Approaches →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

What it means to extend findings beyond the people who were actually studied.

  • Define generalization and why it drives most research
  • Distinguish statistical, theoretical and analytic generalization
  • Identify the conditions that justify each kind
  • Recognize the common over-generalizations and their warning signs

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,351 words). Read along, or search it.

Sampling and Generalization

Linking Population, Sample, and Generalization

A study reports that a treatment works. Works for whom, exactly? For the people in the study, certainly. That much is measured. Everything beyond that is an inference, and inferences can be wrong. This section is about which of those inferences you are entitled to make. You will define generalization, and see why it drives most quantitative research. You will distinguish three different kinds of it. You will learn the conditions that justify each one. And you will recognize the over-generalizations that appear constantly in published work.

The Whole Point

Nobody Studies a Sample to Learn About the Sample

Researchers do not study samples in order to learn about samples. They study samples to learn about something larger. Populations, processes, mechanisms, settings. The sample is a stand-in for all of that. Generalization is the name for that inferential leap. From what we found in the people we studied, to what is probably true beyond them. Consider what research would look like without it. Every study would describe only the particular people who happened to participate. It would be a diary, not a science, and it would guide nobody. But the leap is not free, and that is the theme of this section. Generalization is justified only to the extent that the sample, the design, and the analysis support it. A study of self-selected online respondents tells us about self-selected online respondents. Extending it to the general public requires assumptions, and those assumptions may not hold.

Not One Move

Three Kinds of Generalization

Here is the first thing most students get wrong about generalization. It is not a single move. It is a family of related moves. Three forms recur throughout social science writing. Statistical generalization claims the sample's findings hold in the broader population. That requires probability sampling from a defined sampling frame. Theoretical generalization claims the mechanism identified in the study operates more broadly. That requires a coherent theoretical account of why it should. Analytic generalization claims that insights from one case illuminate similar cases. That requires detailed description, so readers can judge transferability for themselves. Notice the third column. Each kind is earned by different evidence. Having one does not give you the others.

By the Numbers

Statistical Generalization

Take statistical generalization first, because it is the most formal. It extends sample findings to a defined population, using inferential statistics. And it rests entirely on one requirement. The sample must have been drawn by probability sampling. Every member of the target population needed a known, non-zero chance of selection. Here is the part worth pausing on. Confidence intervals, margins of error, and p-values all assume probability sampling. Run them on a convenience sample and the software will happily produce numbers. The arithmetic still works. The interpretation does not. When the assumptions genuinely are met, statistical generalization is rigorous and powerful. That word when is doing real work, and it is not a formality.

By Argument

Theoretical and Analytic Generalization

The other two kinds are earned by argument rather than by arithmetic. Theoretical generalization extends the underlying mechanism, not the specific finding. Take a laboratory study showing that loss aversion shapes risky decision-making. That does not automatically generalize to all decisions in all settings. The claim is narrower, and more careful, than it first appears. The cognitive mechanism will operate, with adjustments, in other contexts where similar conditions hold. It is justified by argument from theory, not by the sampling procedure alone. Analytic generalization is different again, and it is most common in qualitative research. It is sometimes called case generalization, or transferability. The researcher provides such rich description of the studied case that readers can judge for themselves. Whether the findings might apply to other cases they encounter. Which means responsibility rests partly with the reader. That reader must evaluate how similar their context really is to the study's context.

Earning the Claim

The Conditions Behind Each Kind

Each kind of generalization rests on a different evidentiary foundation. So each has its own list of conditions. For a statistical claim you need probability sampling. You also need full coverage of the target population by your frame. And a sample size adequate to the design you are running. For a theoretical claim you need a clear specification of the mechanism. Not a vague gesture at one, but a statement of what is actually supposed to be happening. Plus theoretical reasons to expect that mechanism to operate elsewhere. For an analytic claim you need thick description of the case. And explicit articulation of the conditions under which the findings might transfer. The principle underneath all three is the same. Claim only what your evidence supports, and no more.

The Classic Slip

Do Not Mix the Types

Now the single most common error in this whole area. Sliding from one kind of generalization to another, without noticing you did it. Watch how it happens in a real paper. A laboratory experiment demonstrates a mechanism. That is a theoretical generalization. It is well supported, and the design is appropriate for it. Then the discussion section claims the finding applies to a national population. That is a statistical generalization. And nothing in the design licensed it. The second claim does not follow from the first. This slide happens quietly, usually in the last paragraph, often to a press release. Each kind of generalization has to be earned on its own terms. Demonstrating a mechanism is not the same as estimating a population value.

Five Warning Signs

Common Over-Generalizations

Five over-generalizations account for most of the trouble in practice. First, from a convenience sample to the public. Generalizing findings from undergraduate volunteers to all adults. Second, from one cultural setting to all human beings. Treating findings from Western, educated, rich populations as if they described everyone. Third, from a particular moment to all times. Reading economic findings from a boom period as though they described every period. Fourth, from a single study to a settled fact. Treating a striking finding as established before anyone has independently replicated it. Fifth, from an effect's existence to its size. Treating a statistically significant but small effect as though it were practically significant. That last one is the quietest, and it drives an enormous amount of bad advice.

A Field-Wide Problem

The WEIRD Problem

One over-generalization has been documented so thoroughly it has a name. A large share of empirical research draws its samples from WEIRD populations. The letters stand for Western, Educated, Industrialized, Rich, and Democratic. These populations represent only a fraction of humanity. Yet findings from them have often been treated as describing human nature in general. Cross-cultural research has tested that assumption directly. And many supposedly universal findings turned out to vary substantially across cultures. Which calls those generalizations into question. Notice why this is a generalization problem, not merely a sampling problem. The sampling was often fine for the population it described. The error was in how far the conclusion was allowed to travel. Responsible work today either samples more diversely, or explicitly limits its claims.

Honest Reporting

Who Did You Not Study?

There is a simple discipline that prevents most of this. An honest report answers two questions about its sample. Who was studied? That describes the participants who provided the data. Every paper answers that one, because it flatters the study. Who was not studied? That describes the populations the findings cannot safely reach. Not without further work, at any rate. Strong research takes both questions equally seriously. One more practice belongs in the same category. Replication is the independent reproduction of a finding by other researchers. It strengthens claims of generality and robustness in a way no single study can. Treat it as part of how generalizable knowledge gets built, not as an optional extra.

A Reader's Checklist

Six Questions That Test Generalizability

Finish with a checklist you can run on any paper you read. From what target population was the sample drawn? That establishes the population statistical generalization could even apply to. How was the sample selected? That separates probability sampling from convenience or purposive sampling. Who was excluded, and why? That identifies populations the findings cannot describe. What conditions did the study create or observe? That anchors the findings to particular settings, times, and contexts. What theoretical mechanism is being claimed? That lets you assess whether the mechanism should be expected to generalize. And finally, has the finding been replicated? Independent replications strengthen claims of robustness across settings. Six questions. They will tell you more about a paper than its abstract will.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Linking Population, Sample, and Generalization

What are the types of generalization in research?
Three common types are statistical, theoretical, and analytic generalization. Statistical generalization extends sample findings to a defined population and requires probability sampling from a sampling frame that covers it. Theoretical generalization claims a mechanism operates more broadly and requires a clear theoretical account of why. Analytic generalization, or transferability, lets readers judge whether insights from a richly described case apply to similar cases. Each must be earned separately.
Can you generalize from a convenience sample?
Not in the statistical sense, because confidence intervals, margins of error, and p-values assume probability sampling, where every member of the target population had a known, non-zero chance of selection. Software will still produce these numbers for a convenience sample, but they cannot be read as population estimates. Findings from undergraduate volunteers, for example, cannot simply be extended to all adults without further assumptions or replication.
How can you tell whether a study's findings generalize beyond its sample?
You judge generalizability by asking how the sample was drawn, who was left out, and whether the finding has been replicated. Identify the target population the sample came from and whether selection used probability sampling or convenience or purposive methods. Ask who was excluded and why, and note the settings, times, and conditions the study created or observed. Check whether a clear theoretical mechanism explains why the effect should hold elsewhere, and whether independent researchers have reproduced it.
What are common examples of over-generalization in research?
Common over-generalizations include extending convenience samples to the public, findings from one culture to all humans, results from one time period to all times, a single unreplicated study to settled fact, and a statistically significant effect to a practically important one. A related error is sliding from a laboratory finding about a mechanism to a claim about a national population, which the design never supported.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →