Building Blocks and Requirements of External Validity

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 11.1 Free with a free account

Building Blocks and Requirements of External Validity

Research Methods in Psychology · Chapter 11: External Validity and GeneralizationSection 11.1 · lesson 65 of 9411 min · free with a free account

← Techniques for Controlling Extraneous VariablesNext lesson: Why Generalization Matters in External Validity →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

The conditions under which a finding can be expected to extend beyond the study itself.

  • Define external validity and its place among the validities
  • Identify the four dimensions of generalization
  • Distinguish target, accessible and achieved populations
  • State the three requirements for a defensible claim

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,465 words). Read along, or search it.

External Validity and Generalization

Building Blocks and Requirements of External Validity

Suppose a study is beyond reproach. The causal claim is airtight. There is still one enormous question left, and it decides whether the finding matters. Does it apply to anybody outside that study? That is external validity, and this chapter is about it. You will define external validity and place it among the other validities. You will identify four separate dimensions along which findings may or may not travel. You will distinguish the target population from the accessible one and the achieved sample. And you will state the three requirements for claiming that a finding generalizes at all.

A Different Question

To Whom, Where, When, and How?

The previous chapter asked one question, over and over. Did the independent variable, and nothing else, produce the change we observed? This chapter assumes that question is settled, and asks a different one. To whom, where, when, and under what circumstances does the claim continue to hold? Take a memory study with second-year undergraduates in a quiet laboratory. It may have established firmly that a particular encoding strategy improves recall. But notice how heavily hedged that finding actually is. For those participants. In that setting. On those materials. At that point in time. Whether the strategy would help older adults is a separate question. So is whether it helps primary-school children, or an office worker preparing a presentation. Or anyone at all, fifty years from now.

Not an Afterthought

Why This Is Not a Secondary Concern

So here is the formal definition. External validity is the degree to which a study's findings extend beyond their origins. Beyond the specific participants, settings, operationalizations, and historical moment. External validity is sometimes treated as a secondary concern. An afterthought to the demanding work of internal-validity control. That treatment is misleading, and it is worth saying why. A finding with perfect internal validity but no external validity is a curiosity. It tells us about a specific population in a specific moment, and nowhere else. Now think about who actually reads social research. Clinicians, educators, policy-makers, and ordinary citizens. Every one of them wants findings that travel. And the work of establishing whether a finding travels is the work of external validity.

Four Dimensions

Four Separate Empirical Questions

External-validity analysis examines four distinct dimensions. Four ways in which a finding may, or may not, extend. Population. The participants, and their characteristics. Setting. The locations and the contexts. Operations. The specific measures and manipulations used. And time. The historical moment, and whether the finding recurs. Each of those is a separate empirical question. Which means a study may be strong on one dimension and weak on another. Treating them separately is what lets you diagnose a finding precisely. You can say where it can confidently be applied, and where it cannot.

Dimension One

Population Generalization

Population generalization concerns the people to whom a finding extends. Take a study conducted with first-year university students in one country. It may not extend to older adults, or to children. It may not extend to people from different cultural or socioeconomic contexts. It may not even extend to students at different institutions. That last one surprises people, and it should not. Institutions select their students, so different institutions hold different populations. This is the dimension most often invoked when researchers worry about a sample. It is also the dimension people stop at. As though fixing the sample settled the other three, which it does not.

Dimension Two

Setting Generalization

Setting generalization concerns the contexts to which a finding extends. A memory effect demonstrated in a quiet laboratory cubicle is a specific achievement. It may not appear in a noisy open-plan office. Or on a smartphone, on a moving train. The same problem appears in clinical work, and with higher stakes. Therapeutic effects shown in a tightly controlled clinical trial may not appear in routine practice. In an ordinary outpatient clinic, scheduling differs, supervision differs, and the patient mix differs. The principle underneath both examples is worth stating plainly. The laboratory or research setting is itself a context. It is not a neutral absence of context, however much it feels like one. So findings that depend on that context will not necessarily appear anywhere else.

Dimension Three

Operations Generalization

Operations generalization concerns the specific measures and manipulations you used. The procedures through which the constructs were operationalized. Suppose one particular questionnaire shows an effect of stress on attention. That finding may not extend to physiological measures of stress. Nor to attention measured behaviorally rather than by self-report. The reason this dimension exists at all goes back to the measurement chapter. The construct stress is not the same thing as any single operationalization of it. Any specific implementation reflects the construct only imperfectly. So establishing generalization across operationalizations is a demanding requirement. It is typically addressed through programs of related studies. Rather than inside any single study, which can only use the measures it used.

Dimension Four

Temporal Generalization

Temporal generalization asks whether a finding holds at different points in time. And the case for worrying about it is simply that the world changes. Cohorts change. Cultural attitudes shift. Technologies replace earlier technologies entirely. A finding about how people respond to advertising in nineteen eighty-five. That may bear limited relevance to twenty twenty-five. Or take a finding about social-media behavior, established in a single year. It reflects platforms that may no longer exist five years later. This dimension is rarely addressed within any single study, for obvious reasons. It emerges as a question whenever findings are applied long after the original work. Which, given publication timelines and textbook cycles, is most of the time.

Three Nested Layers

Target, Accessible, Achieved

The population dimension is best analyzed through three nested layers. The target population is the set of people you ultimately want to draw conclusions about. All adults experiencing chronic insomnia, for instance. Or all primary-school children learning to read. Crucially, the target population is defined by the research question. It is not defined by what happens to be convenient. The accessible population is the subset you can practically reach for recruitment. A study of insomnia might draw from a single sleep clinic. A study of literacy might draw from schools in one district. And the achieved sample is who actually agreed, and stayed to the end. Volunteer bias, attrition, and scheduling constraints narrow things further at that stage. That achieved sample is the only group whose responses contribute to your findings. So external validity depends on how that group relates to the target population.

A Deeper Check

Why the Three-Layer Analysis Matters

Researchers sometimes report that the sample was representative. As if a single check on demographic distribution settled the matter. The three-layer analysis pushes considerably deeper than that. Because a sample can match the accessible population perfectly. While that accessible population differs sharply from the target population. Return to the sleep clinic to see how this plays out. A clinic-based study may draw a sample demographically typical of that clinic. Every box ticked. Age, sex, income, all matching the clinic's own profile. And it may still tell us very little about insomnia in the general population. Because people who present at sleep clinics differ systematically from those who do not. Many people manage their symptoms without ever seeking clinical care. So external-validity analysis must examine each layer of narrowing, not only the last one.

The Requirements

Three Conditions for a Defensible Claim

Three requirements must be met before you can responsibly claim a finding extends. Each is straightforward to state, and demanding to satisfy. The first is internal validity. The within-study causal claim must be sound. It is established by the design and procedural defenses of the previous chapter. The second is adequate representation. Your sample, settings, and operationalizations must adequately represent the targets of interest. That comes from deliberate sampling strategy, multiple settings, and varied operationalizations. The third is replication across conditions. The finding should appear consistently with different participants and in different settings. Which requires a program of related studies, rather than any single study. Note that the first requirement is logically prior to the other two. Without an established causal relationship within the study, there is nothing to generalize. The second and third are matters of degree, and no study fully satisfies either.

The Trade-Off

Resolved Across Studies, Not Within Them

The earlier chapter noted that internal and external validity often pull in opposite directions. Now the source of that tension should be clear. The very features that make a study internally valid are the constraining ones. Tight control of extraneous variables. Standardized procedures. Restricted samples. Each of those limits how confidently the findings extend elsewhere. But the trade-off is not absolute, and that matters. Strong research programs address it across studies, rather than within any one of them. A laboratory study establishes the causal relationship under controlled conditions. A field study then tests whether that relationship survives under naturalistic conditions. And a series of studies with varied populations tests generalization directly. Along the specific dimensions that actually matter for the question. So recognizing the sources of the trade-off lets you manage it deliberately. Rather than treating it as an unavoidable defect of whatever design you chose.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Building Blocks and Requirements of External Validity

What is external validity?
External validity is the degree to which a study's findings extend beyond the specific participants, settings, operationalizations, and historical moment in which they were obtained. A finding with strong internal validity but no external validity describes only one group in one setting at one time. Clinicians, educators, and policymakers depend on findings that travel, so external validity is a central concern rather than an afterthought.
What are the four dimensions of external validity?
External validity can be judged along four dimensions: population, meaning whether findings extend to other people; setting, meaning other places and contexts; operations, meaning other measures and manipulations of the same constructs; and time, meaning other historical periods and cohorts. A study can be strong on one dimension and weak on another. A memory effect found in students in a quiet lab, for example, may not hold for older adults or in a noisy office.
What is the difference between a target population and an accessible population?
The target population is everyone a researcher ultimately wants to draw conclusions about, defined by the research question, such as all adults with chronic insomnia. The accessible population is the subset that can actually be reached, such as patients at one sleep clinic. The achieved sample is who actually agreed and completed the study. A sample can match the accessible population closely yet still differ sharply from the target population.
What is required to generalize research findings?
Three requirements must be met. First, internal validity: the causal relationship within the study must be sound, since otherwise there is nothing to generalize. Second, adequate representation: samples, settings, and operationalizations must reasonably represent the targets of interest. Third, replication across conditions: the finding should recur with different participants and settings, which requires a program of studies. The last two are matters of degree that no single study fully satisfies.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →