The Logic Behind Matched-Group Designs, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 12.5 Free with a free account

The Logic Behind Matched-Group Designs

Research Methods in Psychology · Chapter 12: Experimental Design FundamentalsSection 12.5 · lesson 74 of 949 min · free with a free account

← Setting Up Two-Group ExperimentsNext lesson: Designing Studies with Multiple Groups →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

Pairing participants to sharpen the comparison between conditions.

  • Explain how matching improves precision
  • Distinguish matched-pair from matched-block designs
  • Apply matching using one or more baseline variables
  • Recognize the analytic consequences of a matched design

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,193 words). Read along, or search it.

Experimental Design Fundamentals

The Logic Behind Matched-Group Designs

Random assignment is powerful, but it works on average, and averages take time to arrive. This section covers a technique that helps randomization along, matching. By the end you will explain the rationale for matched-group designs. You will see specifically how they improve precision. You will distinguish matched-pair designs from matched-block and other matched approaches. You will apply matching procedures using one or more pre-treatment variables. You will identify when matching adds value, and when it does not. And you will recognize what a matched design does to your choice of statistical test.

The Problem

Two Costs of Individual Differences

Random assignment distributes individual differences across conditions on average. On average is doing a lot of work in that sentence. With finite samples, chance can still produce groups that differ noticeably. And the smaller the sample, the larger that residual imbalance is likely to be. That is the first cost, and it is the obvious one. The second cost is subtler, and often more damaging. Individual differences also contribute to the variability within each group. That variability is the background against which you have to see the treatment effect. So even when the groups are perfectly equivalent on average, you have a problem. The noise produced by within-group differences can obscure a real effect entirely.

The Fix

What Matching Buys

Matched-group designs address both problems at once. The idea is simple. Pair participants who are similar on relevant baseline characteristics. Then assign the members of each pair to different conditions. Notice what changes about the researcher's relationship to those characteristics. They are now controlled directly, rather than left to randomization to even out. Directly is the operative word. You are no longer waiting on the law of large numbers. The result is a more precise comparison between conditions. And often, a more powerful test of the manipulation's effect. Same participants, same manipulation, better resolution.

The Procedure

Matching in Three Steps

The procedure has three steps. First, measure all participants on one or more variables. Variables you expect to correlate with the dependent variable. Often that is the dependent variable itself, measured at baseline. Second, pair each participant with the most similar other participant. Similar on the matching variable, specifically. Pairs that cannot be matched within an acceptable tolerance may be excluded. Third, within each pair, randomly assign one member to treatment and the other to control. And here is the step people forget. The pair structure is preserved through the analysis. Instead of comparing group means, the researcher compares within-pair differences.

Variant One

Matched-Pair Designs

The simplest matched design pairs each participant with exactly one other. With two conditions, that gives you a sample of n pairs, and two n participants. Matched-pair designs work well under two conditions. When sample sizes are modest, and when a strong baseline variable exists. That variable also has to be easy to measure, or the whole thing gets expensive. Their analytic counterpart is the paired-samples test. It exploits the matching to reduce error variance. Which is exactly the point of building the pairs in the first place.

Variant Two

Matched-Block Designs

What if you have more than two conditions? Then participants are grouped into blocks, rather than pairs. The block size equals the number of conditions. Within each block, all members are similar on the matching variable. And one member is randomly assigned to each condition. Three conditions produce blocks of three. Four conditions produce blocks of four. The logic scales naturally, with nothing new to learn. And it preserves the precision benefits across multi-condition designs. A matched pair is really just a block of size two.

Variant Three

Multivariate Matching

Matching can also be performed on several variables simultaneously. A researcher might match participants on age and pre-test score together. Or compute a propensity score, summarizing several variables into one number. Then match on that single dimension instead. Multivariate matching is especially valuable in one situation. When several baseline variables each contribute meaningful variance to the outcome. But it becomes harder as the number of matching variables grows. Very tight matches across many dimensions become impossible to find. At some point you are searching for a twin, and there is only one person in the room.

A Critical Distinction

Matching Is Not a Substitute for Randomization

Now for a mistake worth stopping on, because it is common and serious. Treating matching as an alternative to random assignment. It is not an alternative. It is a supplement. Matching pairs participants who are similar on the matching variable. Random assignment is still required, to decide which member of each pair gets which condition. Consider what happens when you match without randomizing. Some non-experimental studies match treated and untreated groups on observable variables. That addresses only the variables explicitly matched on. Everything you did not measure is still free to explain the result. Random assignment addresses everything, including variables nobody thought to measure.

When It Helps

Matching Is Not Always Worth It

Matched designs are not always worth the additional effort they require. Their value depends on the strength of the baseline to outcome relationship. When that correlation is strong, matching helps a great deal. It removes a substantial portion of error variance from the comparison. When the correlation is weak, the benefit is limited. Little error variance is removed, and the effort outweighs the benefit. Sample size matters too, in the direction you would expect. With a small sample, random assignment alone is most likely to leave a noticeable imbalance. So matching helps most exactly where randomization is weakest. With a very large sample, random assignment already produces near-equivalent groups.

The Practical Test

Can You Actually Measure It

Two practical conditions finish the picture. If a suitable matching variable is available, matching helps. It is feasible, and likely to produce well-balanced pairs. But if the matching variable is hard to measure, matching does not help. The cost of measurement outweighs the precision you gain. So run two questions, in order, before committing to a matched design. Does the matching variable predict the outcome strongly? And can you measure it cheaply enough to be worth the trouble? A no to either question ends the case for matching.

Analysis

A Matched Design Changes the Test

A matched design changes the structure of your data. And that changes which statistical test is appropriate. Pairs are not independent observations. Their members share variance by construction, because you built them that way. So standard analyses that assume independence cannot be applied without distortion. That rules out the independent samples t test. It also rules out between-subjects analysis of variance. Instead, use paired-samples tests for two-condition designs. And repeated measures analysis of variance for multi-condition designs, with the block as the random factor. When the matching variable correlates strongly with the outcome, these tests are more powerful. So the design and the analysis have to be chosen together, not one after the other.

When Matching Fails

Decide the Tolerance in Advance

Finally, what happens when matching fails. Sometimes participants in one condition cannot be matched to anyone in another. At least not within an acceptable tolerance. The obvious move is to exclude the unmatched participants. But that reduces your sample size, which you were already short of. And it may threaten the representativeness of the sample that remains. The people who are hardest to match are rarely a random subset. So the researcher must decide in advance how strict the matching tolerance will be. And decide in advance how to handle participants who cannot be matched. Reporting those decisions transparently is part of good design practice.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About The Logic Behind Matched-Group Designs

What is a matched-groups design?
A matched-groups design pairs or groups participants who are similar on a baseline characteristic that predicts the outcome, then randomly assigns the members of each pair or block to different conditions. This controls that characteristic directly instead of leaving it to chance. The result is better balance, less error variance, and often a more powerful test, especially in small samples where randomization alone can leave a noticeable imbalance.
How do you create matched pairs or blocks in an experiment?
First, measure all participants on a variable expected to correlate with the dependent variable, often a baseline score on the outcome itself. Second, pair each person with the most similar other person; with more than two conditions, form blocks whose size equals the number of conditions. Third, randomly assign one member of each pair or block to each condition. The pair or block structure is then kept through the analysis.
Can matching replace random assignment?
No, matching supplements random assignment rather than replacing it. Matching makes pair members similar on the variables that were measured, but random assignment is still needed to decide which member receives which condition. Non-experimental studies that match treated and untreated groups without randomizing control only the variables they matched on, so anything unmeasured can still explain the result. Random assignment also balances unmeasured variables, on average.
What statistical test should be used for a matched-pairs design?
A two-condition matched-pairs design is analyzed with a paired-samples t test, and a matched design with more conditions uses repeated-measures analysis of variance. Matched observations are not independent, so an independent-samples t test or between-subjects ANOVA would distort the results. Paired analyses compare within-pair differences, which removes variance explained by the matching variable and gives more power when that variable correlates strongly with the outcome.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →