Techniques for Controlling Extraneous Variables, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 10.7 Free with a free account

Techniques for Controlling Extraneous Variables

Research Methods in Psychology · Chapter 10: Internal Validity and Its ThreatsSection 10.7 · lesson 64 of 9411 min · free with a free account

← Internal Threats: Selection, Maturation, and Their Combined EffectsNext lesson: Building Blocks and Requirements of External Validity →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

From randomization to replication: the toolkit for protecting causal inference.

  • Distinguish design, procedural and statistical control
  • Apply random assignment, matching and blocking
  • Apply standardization and counterbalancing
  • Recognize what statistical adjustment cannot fix

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,394 words). Read along, or search it.

Internal Validity and Its Threats

Techniques for Controlling Extraneous Variables

This chapter has spent six sections on what can go wrong. This final section asks the opposite question. What do researchers actually do to keep things from going wrong? You will distinguish design, procedural, and statistical control. You will apply random assignment, matching, and blocking. You will apply standardization and counterbalancing during data collection. And you will learn the single most important limitation of statistical adjustment. Which is that it cannot repair a design that was broken before the data existed.

Turning It Around

Three Families of Control

The techniques fall into three broad families. Design control works by structuring the study so that threats cannot operate. Procedural control works by standardizing how the study is administered. So that irrelevant variation is held constant across conditions. Statistical control works at the analysis stage, after data collection. It partitions out the contribution of measured covariates. Notice that those three operate at three different moments in a project. Before, during, and after. Strong studies typically combine all three. Because each family addresses a different kind of threat, and none is sufficient alone.

Design Control

Random Assignment

Design control is the strongest of the three. It prevents threats from operating in the first place. And its most important technique is random assignment. You allocate participants to conditions by a chance procedure. A random-number generator, or in a small study, a coin flip. It equates groups in expectation on every variable simultaneously. Including variables the researcher has not measured, or even thought of. With a sufficiently large sample, the groups will differ only by sampling variability. That protection against unmeasured confounds is unique to randomization. Which is exactly why it is the defining feature of the true experimental design.

A Constant Confusion

Assignment Is Not Sampling

One confusion is worth clearing up before going further. Random assignment must be distinguished from random sampling. Random sampling concerns how participants are drawn from a population. It supports external validity, the reach of your conclusions. Random assignment concerns how participants already in the study are placed into conditions. It supports internal validity, the trustworthiness of your causal claim. A study can have one without the other, in either combination. Both are desirable, for entirely different reasons. And confusing them leads people to claim the wrong kind of strength for their design.

Design Control

Matching and Blocking

When random assignment is impossible, two design techniques take its place. Matching pairs participants across conditions on specified variables. That ensures the groups are equivalent on those variables at baseline. It is most useful in quasi-experimental and applied studies. Blocking is matching's larger-scale relative, and it is more powerful. Participants are first sorted into blocks based on a relevant variable. Age groups, for instance, or baseline scores. Then they are randomly assigned to conditions within each block. So blocking guarantees balance on the blocking variable across conditions. And it reduces the noise that variable contributes, which improves statistical power. The Solomon four-group design and the Latin-square design are sophisticated blocking schemes. It protects only against confounds on the matched variables. So choose those variables strategically, picking the ones most strongly related to your outcome.

Design Control

Within-Subjects Designs

Within-subjects designs, also called repeated-measures designs. They use each participant as their own control. Every participant is exposed to every condition. And that has a remarkable consequence. Because the same person appears in every condition, individual differences are perfectly equated. Not approximately equated. Perfectly. So selection, and most maturation threats, are neutralized by design. There is no free lunch, of course. Within-subjects designs introduce order effects. The position in which a condition is encountered may itself affect performance. Which requires counterbalancing, a procedural remedy we come to next.

Procedural Control

Standardization and Counterbalancing

Procedural control techniques are applied during data collection. Standardization means delivering the study the same way to every participant. Scripted instructions, identical materials, equivalent equipment, consistent session structure. The closely related principle is constancy of conditions. Hold the physical and temporal setting as nearly identical as you can. Same room, same time of day where possible, same lighting, same materials. You vary the order in which conditions are presented across participants. Full counterbalancing uses every possible order equally often. But the number of orders is the factorial of the number of conditions. Latin-square designs offer a more economical alternative. Each condition appears equally often in each ordinal position, without every permutation. And be clear about what counterbalancing achieves. It does not eliminate order effects. It distributes them equally across conditions, so they do not bias the comparison.

Procedural Control

Blinding and the Two Checks

Blinding is the second major procedural technique. It conceals the experimental condition from participants, experimenters, or analysts. Single-blind designs protect against participant expectancy effects. Double-blind designs additionally protect against experimenter effects and recording bias. Triple-blind designs extend the protection to the data analysts. A manipulation check verifies that your manipulation actually did something. That it altered the construct it was intended to alter. Did participants in the high-arousal condition really feel more aroused? Manipulation checks primarily support construct validity. But they also help diagnose internal-validity problems. By ruling out the explanation that the manipulation was simply ineffective. And attention checks identify participants who did not engage with the task. Their data can be excluded, so uninformative responses do not dilute the comparison.

Statistical Control

Adjusting at the Analysis Stage

Statistical control operates after data collection is complete. It removes variation associated with measured covariates. So the remaining variation can be attributed more cleanly to the manipulation. It adjusts the dependent variable for one or more covariates before testing the effect. Covariates are variables expected to influence the outcome but not theoretically central. Baseline scores, age, prior experience. More general regression techniques allow control for several covariates at once. And hierarchical, or multilevel, models handle a specific structural problem. Participants nested within groups, such as classrooms, clinics, or schools. Propensity-score methods are designed for observational studies. Each participant's propensity to receive the treatment is estimated from observed characteristics. Then similar participants are matched or weighted in the analysis. That approximates the balance random assignment would have given you. But only on the variables included in the model.

A Warning

Statistical Control Cannot Repair a Broken Design

Which leads to the most important warning in this section. It is tempting to imagine that statistical adjustment can rescue any weak study. You simply control for the relevant covariates and the problem goes away. That temptation should be resisted, and here is why. Statistical adjustment can only correct for variables that have been measured. And even then, only if the statistical model is correctly specified. Unmeasured confounds remain exactly where they were. And the researcher rarely knows for certain which confounds matter. So strong design choices do work that statistical adjustment cannot replace. Random assignment where possible. Careful matching where it is not. The principle fits into five words. Adjust where you must, design where you can.

Putting It Together

Which Techniques for Which Concern

For pre-existing group differences, reach for random assignment first. If that is impossible, match or block on the strongest predictors of the outcome. Then add statistical adjustment for measured baseline differences. For order effects in repeated measures, use counterbalancing. Latin-square or full permutation, plus statistical tests for order effects in the analysis. For experimenter and demand effects, use double-blinding where feasible. Plus scripted standardization, and cover stories that conceal the hypothesis without violating consent. For history and maturation, use a control group over the same interval and setting. And for long-term studies, add multiple measurement waves. For differential attrition, reduce burden and report attrition by condition. Then run an intent-to-treat analysis, and sensitivity analyses for plausible attrition patterns. And for regression to the mean, use a comparison group selected on the same criterion. Plus multiple baseline measurements, which let you estimate the regression component directly.

The Final Defense

Replication, and the Limits of Control

No single study, however well controlled, settles a causal question. Every study is vulnerable to some configuration of threats. Every researcher carries blind spots, and every sample is to some extent peculiar. So the most powerful defense is replication. Repeating the study with different samples, settings, and research teams. Direct replications test whether the same procedure produces the same result. Conceptual replications test whether different operationalizations produce convergent results. A finding that survives both has withstood what no individual study can eliminate. And it is honest to acknowledge that perfect internal validity is unattainable. Every experimental setting is in some respects artificial. Every measurement is approximate, and every randomization occurs in a finite sample. The realistic goal is not perfect control, but adequate control. Enough to make the alternative explanations implausible. Next, the chapter turns outward, to how far a finding can travel beyond its own study.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Techniques for Controlling Extraneous Variables

What are the three types of control in experimental research?
Researchers control extraneous variables through design control, procedural control, and statistical control. Design control structures the study so threats cannot operate, mainly through random assignment, matching, blocking, and within-subjects designs. Procedural control standardizes how the study is run, through scripted instructions, constant conditions, counterbalancing, and blinding. Statistical control adjusts for measured covariates after data collection. Each acts at a different stage, so strong studies usually combine all three.
What is the difference between matching and blocking in experimental design?
Matching pairs participants across conditions on chosen variables so the groups are equivalent on those variables at baseline. Blocking first sorts participants into blocks based on a relevant variable, such as age group or baseline score, and then randomly assigns them to conditions within each block. Blocking guarantees balance on the blocking variable, keeps randomization, and reduces the noise that variable contributes, which improves statistical power.
What is a manipulation check?
A manipulation check is a measure that verifies the manipulation actually changed the construct it was meant to change, for example asking whether participants in a high-arousal condition really felt more aroused. It mainly supports construct validity, but it also helps internal validity by ruling out the explanation that a null result came from an ineffective manipulation. It differs from an attention check, which identifies participants who did not engage with the task.
Can statistical control fix a poorly designed study?
No, statistical adjustment cannot fully rescue a weak design. Techniques such as analysis of covariance, regression, multilevel models, and propensity-score methods adjust only for variables that were measured, and only if the model is correctly specified. Unmeasured confounds remain. That is why random assignment, or careful matching when randomization is impossible, does work that statistics cannot replace. The principle is often summed up as adjust where you must, design where you can.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →