Setting Up Two-Group Experiments, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 12.4 Free with a free account

Setting Up Two-Group Experiments

Research Methods in Psychology · Chapter 12: Experimental Design FundamentalsSection 12.4 · lesson 73 of 949 min · free with a free account

← The Purpose and Power of Control GroupsNext lesson: The Logic Behind Matched-Group Designs →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

The simplest experimental architecture and the questions it can answer.

  • Describe posttest-only and pretest-posttest structures
  • Weigh the strengths and weaknesses of each variant
  • Choose a variant from the question and the constraints
  • Name what a two-group design cannot do

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,209 words). Read along, or search it.

Experimental Design Fundamentals

Setting Up Two-Group Experiments

You can build an experiment out of almost nothing, and it will still work. This section is about that minimum, the two-group experiment. By the end you will describe both of its structures. The posttest-only version, and the pretest posttest version. You will identify the strengths and weaknesses of each variant. You will apply random assignment correctly to a two-group design. You will recognize the threats these designs do, and do not, protect against. And you will choose between the variants using the question and the practical constraints.

The Minimum Viable Experiment

Why Start with Two Groups

The two-group experiment is the smallest design that can support a causal inference. Take away anything, and the inference stops working. It contains exactly three things. One independent variable, manipulated across two levels. Random assignment of participants between those two levels. And a measured outcome. That is it. Nothing else is required for a valid causal claim. And despite its simplicity, this design underlies a remarkable share of published research. So understanding it well is both a starting point and a foundation. Every more complex design in this chapter is an elaboration of it.

Two Variants

Measured Once, or Measured Twice

Two-group experiments come in two basic forms. The distinction is a single question. Are participants measured before the manipulation, as well as after? In the posttest-only form, you assign at random, treat, and measure once. In the pretest posttest form, you measure first. Then assign at random, treat, and measure a second time. One extra measurement occasion. That is the whole structural difference. But it changes what you can verify, what you can compute, and what you risk. So the choice is not cosmetic.

Variant One

The Posttest-Only Design

Take the posttest-only design first. Participants are randomly assigned to one of two conditions. They are exposed to their assigned treatment. Then the dependent variable is measured. There is no measurement beforehand at all. That means the logic of this design depends entirely on random assignment. Because participants are placed into conditions by chance, any difference afterwards has one home. It can be attributed to the manipulation, rather than to pre-existing differences. The design is efficient and fast. It also avoids all the complications that pretesting brings with it. What it gives up is participant-level information about how much each individual changed.

Variant Two

The Pretest-Posttest Design

Now the pretest posttest design. Participants are measured on the dependent variable before the manipulation. They are randomly assigned, exposed to the treatment, and measured again afterward. That extra occasion buys three things. First, the researcher can verify that random assignment produced equivalent groups at baseline. You are no longer trusting the procedure. You are checking it. Second, change scores can be computed for each participant individually. Subtract the pretest from the posttest, and you have a personal effect size. Third, pretests enable stratified analysis. You can examine whether the treatment works differently for people who started high or low.

Head to Head

Strengths and Weaknesses

Put the two variants side by side. Posttest-only is simple, with fewer measurement occasions. It avoids testing effects, and it is ideal when pretesting is impractical or sensitizing. Against that, it cannot verify baseline equivalence. It cannot compute individual change, and it has less power for small effects. Pretest posttest allows verification of baseline equivalence. It supports the analysis of change, and it is useful when sample size is modest. Against that, the pretest may sensitize participants. Testing effects can inflate apparent change, and the design needs more time and resources. Notice that neither column is a winner. Each strength has a matching cost.

The Hidden Cost

Pretest Sensitization

Sensitization deserves a closer look, because it is easy to underestimate. Asking participants about a topic before the manipulation can change how they respond to it. Consider a questionnaire about racial attitudes, given before the study proper. It may prime participants to think about the topic in ways they otherwise would not. That alters their reactions to the material that follows. The measurement has become part of the manipulation. So when sensitization is likely, a posttest-only design is usually preferable. But when the dependent variable is stable, and measurement is unobtrusive, the calculation flips. Then the pretest adds information without introducing distortion.

Choosing

Four Considerations

Four considerations settle the choice in practice. The first is sample size. Smaller samples benefit more from a pretest, because it improves statistical power. It does that by accounting for individual differences already present. The second is the sensitivity of the topic. If pretesting is likely to alter responses, posttest-only avoids that contamination. The third is the stability of the outcome. Stable outcomes can usefully be pretested. Volatile outcomes produce noisy pretests, which reduce precision rather than improve it. The fourth is whether you need to verify the random assignment itself. When samples are modest, or stakes are high, that check is worth the extra session.

Power

Why a Pretest Buys Precision

One term in that list deserves a definition, because it recurs everywhere. Statistical power is the probability that a study will detect an effect. An effect of a specified size, assuming one truly exists. So a low powered study can miss a real effect entirely, and often does. A pretest raises power by a specific mechanism, worth understanding. It lets the analysis account for individual differences that were already there. Removing that variation from the comparison makes a real effect easier to see. Which is why the pretest matters most in small samples. With a large sample, random assignment alone already produces near-equivalent groups. So the extra measurement earns less, and may not be worth its cost.

The Boundary

What Two-Group Designs Cannot Do

The two-group experiment is powerful, and it is bounded. Knowing the boundary is what stops you from over-claiming. It can establish that a manipulation produced a difference between conditions. But it cannot decompose that difference into underlying mechanisms. It can compare an active treatment against a single comparison. But it cannot examine how the treatment interacts with other variables. And it can show that an effect exists. But it cannot reveal a dose response relationship across multiple levels of the independent variable. For those questions you need more conditions. That is the job of the multi-group designs and the factorial designs later in this chapter.

Not a Stepping Stone

The Case for Staying Simple

Researchers sometimes treat the two-group design as a stepping stone. Something to outgrow, and replace with something more complex. In practice, it remains the right tool for many of the questions psychology asks. Does an intervention work? Does a manipulation produce a predicted shift? Does the presence of a stimulus change behavior? Every one of those is a two-group question. And here is the comparison worth remembering. A clean two-group experiment, with strong randomization and adequate power, is often more informative. More informative than an over-specified design with underpowered cells. Complexity that outruns your sample size is not sophistication. It is noise.

Vocabulary

The Terms This Section Depends On

Four terms carry most of the weight in this section. Baseline equivalence is two groups being statistically similar on the outcome measure. Similar before the manipulation begins, not after. Pretest sensitization is the risk that taking a pretest changes later responses. It is the reason a more informative design can sometimes be the worse choice. A change score is computed by subtracting the pretest from the posttest. One number per participant, rather than one number per group. And statistical power is the probability of detecting a real effect of a given size. Hold those four, and the rest of the chapter has somewhere to attach.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Setting Up Two-Group Experiments

What is a two-group experiment in psychology?
A two-group experiment is the simplest design that can support a causal conclusion. It needs one independent variable manipulated at two levels, random assignment of participants to those two levels, and a measured outcome. Because chance decides who gets which condition, a difference in the outcome can be attributed to the manipulation rather than to pre-existing differences. Multi-group and factorial experiments build on this same structure.
What is the difference between a posttest-only design and a pretest-posttest design?
The difference is one extra measurement: a pretest-posttest design measures the dependent variable before the manipulation as well as after it, while a posttest-only design measures it once, afterward. The pretest lets researchers check baseline equivalence, compute a change score for each participant (posttest minus pretest), and see whether the treatment works differently for people who started high or low. Posttest-only is faster and avoids testing effects.
What is pretest sensitization?
Pretest sensitization is the risk that taking a pretest changes how participants respond to the treatment or to the later measure. For example, a racial attitudes questionnaire given before a study can prime people to think about the topic in ways they otherwise would not, so the measurement becomes part of the manipulation. When sensitization is likely, a posttest-only design is usually the better choice.
When should a researcher add a pretest to a two-group experiment?
A pretest is most useful when the sample is small, the outcome is stable, measurement is unobtrusive, and verifying that random assignment produced equivalent groups matters. In small samples a pretest raises statistical power by accounting for individual differences that already existed. It helps less when pretesting could alter responses on a sensitive topic, when the outcome is volatile, or when a large sample already makes the groups nearly equivalent.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →