The Logic of Pretest-Posttest Designs, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 7.4 Free with a free account

The Logic of Pretest-Posttest Designs

Research Methods in Psychology · Chapter 7: Quasi-Experimental and Time-Based DesignsSection 7.4 · lesson 43 of 9410 min · free with a free account

← Tracking Change: Longitudinal and Cross-Sectional MethodsNext lesson: How Time Shapes Research Design Choices →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

Comparing before and after to estimate the effect of an intervention.

  • Explain the logic of before-and-after comparison
  • Distinguish the three variants by inferential strength
  • Match each variant to the threats it does and does not address
  • Apply techniques that strengthen the inference

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,366 words). Read along, or search it.

Quasi-Experimental and Time-Based Designs

The Logic of Pretest-Posttest Designs

Did the thing we tried actually work? That question drives most evaluation research, and it sounds easy to answer. Measure before, measure after, and compare. This section is about why that comparison is harder than it looks. And about the three designs that make it trustworthy. You will learn the basic logic of before-and-after comparison. You will distinguish three variants, ordered by how much they let you conclude. You will match each variant to the threats it handles, and the ones it does not. And you will learn when a weak design is still the right one to use.

Before and After

The Simplest Question in Applied Research

Real-world questions about interventions take a simple form. Did things change after the intervention, compared with before? Did test scores improve after the new tutoring program started? Did emergency room wait times fall after the triage protocol was reformed? Pretest-posttest designs answer these by measuring the outcome twice. Once before the intervention, once after, and then comparing. The logic is intuitive. The inference is where the difficulty lives. A difference between pretest and posttest can come from many sources. The simple passage of time. Simultaneous external events. Normal development, or regression to the mean. The intervention is only one candidate explanation among several. The whole craft of this design is eliminating the others.

Three Variants

A Ladder of Inferential Strength

Strong pretest-posttest designs add features to rule out those rivals. There are essentially two things you can add. You can add a comparison group. Or you can add more measurement points. Or, best of all, you can add both. That gives three variants, forming a ladder of inferential strength. At the bottom, the single-group design. One group, two measurements. In the middle, the non-equivalent control group design, which adds a comparison group. At the top, the interrupted time series, which adds many measurements over time. Each rung buys you the ability to eliminate a specific set of alternative explanations. Knowing which rung a study stands on tells you how much to believe it.

The Weakest Variant

Single-Group Pretest-Posttest

Start at the bottom of the ladder. The single-group pretest-posttest design measures one group, before and after. There is no comparison group at all. You have almost certainly seen this design in the wild. A company measures employee morale. It introduces a wellness program, and re-measures morale six months later. It is used constantly, because it is practical and cheap. It is also the weakest pretest-posttest variant there is. Without a comparison group, the researcher cannot rule out any of the major threats. Any change might be the intervention. It might equally be something else entirely. So findings from this design should be treated as suggestive, not conclusive.

What Goes Unruled-Out

Five Threats the Single Group Cannot Touch

Five specific threats stay live in a single-group study. History. Other events between the two measurements may have caused the change. If morale improved because the company also got a big contract, that is history. Maturation. Participants may have changed naturally over time. Testing. Taking the pretest itself may have influenced posttest performance. People remember the questions, or they think about the topic differently afterwards. Instrumentation. Changes in the measurement process between waves produce spurious effects. And regression to the mean, which is the trap built into program selection. If you chose the group because their pretest scores were extreme, watch out. Those scores will drift toward average on retest, whatever you do to them. Notice that all five predict improvement. None of them require your program to work.

Adding a Comparison

The Non-Equivalent Control Group Design

Adding a comparison group changes the picture dramatically. In the non-equivalent control group design, two groups are measured at pretest and posttest. Only one of them receives the intervention. The word non-equivalent is doing important work in that name. The groups were not formed by random assignment. They may differ in important ways before anything happens. Even so, the comparison earns its keep, and here is exactly why. If both groups change in the same way, that change cannot be due to the intervention. The comparison group reveals what would have happened without it. That is the counterfactual, and it is the central idea of causal inference. So the treatment effect is the difference in change between the two groups. It is not the change in the treatment group alone. That distinction is everything.

Gains and Gaps

What This Variant Fixes, and What It Leaves

Weigh what this variant fixes against what it leaves open. On the credit side, it controls history and maturation. That holds to the extent that both groups are exposed to the same external events. It controls testing too, as long as both undergo the same measurement process. And it provides a natural way to estimate the size of the effect. All of that, while remaining practical enough for real evaluation work. Now the debit side. The main weakness is selection. Because groups were not randomly assigned, they may differ in ways that affect outcomes. Choose the comparison group badly, from people very unlike the treated group, and it misleads. There is also a subtler problem called selection by history. External events may affect the two groups differently, because of what already differed between them. So a shared event does not always cancel out. Sometimes it lands unevenly.

Adding Time Points

The Interrupted Time-Series Design

The interrupted time-series design strengthens the same logic a different way. Instead of adding a comparison group, it adds many measurement points. You measure the outcome repeatedly before the intervention, to establish a trend. Then repeatedly after, to see whether that trend changes. A genuine effect shows up as a discontinuity at the moment of the intervention. And a discontinuity can take two forms, which are worth naming separately. A level change is an immediate shift at the time of the intervention. A step up, or a step down. A slope change is a change in the rate at which the outcome moves over time. The line bends, starting at the intervention. In either case, the test is the same. The change cannot be explained by extrapolating the pre-intervention pattern forward.

Why It Works

Four Features That Do the Work

Four features do the work in a time-series design. Multiple pre-intervention measurements establish the baseline trend. They reveal maturation, seasonal patterns, and any trend already underway. Multiple post-intervention measurements distinguish a sustained effect from a short-term blip. Testing for level and slope changes shows whether outcomes shifted immediately, gradually, or both. And an optional comparison series, an untreated series alongside, strengthens things further. This design has one particular strength worth singling out. It addresses maturation directly, which almost nothing else does so cleanly. Any natural trend is already visible in the pre-intervention data. So you can extrapolate it, and estimate what would have happened anyway. It is also robust to one-time historical events. Those produce blips, not the sustained discontinuities that real interventions produce.

Good Practice

Six Ways to Strengthen the Inference

Whichever variant you use, six practical strategies improve credibility. Use the same instruments at pretest and posttest. That eliminates instrumentation as an alternative explanation, cheaply and completely. Add a comparison group whenever it is possible. That handles history, maturation, and testing effects. Add more measurement points, before and after. That reveals trends, and separates sustained change from transient change. Pre-register your hypotheses and your analytic decisions before you collect data. That reduces the temptation to fit the analysis to whatever pattern happens to appear. Report effect sizes, not only statistical significance. Practical magnitude and statistical confidence are different questions, and readers need both. And acknowledge the validity threats you did not address. That lets a reader weigh your conclusions accurately, which is the point of publishing them.

Choosing Honestly

When a Simple Design Is Enough

One last piece of judgment, and it is not what students usually expect. The single-group design is sometimes genuinely the right choice. In exploratory pilot work. In clinical case studies. In applied evaluations where a comparison group simply is not feasible. Two terms tie the whole section together. The counterfactual is what would have happened without the intervention. Every variant estimates it differently, from a comparison group or from an extrapolated trend. The baseline is the level of the outcome before the intervention, your reference point for change. So here is the principle to carry away. Use the strongest design your circumstances allow. Then interpret your findings in light of the design you actually used. Overstating conclusions from a weak design is the error to avoid. Using a weak design honestly is sometimes unavoidable, and that is a different thing entirely.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About The Logic of Pretest-Posttest Designs

Why is the one-group pretest-posttest design considered weak?
Without a comparison group, any change between pretest and posttest could come from something other than the intervention. Outside events (history), natural development (maturation), practice from the pretest (testing), changes in measurement (instrumentation), and regression to the mean in groups chosen for extreme scores can all produce change. The design is cheap and practical, but its findings should be treated as suggestive rather than conclusive.
How do you measure the treatment effect in a non-equivalent control group design?
The treatment effect is the difference in change between the treated group and the comparison group, not the change in the treated group alone. The comparison group estimates the counterfactual, meaning what would have happened without the intervention. This helps control history, maturation, and testing when both groups share the same events and measures, but selection remains the main weakness, because the groups were not randomly assigned.
What is an interrupted time-series design?
An interrupted time-series design measures an outcome many times before and after an intervention to see whether the established trend changes. An effect can appear as a level change, an immediate step up or down, or a slope change, a shift in the rate of change starting at the intervention. Multiple pre-intervention points reveal maturation and seasonal patterns, and an untreated comparison series strengthens the design further.
How can researchers make a pretest-posttest study more credible?
Use the same instruments at pretest and posttest to rule out instrumentation effects, and add a comparison group whenever possible to address history, maturation, and testing. Add more measurement points before and after to reveal trends. Pre-register hypotheses and analysis plans, report effect sizes alongside statistical significance, and openly acknowledge validity threats that could not be addressed, so readers can weigh the conclusions accurately.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →