Participant-Related Factors That Affect Internal Validity

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 10.4 Free with a free account

Participant-Related Factors That Affect Internal Validity

Research Methods in Psychology · Chapter 10: Internal Validity and Its ThreatsSection 10.4 · lesson 61 of 9410 min · free with a free account

← Physical and Researcher Influences on Internal ValidityNext lesson: Internal Threats: History, Instrumentation, Mortality, Regression, and Testing →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

How the people in a study, and what they bring with them, shape its conclusions.

  • Identify participant characteristics that threaten validity
  • Define participant reactivity and its principal forms
  • Distinguish placebo, Hawthorne and John Henry effects
  • Apply cover stories, attention checks and matched controls

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,407 words). Read along, or search it.

Internal Validity and Its Threats

Participant-Related Factors That Affect Internal Validity

Give a group of people an inert pill and some of them will genuinely feel better. Tell a control group they are the control group, and some will try harder to compete. Neither of those is a measurement error. It is the participants responding to the study itself. This section is about the threats people bring with them. You will identify the participant characteristics that most often threaten internal validity. You will define participant reactivity, and its four classic forms. You will distinguish the placebo, Hawthorne, and John Henry effects, which are often blurred. And you will learn the standard tools for managing all of it.

Not Inert

A Participant Is Not a Vessel

A research participant is not a passive vessel. You do not simply pour a manipulation in and read out the result. Participants arrive at studies with histories, motivations, abilities, and expectations. They form impressions of the study and of the researcher. They generate their own hypotheses about what is being measured. And they adjust their behavior accordingly, usually without announcing it. They also become bored, tired, hungry, anxious, or distracted. All within the course of an hour-long session. They sometimes guess the hypothesis, and sometimes act on what they guess. Each of those phenomena can threaten internal validity in the same way. By introducing systematic differences across conditions that compete with the manipulation.

What They Bring

Pre-Existing Differences

Start with what participants bring in with them. Age, education, intelligence, personality, prior experience with the task. Motivation, and physiological state. All of these vary across people. And the rule from the previous section applies exactly. Randomly distributed across conditions, those variables add noise but do not bias anything. Systematically tied to condition, they become confounds. Two processes create that systematic tie most often. The first is self-selection, where participants choose the condition they end up in. People who volunteer for an exercise study differ measurably from those who decline. People who finish a six month diet trial differ from those who quit partway. And those differences may be exactly what would produce the predicted effect anyway. The second is differential recruitment across conditions. Recruit your treatment group through a specialist clinic, and your control group through an advertisement. Those groups now differ on far more than your manipulation. Random assignment is the remedy, because it hands the sorting to chance.

Being Watched

Four Forms of Participant Reactivity

Now a different family of threat, which arises from being studied at all. Participant reactivity means changes in behavior produced by the awareness of being studied. Not by the manipulation. By the observation itself. It takes four distinguishable forms, and the differences between them matter. The Hawthorne effect is behavior changing simply because someone is watching. The placebo effect is improvement arising from expectation alone. The John Henry effect is a control group working harder to compete. And resentful demoralization is a control group giving up because it feels disadvantaged. Notice that the last two involve the control group, not the treatment group. Which is a place most people never think to look for bias.

Form One

The Hawthorne Effect

The Hawthorne effect is named after industrial productivity studies of the nineteen twenties. Those were conducted at the Hawthorne Works. It refers to the tendency of people whose behavior is observed to perform differently. Usually better than they otherwise would. One caveat about the name is worth knowing. Whether the original Hawthorne data actually demonstrated such an effect is now debated. But the phenomenon the name refers to is real enough. People who know they are being watched often work harder. Or behave more cooperatively, or produce socially desirable responses. Now here is the subtle part about whether it threatens your study. If it operates uniformly across conditions, it inflates baseline performance. But it does not by itself bias the comparison between groups. The threat to internal validity arises when it operates differently across conditions. For instance, when it is stronger in the treatment group than in the control group.

Form Two

Placebo Effects

A placebo effect is improvement arising from the expectation of improvement. Rather than from any active ingredient of the treatment. It is best documented in medicine. Patients given inert pills often report meaningful symptom relief. That relief is genuine. It just did not come from the pill. But placebo effects operate in psychological and educational research as well. A new therapy may appear to work partly because participants believe it will. The standard remedy is a placebo control condition. The comparison group receives an apparently active treatment. One with no theoretically relevant ingredient in it. That holds the placebo effect constant across conditions. So any difference between the groups can be attributed to the active ingredient.

Forms Three and Four

What the Control Group Does

The last two forms involve participants in the control condition. The John Henry effect describes control participants who know they are controls. They work harder than usual, as though to compete with the treatment being denied them. And the consequence is specific. That performance lift masks a real treatment benefit. Your treatment worked, and the control group caught up out of pride. Resentful demoralization is the opposite reaction to the same knowledge. Control participants who feel disadvantaged may perform worse than they otherwise would. Which exaggerates the apparent treatment effect instead. Both arise from the same root cause. Participants know which condition they are in, and assign meaning to that knowledge. Both are reduced by blinding, and by careful management of expectations. But notice that they push your estimate in opposite directions. So knowing which one is operating matters as much as knowing that one is.

During the Session

Transient States

Beyond what participants bring and how they react, there are transient states. Things that develop during the session itself. Three matter most. Fatigue is the first. Performance declines across a long session, contaminating the later measurements. And if conditions are administered in a fixed order, fatigue confounds the manipulation directly. Manage it by counterbalancing condition order, shortening sessions, and scheduling breaks. Boredom is the second, and it is more insidious. Participants stop attending and begin responding automatically. The data become meaningless even though the measurements remain on file. Manage it with attention-check items, varied stimulus types, and appropriate task length. The third is anxiety, or evaluation apprehension. Concern about being evaluated alters performance, especially where participants feel judged. Standardize the instructions, reassure about confidentiality, and measure unobtrusively where ethical.

The Defenses

Three Families

The defenses fall into three families, and each targets a different threat. The first is random assignment. It neutralizes pre-existing differences between participants. By breaking the link between who someone is and which condition they receive. The second is concealment of the hypothesis. That neutralizes reactivity and demand characteristics. If participants cannot tell what you predict, they cannot cooperate with it. The third is standardization of the participant experience. That neutralizes within-session state effects. And it is the most practical of the three, and the most often neglected. Scripted instructions. Breaks at standard intervals. Attention-check items. Consistent session length, and consistent time-of-day scheduling.

Two Standard Tools

Cover Stories and Attention Checks

Two practical tools are nearly universal in psychological research. The first is the cover story. It describes the study's purpose to participants in terms that are accurate. But that do not reveal the actual hypothesis. You might call a study of stereotype activation a study of person perception. That is true, and it is not the whole truth, which is the point. The second tool is the attention check. It is a short item embedded in a questionnaire. It might ask participants to select a particular response. Or simply test whether they have read the instructions at all. Participants who fail are excluded from analysis. That removes data points reflecting disengagement rather than your manipulation. Two conditions apply to both tools. Cover stories must avoid deception beyond what informed consent permits. And both should be planned in advance, and reported in the methods section.

An Honest Limit

What Cannot Be Controlled

It is worth saying plainly how far any of this can go. No study can eliminate all participant-related threats. Participants are people. They will continue to bring expectations, fatigue, and prior knowledge into studies. Regardless of what the researcher does about it. So the realistic goal of internal-validity work is not to remove every threat. It is to ensure that the remaining threats do not differ systematically across conditions. That distinction is genuinely freeing once you absorb it. Picture a study where every participant is equally tired and equally curious. Equally aware of being studied, all the way through. That may still be a strong study, because its comparisons remain unbiased. Even though the absolute level of fatigue and reactivity is nowhere near zero.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Participant-Related Factors That Affect Internal Validity

What is participant reactivity in research?
Participant reactivity is a change in behavior caused by awareness of being studied rather than by the manipulation. Four classic forms are the Hawthorne effect, where being observed changes performance; the placebo effect, where expecting improvement produces improvement; the John Henry effect, where a control group works harder to compete; and resentful demoralization, where a control group performs worse because it feels disadvantaged. Reactivity threatens internal validity when it differs across conditions.
What is the difference between the John Henry effect and resentful demoralization?
Both are reactions of control-group participants who know they are not receiving the treatment, but they push results in opposite directions. In the John Henry effect, controls work harder to compete, which can mask a real treatment benefit. In resentful demoralization, controls feel disadvantaged and perform worse, which exaggerates the apparent treatment effect. Blinding participants to their condition and managing expectations reduce both.
How do participant characteristics threaten internal validity?
Participant characteristics such as age, motivation, intelligence, personality, and prior experience threaten internal validity when they are tied systematically to condition. Spread randomly across groups, they add only noise. Two processes usually create a systematic link: self-selection, where people choose their own condition, and differential recruitment, such as drawing the treatment group from a specialist clinic and controls from an advertisement. Random assignment breaks the link by leaving the sorting to chance.
What is an attention check in a survey?
An attention check is a short item embedded in a questionnaire to confirm that participants are reading, for example by asking them to select a particular response. Those who fail are usually excluded, removing data that reflect disengagement rather than the manipulation. Attention checks are often paired with a cover story, an accurate description of the study's purpose that does not reveal the hypothesis. Both should be planned in advance and reported in the methods section.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →