Physical and Researcher Influences on Internal Validity

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 10.3 Free with a free account

Physical and Researcher Influences on Internal Validity

Research Methods in Psychology · Chapter 10: Internal Validity and Its ThreatsSection 10.3 · lesson 60 of 9411 min · free with a free account

← Spotting Extraneous and Confounding VariablesNext lesson: Participant-Related Factors That Affect Internal Validity →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

How the room, the materials, and the investigator quietly shape what a study finds.

  • Identify setting features that threaten internal validity
  • Define experimenter effects and how they operate
  • Distinguish demand characteristics from experimenter effects
  • Apply blinding and standardization as defenses

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,405 words). Read along, or search it.

Internal Validity and Its Threats

Physical and Researcher Influences on Internal Validity

Rats run mazes faster when the person timing them expects it. The rats were assigned at random. The expectation belonged only to the experimenter. That result tells you something uncomfortable about who is actually in a study. This section covers the two least visible threats to internal validity. The physical setting, and the researcher running it. You will identify the setting features that can bias an outcome. You will distinguish experimenter effects from demand characteristics, which are often confused. And you will learn the two defenses that neutralize most of these threats.

The Container

The Setting Is Never Neutral

It is tempting to think of a study as happening in a neutral container. The manipulation, the measure, and the participants do all the interesting work. Everything else is just the room it happened in. In practice, the container is itself a variable. The room, the equipment, the materials handed out. The time of day, the noise level, and the investigator who administers everything. All of those are part of the situation that shapes how participants respond. Now apply the confound test from the previous section. When any of these features differs systematically across conditions, it becomes a competitor. It competes with the independent variable for credit, exactly as a participant characteristic would.

The Physical Setting

Four Classes of Feature

Physical-setting threats are easy to overlook, because they feel incidental. Nobody writes a hypothesis about lighting. But they have a long history of biasing study outcomes. Four classes of feature deserve attention. Location. The room, the building, or the online platform. Equipment. Devices, software, and measuring instruments. Ambient conditions. Light, noise, temperature, and time of day. And materials. The stimuli, the forms, and the instructions. Run through those four whenever you design or evaluate a procedure. The question each time is the same. Does this differ across my conditions?

Setting Threats

Location and Equipment

Start with location, because where a study runs is rarely a neutral choice. A study conducted in a clinical-looking laboratory may produce different responses. Different from one conducted in a familiar classroom. One room slightly warmer, one with brighter lighting. Those affect mood, fatigue, and concentration, which affect performance. Here is the failure mode. Suppose one condition runs in a quiet morning session. And the other in a noisy afternoon session. Location-related factors are now confounded with your manipulation. Equipment brings a second family of problems, especially drift and calibration error. A reaction-time study run on different computers may pick up millisecond differences. Differences from the hardware, not from the manipulation. And an online survey rendered differently across browsers produces platform effects. Response patterns that reflect the screen, rather than the question.

Setting Threats

Ambient Conditions and Materials

Ambient conditions are the third class. Light, noise, temperature, and time of day. All of them influence cognitive performance, mood, and physiology. The danger arises when ambient conditions correlate with experimental condition. Imagine treatment sessions scheduled for early morning. And control sessions scheduled for late afternoon. The resulting circadian and energy differences may masquerade as treatment effects. The fix is usually cheap. Counterbalance the schedule across conditions. Or simply hold the ambient features constant. Materials are the fourth class, and they follow the same logic. If conditions use materials of different lengths, complexity, or visual design. Then the manipulation is no longer the only thing that differs. Even subtle wording differences in instructions can change how participants read the task.

The Researcher

Experimenter Effects

Now to the researcher, who is also part of the study. Experimenter effects describe how a researcher's expectations shape what happens. Both the participants' behavior, and the recording of the data. They are sometimes called experimenter expectancy, or Rosenthal effects. Robert Rosenthal's classic studies made the point unforgettably. Experimenters were told their rats were maze-bright, or maze-dull. The maze-bright group obtained faster maze-running times. Even though the rats had been randomly assigned to those labels. So the difference was entirely in the experimenters' heads, and it still showed up in the data. Their expectations had altered how they handled the animals. How they timed the runs, and how they interpreted ambiguous behavior. Similar effects are well documented with human participants. Tone of voice, eye contact, posture, and choice of words all communicate expectations.

The Participant's Reading

Demand Characteristics

Demand characteristics are the second researcher-related threat, and they work differently. They are cues within a study that signal to participants what is expected of them. Most participants want to be helpful. They want to appear competent, and to produce data that please the researcher. So when the design or procedure makes the hypothesis transparent, they respond to it. Often unconsciously shifting their behavior to confirm it. Here is the key distinction from experimenter effects. Demand characteristics need not be transmitted by the researcher's behavior at all. The cover story can give it away. So can the order of the questions. So can the labeling of conditions, or the presence of distinctive equipment. A perfectly blank-faced experimenter cannot prevent a transparent design from leaking.

Orne's Analysis

Three Roles a Participant May Adopt

Martin Orne's analysis described several roles participants may adopt. The good participant tries to confirm what the researcher seems to want. That is the one people expect. The negativistic participant deliberately disconfirms it instead. Some people simply do not enjoy being predictable. And the apprehensive participant does something different again. They try to look mentally healthy, or socially desirable, rather than answer truthfully. Each role distorts the data in a different direction. But every one of them is a response to demand, rather than to your manipulation. And do not assume the directions cancel out. There is no reason they should. The remedy is structural rather than statistical. Design studies in which the hypothesis is not obvious to the participants.

At the Coding Sheet

Recording and Observer Bias

There is a third researcher threat, and it does not involve the participants at all. Even when participants behave identically across conditions, the data can still differ. An experimenter who knows the hypothesis may record ambiguous responses differently. Picture the same behavior, coded twice. For a treatment-group participant, it gets coded as engaged. For a control-group participant, the identical behavior gets coded as distracted. Because the coder expects the treatment to increase engagement. This is called observer bias. It is most acute when the dependent variable requires judgment or interpretation. Or open-ended coding of any kind. And it is one of the strongest arguments for blinding in observational studies.

Defense One

Standardization

Two complementary defenses neutralize most of these threats. The first is standardization. It means writing down, in advance, the protocol everyone administering the study will follow. That includes the instructions, word for word. It includes the equipment settings and the room arrangement. And it includes the responses to common participant questions. Researchers often use scripted introductions, and scripted answers to anticipated queries. Be clear about what standardization can and cannot achieve. It does not eliminate all sources of variation. Nothing does. What it ensures is that whatever variation remains is not tied to experimental condition. And that is the property that actually matters for internal validity.

Defense Two

Blinding

The second defense is blinding, sometimes called masking. It means concealing the experimental condition from someone in the study. So that their expectations cannot influence behavior or data recording. In a single-blind design, the participant does not know which condition they are in. That addresses demand characteristics, and placebo and expectancy effects. In a double-blind design, the administering experimenter does not know either. That adds protection against experimenter effects and recording bias. In a triple-blind design, the data analyst is also unaware until analysis is complete. Which addresses analytical bias on top of everything else. Double-blinding is the gold standard in clinical research. Sometimes full blinding is impossible, as in psychotherapy research. Therapist and patient inevitably know which therapy is being used. So you blind the data coders and analysts instead. They can evaluate outcomes without knowing which condition produced them.

By Setting

Different Contexts, Different Dangers

Different research contexts pose different versions of these threats. In the laboratory, the dangers are artificiality and demand characteristics. Plus equipment differences across testing stations. The defenses are cover stories, scripted procedures, and calibrated equipment. Along with automated stimulus delivery, which removes the experimenter from the loop. In a field study, ambient conditions are uncontrollable. Populations vary across sites, and observers provoke reactivity. So you use multiple measurement waves, unobtrusive observation, and multi-site standardization. In an online study, the dangers are device variability and attentional drift. Along with identity uncertainty and platform-specific rendering. The defenses are attention-check items, device data capture, and time-on-task limits. And in clinical settings, therapist allegiance and patient expectations dominate. Those are met with manualized protocols, blinded outcome assessors, and treatment-fidelity ratings.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Physical and Researcher Influences on Internal Validity

What is the experimenter expectancy effect?
The experimenter expectancy effect, also called the Rosenthal effect, occurs when a researcher's expectations influence participants' behavior or the recording of data. In Robert Rosenthal's classic study, experimenters told their rats were maze-bright obtained better maze performance than those told their rats were maze-dull, even though the labels had been assigned at random. With human participants, expectations can be conveyed through tone of voice, eye contact, posture, and wording.
What is the difference between experimenter effects and demand characteristics?
Experimenter effects originate in the researcher, while demand characteristics can arise anywhere in the study's design. Experimenter effects occur when a researcher's expectations shape participants' behavior, through tone of voice, eye contact, posture, or wording, or how ambiguous data get recorded. Demand characteristics are cues such as the cover story, question order, condition labels, or distinctive equipment that reveal the hypothesis, so they can operate even with a neutral experimenter. Blinding experimenters targets the first; concealing the hypothesis targets the second.
What is the difference between single-blind, double-blind, and triple-blind studies?
In a single-blind study, participants do not know which condition they are in, which addresses demand characteristics and placebo effects. In a double-blind study, the experimenters administering the conditions do not know either, adding protection against experimenter effects and recording bias. In a triple-blind study, the data analysts are also kept unaware until analysis is complete. When full blinding is impossible, as in psychotherapy research, outcome coders and analysts can still be blinded.
How can the physical setting threaten internal validity?
The physical setting threatens internal validity when it differs systematically across conditions. Four kinds of feature matter: location, such as the room or online platform; equipment, such as computers or instruments that drift; ambient conditions, such as light, noise, temperature, and time of day; and materials, such as stimuli and instructions. If treatment sessions run in the morning and control sessions in the afternoon, time-of-day differences can masquerade as a treatment effect.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →