Can We Trust the Results? Validity, Reliability, and Correlation

UPI Study › Courses › Educational Psychology › Free lessons › Lesson 10.2 Free with a free account

Can We Trust the Results? Validity, Reliability, and Correlation

Educational Psychology · Chapter 10: Research Methods and Evidence in EducationSection 10.2 · lesson 93 of 1009 min · free with a free account

← How Do Researchers Study Education? Research Designs and ExperimentsNext lesson: What Do the Numbers Tell Us? Inferential Statistics in Education →

Earn 3 college credits for Educational PsychologyRecommended for credit by NCCRS · Psychology 120 · self-paced · one payment, lifetime access
Enroll in Educational Psychology · $250 →Download the syllabus (PDF)

In this lesson

Internal and external validity, threats, reliability, and reading a correlation.

  • Explain internal and external validity
  • Name the common threats to internal validity
  • Relate reliability to validity in measurement
  • Interpret a correlation coefficient correctly

Transcript

Everything said in the video, section by section (1,134 words). Read along, or search it.

Research Methods and Evidence in Education

Can We Trust the Results?

A study can be perfectly conducted and still tell you nothing useful. It can also be sloppy and produce a number people quote for years. This section is about telling those two apart. We start with internal validity, and external validity. Then the standard threats to each of them. Then reliability, and how it relates to validity. Then correlation, and how to read a correlation coefficient. And finally the principle everyone quotes and few apply. Correlation does not imply causation. Knowing that sentence is easy. Spotting the lurking variable in a real study is the skill.

Internal Validity

Did the Treatment Cause the Change?

Validity refers to whether a study measures what it claims to measure. And whether the conclusions drawn from it are accurate and meaningful. Two forms of validity are central in educational research. The first is internal validity. It asks whether the observed outcomes can truly be attributed to the independent variable, and not to something else. Those something-elses have a name. Threats to validity. A study high in internal validity gives strong evidence. Evidence that the treatment, and only the treatment, caused the change. Note how demanding that phrase is. And only the treatment.

External Validity

Does It Hold Anywhere Else?

External validity asks a completely different question. It concerns whether findings generalize beyond the study. Beyond the specific participants, settings and conditions used. And here is the uncomfortable part. A study can have high internal validity and low external validity. Its causal conclusions are sound, and they apply almost nowhere. That happens when the sample came from one demographic group. Or when the research setting was highly artificial. So artificial it bears little resemblance to a real classroom. Researchers have to balance the two forms deliberately. Because the tight controls that raise internal validity can reduce the ecological realism external validity needs.

Threats, Part One

Four Things That Are Not the Treatment

Several well-documented threats can undermine internal validity. The first is history. That means external events occurring during the study. Events that affect the outcomes independently of your treatment. A district implementing a new literacy policy mid-study, for instance. The second is maturation. Participants naturally develop or change over the study period. Children in particular do a great deal of that. The third is testing effects. Participants improve simply from having taken the same test before. And the fourth is regression to the mean. Select extreme scorers, and their scores drift toward average. That happens on retesting, regardless of any intervention at all.

Threats, Part Two

Who Was in the Study, and Who Stayed

Three more threats concern who was studied, and who stayed. Selection bias is perhaps the most common in educational research. When participants are not randomly assigned, the groups may differ systematically before the study even begins. Attrition is the loss of participants over time. It skews results under one specific condition. When the students who drop out differ from those who remain. And instrumentation threats arise from the measurement tools. When those tools change between pre-test and post-test. Score differences then become impossible to interpret. Being aware of all these threats has a practical payoff. It lets researchers design studies that minimize their impact.

Eight Threats

Internal and External, Side by Side

It helps to see the threats collected in one place. History is internal. External events affect outcomes during the study. Maturation is internal. Natural development changes outcomes over time. Testing is internal. Prior exposure improves future performance. Selection bias is internal. Groups differ before the study begins. Regression to the mean is internal, and easy to mistake for an effect. Extreme scorers move toward average on retesting. Attrition is internal. Systematic dropout changes group composition. Two more deserve a mention here. The Hawthorne effect is also internal. Participants change behavior because they know they are observed. And a restricted sample is an external threat. A non-representative sample limits how far findings generalize.

Reliability

Consistency Across Time, Raters, and Items

Reliability refers to the consistency of a measurement. Consistency across time, across raters, or across test forms. There are three forms, matching those three sources. Test-retest reliability asks about time. Does the measure give similar results on two different occasions? Given to the same participants both times. Inter-rater reliability asks about people. Do two or more scorers evaluate the same performance consistently? And internal consistency asks about the items themselves. Do all the items on a scale assess the same underlying construct? That one is often expressed as Cronbach's alpha.

The Relationship

Necessary, But Not Sufficient

Now the relationship between validity and reliability. It runs in one direction only, and that asymmetry matters. A measurement tool can be reliable without being valid. It may consistently measure entirely the wrong thing. But a valid measurement must also be reliable. An inconsistent measure cannot accurately capture anything. So high reliability is necessary, but not sufficient, for validity. A scale must first measure consistently. Only then can it accurately reflect the construct of interest. This matters most where the stakes are highest. Grade promotion. Placement in advanced programs. Teacher evaluation. An unreliable test produces scores that fluctuate from measurement error. Rather than from true differences in student knowledge.

Correlation

Direction and Strength, in One Number

Correlational research examines relationships between variables. Two or more variables, without manipulating any of them. The result is summarized in a correlation coefficient, written as r. It expresses two things at once. The direction of the linear relationship, and its strength. And it ranges from minus one point zero to plus one point zero. A positive correlation means both variables move together. As one increases, so does the other. Reading time and vocabulary size, for example. A negative correlation means they move in opposite directions. Absenteeism and academic performance, for instance. Scatterplots are the standard way of showing these relationships. Each point represents one participant's scores on both variables.

The Central Principle

Correlation Does Not Imply Causation

Now the central principle of the whole section. Correlation does not imply causation. Even a strong positive correlation between two variables does not mean that one of them causes the other. A third variable may be driving both of them. It is sometimes called a lurking variable. Here is the example worth remembering. There is a positive correlation between books in a home and student academic achievement. That does not mean the books themselves cause the achievement. Both may reflect family socioeconomic status. And parental education, sitting underneath both measures. Only experimental designs with proper controls establish causation.

Classroom Example

Reading a Scatterplot With Students

Here is a teacher building this skill in her own students. She shows the class a scatterplot with two axes. Daily study hours along the bottom, quiz scores up the side. The data come from thirty students. And they show a moderate positive correlation. An r of zero point six two. Then she asks the question that does the teaching. Does studying cause the higher scores? Or might a third variable, such as student motivation, explain it? That exercise develops critical thinking about causal inference. Assessing measurement quality takes several tools together. Reliability coefficients, and standard errors of measurement. Plus validity evidence from content reviews and construct analyses. No single index is sufficient. Validity is an ongoing argument.

About Educational Psychology at UPI Study

Educational Psychology (Psychology 120) is a self-paced online course of 100 video lessons in 11 chapters, from memory, conditioning and child development to motivation, individual differences, assessment, instruction and educational technology. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Why take it here

Why students take Educational Psychology with UPI Study

How people learn, and how to teach themEducational Psychology explains how students learn, develop and stay motivated, and how teachers assess and support them. It is a standard course in education and teaching degrees and a common psychology elective.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Educational Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Can We Trust the Results? Validity, Reliability, and Correlation

What is the difference between internal and external validity?
Internal validity is whether a study's outcomes can truly be attributed to the independent variable, while external validity is whether its findings generalize to other people, settings and conditions. A study can be high in one and low in the other: tight laboratory controls may support sound causal conclusions that apply to few real classrooms. Researchers have to balance the two deliberately.
What are common threats to internal validity?
Common threats to internal validity include history, maturation, testing effects, regression to the mean, selection bias, attrition and instrumentation. Maturation means children change naturally over a study, testing effects mean students improve from retaking a test, and regression to the mean means extreme scorers drift toward average on retesting. The Hawthorne effect, where people change because they know they are observed, is another.
Can a test be reliable but not valid?
Yes, a test can be reliable without being valid, because it may consistently measure the wrong thing. The reverse does not hold: a valid measure must also be reliable, since an inconsistent measure cannot capture anything accurately. Reliability is therefore necessary but not sufficient for validity. It is checked through test-retest reliability, inter-rater reliability and internal consistency, often reported as Cronbach's alpha.
Why doesn't correlation imply causation?
Correlation does not imply causation because a third, lurking variable may be driving both measures. For example, the number of books in a home correlates positively with student achievement, but both may reflect family socioeconomic status and parental education. A correlation coefficient, r, ranges from -1.0 to +1.0 and shows only direction and strength; establishing cause requires experimental designs with proper controls.

About the Educational Psychology course

How many college credits is Educational Psychology?
Educational Psychology (Psychology 120) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Educational Psychology cost and what is included?
$250, paid once, with lifetime access: all 100 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Educational Psychology take?
It is self-paced with no deadlines. The 100 lessons total about 15 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Educational Psychology.

100 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Educational Psychology →Download the syllabusEvery course with PRO →
Educational Psychology
3 credits · NCCRSEnroll →