UPI Study › Courses › Educational Psychology › Free lessons › Lesson 10.3 Free with a free account
What Do the Numbers Tell Us? Inferential Statistics in Education
Educational Psychology · Chapter 10: Research Methods and Evidence in EducationSection 10.3 · lesson 94 of 1009 min · free with a free account
Free preview
Create a free account to keep watching
It takes a minute and costs nothing. Your free account unlocks all of Chapter 1 and 3 more lessons of Educational Psychology.
Lesson complete
Earn the 3 college credits for Educational Psychology
Enrolling adds a quiz after each lesson, the graded work and the official transcript. Recommended for credit by NCCRS.
Next lesson: Research with Integrity: Ethical Guidelines in Educational Research →← Can We Trust the Results? Validity, Reliability, and CorrelationNext lesson: Research with Integrity: Ethical Guidelines in Educational Research →
In this lesson
Hypothesis testing, p-values, t-tests, ANOVA, confidence intervals and effect size.
- Explain hypothesis testing and the null hypothesis
- Interpret a p-value and statistical significance
- Describe when to use a t-test and when to use ANOVA
- Distinguish statistical from practical significance
Transcript
Everything said in the video, section by section (1,266 words). Read along, or search it.
Research Methods and Evidence in Education
What Do the Numbers Tell Us?
A study reports that a new reading program worked. What does that sentence actually rest on? It rests on a sample, and on a set of statistical decisions. This section is about those decisions. We start with the logic of hypothesis testing. Then the p-value, which is the most misread number in research. Then t-tests and analysis of variance. Then confidence intervals, and effect size. And finally the difference between significant and important. Those last two words are not synonyms. Confusing them is the most common error in reading education research.
Sample To Population
Why We Cannot Just Count Everyone
Inferential statistics let researchers draw conclusions about populations. And they do it using data collected from samples. The reason is practical. You almost never get the whole population. All students in a state. All teachers in a country. Studying every one of them is not possible. So researchers work with a sample instead. Then they use statistical reasoning to infer what it says about the group. The foundation of that process is hypothesis testing. Hypothesis testing is a formal procedure, not a hunch. It evaluates claims about population parameters using sample data. The formality is the point. It makes the reasoning checkable.
Null And Alternative
The Claim You Try to Rule Out
Every test begins with a claim you are trying to rule out. That claim is the null hypothesis. It asserts that there is no effect, no difference, no relationship. Whatever the default no-change position is for the question at hand. The alternative hypothesis is what the researcher is actually testing. That a new curriculum improves scores. That boys and girls differ in mathematics anxiety. Or that attendance predicts graduation rates. Statistical tests then ask one question about the data. How likely would this data be, if the null hypothesis were true? And notice what rejecting the null does not do. It does not prove the alternative. It says the evidence against the null is strong.
The p-Value
A Probability About Data, Not About Truth
The p-value is the most misread number in all of research. So here is what it actually is. It is the probability of obtaining a test statistic as extreme as the one observed. And that probability assumes the null hypothesis is true. Take a p-value of zero point zero three. That means if there were truly no effect, there would be a three percent chance. A three percent chance of seeing data this extreme by random chance. Here is what it does not mean. It does not mean there is a ninety-seven percent probability the effect is real. That is a claim about the world, and the p-value is a claim about the data. If the probability falls below a set threshold, we reject the null. That threshold is the significance level, alpha, usually set at zero point zero five.
t-Tests
Comparing Two Means
The independent-samples t-test compares two independent groups. It compares their means on a continuous outcome variable. A researcher might compare mean reading scores between two sets of students. Those who received a tutoring intervention, and those who did not. The paired-samples t-test is for a different situation. It is used when the same group is measured twice. Before an intervention, and after it. That design asks whether scores changed significantly over time. Each test produces a t-statistic. And the associated p-value indicates whether the difference is statistically significant. The distinction is about the data, not the question. Two groups once, or one group twice.
ANOVA
Three Groups or More
Two groups is the easy case. Three or more needs a different tool. That tool is analysis of variance, usually called ANOVA. ANOVA works by splitting the total variability in the data. Some variability sits between the groups, and is due to the treatment. Some sits within the groups, from individual differences and error. The F-statistic is the ratio of those two. Between-group variance, divided by within-group variance. A significant F tells you that at least one group mean differs. But it does not tell you which one. That is the part people forget, and it matters. To find out, you run post-hoc tests. Tukey's honestly significant difference, or Bonferroni corrections.
Worked Example
Reading a t-Test Result
Here is a complete result, read the way a researcher would read it. A researcher tests whether a new vocabulary program improves scores. The control group mean is seventy-two point four. The intervention group mean is seventy-eight point one. So the raw gap is a bit under six points. The t-test yields a t-statistic of two point four three. With fifty-eight degrees of freedom, and a p-value of zero point zero one eight. Because that p-value is below zero point zero five, the null is rejected. The conclusion is a statistically significant improvement. But one more number is reported, and it does different work. Cohen's d is zero point six one, which is a medium practical effect. Both numbers matter. Neither one is sufficient on its own.
Confidence Intervals
A Range, Not a Verdict
A p-value gives you a verdict. A confidence interval gives you a range. It provides a range of plausible values for a population parameter. And it is built from the sample data you actually collected. The phrase ninety-five percent confidence has a precise meaning. If the same study were repeated one hundred times. Approximately ninety-five of the resulting intervals would contain the true value. The width of the interval carries information too. A narrow interval signals high precision. A wide interval reflects greater uncertainty. This is why intervals are increasingly preferred over p-values alone. They communicate the direction of an effect, and its precision. A p-value gives you neither of those.
Effect Size
How Big, Independent of How Many
Effect size answers a question the p-value cannot. Not whether an effect exists, but how big it is. It is a standardized measure of the practical magnitude of an effect. And crucially, it is independent of sample size. Cohen's d is the most common effect size for comparing two means. Three values act as conventional benchmarks. Zero point two is interpreted as a small effect. Zero point five is a medium effect. And zero point eight is a large effect. Those are conventions, not laws of nature, but they are widely shared. Reporting effect sizes alongside p-values is now best practice. In educational research, a p-value on its own is an incomplete report.
Two Kinds Of Significance
Unlikely Is Not the Same as Important
Now the distinction this whole section has been building toward. Statistical significance and practical significance are not the same thing. Statistical significance tells you an effect is unlikely due to chance. Practical significance tells you whether it is large enough to matter. And here is the trap. Sample size drives the first one. With a large enough sample, even a trivial difference becomes significant. So a study can be statistically significant and practically negligible. The reverse happens too, and it is less discussed. A study with a small sample may fail to reach significance. Even when the underlying effect is practically meaningful. Consider a zero point two point increase on a hundred-point test. That can be statistically significant, and it changes nothing for a student.
At A Glance
Six Tests, and When Each Applies
Six tests, and the situation each one is built for. The independent t-test compares two independent groups. It gives you a t-statistic, as in a new curriculum against a control. The paired t-test handles one group at two time points. Also a t-statistic, as in a pre and post intervention study. One-way analysis of variance takes three or more independent groups. It returns an F-statistic, as when comparing three teaching methods. Two-way analysis of variance handles two factors at once. It tests main effects and interactions, like grade and gender together. Pearson r measures linear association between two continuous variables. Test score against attendance, for example, reported as a coefficient. And chi-square compares frequency distributions across categories. Pass rate by school type is the standard educational case.
About Educational Psychology at UPI Study
Educational Psychology (Psychology 120) is a self-paced online course of 100 video lessons in 11 chapters, from memory, conditioning and child development to motivation, individual differences, assessment, instruction and educational technology. It carries a NCCRS credit recommendation for 3 semester credits.
Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →
Why take it here
Why students take Educational Psychology with UPI Study
Where the credit goes
Universities that work with NCCRS
Educational Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Thomas Edison State UniversitySigned partner
University of the PeopleSigned partner
Charter Oak State CollegeSigned partner
Strayer UniversitySigned partner
Columbia Southern UniversitySigned partner
Indiana TechSigned partner
Penn StateWorks with NCCRS
Southern New Hampshire UniversityWorks with NCCRS
SUNYWorks with NCCRS
+many more cooperating universitiesFind my college →Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.
FAQ
Questions students ask us
About What Do the Numbers Tell Us? Inferential Statistics in Education
What does a p-value actually mean?
What is the null hypothesis?
When should you use a t-test versus ANOVA?
What is the difference between statistical significance and practical significance?
About the Educational Psychology course
How many college credits is Educational Psychology?
How much does Educational Psychology cost and what is included?
Which universities accept UPI Study credits?
How long does Educational Psychology take?
How are the exams done?
How do my credits reach my college?
Ready for the full course? Earn 3 credits in Educational Psychology.
100 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.
3 credits · NCCRSEnroll →