Compared to Others vs. Compared to a Standard: Norm vs. Criterion

UPI Study › Courses › Educational Psychology › Free lessons › Lesson 8.7 Free with a free account

Compared to Others vs. Compared to a Standard: Norm vs. Criterion

Educational Psychology · Chapter 8: Assessment, Testing, and Making Sense of ResultsSection 8.7 · lesson 76 of 1009 min · free with a free account

← The Bell Curve and Standard Deviation: What the Numbers Tell UsNext lesson: Average, Middle, and Most Common: Stats Every Teacher Should Know →

Earn 3 college credits for Educational PsychologyRecommended for credit by NCCRS · Psychology 120 · self-paced · one payment, lifetime access
Enroll in Educational Psychology · $250 →Download the syllabus (PDF)

In this lesson

Norm-referenced and criterion-referenced interpretation, and why the choice decides the answer.

  • Define norm-referenced and criterion-referenced assessment
  • Explain why norm-referencing cannot certify mastery
  • Describe cut scores and defensible standard setting
  • Choose the right interpretive framework for a purpose

Transcript

Everything said in the video, section by section (1,151 words). Read along, or search it.

Assessment, Testing, and Making Sense of Results

Compared to Others, or Compared to a Standard?

A school reports that its students rank at the sixtieth percentile. It also reports that forty percent of them cannot decode at grade level. Both statements are true, about the same children, in the same year. This section explains how that happens, and which number to act on. The distinction is between two ways of interpreting a score. Norm-referenced, which compares a student to other students. And criterion-referenced, which compares a student to a standard. We will look at what each is designed to do. And at cut scores, which are where criterion-referencing gets difficult. One line summarizes the whole section, and it is worth keeping. Norm-referencing asks who is best. Criterion-referencing asks who is ready.

Norm-Referenced

How Does This Student Compare?

Norm-referenced assessments interpret a score through comparison. Comparison to the performance of a defined norm group. That group is typically a large, representative national sample. Tested at the time the assessment was standardized. The primary question such a test answers is straightforward. How does this student perform relative to others? Percentile ranks are the characteristic reporting format. So are standard scores, with predetermined means and standard deviations. Intelligence tests are norm-referenced instruments. So are nationally standardized achievement batteries, like the Iowa Assessments. And so are many college admissions tests.

Its Advantage

A Common Yardstick Across Systems

The advantage of norm-referenced interpretation is comparability. It enables meaningful comparison across schools and districts. And across time periods, which is often what a district wants. Suppose a school learns it ranks at the thirty-fifth percentile in reading. Ranked against a national norm group. That school now knows it is below average. And it has a benchmark against which to measure improvement. That is a genuine service, and it should not be dismissed. Without an external reference point, a school has only one comparison available. It can compare itself to its own past. Which is useful, but it cannot tell you whether the past was any good.

Its Limitation

Somebody Has to Be Below Average

Now the limitation, and it is philosophical rather than technical. Norm-referencing is designed to produce a spread of scores. That is not a side effect. It is the design goal. Which means some students will always fall below average. By definition, no matter how much everyone has actually learned. Teach a class brilliantly and half of them still rank in the bottom half. That design feature has a direct consequence for what these tests can do. It makes them poorly suited for evaluating mastery. Specifically, whether everyone has mastered a particular skill set. You cannot answer, did they all learn it, with an instrument built to guarantee that some rank lower than others.

Criterion-Referenced

Has This Student Mastered the Content?

Criterion-referenced assessment works from a completely different logic. It interprets a score relative to a predefined standard. Or to a set of learning objectives. And it does so entirely independent of how other students performed. The primary question is: has this student demonstrated mastery? Mastery of the intended content, specifically. Cut scores are the defining feature of this approach. A cut score is a threshold separating mastery from non-mastery. And here is the reversal that makes the difference concrete. If every single student scores above the cut score, that is a success. Not a problem, and not evidence that the test was too easy. Under norm-referencing, that same result would look like a broken test.

Mastery Learning

The Philosophy Underneath

There is a philosophy sitting underneath criterion-referencing. It aligns naturally with mastery learning theory. That theory holds that virtually all students can achieve high standards. Given sufficient time, and appropriate instruction. Notice how well that fits an assessment with a fixed threshold. Suppose a criterion-referenced test shows eighty percent have mastered a standard. The school then knows what it has accomplished. It does not know how its students compare to other schools. Which, for this purpose, it does not need to know. A driving test is criterion-referenced. So is a licensing examination. So are most classroom unit tests. And most state standards-based assessments.

Cut Scores

The Hardest Number to Justify

The practical difficulty with criterion-referencing is the cut score itself. Setting it defensibly is genuinely hard. Set the cut score too low, and the standard becomes meaningless. Everybody passes, and passing no longer certifies anything. Set it too high, and passing becomes nearly impossible. Then the test measures something other than what it claims to. So how is a defensible threshold actually established? Through standard-setting studies, involving expert panels. That is the standard methodology in the field. The important word in that phrase is defensible. The threshold is argued for, and documented. It is not simply chosen.

Compared

Six Dimensions of Difference

Six dimensions separate the two approaches cleanly. Primary question. How does this student compare, against, has this student mastered? Score reporting. Percentile ranks and standard scores on one side. Percent correct, pass or fail, and proficiency levels on the other. Design goal. One maximizes score variability. The other maximizes content alignment. Examples. The SAT, the Iowa Assessments, and IQ tests. Against state standards tests, and driving tests. Best used for. Selection, ranking and national comparison. Against certification, accountability, and mastery. And key limitation, which is the row worth memorizing. One does not show mastery level. The other does not show relative standing.

Key Idea

Who Is Best, or Who Is Ready?

All of that reduces to two questions that sound similar and are not. Norm-referencing asks: who is best? Criterion-referencing asks: who is ready? Those are fundamentally different questions. They require different test designs, and different interpretations. Consider a school system wanting to know one thing. Can our students read at grade level? That system should use a criterion-referenced interpretation. Now consider a college selecting the top ten percent of applicants. That college needs norm-referenced data. Get this backwards and the numbers will be perfectly accurate. Perfectly accurate answers to a question nobody actually asked.

Classroom Example

Two Tests, Two Answers, No Contradiction

Here is one teacher running both frameworks at once. A second-grade teacher administers two reading assessments. One is a norm-referenced standardized reading test. The other is a criterion-referenced district benchmark. The norm-referenced test brings encouraging news. Sixty percent of her class scores above the national median. The criterion-referenced benchmark tells a harder story. Only forty percent have mastered the second-grade phonics decoding standard. It is tempting to decide one of those results must be wrong. But they are not contradictory at all. They answer different questions about the same students. And together they give a much richer picture of class performance.

Choosing

The Error This Prevents

So consider what happens when a school uses only one of them. A school applies norm-referenced logic to a mastery skill. Its students rank at the sixtieth percentile nationally. So it concludes, reasonably enough, that it is doing fine. Meanwhile forty percent of those students are below grade-level mastery. The framework did not lie. It answered a different question. So the selection rule is simple enough to remember. Choose norm-referenced for comparison, selection, or ranking. Choose criterion-referenced for certifying mastery of a defined skill set. And here is the good news to end on. Many modern assessments report both types of derived score. The same test can carry both interpretations at the same time.

About Educational Psychology at UPI Study

Educational Psychology (Psychology 120) is a self-paced online course of 100 video lessons in 11 chapters, from memory, conditioning and child development to motivation, individual differences, assessment, instruction and educational technology. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Why take it here

Why students take Educational Psychology with UPI Study

How people learn, and how to teach themEducational Psychology explains how students learn, develop and stay motivated, and how teachers assess and support them. It is a standard course in education and teaching degrees and a common psychology elective.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Educational Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Compared to Others vs. Compared to a Standard: Norm vs. Criterion

What is the difference between norm-referenced and criterion-referenced assessment?
A norm-referenced assessment compares a student's score to the performance of other students in a norm group, while a criterion-referenced assessment compares it to a predefined standard or set of learning objectives. The first is usually reported as percentile ranks or standard scores; the second as percent correct, pass or fail, or proficiency levels. In short, norm-referencing asks who is best, and criterion-referencing asks who is ready.
Why can't a norm-referenced test show whether students have mastered a skill?
A norm-referenced test cannot show mastery because it is designed to spread scores out, so some students fall below average no matter how much everyone has learned. Since rank is relative, a class that learned a skill well can still have half its students in the bottom half. That makes these tests poorly suited to answering whether every student has mastered a particular skill set.
What is a cut score and how is it set?
A cut score is the threshold on a criterion-referenced test that separates mastery from non-mastery. Setting it is hard: too low and passing certifies nothing, too high and passing becomes nearly impossible. A defensible cut score is usually established through standard-setting studies with expert panels, so the threshold is argued for and documented rather than simply chosen.
When should a school use norm-referenced versus criterion-referenced scores?
Norm-referenced scores fit comparison, selection and ranking, such as a college choosing the top ten percent of applicants, while criterion-referenced scores fit certifying mastery, such as checking whether students read at grade level. Using the wrong one misleads: a school at the sixtieth percentile nationally may still have forty percent of its students below grade-level mastery. Many modern tests report both kinds of score.

About the Educational Psychology course

How many college credits is Educational Psychology?
Educational Psychology (Psychology 120) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Educational Psychology cost and what is included?
$250, paid once, with lifetime access: all 100 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Educational Psychology take?
It is self-paced with no deadlines. The 100 lessons total about 15 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Educational Psychology.

100 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Educational Psychology →Download the syllabusEvery course with PRO →
Educational Psychology
3 credits · NCCRSEnroll →