The Core Principles Behind Sampling, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 8.1 Free with a free account

The Core Principles Behind Sampling

Research Methods in Psychology · Chapter 8: Sampling and GeneralizationSection 8.1 · lesson 45 of 949 min · free with a free account

← How Time Shapes Research Design ChoicesNext lesson: Linking Population, Sample, and Generalization →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

Why we sample, what makes a good sample, and how the key concepts fit together.

  • Distinguish population, frame, sample and unit of analysis
  • Name the three principles of a good sample
  • Separate sampling error from sampling bias
  • Walk a sampling plan through its six decisions

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,337 words). Read along, or search it.

Sampling and Generalization

The Core Principles Behind Sampling

No study measures everyone. Every study measures some people and talks about more. That gap between who you measured and who you meant is where sampling lives. This section builds the vocabulary and the principles for crossing it safely. You will distinguish population, target population, sampling frame, and sample. You will learn the three principles that separate good samples from poor ones. You will separate sampling error from sampling bias, which behave very differently. And you will walk a sampling plan through its six decisions. One warning up front. The most famous sampling disaster in history had two million respondents.

Why We Sample

Studying Some to Learn About All

Researchers almost never study every member of the group they care about. Instead they study a smaller subset, called a sample. Then they use information from that sample to draw conclusions about the larger group. That larger group is the population. The reasons for sampling are entirely practical. Studying every adult in a country is rarely affordable. Studying every patient in a hospital system is often impossible. And studying every employee in a global corporation is sometimes just unnecessary. Because here is the remarkable part. A well-chosen sample can yield strikingly accurate inferences about a much larger population. That is the whole promise of the method.

The Catch

Two Words Doing a Great Deal of Work

Now look again at two words in that sentence. Well chosen. Those two words are doing a great deal of work. Because samples can mislead, and they usually do it quietly. A poll of online readers tells us about online readers. It does not tell us about the public. A study of students from one university tells us about that university. It does not tell us about students in general. In both cases the arithmetic is fine. The reach of the conclusion is not. So the science of sampling has two jobs, not one. The first is drawing samples that support credible inferences. The second is recognizing the limits of those inferences when the sampling was imperfect. That second job matters more, because imperfect sampling is the normal case.

The Vocabulary

Five Concepts That Are Constantly Confused

Five concepts do the structural work here, and they are constantly confused. The population is the complete set of units you want to draw conclusions about. People, organizations, or events. The target population is the specific subset your study is actually designed to study. It is defined by inclusion and exclusion criteria that you choose deliberately. The sampling frame is the accessible list, register, or procedure. It operationally identifies the members of that target population. The sample is the subset of the frame that actually gets selected and studied. And the unit of analysis is the kind of entity being sampled. Individual people, or households, or classrooms, or organizations. Getting these five straight is the first step to thinking clearly about any sample.

Nested Layers

How the Layers Sit Inside One Another

These concepts are not a list. They are nested layers. The outermost layer is the population, everyone you care about. Inside it sits the target population, narrowed by your inclusion criteria. Inside that sits the sampling frame, the part you can actually reach. And inside that sits your sample, the part you actually studied. Read the diagram inward, and each layer is defined by the one outside it. The sample is selected from the frame. And the frame ideally covers the target population. Sit with that word ideally, because it is carrying a great deal of risk. Almost every serious sampling problem is a gap between two of these layers.

Where It Breaks

Frame Gaps

So consider the gap between the frame and the target population. If the frame does not fully cover the target population, the sample cannot represent it. That is not a probability statement. It is a structural fact. No amount of careful work later in the study repairs it. The classic example is a phone survey using only landline numbers. It excludes every household without a landline. That is a substantial group, and an increasingly mobile-only one. So the findings describe landline households. They do not describe the broader population the researcher hoped to study. Frame gaps are among the most common sources of misleading sampling. And they are easy to miss, because a study with a frame gap looks entirely normal from the outside.

Three Principles

What Separates a Strong Sample from a Weak One

Three principles separate strong samples from weak ones. The first is representativeness. The sample reflects the population's characteristics on dimensions relevant to your question. A representative voter sample would mirror the distribution of age, region, education, and income. Note the qualifier. Relevant dimensions, not all dimensions. Perfect representativeness is rare, so aim for closeness on what matters to the question. The second principle is adequate size. Large enough to support the inferences you want to draw. Adequate is not a fixed number, and anyone who quotes you one is oversimplifying. It depends on the size of the effect, the variability in the population. It also depends on the precision required, and the kind of analysis planned. The third principle is an appropriate selection method. A study aiming for population estimates needs probability sampling. A study aiming for in-depth qualitative understanding may deliberately choose information-rich cases. A method that is excellent for one purpose can be entirely wrong for another.

Two Different Errors

Sampling Error Is Not Sampling Bias

Two distinct kinds of error arise in sampling, and they are managed differently. Sampling error is the random difference between a sample statistic and the true population value. It arises because every sample includes only some members of the population. And chance affects which ones. Sampling error is unavoidable, but it is controllable. Increase the sample size, or use a more efficient sampling design, and it shrinks. Confidence intervals and standard errors are how we quantify its likely size. Sampling bias is a completely different animal. It is systematic distortion, arising when the sample is not representative of the population. And here is the crucial asymmetry. Unlike sampling error, sampling bias does not shrink as the sample grows. It is reduced by better sampling design, and by nothing else.

The Classic Warning

Big Is Not the Same as Unbiased

That asymmetry sounds abstract, so make it concrete. A million-respondent online poll of self-selected users is exactly as biased as a thousand-respondent one. The wrong people are simply being counted more accurately. The most famous demonstration of this is nearly a century old. In nineteen thirty-six, the Literary Digest poll predicted that Alfred Landon would defeat Franklin Roosevelt. That was for the United States presidency. The prediction rested on more than two million responses. By any standard, that is a colossal sample. Roosevelt won in a landslide. The poll's respondents had been drawn from telephone directories and automobile registrations. Which missed the much larger population that owned neither a phone nor a car. So remember the lesson in one line. Size cannot rescue a biased sample.

The Plan

Six Decisions in Every Sampling Plan

Whatever method you eventually choose, every sampling plan moves through six decisions. First, define the population. Specify exactly who or what the study is about. Second, specify the target population. Identify the inclusion or exclusion criteria that narrow it down. Third, construct or identify the sampling frame. Find or create the list, register, or procedure that identifies target members. Fourth, choose a sampling method. Decide between probability and non-probability approaches, then select a specific technique. Fifth, determine the sample size. Use power analysis, precision goals, or saturation criteria, depending on the study. Sixth, recruit and verify. Implement the plan, then check the sample you actually got for coverage problems and bias. That last step is the one people skip, and it is the one that catches the disasters.

Carrying It Forward

The Two Words to Remember

Two terms will structure the rest of this chapter. In probability sampling, every member of the target population has a known chance of selection. Known, and non-zero. Both halves of that phrase matter. In non-probability sampling, selection rests on judgment, convenience, or other non-random criteria. And one more term to keep close. A frame gap. That is a failure of the sampling frame to fully cover the target population. If you take one diagnostic question away from this section, make it this one. Who could not possibly have ended up in this sample? Answer that honestly, and most sampling weaknesses become visible immediately.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About The Core Principles Behind Sampling

What is the difference between a population, a sampling frame, and a sample?
The population is the complete set of units a researcher wants to draw conclusions about. The target population is the subset defined by inclusion and exclusion criteria. The sampling frame is the accessible list, register, or procedure that identifies members of the target population. The sample is the subset of the frame actually selected and studied. A gap between the frame and the target population limits what the sample can represent.
What is the difference between sampling error and sampling bias?
Sampling error is the random difference between a sample statistic and the true population value, while sampling bias is systematic distortion from an unrepresentative sample. Sampling error shrinks as the sample grows and is quantified with standard errors and confidence intervals. Sampling bias does not shrink with size; it is reduced through better sampling design. A huge self-selected online poll is just as biased as a small one.
Why did the 1936 Literary Digest poll fail?
The Literary Digest poll predicted that Alf Landon would defeat Franklin Roosevelt despite more than two million responses, yet Roosevelt won in a landslide. Its mailing lists drew heavily on sources such as telephone directories and automobile registrations, which underrepresented poorer voters during the Depression, and most people who received ballots never returned them. The lesson is that a large sample cannot rescue a biased one.
What makes a sample good in research?
A good sample meets three principles: representativeness, adequate size, and an appropriate selection method. It should reflect the population on characteristics relevant to the research question, be large enough for the planned inferences, and be chosen by a method suited to the goal. Population estimates need probability sampling, while in-depth qualitative studies may deliberately select information-rich cases. Adequate size depends on effect size, variability, and required precision.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →