Determining the Appropriate Sample Size, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 8.5 Free with a free account

Determining the Appropriate Sample Size

Research Methods in Psychology · Chapter 8: Sampling and GeneralizationSection 8.5 · lesson 49 of 9410 min · free with a free account

← Recognizing Common Sampling ChallengesNext lesson: Foundations of Measurement: Qualitative and Quantitative Approaches →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

Choosing how many participants to study, and why the answer is rarely simple.

  • Name the factors that actually drive sample size
  • Distinguish power, precision and saturation approaches
  • Define effect size, alpha, power and precision
  • Recognize the costs of under- and over-powered studies

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,506 words). Read along, or search it.

Sampling and Generalization

Determining the Appropriate Sample Size

How many participants do I need? It is the most common methodological question there is, and it has no single answer. What it has instead is three different logics, each suited to a different goal. In this section you will learn all three. You will define effect size, alpha, power, and precision, and see how they interlock. You will apply the rules of thumb, while knowing exactly where they stop working. And you will see why a study can be too large, not merely too small. That last idea surprises most people, and it is both a statistical and an ethical point.

The Honest Answer

Why There Is No Universal Number

Start with why the honest answer is, it depends. It depends on what the study is for. On what kind of analysis is planned. On what kind of effect is expected. And on how precise the estimates need to be. One estimates the prevalence of a rare condition. One compares two interventions. One seeks thematic saturation in a qualitative design. Every one of those needs a different sample size. So be suspicious of anyone who offers you a universal magic number.

Three Logics

Power, Precision, Saturation

Three broad approaches dominate practice. Power analysis asks how many participants are needed to detect an effect. An effect of a given size, with a given probability of success. Precision-based approaches ask a different question entirely. How many are needed to estimate a population value within a given margin of error? Saturation-based approaches, used in qualitative research, ask a third question. How many cases before additional data stop yielding new insights? Picking the wrong logic is a worse error than picking a slightly wrong number. One pattern runs through all three. The smaller the effect, and the higher the power you want, the larger the sample. The stricter the alpha level, and the narrower the confidence intervals, the larger again.

The Standard Tool

The Four Components of a Power Analysis

Power analysis is the most widely used approach in studies that test hypotheses. It rests on four interrelated quantities. Effect size is the magnitude of the difference or relationship you aim to detect. Alpha is the probability of rejecting the null hypothesis when it is actually true. That is the Type One error rate, and it is commonly set at point zero five. Power is the probability of detecting an effect when one is truly present. It is commonly set at point eight zero, which is eighty percent. And sample size is the number of participants needed to reach that power. Now the structural fact that makes the whole method work. These four quantities are interrelated. Any three of them determine the fourth. That is exactly why you can fix three and solve for the one you want.

In Practice

How the Calculation Is Actually Run

In practice the calculation runs in a fixed order. First you specify alpha, most often point zero five. Then you specify the power you want, most often point eight zero. Then you estimate the expected effect size. That estimate comes from prior research, or from a pilot study. Or from a theoretical minimum for what would count as practically significant. Then you solve for the sample size. Free software such as G Power will do it, and so will standard statistical packages. They cover the common analyses. T-tests, analyses of variance, regressions, and chi-square tests. Notice where the weak link sits in that sequence. Three of the four inputs are conventions you simply choose. The effect size estimate is the one you have to justify, and it drives the answer.

The Lever

Effect Size Decides Everything

So how much does effect size actually matter? Of the four components, it has by far the largest influence on the required sample. Put numbers on it, and the point becomes vivid. Take a t-test, with alpha at point zero five and power at point eight zero. To detect a large effect, a Cohen's d of point eight, you need about twenty-six participants per group. To detect a small effect, a Cohen's d of point two, you need roughly three hundred and ninety-three per group. That is more than fifteen times as many people. For the same design, the same test, and the same conventions. The only thing that changed was what you expect to find. So the same study can go from feasible to impossible on that assumption alone. Which is why an honest effect-size estimate matters more than any other input you provide.

Estimation, Not Testing

Precision-Based Sample Size

Sometimes you are not testing a hypothesis at all. You are estimating something, such as the prevalence of a behavior in a population. For estimation, sample size is driven by the precision you want. Precision is expressed as the width of the confidence interval around the estimate. Narrower intervals require larger samples. That relationship is the whole story. The margin of error is the half-width of that interval. It is usually reported as a number of percentage points. For a single population proportion, three quantities set the requirement. The margin of error you want, and the confidence level you want. And the variability of the characteristic in the population. At maximum variability, here is the number worth memorizing. A ninety-five percent interval with a margin of plus or minus three percentage points. That needs roughly one thousand and sixty-seven respondents. Which is why so many national polls land right around that figure.

The Cost Curve

What Each Extra Point of Precision Costs

Now watch what precision actually costs. All of these assume ninety-five percent confidence, at maximum variability. A margin of plus or minus ten percentage points needs about one hundred people. Plus or minus five points needs about four hundred. Plus or minus three points needs about one thousand and sixty-seven. Plus or minus two points needs about two thousand four hundred. And plus or minus one point needs about nine thousand six hundred. Look at the shape of that progression rather than the individual numbers. Halving the margin of error roughly quadruples the sample you need. Precision is not bought linearly. It gets expensive fast. That single fact explains most of the sample sizes you will ever see in survey research.

A Different Logic

Saturation in Qualitative Research

Qualitative research runs on an entirely different logic. The criterion is neither statistical power nor precision. It is theoretical saturation. That is the point at which additional cases produce no new categories, themes, or insights. Sample sizes are therefore much smaller than in quantitative work. Phenomenological interviews typically run from five to twenty-five participants. That depends on how heterogeneous the experience is. Grounded theory typically runs twenty to fifty, continuing until categories are saturated. A single case study is one case, studied through multiple data sources. Multiple-case studies run four to fifteen cases, depending on theoretical purpose. And ethnography takes an entire community or setting, sampled opportunistically. Treat all of those as guidelines rather than rules. A homogeneous population studied in depth may saturate at five interviews. A complex and varied phenomenon may require dozens.

Beyond the Formula

Five Practical Considerations

The formula gives you a number. Five practical issues then adjust it. Anticipated attrition comes first. Recruit more than the analytic minimum, because some participants will drop out. Subgroup analyses come second, and this one catches people out. Your sample must support the smallest subgroup you care about. Not merely the overall total, which can look comfortable while a subgroup is starved. Multiple comparisons come third. Studies testing many hypotheses need larger samples to maintain power across those tests. Budget and feasibility are fourth, and they are real constraints, not excuses. A study with infeasible sample requirements may need redesigning rather than shrinking. And fifth, publication and review standards. Some fields and journals expect minimum sample sizes, so anticipate them early.

Both Directions

Too Few, and Too Many

Getting sample size wrong hurts in both directions. An underpowered study lacks the participants to detect effects that are actually there. Most underpowered studies fail to find their hypothesized effect, even when it is real. That produces spurious null findings. And then there is a second, nastier problem. When an underpowered study does detect an effect, that effect tends to be overestimated. The phenomenon has a name. The winner's curse. So repeated low-powered work distorts a literature in two directions at once. Inflated effect estimates where results reached significance, and misleading nulls everywhere else. Overpowered studies have their own failure mode. They waste resources, and they tend to find significant differences too small to matter. And they raise a genuine ethical question. It is not acceptable to enroll more participants than the research question requires. Especially when participation involves risk, burden, or scarce resources.

Calibration

Goldilocks, Not Maximum

So retire the instinct that bigger is always safer. The right sample size is the smallest that supports the intended inference. With adequate precision, and adequate power. More is always better is wrong methodologically, and wrong ethically as well. Determining sample size is a careful calibration, not a maximization problem. Finally, report the decision, because readers cannot evaluate what you do not describe. Say which approach you used. Power analysis, precision, or saturation. State the assumptions you made about effect size, alpha, power, and expected variability. State what the calculation actually produced. And if the sample you ended up with was smaller than planned, say so. Then say how that shortfall affects the strength of your conclusions.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Determining the Appropriate Sample Size

How do you calculate sample size with a power analysis?
You fix three of four linked quantities and solve for the fourth, the sample size. Researchers usually set alpha at .05 and power at .80, then estimate the expected effect size from prior research, a pilot study, or the smallest effect that would matter in practice. Free software such as G*Power, or a standard statistics package, then returns the number of participants needed for t-tests, ANOVA, regression, or chi-square tests.
Why does effect size matter so much when choosing a sample size?
Effect size has the largest influence of any input on the required sample. For a two-group t-test with alpha at .05 and power at .80, detecting a large effect (Cohen's d of 0.8) needs about 26 participants per group, while a small effect (d of 0.2) needs close to 400 per group, about 15 times as many. An optimistic effect-size guess can leave a study badly underpowered.
How many people do you need for a survey with a 3 percent margin of error?
About 1,067 respondents are needed for a 95 percent confidence interval of plus or minus 3 percentage points, assuming maximum variability (a 50/50 split). Precision gets expensive fast: plus or minus 5 points needs about 385 people, and plus or minus 2 points about 2,401. Halving the margin of error roughly quadruples the required sample, which is why many national polls survey around 1,000 people.
Can a research sample be too large?
Yes, an overpowered study wastes resources and tends to find statistically significant differences too small to matter. It also raises an ethical problem, because enrolling more participants than the question requires exposes extra people to risk or burden. Underpowered studies fail the other way: they miss real effects, and when they do reach significance they tend to overestimate the effect, a pattern known as the winner's curse.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →