Recognizing Main Effects in Factorial Studies, Explained

UPI Study › Courses › Research Methods in Psychology › Free lessons › Lesson 13.3 Free with a free account

Recognizing Main Effects in Factorial Studies

Research Methods in Psychology · Chapter 13: Factorial, Within-Subject, and Small-n DesignsSection 13.3 · lesson 79 of 949 min · free with a free account

← Making Sense of Interactions in Factorial DesignsNext lesson: Applying Multivariate Experimental Designs →

Earn 3 college credits for Research Methods in PsychologyRecommended for credit by NCCRS · Psychology 111 · self-paced · one payment, lifetime access
Enroll in Research Methods in Psychology · $250 →Download the syllabus (PDF)

In this lesson

What the marginal averages tell, and when they should be read with caution.

  • Define a main effect and compute it from cell means
  • Recognize main effects on an interaction graph
  • Judge when a main effect must be qualified
  • Separate statistical significance from theoretical meaning

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,247 words). Read along, or search it.

Factorial, Within-Subject, and Small-n Designs

Recognizing Main Effects in Factorial Studies

Main effects look like the simple part of a factorial design. They are simple to compute, and easy to misread. By the end of this section you will define a main effect precisely. And compute one from a table of cell means. You will recognize main effects on an interaction graph, by attending to the marginal averages. You will determine when a main effect stands alone, and when it must be qualified. You will distinguish the statistical significance of a main effect from its theoretical meaning. And you will report main effects clearly, alongside any interaction that accompanies them.

The Definition

An Average Across Everything Else

A main effect is the average effect of one independent variable. Averaged across the levels of all the other independent variables in the design. Make that concrete with a two by two study, with factors A and B. The main effect of A compares two averages. The average outcome across everyone who received level A one. Regardless of what level of B they happened to get. Against the average outcome across everyone who received level A two. Those two averages have a name. They are the marginal means for factor A. You compute them from the rows or columns of the cell-mean table. Which is why they sit in the margins, and why the name stuck.

How Many

One Main Effect Per Factor

Every independent variable in a factorial design has its own main effect. So a two by three design has two main effects. A two by two by two design has three. The count follows the number of factors, not the number of cells. And each main effect addresses a separate question. Whether that variable, on average, produces a difference in outcomes across its levels. Notice the phrase on average appearing again. It is the source of everything useful about main effects. And of everything dangerous about them.

On the Graph

Two Places to Look

The interaction graph is not just for interactions. It is the easiest place to see main effects too, once you know where to look. There are two places, one for each factor. The factor on the horizontal axis shows its main effect as the average slope of the lines. Lines that climb together from left to right mean the outcome rises with that variable. The factor drawn as separate lines shows its main effect differently. It appears as the vertical separation between the lines. When one line sits consistently above another, that variable has a main effect on the outcome. Slope for one factor, separation for the other. Reading both from the same graph trains your eye to see structure quickly.

The Condition

Interpretability Depends on the Interaction

Now the question that governs everything else in this section. Whether a main effect can be interpreted depends on whether an interaction is present. Suppose the lines on the interaction graph are roughly parallel. No interaction, or only a small ordinal one. Then the main effect describes the average effect of the variable. And that average is a reasonable summary of what happens at every level of the other factor. Now suppose the lines are non-parallel, and especially suppose they cross. The main effect still computes. It just stops meaning what you want it to mean. It summarizes an average that may not reflect any actual condition in the study. A midpoint between conditions where the effect runs in opposite directions.

The Decision Table

Four Patterns, Four Verdicts

That reasoning sorts into four patterns, each with its own verdict. With no interaction, and parallel lines, the main effect is fully interpretable. It describes a consistent effect across conditions. With an ordinal interaction, non-parallel but not crossing, interpret with care. The main effect captures the average, but magnitude varies across conditions. With a disordinal interaction, where the lines cross, the main effect is often misleading. It averages opposite-direction effects, so report the simple effects instead. And there is a fourth case worth naming. A significant interaction alongside small main effects. Set the main effects aside. The interaction is the story, and the rest just confuses readers.

Order Matters

Examine the Interaction First

All of that implies an order of operations, and it is not the intuitive one. Standard practice in factorial analysis is to examine the interaction first. Before the main effects, not after them. If the interaction is significant, the main effects are interpreted in its light. And sometimes set aside entirely, in favor of simple effects. If the interaction is not significant, then the main effects can stand in their own right. Reversing that order is where most misreadings begin. Reporting main effects without first checking the interaction is extremely common. And it is one of the most common ways factorial results get misread.

Significance

What a Significant Main Effect Actually Says

Now separate two things that are easy to conflate. A statistically significant main effect tells you something quite narrow. The marginal averages differ by more than sampling variability alone would produce. That is the entire claim. It does not tell you whether the difference is large. It does not tell you whether the difference is theoretically interesting. And it says nothing about whether the difference is practically important. The reason is that significance depends heavily on sample size. With large samples, even tiny differences become statistically significant. With small samples, even large differences may fail to reach significance.

Effect Sizes

The Number That Answers 'How Much'

The answer to how much is the effect size. Reporting main effects without one leaves readers unable to judge whether a finding matters. An effect size is a standardized measure of magnitude, independent of sample size. That independence is exactly what makes it useful. Cohen's d is one common choice. It expresses the difference between two means in standard deviation units. Partial eta squared is the usual choice in analysis of variance. It expresses the proportion of variance attributable to a factor. Specifically, after the other factors have been removed.

Judgment

Statistics Cannot Decide What Matters

Effect sizes also help with a distinction that no p value can make. Consider a small main effect in a high stakes context. A small reduction in mortality, for instance. That effect may be tiny in standardized units and still be enormously important. Now consider a large main effect in a context where small differences do not matter. The number is bigger and the finding means less. So importance is not a property of the statistic at all. The statistical machinery of factorial designs cannot make this judgment. The researcher must, and must argue for it explicitly.

Reporting

Four Pieces of a Clean Report

A clean report of a main effect includes four pieces of information. First, the marginal means, one for each level of the factor. Readers cannot reconstruct those from anything else, so they have to be there. Second, the test statistic, with its degrees of freedom and p value. Third, an effect size estimate, so magnitude is visible alongside detectability. Fourth, a brief verbal description that situates the result inside the design. And when an interaction is also present, that description has a job to do. It should either qualify the main effect appropriately. Or defer to simple-effects analysis, as the more informative breakdown.

A Better Practice

Reports Should Argue, Not Dump

One last habit, about what to emphasize. Many published factorial studies report every effect in equal detail. All main effects, all interactions, regardless of which is theoretically central. The result reads as a data dump rather than an argument. A clearer practice starts before the analysis. Identify which effects answer the central research question. Then give those effects the most prominent treatment in the write-up. Other effects can be reported briefly, or in supplementary materials. And here is the essential qualifier on that advice. Selectivity guided by pre-specified questions produces clearer science. Selectivity guided by the results you happened to get does the opposite.

About Research Methods in Psychology at UPI Study

Research Methods in Psychology (Psychology 111) is a self-paced online course of 94 video lessons in 16 chapters, from the scientific method and research ethics to designs, data collection, sampling, measurement, validity, experiments, descriptive statistics and hypothesis testing. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Psychology and social science courses Every course with PRO →

Compare: Research Methods vs Statistics

Why take it here

Why students take Research Methods in Psychology with UPI Study

How psychologists know what they knowResearch Methods in Psychology teaches how psychological studies are designed, run and judged: research ethics, surveys and observation, correlational and experimental designs, sampling, reliability and validity, and the statistics used to test hypotheses. It is a core course in psychology and many social-science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Research Methods in Psychology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Recognizing Main Effects in Factorial Studies

What is a main effect in a factorial design?
A main effect is the average effect of one independent variable, averaged across the levels of the other independent variables. It compares marginal means, the row or column averages of the cell-mean table. For example, with equal cell sizes, if the A1 cells have means of 10 and 20 and the A2 cells have means of 30 and 40, the marginal means for A are 15 and 35. Each factor has its own main effect.
How do you see main effects on an interaction graph?
The factor on the horizontal axis shows its main effect as the average slope of the lines, and the factor drawn as separate lines shows its main effect as the vertical separation between them. Lines that rise together from left to right suggest a main effect of the horizontal-axis factor. One line sitting consistently above another suggests a main effect of the factor shown by the lines.
Can you interpret a main effect when there is a significant interaction?
It depends on the type of interaction, which is why the interaction should be examined before the main effects. With parallel lines and no interaction, a main effect is fully interpretable. With an ordinal interaction, it can be interpreted with care because the size of the effect varies. With a disordinal interaction, where the lines cross, the main effect averages opposite-direction effects and is often misleading, so simple effects should be reported instead.
What should a report of a main effect include?
A clear report of a main effect includes the marginal means for each level of the factor, the test statistic with its degrees of freedom and p value, an effect size such as partial eta squared, and a short description placing the result in the design. Significance shows only that the marginal means differ more than sampling variability would produce, not how large or important the difference is. If an interaction is present, the description should qualify the main effect.

About the Research Methods in Psychology course

How many college credits is Research Methods in Psychology?
Research Methods in Psychology (Psychology 111) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Research Methods in Psychology cost and what is included?
$250, paid once, with lifetime access: all 94 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Research Methods in Psychology take?
It is self-paced with no deadlines. The 94 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Research Methods in Psychology.

94 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Research Methods in Psychology →Download the syllabusEvery course with PRO →
Research Methods in Psychology
3 credits · NCCRSEnroll →