Reasoning Backward From Evidence, Explained with Examples

UPI Study › Courses › Quantitative Analysis › Free lessons › Lesson 6.3 Free with a free account

Reasoning Backward From Evidence

Quantitative Analysis · Chapter 6: Bayesian Revision and UtilitySection 6.3 · lesson 28 of 5512 min · free with a free account

← Business Decision StrategiesNext lesson: Is More Data Worth It? →

Earn 3 college credits for Quantitative AnalysisRecommended for credit by NCCRS · Math 250 · self-paced · one payment, lifetime access
Enroll in Quantitative Analysis · $250 →Download the syllabus (PDF)

In this lesson

Bayes' Theorem - defined, derived, and applied.

  • Derive Bayes' Theorem from conditional and total probability
  • Separate prior, likelihood, marginal and posterior
  • Apply the update to quality, credit and supply decisions
  • Compute a posterior and expose the base rate fallacy

Transcript

Everything said in the video, section by section (1,813 words). Read along, or search it.

QAN-201 - Quantitative Analysis · Section 6.3

Reasoning Backward From Evidence

Bayes' Theorem is the mathematics of changing your mind. It answers a question that ordinary probability does not. Not simply, what is the probability, but rather, how should that probability change once I observe this evidence? That reversal is the entire idea of this section. You usually know the forward direction, how likely the evidence is when a hypothesis happens to be true. What you actually need is the backward direction. Given the evidence in front of you, how likely is the hypothesis now? We will derive the theorem from first principles, in four logical steps. Then apply it to quality control, credit risk, and supply reliability. And finish with an inspection problem that almost everyone gets badly wrong.

The Question

What Bayes' Theorem Actually Answers

Bayes' Theorem is the mathematical foundation of probabilistic reasoning with evidence. It formalises something you already do, you hold a belief, information arrives, and the belief moves. The theorem says exactly how far it should move, and in which direction. There are two directions of reasoning here, and keeping them apart is essential. The forward direction asks how likely the evidence is, assuming a hypothesis is true. That is usually the measurable quantity, a test's detection rate, or a supplier's historical failure rate. The backward direction asks how likely the hypothesis is, now that the evidence has been seen. That is what a decision maker actually needs. And here is the trap, they are not the same number. Confusing the two is the single most expensive mistake in this whole chapter.

Formal Derivation

Three Steps From Definition to Denominator

The theorem is not an axiom, it is a consequence. It emerges naturally from the definition of conditional probability, combined with the total probability theorem. The derivation proceeds in four logical steps, and here are the first three. Step one is the definition itself. The probability of A given B equals the joint probability of A and B, divided by the probability of B. Nothing clever yet, that is simply what conditional probability means. Step two substitutes the joint probability, the probability of A and B together equals the probability of B given A, times the probability of A. Push that into step one, and the reversal has already happened. Step three builds the denominator, and for mutually exclusive, exhaustive hypotheses the probability of B is a sum. Sum over every hypothesis, of the likelihood times the prior, because nobody ever hands you the marginal directly. Note the condition carefully, the hypotheses must be mutually exclusive and exhaustive. Drop either requirement, and the sum stops covering every route the evidence could have taken.

Step 4

The Full Theorem, and How to Read It

Now substitute step three into step two, and the theorem appears in full. The posterior probability of hypothesis A sub i, given evidence B. Equals the likelihood times the prior, divided by a sum over all hypotheses. That sum is the same likelihood times prior product, added up across every competitor. Read the numerator first, it is the likelihood multiplied by the prior. A hypothesis scores well only on two conditions. It has to have been plausible to begin with, and it has to explain what you actually saw. If either factor sits near zero, the whole product collapses. Now the denominator, the total probability of the evidence across every competing hypothesis. Dividing by it forces all the posteriors to add up to one. So the denominator is not new information, it is a normaliser.

Key Terminology

Four Names You Must Keep Apart

Four names, and confusing any two of them produces a wrong answer. The prior is your belief about a hypothesis before observing any evidence, written P of A sub i. The likelihood is the probability of observing the evidence if that hypothesis is true. Written P of B given A sub i. The marginal probability is the total probability of the evidence, across all hypotheses, written simply P of B. And the posterior is your updated belief, after the evidence has been observed. Written P of A sub i given B. Notice that the likelihood and the posterior look almost identical on the page. The conditioning bar just points the other way, and that similarity is exactly why people swap them. One sentence holds the whole theorem together. The posterior weights the prior by the likelihood, then normalises across all competing hypotheses.

In Business

One Structure, Three Different Decisions

An abstract formula becomes a decision engine the moment you attach it to a real problem. And the structure is identical every time. A prior belief about the state of the world, new evidence, and a revised belief. Take quality control first, a test signals defective, and you want to know whether the item really is. Defect rates are typically low, so even a small false positive rate generates a great many false alarms. Bayes' Theorem quantifies that precisely, and the answer is almost always lower than intuition suggests. Credit risk works the same way, a lender starts with a prior default probability for a class of borrower. Then evidence arrives, a missed payment, or a changed debt to income ratio. The revised posterior default probability is what informs the credit decision. Supply chain risk is the third case, a supplier delivers late. Is that systemic, calling for tighter oversight or a switch, or simply a random delay? The prior systemic failure rate, and the likelihood under each scenario, settle it.

Sequential Updating

Each Posterior Becomes the Next Prior

Evidence rarely arrives all at once, so Bayes' Theorem is built to be applied again and again. The rule is simple, and it is the most useful thing in this section. The posterior from round one becomes the prior for round two. Start with your initial belief, before any evidence at all. The first signal arrives, and you apply the theorem once. That gives a posterior, the probability of the hypothesis given the first piece of evidence. Now stop thinking of it as a posterior, it is simply your new prior. The second signal arrives, and you apply exactly the same theorem again. The final result reflects all the accumulated evidence. And here is the property worth remembering, the order in which evidence arrives does not change the end result. Two signals, processed in either sequence, land on the same final posterior.

Worked Problem

Quality Inspection on the Assembly Line

Now a full worked problem, and it is the classic one. A manufacturer's quality inspector tests products coming off an assembly line. The inspection process has three published characteristics. First, the true positive rate, also called sensitivity, is ninety per cent. If a product is genuinely defective, the test flags it as defective ninety per cent of the time. Second, the false positive rate is five per cent. If a product is perfectly fine, the test still flags it as defective five per cent of the time. Third, the base defect rate, which is our prior, is two per cent. Two per cent of all products coming off the line are actually defective. Now the question, a product fails inspection, what is the true probability that it is actually defective? Before we compute, write down your own guess.

Step 1

Construct the Probability Tree

Step one, construct the probability tree. Do no arithmetic yet, just name every given value and attach a symbol to it. The tree branches twice, first on the true state of the product, then on what the test says. The probability that a product is truly defective is zero point zero two. So the probability that it is not defective is zero point nine eight. Those two are the first branching, and they must sum to one. The test fails, given a defective product, with probability zero point nine, the true positive rate. The test fails, given a sound product, with probability zero point zero five, the false positive rate. Two more branches follow for free, from the complements. A defective item passes with probability zero point one. And a sound item passes with probability zero point nine five. Each pair of branches leaving a node adds to one, which is the check on your tree.

Step 2

Apply Bayes' Theorem

Step two, apply the theorem, and notice that only two branches of the tree end in a failed inspection. Multiply along each branch, add them together, then divide. The first path is defective, and then fails. Zero point zero two times zero point nine, which is zero point zero one eight. The second path is not defective, and then fails anyway. Zero point nine eight times zero point zero five, which is zero point zero four nine. Add those two joint probabilities, and the total probability of a failed test is zero point zero six seven. That is the marginal probability of the evidence, the denominator of Bayes' Theorem. Now divide the path we care about by the total. Zero point zero one eight, divided by zero point zero six seven, gives roughly zero point two six nine. Or twenty six point nine per cent. Look at those two joint probabilities again, and notice which one is bigger. The false alarms outweigh the true detections by nearly three to one.

Step 3

Why 26.9% Feels Wrong: the Base Rate Fallacy

Twenty six point nine per cent. Compare that against the guess you wrote down. Most people, on hearing a ninety per cent detection rate, assume a failed item is defective with about ninety per cent probability. The true answer is less than a third of that. This gap is not a rounding detail, it is the entire result. So why does intuition fail so badly here? Because the defect rate is only two per cent. Think in counts rather than probabilities, it makes the trap visible. Out of every thousand products, roughly twenty are actually defective. But forty nine perfectly good products will also trigger a false alarm. False positives numerically dominate, even for a good test, when the condition being tested is rare. That is the base rate fallacy, ignoring a low prior overestimates the posterior after a positive signal. And the rarer the condition, the worse the error gets.

What It Costs

Business Implications and What to Carry Forward

This is not an academic curiosity, it has a price tag. Blindly rejecting every failed item throws away roughly seventy three per cent of them, items that are actually fine. That drives up rework costs, it creates delays, and it lengthens customer wait times. Which points straight at the fix. A two stage inspection, or a human re-check of flagged items, is often more economical. Let the cheap test narrow the field, and let a second look do the deciding. Four things to carry out of this section. First, the theorem derives from the definition of conditional probability, and the total probability theorem. It is a consequence, not an assumption. Second, the posterior weights the prior by the likelihood, then normalises across all competing hypotheses. Third, it applies anywhere evidence should update a belief, quality control, credit risk, supply reliability. Fourth, sequential updating is possible, apply the theorem once, then use the posterior as the prior for the next piece of evidence. So if that same item fails a second, independent test, run it again with twenty six point nine per cent as the prior.

About Quantitative Analysis at UPI Study

Quantitative Analysis (Math 250) is a self-paced online course of 55 video lessons in 13 chapters, from decision analysis and simulation to inventory and queuing models, project scheduling, linear programming and transportation models. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Math courses Every course with PRO →

Compare: Statistics vs Quantitative Analysis

Why take it here

Why students take Quantitative Analysis with UPI Study

Decide with numbersQuantitative Analysis teaches the models managers use to decide under uncertainty: decision trees, simulation, inventory and queuing models, project scheduling and linear programming. It is part of many business, management, operations and supply-chain programs.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Quantitative Analysis carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Reasoning Backward From Evidence

What does Bayes' theorem calculate?
Bayes' theorem calculates a posterior probability: how likely a hypothesis is after you observe some evidence. It reverses the direction of conditioning. You usually know the likelihood P(B \mid A_i), how probable the evidence is if the hypothesis is true, and the theorem converts it into P(A_i \mid B) by multiplying the likelihood by the prior and dividing by the total probability of the evidence across all mutually exclusive, exhaustive hypotheses.
What is the difference between the prior, likelihood, marginal and posterior in Bayes' theorem?
The prior P(A_i) is your belief before seeing evidence; the likelihood P(B \mid A_i) is the chance of the evidence if the hypothesis is true; the marginal P(B) is the total probability of the evidence summed over all hypotheses; and the posterior P(A_i \mid B) is the updated belief. Likelihood and posterior look alike, but the conditioning bar points in opposite directions, which is why they are often swapped.
If a test with 90% sensitivity flags a defect, how likely is the item to actually be defective?
It depends heavily on the base rate. With a 2% defect rate, 90% sensitivity and a 5% false positive rate, P(\text{fail}) = 0.02 \times 0.9 + 0.98 \times 0.05 = 0.067, so P(\text{defective} \mid \text{fail}) = 0.018 / 0.067 \approx 26.9\%. False alarms (0.049) outnumber true detections (0.018) by nearly three to one, so most flagged items are actually fine.
What is the base rate fallacy?
The base rate fallacy is ignoring a low prior probability and so overestimating the posterior after a positive signal. People hear that a test catches 90% of defects and assume a flagged item is about 90% likely to be defective. When the condition is rare, false positives from the large healthy group can swamp the true positives. Thinking in counts, such as per 1,000 items, makes the effect visible, and the rarer the condition, the larger the error.

About the Quantitative Analysis course

How many college credits is Quantitative Analysis?
Quantitative Analysis (Math 250) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Quantitative Analysis cost and what is included?
$250, paid once, with lifetime access: all 55 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Quantitative Analysis take?
It is self-paced with no deadlines. The 55 lessons total about 11 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Quantitative Analysis.

55 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Quantitative Analysis →Download the syllabusEvery course with PRO →
Quantitative Analysis
3 credits · NCCRSEnroll →