UPI Study › Courses › Introduction to Artificial Intelligence › Free lessons › Lesson 1.6 Free with a free account
Core Ideas in Probability and Chance
Introduction to Artificial Intelligence · Chapter 1: Foundations of AI and Core ConceptsSection 1.6 · lesson 6 of 6915 min · free with a free account
Free preview
Create a free account to keep watching
It takes a minute and costs nothing. Your free account unlocks all of Chapter 1 and 3 more lessons of Introduction to Artificial Intelligence.
Lesson complete
Earn the 3 college credits for Introduction to Artificial Intelligence
Enrolling adds a quiz after each lesson, the graded work and the official transcript. Recommended for credit by NCCRS.
Next lesson: Measuring Data Quality: Precision, Accuracy, and Pitfalls →← Techniques for Computing Logarithmic ValuesNext lesson: Measuring Data Quality: Precision, Accuracy, and Pitfalls →
In this lesson
Probability rules, conditional chance, expectation, spread, and four common distributions.
- Apply the probability axioms, the complement rule, and the addition rule to events
- Calculate conditional probabilities and distinguish independence from mutual exclusivity
- Compute the expectation, variance, and standard deviation of a random variable
- Identify uniform, Bernoulli, binomial, and Gaussian distributions and their means
Free study notes
Get this lesson's notes by email
The key points and 4 practice questions with answers, in your inbox. One email, free.
One email. Unsubscribe in one click.
Transcript
Everything said in the video, section by section (2,224 words). Read along, or search it.
Opening Question
Putting Numbers on Uncertainty
A spam filter says a message is 93% likely to be junk. A navigation app gives you a 70% chance of arriving on time. Almost nothing an AI system meets is certain: blurry images, sensor readings a little off, clicks that may or may not come. Probability describes that uncertainty with numbers, instead of guesses. So how do we attach those numbers, and how should new information change them?
Sample Space
Experiments, Outcomes, and Sample Spaces
Start with an experiment: any process with an uncertain result. Rolling a die, checking an email for spam, reading a temperature. Each possible result is an outcome. The set of all possible outcomes is the sample space, usually written as a capital S. For a die, S is one through six. For a single email, S has just two outcomes: spam, or not spam.
Events
Probability as a Simple Fraction
An event is any collection of outcomes we care about. An even roll is the event A: two, four and six. A roll greater than four is the event B: five and six. When every outcome is equally likely, an event's probability is a simple fraction: its outcomes, divided by all the outcomes. So P(A) is 3/6, or 1/2. And P(B) is 2/6, or 1/3.
Combining Events
And, Or, and Not
Events can be combined. A and B means both happen: here, only the six. A or B means at least one happens: two, four, five or six. And not A means A doesn't happen: one, three or five.
Probability Axioms
Three Rules Everything Builds On
Every assignment of probabilities must obey three basic rules, called the probability axioms. Everything else is built on them. Rule one: a probability is never negative. Rolling a three has probability one sixth, a positive number. Rule two: the whole sample space has probability one. Something on the list must happen. Rule three: if two events can't happen together, their probabilities add. A one or a two: one sixth plus one sixth, a third.
Derived Rules
Complement and Addition Rules
Three handy facts follow directly from the axioms. First, every probability lies between zero and one. Second, the complement rule. The chance that A doesn't happen is whatever is left over: one minus P of A. For the die, P of odd is one minus a half, which is a half. Third, when events can overlap, the addition rule stops the overlap from being counted twice. P of A or B equals P of A, plus P of B, minus P of A and B. For even, and greater than four: three sixths, plus two sixths, minus one sixth. That's four sixths, or two thirds.
Conditional Probability
Shrinking the Sample Space
Probabilities change when we learn something new. Conditional probability is the chance of an event, once we know another event has happened. It's written P of A given B. Learning B shrinks the sample space to B's outcomes, and we ask what share are also in A. The formula: P of A given B equals P of A and B, divided by P of B, as long as P of B isn't zero. A friend peeks at the die, and says the roll is greater than four. Only five and six are left. Just one of those, the six, is even. So P of even given greater than four is one sixth, divided by two sixths: one half.
Worked Example
The Word Winner Lifts Spam to 96 Percent
Now a realistic case. A team checks a thousand emails for the word winner, and for spam. Two hundred forty are spam with winner, and ten are good emails with it. Sixty spam emails lack it, and so do six hundred ninety good ones. Overall, 300 of the 1,000 are spam, so P(spam) is 0.30. But once we know the email contains winner, we look only at that row: 240 out of 250, or 0.96. One word lifts the chance of spam from 30% to 96%. That jump is exactly what makes the word useful evidence.
Common Mistake
Reversing the Condition Changes the Answer
Order matters. Flip the question around: what share of spam emails contain the word? That's P of winner given spam. Now we look only at the spam column: 240 out of 300, or 0.80. Same table, a different starting point, and a different answer. Mixing up P of A given B, and P of B given A, is one of the most common mistakes. A rule for converting one into the other does exist. It's called Bayes' rule, and section 4.2 covers it.
Independence
When Probabilities Multiply
Two events are independent when knowing one happened doesn't change the chance of the other: P of A given B equals P of A. An easier form to use: for independent events, P of A and B is P of A times P of B. Two coin flips are independent, so two heads has chance a half times a half: one quarter. Drawing two cards without putting the first back is different. If the first is a king, fewer kings remain for the second. Our die events are independent: P of even given greater than four was one half, the same as P of even. But spam and winner are dependent: 0.30 overall, 0.96 with the word. That's what lets a filter learn from words.
Key Distinction
Mutually Exclusive Is Not Independent
Don't confuse independent with mutually exclusive. Mutually exclusive events can't both happen, so P of A and B is exactly zero. The roll is one, and the roll is six, are mutually exclusive. A single roll can't show both. The two ideas are nearly opposites. If two possible events are mutually exclusive, they're dependent. Learning that one happened tells you for certain that the other didn't.
Quick Check
Was Rain Forecast on a Rainy Day
Quick check. A weather app looks back over a hundred days, with illustrative numbers. On thirty days, it forecast rain, and it rained. On ten, it forecast rain, and it stayed dry. On five, it rained with no forecast. And on fifty-five, there was no forecast, and no rain. Today it rained. What's the chance that rain had been forecast? Pause, and decide. Thirty out of thirty-five, about 86%. Knowing it rained, we look only at the rained column: thirty plus five, thirty-five days. Rain was forecast on thirty of them. Thirty out of forty is the tempting answer, but it starts from the forecast row. It answers a different question: when rain is forecast, how often does it rain?
Random Variables
Discrete and Continuous Random Variables
Now, numbers produced by chance. A random variable is a rule that turns each outcome into a number, usually written with a capital letter. If X is the number showing on a die, X can be one through six. If Y counts the heads in three coin flips, Y can be zero, one, two or three. Its probability distribution lists each possible value, with its probability. A discrete random variable takes separate values you can count, like a die roll, or the clicks on an ad. A continuous one takes any value in a range, like a temperature, or a load time. Its probabilities are areas under a curve.
Expectation
The Long-Run Average
The expectation of a random variable, written E of X, is its long-run average. To find it, multiply each value by its probability, and add the results. For a fair die: one times a sixth, plus two times a sixth, and so on, up to six times a sixth. That adds up to twenty-one sixths, or three point five. No single roll can show three point five. It's the average over many rolls, not a result you'll see. One more: a game pays a bonus of ten coins with probability zero point two, and nothing otherwise. Ten times zero point two, plus zero times zero point eight: two coins per try, on average.
Variance
Two Sensors With the Same Average
Two quantities can share an average, and still behave very differently. Picture two temperature sensors, both averaging twenty degrees. Sensor A reads nineteen, twenty, or twenty-one, each equally likely. Sensor B reads ten, twenty, or thirty. Variance measures how far values typically stray from the average. It's the average of the squared distances from the mean. For A, the distances are minus one, zero and plus one. Squared, that's one, zero and one. Averaged: two thirds, about zero point six seven. For B, the distances are minus ten, zero and plus ten. Squared, a hundred, zero and a hundred. Averaged: about sixty-six point six seven. Squaring removes the minus signs, and makes big misses count more.
Standard Deviation
Spread Back in Original Units
The standard deviation is the square root of the variance. It brings the measure back to the original units, like degrees, so it's easier to read. Sensor A's standard deviation is about zero point eight two degrees. Sensor B's is about eight point one six. Same expectation, but B is far less trustworthy for any single reading. The average can't tell them apart. The spread can. And the fair die? Its center is three point five. Its variance is about two point nine two, and its standard deviation about one point seven one.
Uniform Distribution
Every Outcome Equally Likely
Some patterns of chance appear so often that they have names, and ready-made formulas. Four of them cover most needs. In a uniform distribution, every outcome is equally likely. A fair die is a discrete example. A random number anywhere between zero and one is a continuous one. If the values run from a to b, the mean is the midpoint: a plus b, over two. For the die, one plus six, over two, is three point five.
Bernoulli Trial
One Yes-or-No Experiment
A Bernoulli trial is a single experiment with exactly two outcomes: success, worth one, and failure, worth zero. Success has probability p. Did the user click the ad? Is this email spam? Its mean is p, and its variance is p times one minus p. If the chance of a click is zero point one, the mean is zero point one, and the variance is zero point one times zero point nine: zero point zero nine. The variance is largest when p is a half: a fair coin is the least predictable. It shrinks toward zero as p nears zero or one.
Binomial Distribution
Counting Successes in n Trials
Now repeat a Bernoulli trial n times, independently, with the same p each time, and count the successes. That count follows a binomial distribution. The chance of exactly k successes is n choose k, times p to the k, times one minus p, to the power n minus k. n choose k counts the ways to pick which k of the n trials are the successes. The mean is n times p, and the variance is n times p times one minus p.
Worked Example
Exactly Two Spam Emails Out of Five
Five emails arrive, and each one independently has a 30% chance of being spam. What's the chance that exactly two are spam? Step one: five choose two is ten. There are ten ways to choose which two emails are the spam ones. Step two: each such arrangement has probability zero point three squared, times zero point seven cubed. That's zero point zero nine, times zero point three four three, which is zero point zero three zero eight seven. Step three: ten times that is zero point three zero eight seven, about thirty-one percent. Drawn for every count from zero to five, one spam email is the most likely, and two is close behind. The expected count is five times zero point three: one and a half.
Gaussian Distribution
The Bell Curve and the 68-95-99.7 Rule
The Gaussian, or normal, distribution is the famous bell curve: continuous, and symmetric around its mean. Two numbers describe it fully. The mean sets the center, and the standard deviation sets the width: small gives a tall, narrow bell, large a low, wide one. A handy rule of thumb: about 68% of values fall within one standard deviation of the mean, about 95% within two, and about 99.7% within three. So a sensor reading twenty degrees, with a standard deviation of zero point five, lands between nineteen and twenty-one about ninety-five percent of the time. The bell is common because many quantities are sums of lots of small, independent effects, like the tiny disturbances that add noise to a sensor. That's why Gaussian noise is the standard model for measurement error. And one exact value has probability zero. Probabilities are areas under the curve, between two values.
Quick Check
Counting a Robot's Successful Grabs
Quick check. A warehouse robot tries to grab a box eight times. Each grab succeeds with chance zero point seven five, independently of the others. Which pattern describes the number of successful grabs, and what's its average? Pause, and decide. Binomial, with an average of six. It counts successes in eight independent tries, each with the same chance. So n is eight, p is zero point seven five, and the mean, n times p, is six. Bernoulli is tempting, but it describes a single grab, not the count over eight. And uniform would need every count, from zero to eight, to be equally likely. They aren't.
Key Takeaways
Axioms, Conditions, Spread, and Distributions
To sum up. A sample space lists every outcome, events are sets of outcomes, and probabilities obey three axioms. Conditional probability looks only at the cases where B happened. And P of A given B isn't P of B given A. Independent events leave each other's chances alone, so their probabilities multiply. Expectation is the long-run average. Variance and standard deviation measure the spread. And uniform, Bernoulli, binomial and Gaussian describe equal chances, one yes-or-no trial, counts of successes, and bell-shaped noise.
Reflection
Two Spam Words Used Together
Here's something to think about. A spam filter sees two words in one email: winner, and free. If it treats them as independent, it simply multiplies their evidence. But spammers often use the two together. Would the filter end up too sure, or not sure enough?
About Introduction to Artificial Intelligence at UPI Study
Introduction to Artificial Intelligence (Computer Science 373) is a self-paced online course of 69 video lessons in 9 chapters, from intelligent agents and search to game strategy, neural networks, logic and reasoning, machine learning, language, robotics and AI ethics. It carries a NCCRS credit recommendation for 3 semester credits.
Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Enroll now → Course details Download the syllabus All Computer Science courses Every course with PRO →
Compare: AI vs Current Trends in CS and IT
Why take it here
Why students take Introduction to Artificial Intelligence with UPI Study
Where the credit goes
Universities that work with NCCRS
Introduction to Artificial Intelligence carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Thomas Edison State UniversitySigned partner
University of the PeopleSigned partner
Charter Oak State CollegeSigned partner
Strayer UniversitySigned partner
Columbia Southern UniversitySigned partner
Indiana TechSigned partner
Penn StateWorks with NCCRS
Southern New Hampshire UniversityWorks with NCCRS
SUNYWorks with NCCRS
+many more cooperating universitiesFind my college →Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.
FAQ
Questions students ask us
About Core Ideas in Probability and Chance
What is conditional probability?
What is the difference between independent and mutually exclusive events?
What is the difference between variance and standard deviation?
How do you calculate a binomial probability?
About the Introduction to Artificial Intelligence course
How many college credits is Introduction to Artificial Intelligence?
How much does Introduction to Artificial Intelligence cost and what is included?
Which universities accept UPI Study credits?
How long does Introduction to Artificial Intelligence take?
How are the exams done?
How do my credits reach my college?
Ready for the full course? Earn 3 credits in Introduction to Artificial Intelligence.
69 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.
3 credits · NCCRSEnroll →