Maximizing Outcomes: Utility-Driven Agent Behavior, Explained

UPI Study › Courses › Introduction to Artificial Intelligence › Free lessons › Lesson 2.8 Free with a free account

Maximizing Outcomes: Utility-Driven Agent Behavior

Introduction to Artificial Intelligence · Chapter 2: Intelligence, Cognition, and Agent DesignSection 2.8 · lesson 16 of 6914 min · free with a free account

← Agents That Plan Toward ObjectivesNext lesson: Agents That Improve Through Experience →

Earn 3 college credits for Introduction to Artificial IntelligenceRecommended for credit by NCCRS · Computer Science 373 · self-paced · one payment, lifetime access
Enroll in Introduction to Artificial Intelligence · $250 →Download the syllabus (PDF)

In this lesson

How utility-based agents use expected utility, risk attitudes, and weighted trade-offs.

  • Explain how a utility function lets an agent rank outcomes that a goal test cannot separate
  • Calculate expected utility and apply the maximum expected utility principle
  • Describe how the shape of a utility function sets risk attitude and find a certainty equivalent
  • Combine several attributes into a weighted utility and identify dominated options

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,853 words). Read along, or search it.

Opening Question

Two Taxi Trips, Both on Time

Two taxi drivers both reach the airport on time. One took a calm route, and arrived relaxed. The other raced through heavy traffic, burned extra fuel, and arrived with seconds to spare. Ask only: was the goal reached? Then both succeeded equally. Yet almost everyone would say the first trip was better. So how can an agent tell better from worse, when the results are uncertain, and when the things it cares about pull in different directions?

Utility Function

Scoring Every Outcome With a Number

A goal is a yes-or-no target. A utility function goes further. It gives every possible outcome a number, and a higher number means a more desirable outcome. A utility-based agent chooses actions that lead to outcomes with high utility. That lets it do three things a goal-based agent can't. It can rank two ways that both reach the goal. It can weigh a risky option against a safe one. And it can balance conflicting aims, by combining them into one score.

What the Numbers Mean

Utility Expresses Preference

What do the numbers mean? Utility expresses preference. It isn't a physical quantity like minutes or dollars. A utility of 80 beats 60, and the size of the gap matters when outcomes are uncertain. You may rescale every utility, say from a zero-to-ten scale to zero-to-a-hundred, and no choice changes. What you may not do is stretch the scale unevenly. That changes how outcomes trade off.

Expected Utility

Probability Times Utility for Each Outcome

Most actions don't lead to one certain outcome. So for every possible outcome, the agent combines two things: how likely it is, a probability, and how good it is, a utility. The expected utility of an action: for each outcome, multiply its probability by its utility, then add up the results. It's a probability-weighted average of how good the outcomes are. Here's a choice of route to the airport. On the highway, there's a 60 percent chance of clear traffic, utility 95, and a 40 percent chance of a heavy jam, utility 20. On the back roads, there's a 90 percent chance of an on-time arrival, utility 75, and a 10 percent chance of a slow trip, utility 45.

Worked Example

Highway 65, Back Roads 72

Now compute. Highway: 0.6 times 95 is 57. 0.4 times 20 is 8. Add them: 65. Back roads: 0.9 times 75 is 67.5. 0.1 times 45 is 4.5. Add them: 72. The back roads win, even though the highway has the best single outcome. The highway carries a sizable chance of a very bad trip. The back roads are never spectacular, and never terrible. Expected utility captures that balance in one number.

MEU Principle

Choose the Highest Expected Utility

This gives the maximum expected utility principle: a rational agent should choose the action with the highest expected utility. It turns acting sensibly into a procedure. List the actions. For each, list the outcomes and their probabilities. Give every outcome a utility. Compute each expected utility. Choose the highest. Why call this rational? First, over many similar choices, it gives the best result on average. Second, in 1944, John von Neumann and Oskar Morgenstern showed that if an agent's preferences follow a few consistency rules, it behaves exactly as if it assigns utilities and maximizes their expectation. So the principle isn't an arbitrary recipe. It's what consistent preferences look like.

Quick Check

A Lower Jam Chance Flips the Choice

Quick check. Same two routes, same utilities. But a traffic report now puts the highway's jam chance at just 20 percent. The back roads still score 72. Which route has the higher expected utility now? Pause, and decide. The highway. 0.8 times 95 is 76, plus 0.2 times 20, which is 4. That's 80, ahead of the back roads' 72. Sticking with the back roads is tempting, because they won before. But the values never changed. Only the agent's beliefs did. Push the jam chance up to 45 percent, and the highway falls to about 61. In fact, the highway wins only while the jam chance stays below about 31 percent.

Decisions and Luck

A Sound Choice Meeting Bad Luck

One more idea about uncertainty. Choosing the back roads at a 40 percent jam chance is the right decision. But there's still a 10 percent chance of a slow trip. If that happens, the decision wasn't wrong. It was a sound choice that met bad luck. Judge a decision by the information available when it was made, not by one result. And in practice, agents often estimate probabilities and utilities, because exact values cost too much to compute.

Consistent Preferences

Three Properties Utilities Require

For utilities to exist at all, an agent's preferences must be consistent. Three properties are the foundation. Comparable: for any two outcomes, the agent prefers one, or is indifferent. It never says: I can't tell which is better, and they're not equal either. Transitive: if it prefers A to B, and B to C, then it prefers A to C. And consistent under uncertainty: it prefers a better chance of a good outcome. It never prefers a 50 percent chance of a prize, to a 60 percent chance of the same prize.

Common Mistake

Circular Preferences Can Be Exploited

Why does it matter? Picture an agent that prefers tea to coffee, coffee to juice, and juice to tea. Its preferences run in a circle. Holding coffee, it pays ten cents to swap to tea. Holding tea, it pays ten cents to swap to juice. Holding juice, it pays ten cents to swap back to coffee. It's back where it began, thirty cents poorer. And a trader can repeat the loop forever. Inconsistent preferences can be exploited.

Risk Attitudes

Why the Utility of Money Bends

Now, attitudes to risk. Compare two offers: a sure 50 dollars, or a 50 percent chance of 100 dollars, and nothing otherwise. On average, both are worth 50 dollars. Yet many people take the sure thing. That's no flaw in the principle. It shows that the utility of money isn't simply the amount of money. Each extra dollar usually adds a little less satisfaction than the one before. A utility curve that rises ever more gently captures this, and the curve's shape sets the agent's attitude to risk.

Three Agents

Risk-Averse, Risk-Neutral, and Risk-Seeking

Give three agents the same gamble. Scale each utility so 0 dollars scores 0, and 100 dollars scores 100. For all three, the gamble's expected utility is 0.5 times 100, plus 0.5 times 0. That's 50. The risk-averse agent uses 10 times the square root of the dollars. A sure 50 dollars scores about 70.7, more than 50, so it takes the sure money. The risk-neutral agent's utility is just the dollars. The sure 50 scores 50, a tie, so it's indifferent. The risk-seeking agent uses dollars squared, over 100. The sure 50 scores only 25, so it gambles.

Certainty Equivalent

The Sure Amount Worth the Same

The certainty equivalent of a gamble is the sure amount the agent values just as much. For the risk-averse agent: 10 times the square root of x equals 50. So the square root of x is 5, and x is 25 dollars. It would accept a sure 25 dollars instead of a gamble that pays 50 on average. That 25-dollar gap is what it gives up to avoid risk. It's why people buy insurance. For the risk-seeker, x squared over 100 equals 50, so x is about 70 dollars 71. It keeps the gamble over any sure amount below that. So risk attitude isn't an extra rule. It's already inside the utility function.

Multi-Attribute Utility

Turning Time, Cost, and Safety Into Scores

So far, each outcome had one utility. Real outcomes have several attributes, like time, cost and safety. A multi-attribute utility combines them into one number. A common way is the weighted sum. Convert each attribute to a utility from 0 to 100. Choose weights that add up to 1. Then add weight times utility. Three routes to the airport. Route A, the highway: 25 minutes, a 6-dollar toll, safety 7 out of 10. Route B, city streets: 40 minutes, no toll, safety 8. Route C, the scenic road: 55 minutes, no toll, safety 9. Time utility is 60 minus the minutes, over 40, times 100. Cost utility is 10 minus the dollars, over 10, times 100. Safety utility is the rating times 10. For the highway, that gives 87.5, 40 and 70.

Weighted Sum

The Commuter Picks the Highway

Do the same for B and C. B scores 50, 100 and 80. C scores 12.5, 100 and 90. Now a commuter in a hurry weights time 0.5, cost 0.2, and safety 0.3. For the highway: 0.5 times 87.5 is 43.75. 0.2 times 40 is 8. 0.3 times 70 is 21. The total is 72.75. City streets come to 69, and the scenic road to 53.25. So the commuter takes the highway.

Quick Check

A Budget Traveler's Weights

Quick check. A budget traveler uses the same three routes and the same utilities, but different weights: time 0.2, cost 0.5, safety 0.3. Which route has the highest weighted utility for this traveler? Pause, and decide. City streets. 0.2 times 50 is 10, plus 0.5 times 100, which is 50, plus 0.3 times 80, which is 24. That's 84. The scenic road is tempting: it's free, and the safest. But its time utility is only 12.5, so it reaches 79.5. The highway drops to 58.5. Same routes, same utilities. Only the weights changed, and so did the winner.

Dominance

Dropping a Route Beaten on Every Attribute

The weighted sum works when each attribute can be judged on its own: how you feel about a travel time doesn't depend on the toll. When attributes interact, like: I'll pay a high price only for high quality, a more careful combination is needed. And sometimes you need no weights at all. Add Route D: 50 minutes, a 2-dollar toll, safety 7. City streets beat it on all three: faster, cheaper and safer. So D is dominated. It can be dropped before anyone picks a weight. A, B and C aren't dominated. None of them is beaten on all three.

Trade-Offs

Weights as Exchange Rates of Value

Among A, B and C, the aims conflict. The fastest route costs the most, and the safest is the slowest. Gaining one means giving up some of another. That's a trade-off. The weights act as exchange rates. The commuter's weights say saving time is worth paying for. The budget traveler's say the opposite. No formula says what the weights should be. They reflect what a person, a company, or a society cares about. A delivery agent told to value only speed will happily take a risky route, because safety was never in its utility. It isn't misbehaving. It's faithfully maximizing what it was given.

Design Advice

Building Utilities With Care

So design utilities with care. Include everything that matters, because anything left out is treated as worthless. Make the weights explicit, so others can see them and challenge them. Test sensitivity. Nudge the weights, and see whether the choice flips, as the jam chance flipped the route. And check the results against common sense, before relying on them.

Key Takeaways

Utility, Expectation, Risk, and Weights

To sum up. A utility function scores outcomes with numbers, so an agent can rank options a goal test can't separate. Expected utility weights each outcome's utility by its probability, and a rational agent picks the highest. Preferences must be consistent, and the curve of the utility function sets the agent's attitude to risk. And multi-attribute utility combines several aims, with weights that reflect values.

Reflection

Who Sets a Hospital Agent's Weights

Here's something to think about. A hospital's scheduling agent weighs patient waiting time, staff overtime, and equipment cost. Who should decide the weights? And what could go wrong if it counted waiting time, but ignored how urgent each patient's condition is?

About Introduction to Artificial Intelligence at UPI Study

Introduction to Artificial Intelligence (Computer Science 373) is a self-paced online course of 69 video lessons in 9 chapters, from intelligent agents and search to game strategy, neural networks, logic and reasoning, machine learning, language, robotics and AI ethics. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Computer Science courses Every course with PRO →

Compare: AI vs Current Trends in CS and IT

Why take it here

Why students take Introduction to Artificial Intelligence with UPI Study

How machines search, reason and learnIntroduction to Artificial Intelligence covers how AI systems are built: intelligent agents, search and heuristics, game strategy, logic, neural networks, machine learning, language and perception, and the ethics of AI. It is a common upper-level course in computer science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Introduction to Artificial Intelligence carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Maximizing Outcomes: Utility-Driven Agent Behavior

What is a utility-based agent?
A utility-based agent uses a utility function that gives every possible outcome a number, with higher numbers meaning more desirable outcomes, and it chooses actions that lead to high utility. Unlike a goal-based agent, which only asks whether a goal is reached, it can rank two ways of reaching the goal, weigh a risky option against a safe one, and balance conflicting aims by combining them into one score.
How do you calculate expected utility?
For each possible outcome of an action, multiply its probability by its utility, then add the results. If a highway has a 60 percent chance of clear traffic worth 95 and a 40 percent chance of a jam worth 20, its expected utility is 57 plus 8, or 65. Back roads with a 90 percent chance at 75 and a 10 percent chance at 45 score 72, so they are the rational choice.
What does risk-averse mean in utility theory?
A risk-averse agent prefers a sure amount over a gamble with the same average payoff, because its utility curve rises ever more gently as money grows. With utility equal to 10 times the square root of the dollars, a sure $50 scores about 70.7, beating a 50 percent chance of $100, which scores 50. Its certainty equivalent for that gamble is just $25, which helps explain why people buy insurance.
How does multi-attribute utility work?
Multi-attribute utility combines several attributes, such as time, cost, and safety, into one score, often as a weighted sum. Convert each attribute to a utility from 0 to 100, choose weights that add up to 1, and add weight times utility. With weights of 0.5 for time, 0.2 for cost, and 0.3 for safety, a highway scoring 87.5, 40, and 70 totals 72.75. Changing the weights can change the winner.

About the Introduction to Artificial Intelligence course

How many college credits is Introduction to Artificial Intelligence?
Introduction to Artificial Intelligence (Computer Science 373) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Introduction to Artificial Intelligence cost and what is included?
$250, paid once, with lifetime access: all 69 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Introduction to Artificial Intelligence take?
It is self-paced with no deadlines. The 69 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Introduction to Artificial Intelligence.

69 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Introduction to Artificial Intelligence →Download the syllabusEvery course with PRO →
Introduction to Artificial Intelligence
3 credits · NCCRSEnroll →