UPI Study › Courses › Introduction to Artificial Intelligence › Free lessons › Lesson 2.6 Free with a free account
How Model-Based Agents Track Their World
Introduction to Artificial Intelligence · Chapter 2: Intelligence, Cognition, and Agent DesignSection 2.6 · lesson 14 of 6910 min · free with a free account
Free preview
Create a free account to keep watching
It takes a minute and costs nothing. Your free account unlocks all of Chapter 1 and 3 more lessons of Introduction to Artificial Intelligence.
Lesson complete
Earn the 3 college credits for Introduction to Artificial Intelligence
Enrolling adds a quiz after each lesson, the graded work and the official transcript. Recommended for credit by NCCRS.
Next lesson: Agents That Plan Toward Objectives →← Reactive Agents: Condition-Action ResponsesNext lesson: Agents That Plan Toward Objectives →
In this lesson
How model-based agents keep an internal state of a partly hidden world.
- Explain why partially observable environments require an internal state
- Distinguish the transition model from the sensor model
- Apply the predict, observe, correct cycle to update an agent's state
- Track a set of possible states from a sequence of actions and percepts
Free study notes
Get this lesson's notes by email
The key points and 4 practice questions with answers, in your inbox. One email, free.
One email. Unsubscribe in one click.
Transcript
Everything said in the video, section by section (1,501 words). Read along, or search it.
Opening Scene
A Car in Your Blind Spot
You're driving, and a car passes you on the left. For a few seconds it slides into your blind spot, and no mirror shows it. You don't decide the car has vanished. You keep a picture in your head: it's still there, a little farther along than where you last saw it. That picture comes from what you saw earlier, and what you know about how cars move. So how can an agent keep track of what it can't see?
Partial Observability
Acting on What Cannot Be Seen
Real environments rarely show everything at once. A camera points one way, walls block the view, and some facts, like what's behind a closed door, can't be sensed at all. When an agent can't observe every relevant part of its world at every moment, the environment is partially observable. An agent that acts on the current percept alone treats anything it can't sense right now, as if it doesn't exist. With empty-looking mirrors, it changes lanes. A model-based agent acts on the percept plus a stored picture of the world. It waits, because it knows a car was just there.
Internal State
The Agent's Best Description of Now
That stored picture is the internal state: the information an agent keeps about the world between percepts. It's the agent's best current description of what the world is like now, including the parts it can't see. For a robot vacuum, it can hold four kinds of things. Things seen earlier, like a chair spotted in the hallway. Its own actions, like which rooms are already cleaned. Changes that happen on their own, like a pet that has wandered into another room. And the agent's own condition: I'm in the kitchen, with 40 percent charge.
Two Cautions
A Best Guess Holding Only What Matters
Two cautions apply to every internal state. First, it's a best guess, not a fact. The agent never sees the whole world, so its picture can be wrong, and a good design lets it be corrected. Second, it holds only what decisions need. A vacuum robot has no reason to store the color of the sofa. A small, focused state is easier to keep accurate.
Agent Structure
Where the State Sits Inside the Agent
Here's where the state sits inside the agent. At each time step, a percept arrives from the environment. The update step combines it with the previous state, the last action, and the agent's models of the world. The result is the current state. If-then rules then choose an action from that state, not from the raw percept. And the new state and the action are remembered for the next update.
Two Models
Transition Model and Sensor Model
To update its state, the agent needs knowledge of how its world behaves: models. A model-based agent keeps two. The transition model answers: how does the world change? Given the world now, and what the agent does, it says what the world will be like next. It covers changes the agent causes, like turning the wheel to move left, and changes that happen on their own, like a passing car that keeps moving ahead. The sensor model answers: how does the world show up in my percepts? It says what the sensors will report, including blind spots, and how reliable the readings are.
Model Sources
Written, Learned, or Both
Models can be written by designers, learned from data, or built from both. Either way, they're simplifications, so they're never perfect. In realistic settings they often include uncertainty, like: the robot usually moves about one meter. Probability tools for that come in later chapters.
Quick Check
A Drone Climbing in Thin Air
Quick check. A drone's software assumes each climb command raises it two meters. In thin mountain air, it really rises only one and a half. Its altitude estimate keeps drifting upward from the truth. Which part of the drone's knowledge is wrong? Pause, and decide. The transition model. It describes how the world changes after an action, and it says each climb adds two meters. That's what's off. The sensor model is tempting, because the estimate is wrong. But nothing here is misread. The prediction itself is too big, so the state drifts. A sensor model error looks different, like sun glare being mistaken for an obstacle. Good percepts, such as an altimeter reading, can correct the drift.
Update Cycle
Predict, Observe, Correct, Act
The state isn't built once. It's updated at every time step, with a simple rhythm: predict, then correct. Predict: apply the transition model to the previous state and the last action. That gives what the world should be like now, even before the agent looks. Observe: a new percept arrives. Correct: compare it with the prediction, using the sensor model. Keep what fits the percept, and drop or revise what doesn't. Act: the rules choose an action from the corrected state. And that action feeds the next prediction.
When Things Disagree
Four Prediction and Percept Situations
Predictions and percepts don't always agree. If they agree, the models are working, so keep the state, with more confidence. If the percept shows something unexpected, the world changed, or the model missed something. Trust a reliable percept for what it covers, and revise the state. If there's no useful percept, because the view is blocked, rely on the prediction, and accept that doubt grows with time. And if the percept fits none of the possibilities, the model or the sensor is probably wrong. Widen or reset the state, then investigate.
State Forms
Best Guess, Possibility Set, or Probabilities
The state itself can be written down in different ways. A single best guess, like: I'm in cell C. Simple and fast, but it can't express doubt. A set of possibilities, like: I'm in B, C or D. That shows doubt, without numbers. Or probabilities, like: sixty percent C, forty percent D. That handles noisy sensors best, but needs probability tools from later lessons.
Worked Example
Locating a Robot With a Bump Sensor
Now a worked case. A small robot sits in a straight corridor of four cells: A, B, C and D. It doesn't know which cell it started in. It has one action, Right, which moves it one cell. And one sensor: a bump sensor that reports bump when a move is blocked by the end wall. The transition model: Right moves A to B, B to C, and C to D. From D, the wall blocks it, so it stays in D. The sensor model: no bump for A, B or C. A bump only from D. The robot keeps its state as a set of possible cells. At the start, with no information, all four are possible. Move one. Each cell leads to the next, and D would bump. The sensor says no bump, so drop D's outcome. The state is B, C or D. Move two: no bump again. Now it's C or D. Move three: bump. Only the outcome from D bumps, so the state is just D.
What It Shows
Models Plus Percepts Reveal Position
Notice what happened. The bump sensor says nothing about cells A, B or C, yet the robot found its position. It came from combining both models with the sequence of percepts. An agent using only the current percept couldn't do this. Each no bump, taken alone, fits almost any cell. And an empty set is a warning. Suppose the robot believes it's in D, moves right, and gets no bump. No cell fits. Either the sensor failed, or the corridor model is wrong. So widen the state, and investigate.
Quick Check
Two Moves in a Five-Cell Corridor
Quick check. A longer corridor has five cells, A through E, with the wall after E. The robot doesn't know where it starts. It moves right twice, and gets no bump both times. What's its state now? Pause, and decide. C, D or E. After the first move, the robot can't be in A, so the state is B through E. After the second, it can't be in B either, so it's C, D or E. E is still possible, because a robot that started in C would be there now. Just E is tempting, but no bump never pins down the end. Only a bump would.
Bringing It Together
Tracking the Picture, Not Yet Planning
Back to the car in your blind spot. Your internal state holds the car. Your transition model says it keeps moving ahead. And your sensor model knows the blind spot gives no percept. So you predict where it is, and correct the moment it reappears in the mirror. Notice what the agent hasn't done: chosen a route, or aimed for a destination. It has only kept an accurate picture. Using that picture to reach goals is a different job: planning.
Key Takeaways
State, Models, and Predict-Correct
To sum up. Agents that see only part of their world keep an internal state, a best guess that includes what they can't see right now. The transition model says how the world changes. The sensor model says how the world shows up in percepts. At every step, the agent predicts with the transition model, and corrects with the new percept and the sensor model. Models plus a sequence of percepts can reveal facts no single percept shows. And when nothing fits, check the models and the sensors.
Reflection
A Simmering Pot in Another Room
Here's something to think about. A pot is simmering in the kitchen while you're in another room. What picture of it do you keep in your head? What do you know about pots, that updates it? And what percept, a smell or a sound, would make you change it?
About Introduction to Artificial Intelligence at UPI Study
Introduction to Artificial Intelligence (Computer Science 373) is a self-paced online course of 69 video lessons in 9 chapters, from intelligent agents and search to game strategy, neural networks, logic and reasoning, machine learning, language, robotics and AI ethics. It carries a NCCRS credit recommendation for 3 semester credits.
Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Enroll now → Course details Download the syllabus All Computer Science courses Every course with PRO →
Compare: AI vs Current Trends in CS and IT
Why take it here
Why students take Introduction to Artificial Intelligence with UPI Study
Where the credit goes
Universities that work with NCCRS
Introduction to Artificial Intelligence carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Thomas Edison State UniversitySigned partner
University of the PeopleSigned partner
Charter Oak State CollegeSigned partner
Strayer UniversitySigned partner
Columbia Southern UniversitySigned partner
Indiana TechSigned partner
Penn StateWorks with NCCRS
Southern New Hampshire UniversityWorks with NCCRS
SUNYWorks with NCCRS
+many more cooperating universitiesFind my college →Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.
FAQ
Questions students ask us
About How Model-Based Agents Track Their World
What is a model-based reflex agent?
What is the difference between a transition model and a sensor model?
How does a model-based agent update its internal state?
How can an agent find facts that no single percept shows?
About the Introduction to Artificial Intelligence course
How many college credits is Introduction to Artificial Intelligence?
How much does Introduction to Artificial Intelligence cost and what is included?
Which universities accept UPI Study credits?
How long does Introduction to Artificial Intelligence take?
How are the exams done?
How do my credits reach my college?
Ready for the full course? Earn 3 credits in Introduction to Artificial Intelligence.
69 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.
3 credits · NCCRSEnroll →