Measuring How Well Systems Perform Over Time, Explained

UPI Study › Courses › Software Engineering › Free lessons › Lesson 12.3 Free with a free account

Measuring How Well Systems Perform Over Time

Software Engineering · Chapter 12: Reliability Engineering and SecuritySection 12.3 · lesson 77 of 1037 min · free with a free account

← Writing Code That Holds Up: Reliable Programming PracticesNext lesson: A Survey of Information Security Categories →

Earn 3 college credits for Software EngineeringRecommended for credit by NCCRS · Computer Science 301 · self-paced · one payment, lifetime access
Enroll in Software Engineering · $250 →Download the syllabus (PDF)

In this lesson

Turning "we want this to be reliable" into numbers that can be tracked.

  • Explain why reliability targets need metrics
  • Distinguish MTTF from MTBF, and calculate MTBF
  • Calculate availability from MTBF and MTTR
  • Explain why ongoing monitoring beats a 1-time calculation

Transcript

Everything said in the video, section by section (964 words). Read along, or search it.

Reliability Engineering and Security

Measuring How Well Systems Perform Over Time

A reliability target is only as useful as the numbers behind it. This section introduces the metrics that turn a wish for reliability into something that can be tracked, verified, and watched over time. And it covers the monitoring practice that keeps those numbers meaningful, long after the first calculation.

From Goals to Numbers

Why Metrics Matter

Reliability targets are supposed to be concrete and measurable, not a vague aspiration. Two kinds of figure make that concrete. How long a system typically runs before something goes wrong. And how much of the time it is actually available for use. Without numbers like these, reliable stays an opinion. With them, it becomes something that can be tracked, compared over time, and checked against an actual target.

MTTF vs. MTBF

How Long Between Problems

Mean time to failure, or M T T F, is the average time a component operates before its first failure. It is used for things that get replaced, rather than repaired, once they fail. Mean time between failures, or M T B F, is the average time between successive failures, for a system that is repaired and returned to service each time. That is the more common figure for software, which is usually restarted or patched and kept running, not discarded. The two sound alike and are often confused. But one is for something used once until it fails, and the other is for something repaired, over and over.

Worked Example

Calculating MTBF

Here is a simple example. A service runs for seven hundred twenty hours across a month, and experiences four failures. Its M T B F for that period is seven hundred twenty divided by four. That is one hundred eighty hours between failures, on average.

Availability

How Much Time It Actually Works

Availability is the proportion of time a system is actually up and usable, out of the total time being measured. Alongside M T B F, this needs one more figure. Mean time to repair, or M T T R, is the average time it takes to restore the system once it has failed. Availability is then M T B F, divided by M T B F plus M T T R. In other words, it is the uptime portion of one full up-then-down cycle.

Figure 1

Up, Down, and 1 Full Cycle

Picture an observation period as a series of segments. The system is up for a stretch, then down while it is repaired, then up again. Each up-then-down pair is one full M T B F cycle. The down portion inside it is the M T T R. Those repair segments are exactly what availability is measured against.

Worked Example

Calculating Availability

Let's calculate one. A system has an M T B F of one hundred eighty hours, and an M T T R of two hours. Its availability is one hundred eighty, divided by one hundred eighty plus two. That is one hundred eighty over one hundred eighty-two. Which comes to about ninety-eight point nine percent.

The Nines

Why 99.9% Is Far From 99%

Availability is often expressed as a number of nines. Ninety-nine percent, ninety-nine point nine percent, and so on. Each additional nine cuts the allowed downtime by an order of magnitude. Ninety-nine percent availability allows roughly three point six five days of downtime a year. Ninety-nine point nine percent allows roughly eight point seven six hours a year. And ninety-nine point nine nine allows under an hour. The percentages look similar at a glance. The gap in real downtime is enormous.

Table 1

The 4 Metrics at a Glance

Here are the four metrics side by side. M T T F is total operating time, divided by the number of units. It applies to things that are not repaired. M T B F is total operational time, divided by the number of failures. It applies to systems that are repaired. M T T R is total repair time, divided by the number of repairs. And availability is M T B F, divided by M T B F plus M T T R.

Monitoring

A Snapshot vs. a Trend

A metric calculated once is only a snapshot. The real value comes from tracking it continuously. Monitoring means collecting data on uptime, downtime, and failures on an ongoing basis, and watching for trends. Is M T B F improving, or getting worse? Is one particular kind of failure becoming more frequent? A single number rarely tells that story. This is the same catch-problems-early instinct, applied to a whole system's operational history. A gradually degrading M T B F is a warning sign worth acting on, long before it turns into a full outage.

Apply It

960 Hours, 8 Failures

Let's apply that. A service runs for nine hundred sixty hours in a period, and experiences eight failures. What is its M T B F? Nine hundred sixty divided by eight is one hundred twenty hours between failures, on average. And then keep watching it. One good number today says nothing about the trend next month.

Key Terms

The Vocabulary of This Section

Let's lock in the key terms. M T T F, mean time to failure, is the average time a non-repairable component operates before its first failure. M T B F, mean time between failures, is the average time between successive failures, for a repairable system restored each time. M T T R, mean time to repair, is the average time it takes to restore a system once it has failed. And availability is the proportion of total time a system is up and usable. It is M T B F, divided by M T B F plus M T T R.

Key Takeaways

What to Carry Forward

To pull this together. Metrics are what make a reliability target checkable, rather than just an aspiration. M T T F applies to things that are replaced. M T B F applies to repaired systems, like most software. Availability is M T B F, divided by M T B F plus M T T R. Each additional nine of availability cuts the allowed downtime by an order of magnitude. And monitoring is what keeps those numbers meaningful, by watching the trend over time.

About Software Engineering at UPI Study

Software Engineering (Computer Science 301) is a self-paced online course of 103 video lessons in 15 chapters, from process models and agile to requirements, design, user interfaces, reliability, security and project management. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Computer Science courses Every course with PRO →

Compare: Software Engineering vs Systems Analysis

Why take it here

Why students take Software Engineering with UPI Study

How professional software gets builtSoftware Engineering covers how teams turn requirements into working, maintainable software: process models, agile, requirements, design, testing, reliability and project management. It is a core course in computer science and software degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Software Engineering carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Measuring How Well Systems Perform Over Time

What is the difference between MTTF and MTBF?
MTTF (mean time to failure) is the average time a non-repairable component runs before its first failure, while MTBF (mean time between failures) is the average time between successive failures of a system that is repaired and returned to service. MTTF suits parts that are replaced once they fail. MTBF suits most software, which is restarted or patched and kept running.
How do you calculate MTBF?
MTBF is total operational time divided by the number of failures in that period. For example, a service that runs 720 hours in a month and fails four times has an MTBF of 720 / 4 = 180 hours between failures on average. A service that runs 960 hours with eight failures has an MTBF of 120 hours.
How is system availability calculated from MTBF and MTTR?
Availability equals MTBF divided by the sum of MTBF and MTTR, where MTTR (mean time to repair) is the average time needed to restore a failed system. With an MTBF of 180 hours and an MTTR of 2 hours, availability is 180 / 182, or about 98.9 percent. It is the uptime share of one full up-then-down cycle.
How much downtime per year does 99.9% availability allow?
Availability of 99.9 percent allows roughly 8.76 hours of downtime a year, compared with about 3.65 days at 99 percent. Each additional nine cuts the allowed downtime by an order of magnitude, so 99.99 percent allows under an hour a year. Percentages that look nearly identical can mean very different amounts of real outage time.

About the Software Engineering course

How many college credits is Software Engineering?
Software Engineering (Computer Science 301) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Software Engineering cost and what is included?
$250, paid once, with lifetime access: all 103 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Software Engineering take?
It is self-paced with no deadlines. The 103 lessons total about 11 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Software Engineering.

103 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Software Engineering →Download the syllabusEvery course with PRO →
Software Engineering
3 credits · NCCRSEnroll →