Is More Data Worth It? | Quantitative Analysis

UPI Study › Courses › Quantitative Analysis › Free lessons › Lesson 6.4 Free with a free account

Is More Data Worth It?

Quantitative Analysis · Chapter 6: Bayesian Revision and UtilitySection 6.4 · lesson 29 of 5510 min · free with a free account

← Reasoning Backward From EvidenceNext lesson: Measuring More Than Money →

Earn 3 college credits for Quantitative AnalysisRecommended for credit by NCCRS · Math 250 · self-paced · one payment, lifetime access
Enroll in Quantitative Analysis · $250 →Download the syllabus (PDF)

In this lesson

Evaluating the Expected Value of Sample Information (EVSI).

  • Define EVSI and bound it between 0 and EVPI
  • Run the five-step pre-posterior calculation
  • Score a source with efficiency, EVSI divided by EVPI
  • Decide whether to buy data using Net EVSI

Transcript

Everything said in the video, section by section (1,513 words). Read along, or search it.

QAN-201 - Quantitative Analysis · Section 6.4

Is More Data Worth It?

Chapter Five asked a clean question. What would perfect information be worth? The answer, EVPI, is a ceiling. It is not a product that anyone can actually sell you. Real data is never perfect. Surveys misclassify. Forecasts err. Signals arrive noisy. So the practical question changes shape. What is this particular imperfect signal worth, before I pay for it? That number is the expected value of sample information, or EVSI. We define it, compute it in five steps, and score it against the ceiling. Then we work a market entry problem all the way to a verdict. Fair warning. The arithmetic printed in the source does not check out, so we recompute it.

Defining EVSI

The Value of a Signal You Can Actually Buy

Start with what EVSI actually asks. By how much can we raise expected monetary value, if we observe this signal? And then choose optimally, in light of whatever the signal says. That second clause is the whole idea. Value arrives through the decision it changes. Formally, EVSI is the best expected value with the sample, minus the best without it. Two bounds follow immediately, and both are worth memorising. EVSI is never negative. At worst you ignore the signal and lose nothing. And EVSI can never exceed EVPI. An imperfect source cannot be worth more than a perfect one. Equality holds in exactly one case, when the signal is perfectly reliable. Notice the framing shift. Gathering data is now an alternative, with a hurdle to clear.

Five Steps

The Pre-Posterior Calculation, End to End

The calculation runs in five steps, and the order matters. Step one. Using the prior probabilities, find the best alternative and its expected value. That baseline is the number the survey has to beat. Step two. For every signal the source might emit, apply Bayes' Theorem. Prior times likelihood gives the joint probability, and normalising gives the posterior. The likelihood function is the only place the source's accuracy enters. Step three. Under each set of posteriors, recompute every alternative and take the best. Step four is the one people skip. Before you buy, you do not know which signal will arrive. So weight each conditional optimum by the marginal probability of its signal, and sum. That sum is the pre-posterior expected value. Step five. Subtract the prior baseline. What remains is EVSI, before any cost.

Efficiency

Scoring a Source Against the Theoretical Maximum

EVSI on its own is a dollar figure without a scale. The same amount could be excellent in one problem and pitiful in another. Efficiency fixes that. Divide EVSI by EVPI. The result always sits between zero and one. Near zero, the signal carries almost no information. The posterior barely moves from the prior, and paying anything for it is wasteful. Near one, the signal almost perfectly resolves the uncertainty. Efficiency earns its keep because it puts rival sources on a common scale. Different signal structures, different price tags, one comparable number. But here is the trap. Efficiency is not a fixed property of the source. It depends on the decision problem, on the prior, and on the payoff structure.

Net Value

The Only Test That Decides Whether to Buy

EVSI is a gross benefit. Now subtract what the information costs. That difference is net EVSI, and it alone decides the purchase. Collect the information only if net EVSI is positive. Three cases arise. Clearly worthwhile, when net EVSI sits comfortably above zero. There the value exceeds cost by a margin you can defend. Gather the data. Borderline, when net EVSI hovers around zero. Investigate the source's quality further, or defer the purchase. Not worthwhile, when net EVSI is negative. Decide without the data. Notice what this framework refuses to assume. It never treats data collection as automatically beneficial. That is the point.

Negative Value

When Information Is Not Worth Buying

Negative net value is far more common than intuition suggests. Four conditions make it likely, and any one of them can be enough. First, the prior already heavily favours one state. The decision is nearly pre-determined, so a signal has little room to move it. Second, the source is low accuracy, so its efficiency is low. The signal rarely changes which action is optimal. Third, the cost is high relative to the payoff gaps between the alternatives. Fourth, time pressure makes gathering data and then deciding infeasible. Delay is itself a cost, even when nobody writes an invoice for it. When any of these bite, act now on current beliefs, and spend nothing. More information is not always better. That is the whole lesson.

Worked Problem

Market Entry: Should the Firm Buy the Survey?

A firm must launch, delay, or abort a new product. A research firm offers a survey signal, favourable or unfavourable, for fifteen thousand dollars. Three market states. High demand at forty per cent, medium at thirty-five, low at twenty-five. Launch pays two hundred thousand under high demand, and eighty thousand under medium. Under low demand, launch loses sixty thousand. Delay pays fifty thousand, fifty thousand, and twenty thousand across the three states. Abort pays nothing, whatever happens. Weight the launch payoffs by the priors and you get ninety-three thousand dollars. Delay comes to forty-two thousand five hundred. Abort is zero. So launch is optimal with no survey at all, and that is the baseline. The source prints eighty-one thousand here, and forty thousand five hundred for delay. Neither follows from this table, so the recomputed values are used from here on.

Step 2

The Survey's Accuracy Profile

Step two needs the likelihood function. How likely is each signal, given each true state of the market? This is the research firm's track record, and nothing more. When demand is truly high, the survey says high eighty per cent of the time. When demand is truly medium, it says high forty per cent of the time. When demand is truly low, it says high only ten per cent of the time. Equivalently, it correctly calls a low market ninety per cent of the time. Moderately accurate, then. Clearly informative, but nowhere near perfect. Watch the direction of conditioning. These are signal probabilities, given the state. The decision maker needs the reverse, and reversing it is what Bayes does.

Step 2

One Bayesian Update Per Signal

Multiply each prior by its likelihood to get a joint probability. For the favourable signal, forty per cent times eighty per cent gives zero point three two. Thirty-five per cent times forty per cent gives zero point one four. Twenty-five per cent times ten per cent gives zero point zero two five. Sum those three joints and you get zero point four eight five. That is the marginal probability of seeing a favourable signal at all. Divide each joint by it, and the posteriors appear. High demand rises from forty per cent to sixty-six per cent. Low demand collapses to roughly five per cent. The unfavourable branch works identically, with a marginal of zero point five one five. There, high demand drops from forty per cent to about fifteen and a half. Always check that posteriors sum to one. The printed ones sum to one point one two eight.

Step 3

The Optimal Action Under Each Signal

Now recompute every alternative under each set of posteriors. Under a favourable signal, launch is worth about one hundred fifty-two thousand dollars. Delay is worth roughly forty-eight thousand. Abort, as always, is zero. Launch wins comfortably there. No surprise. Under an unfavourable signal, launch falls to about thirty-seven thousand five hundred. Delay falls as well, to about thirty-six thousand nine hundred. Launch still wins, but by only five hundred and eighty-three dollars. A razor-thin margin, and it decides the entire result. The beliefs moved enormously. The chosen action did not move at all. The source reports delay as optimal under the unfavourable signal. That rests on its mis-normalised posteriors, so it does not survive correction.

Steps 4 and 5

Weighting, EVSI, and the Verdict

Step four. Weight each conditional optimum by the probability of its signal. The favourable branch arrives forty-eight point five per cent of the time. Weighted, it contributes seventy-three thousand seven hundred dollars. The unfavourable branch contributes nineteen thousand three hundred. Add them. The pre-posterior expected value is ninety-three thousand dollars. Now step five. Subtract the baseline, which was also ninety-three thousand. EVSI is exactly zero. That is not a rounding artefact. Launch was optimal before the survey, and optimal under both possible signals. A signal that never changes your action cannot change your expected payoff. Efficiency is therefore zero, against an EVPI of twenty thousand dollars for this table. Net EVSI is zero minus a fifteen thousand dollar fee. Minus fifteen thousand. The verdict is unambiguous. Do not buy the survey.

What To Carry

The Formula Card, and the Habit Behind It

Five things to carry out of this section. EVSI measures the value of an imperfect signal, which is the realistic case. The recipe is baseline, posteriors, conditional optima, probability weighting, then subtraction. Efficiency is EVSI over EVPI, running from zero to one. Net EVSI subtracts the price, and only that number licenses a purchase. When net EVSI is negative, decide immediately on current beliefs. One caveat on the criterion. All of this runs on expected monetary value. That assumes risk neutrality, which a real manager may not have. Risk aversion, a renegotiated fee, or a cheaper source can all still matter. And the deepest point came out of the worked problem itself. Information has value only when it can change what you do. A signal that shifts your beliefs but not your action is worth exactly nothing.

About Quantitative Analysis at UPI Study

Quantitative Analysis (Math 250) is a self-paced online course of 55 video lessons in 13 chapters, from decision analysis and simulation to inventory and queuing models, project scheduling, linear programming and transportation models. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Math courses Every course with PRO →

Compare: Statistics vs Quantitative Analysis

Why take it here

Why students take Quantitative Analysis with UPI Study

Decide with numbersQuantitative Analysis teaches the models managers use to decide under uncertainty: decision trees, simulation, inventory and queuing models, project scheduling and linear programming. It is part of many business, management, operations and supply-chain programs.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Quantitative Analysis carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Is More Data Worth It?

What is the expected value of sample information (EVSI)?
EVSI is the increase in expected monetary value from observing an imperfect signal, such as a survey, and then choosing the best action given what it says. Formally, it is the best expected value with the sample minus the best expected value without it. EVSI cannot be negative, because you can always ignore the signal, and it cannot exceed EVPI; it equals EVPI only when the signal is perfectly reliable.
How do you calculate EVSI step by step?
First, find the best alternative and its EMV using the prior probabilities. Second, for every possible signal, use Bayes' theorem with the source's likelihoods to get posterior probabilities. Third, recompute each alternative under those posteriors and pick the best. Fourth, weight each conditional best value by the marginal probability of its signal and sum, giving the pre-posterior expected value. Fifth, subtract the prior baseline to get EVSI.
What is the efficiency of sample information, and how is net EVSI used?
Efficiency is EVSI divided by EVPI, a ratio between 0 and 1 that scores a source against the value of perfect information, so rival sources can be compared on one scale. It depends on the prior and the payoffs, not just the source. Net EVSI is EVSI minus the cost of the information: buy the data if net EVSI is clearly positive, treat values near zero as borderline, and decide without the data when it is negative.
Can a survey be worth nothing even if it changes your probabilities?
Yes. Information has value only if it can change your decision. In one market-entry example, launching has an EMV of $93,000 before a survey. A favorable result raises the chance of high demand from 40% to about 66%, and an unfavorable one lowers it to about 15.5%, yet launch stays optimal under both signals. The pre-posterior value is still $93,000, so EVSI is zero and a $15,000 survey has a net EVSI of negative $15,000.

About the Quantitative Analysis course

How many college credits is Quantitative Analysis?
Quantitative Analysis (Math 250) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Quantitative Analysis cost and what is included?
$250, paid once, with lifetime access: all 55 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Quantitative Analysis take?
It is self-paced with no deadlines. The 55 lessons total about 11 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Quantitative Analysis.

55 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Quantitative Analysis →Download the syllabusEvery course with PRO →
Quantitative Analysis
3 credits · NCCRSEnroll →