UPI Study › Courses › Ethics in Technology › Free lessons › Lesson 5.3 Free with a free account
Techniques for Extracting Meaningful Business Intelligence from Enormous Datasets
Ethics in Technology · Chapter 5: Making Sense of Massive Datasets: Big Data, Mining, and the Privacy Trade-OffSection 5.3 · lesson 19 of 11616 min · free with a free account
Free preview
Create a free account to keep watching
It takes a minute and costs nothing. Your free account unlocks all of Chapter 1 and 3 more lessons of Ethics in Technology.
Lesson complete
Earn the 3 college credits for Ethics in Technology
Enrolling adds a quiz after each lesson, the graded work and the official transcript. Recommended for credit by NCCRS.
Next lesson: How Algorithms Dig Through Raw Data to Discover Hidden Patterns and Relationships →← The Four Vs: Volume, Velocity, Variety, and Veracity as the Defining Characteristics of Big DataNext lesson: How Algorithms Dig Through Raw Data to Discover Hidden Patterns and Relationships →
In this lesson
Turn stored information into a useful, accountable decision.
- Distinguish 4 analytics families
- Follow a diagnostic investigation
- Compare stores, pipelines, and dashboards
- Connect business value with privacy governance
Free study notes
Get this lesson's notes by email
The key points and 4 practice questions with answers, in your inbox. One email, free.
One email. Unsubscribe in one click.
Transcript
Everything said in the video, section by section (1,953 words). Read along, or search it.
Chapter 5
Business Intelligence From Large Datasets
Business intelligence turns stored information, into evidence that can improve a decision. Collecting more records does not establish, that anyone has extracted useful value from them. The work includes technologies, processes, and practices, from routine reports to automated recommendations. This lesson distinguishes descriptive, diagnostic, predictive, and prescriptive analytics, by the question each answers. We examine why agreed definitions matter, before interpreting a chart or a forecast. A diagnostic investigation narrows a problem, through comparisons and drill-down into its components. It still needs care, because correlation can suggest an explanation without proving it. We compare warehouses and lakes, including the purpose of shared structure and governance. We also compare pipelines, that transform before loading or within the loaded target environment. Dashboards connect the analysis to people, so clarity, comparison, ownership, and action matter. Industry examples show how value depends, on particular kinds of information and decision responsibility. The same detail and combination create privacy exposure, making governance part of the analytical project.
4 Families
Each Family Answers a Different Question
Descriptive analytics answers what happened, using totals, trends, reports, and current-state displays. Diagnostic analytics asks why it happened, through comparisons and investigation of candidate explanations. Predictive analytics estimates what is likely next, such as demand, risk, or customer departure. Prescriptive analytics asks what to do, recommending or taking actions under stated objectives and constraints. The families can support each other, while requiring different evidence and kinds of judgment. A reliable historical description provides, a stronger starting point than conflicting departmental totals. Investigating a cause requires more than noticing, that two measures changed at the same time. A forecast depends on patterns and assumptions, whose usefulness can change with circumstances. A recommendation adds choices about objectives, feasible options, costs, and affected people. These dependencies do not mean every prediction requires, a proven causal explanation of the past. They do mean shaky definitions and poor data, can undermine apparently sophisticated outputs. Identify what the result claims, and whether the evidence actually supports that kind of claim.
Descriptive Analytics
Agree on the Measure Before Comparing the Number
Descriptive analytics summarizes what has occurred, and often supports an ordinary operational decision directly. It can be a periodic report, a dashboard, or a comparison with the same period last year. The difficult part may be defining the measure, rather than adding the values. Monthly revenue requires an agreement, about when a sale belongs in the reporting period. Refunds and cancellations affect the total, and must be handled consistently across reports. Tax treatment and currency conversion can also change, apparently comparable figures. Customer counts require definitions, about which accounts or people qualify as customers. If three departments use different definitions, their different totals may each follow their own rules. Meetings then become arguments about the number, instead of discussions about what action it supports. A semantic layer maintains shared meanings, so reporting tools can reuse an agreed measure. That agreement needs ownership and documentation, rather than only an attractive-looking chart. Useful descriptive work establishes a dependable position, before more elaborate analysis is built on top.
Diagnostic Analytics
Locate the Variation, Then Test the Explanation
The source's diagnostic example begins, with monthly sales down twelve percent overall. A regional breakdown shows the North down two percent, and the South down three percent. The West is down thirty-one percent, directing attention to that region's more pronounced change. Within the West, in-store sales are flat, while online sales are down fifty-eight percent. Further investigation identifies a checkout page failing, on one browser in the worked example. The regional and channel percentages have different bases, so they should not be added together. Drill-down narrows the investigation, until a candidate issue is specific enough to examine and fix. First check whether variation is unusual, and whether changed logging or definitions produced the apparent change. Then resist the first plausible correlation, because other differences can explain the same pattern. Loyalty-app users might spend more, because already enthusiastic customers chose to install the application. Their spending does not establish, that installation caused the higher purchases. Confirm an explanation through suitable evidence or deliberate testing, rather than treating a drill-down result as proof.
Prediction and Prescription
An Estimate and a Decision Have Different Responsibilities
Predictive analytics estimates demand, customer departure, credit risk, or the likelihood of equipment failure. The result contains uncertainty, whether expressed as a probability, range, or point forecast. Changing conditions can weaken patterns, learned from a history that no longer resembles the current situation. The source illustrates a forecast right four times in five, which would still be wrong regularly. Historical decisions can also encode unfairness, that a model may reproduce rather than correct. Predicting who previously received approval does not establish, who ought to receive approval now. Prescriptive analytics combines estimates with objectives and constraints, to recommend a feasible action. Pricing, delivery routes, inventory replenishment, and scheduling provide, familiar examples of this additional step. A recommendation is advisory, while automatic execution delegates authority to the system. Before delegation, define the objective, accountability for harm, and how failure will be noticed. An objective focused only on revenue or speed, does not automatically represent fairness, safety, or welfare. Responsibility must be designed into the decision process, rather than inferred from the apparent precision of the output.
Analytical Stores
Warehouses and Lakes Emphasize Different Organization
Analytical workloads often scan many records, while operational systems prioritize reliable individual transactions. Separating those workloads can reduce contention, although modern architectures can combine capabilities in several ways. The source compares a warehouse, with a lake as two common organizing patterns. A warehouse emphasizes prepared analytical data, using agreed structures and definitions for reporting. Schema on write describes imposing that organization, before the prepared records are loaded. A lake preserves varied material, with structure interpreted when particular data is used. Schema on read describes that flexibility, rather than a promise that raw files already have reliable meaning. The warehouse can make shared reports convenient, while new requirements may demand additional engineering. The lake preserves detail, but without governance it can become difficult to navigate or trust. A data mart serves a narrower department, such as finance or marketing. A lakehouse combines flexible storage, with additional structure and reliability features associated with analytical tables. These are typical patterns, not universal quality guarantees or rigid limitations on every modern product.
Pipelines
ETL and ELT Put Transformation at Different Stages
ETL stands for extract, transform, load, preparing selected data before it reaches the destination. Extraction gathers source records, while transformation cleans, reshapes, or combines the required information. Loading then places the prepared result, into the intended analytical store. ELT stands for extract, load, transform, moving transformation into the loaded target environment. Affordable storage and scalable target processing, make that sequence useful for some workloads. Keeping suitable original detail can allow, a mistaken transformation to be revised later. It can also support another justified analytical question, without repeating the original collection step. The flexibility does not justify retaining every available field, indefinitely or without a defined purpose. Loaded personal data still needs an inventory, access boundaries, and appropriate retention rules. Both approaches remain useful, depending on target capacity, preparation needs, and governance requirements. The order of operations changes the workflow, without removing responsibility for the information being held.
Tools and Interfaces
A Dashboard Needs Definitions, Context, and a Next Step
Business intelligence reaches a decision-maker through an interface, rather than only through a completed calculation. The connection layer accesses data, and the semantic layer supplies shared definitions of measures. A visual builder presents information, while distribution delivers reports and alerts to the intended audience. The source's dashboard example begins with headline tiles, showing key measures and comparisons. A trend supplies context, instead of treating one present-day number as self-explanatory. A regional breakdown invites the next question, about where a change is concentrated. Filters include period, region, channel, and product, allowing the reader to investigate relevant segments. A working display should also show, when its information was last refreshed. Name an owner, who can respond when the measure or its interpretation appears wrong. Provide a route to detail, so noticing a problem can lead to a useful investigation. The practical test is whether someone can identify, a decision improved by the information on the display.
Dashboard Faults
Make the Important Question Easy to See and Investigate
A dashboard with too many charts can lack hierarchy, leaving the important decision hidden. Lead with a few measures, that match what the intended audience actually needs to decide. A number without comparison offers little context, so include a target, previous period, or suitable benchmark. Vanity measures can appear reassuring, while failing to reveal deterioration in useful activity. The source's example shows two point four million registrations, with a cumulative rising line. That total can obscure inactivity, because registration alone does not establish continuing use. Consider active users, new registrations per period, retention after thirty days, and customer departure. Avoid decorative effects and misleading axes, that exaggerate change or complicate reading. Clear charts should help the reader, judge magnitude and direction honestly. A named owner and visible refresh time, support trust and a response to errors. Drill-down should connect a concerning headline, to the segments and records underneath it. The display becomes intelligence through useful interpretation and action, rather than the number of charts it contains.
Industry Uses
Useful Analysis Depends on the Relevant Evidence
Retail demand forecasting uses transactions, seasonality, promotions, and local conditions to plan stock. Personalized offers add linked purchase and browsing history, supporting choices about what to recommend. Fraud analysis can predict suspicion, from spending, location, device, and merchant patterns. Holding a suspicious payment adds a prescriptive action, beyond the predictive score itself. Readmission-risk analysis uses clinical history, prior admissions, medication, and relevant personal circumstances. That is an analytical use case, rather than a substitute for appropriate clinical judgment. Manufacturing can use sensor history, maintenance, and failure records to estimate equipment problems. Choosing when to service the equipment adds, a decision about costs, risks, and available capacity. These examples illustrate different outputs, whose value depends on the evidence available for the task. They also show why a technique's label, does not describe every step in an operational system. A prediction, recommendation, and executed action can coexist, within one broader business process. Each step needs suitable quality, responsibility, and limits, especially where it affects people's opportunities or safety.
People and Power
A Technical Measure Can Become a Rule About Someone’s Work
Logistics route planning uses positions, orders, road conditions, capacity, time windows, and driver hours. Detailed workforce activity records can also support, descriptive measurement of individual employees. The machinery may resemble other counting systems, while the relationship between the parties changes its ethical stakes. An employee may have limited ability to refuse, and the measure can affect continued employment. The source's route scenario optimizes delivery time, creating risks when other requirements are omitted. Unsafe assumptions, inadequate breaks, or uneven assignments may appear efficient, under that narrow objective. Before automatic assignment, include safety, lawful hours, and breaks among the constraints that matter. Name a person accountable for outcomes, and provide a meaningful way to challenge an assignment. Retain an appropriate override, with decisions recorded and reviewed by someone able to correct the system. Monitor fairness and driver welfare, alongside the delivery-time measure itself. An apparently objective result remains a delegated decision, whose assumptions and consequences require human responsibility.
Capability and Exposure
Detail, Combination, and Retention Need Governance
Detail, combination, and retention can increase analytical usefulness, while also increasing privacy exposure. Individual records support targeted decisions, but they can reveal more than an aggregate summary. Joining purchases, location, and support conversations can create, a richer portrait than any source alone. Load-first retention can preserve material, for questions that have not yet been justified or defined. Governance addresses those choices, through minimization, access limits, quality, documentation, and suitable retention. The source's complaint scenario begins with a reported increase, of forty percent last quarter. Verify definitions and logging first, then drill into branches, products, channels, categories, and time. Test candidate explanations, estimate likely consequences, and choose remedies within cost and capacity constraints. The route from a number to an action, should preserve the limits of each analytical claim. Responsible custody belongs inside the project, because capability does not establish permission to collect or combine everything. Useful business intelligence can be legitimate, when its purpose, evidence, decisions, and information boundaries are accountable.
About Ethics in Technology at UPI Study
Ethics in Technology (Computer Science 195) is a self-paced online course of 116 video lessons in 15 chapters, from cybersecurity basics, privacy and surveillance to big data, hacking, malware, network defense, intellectual property, cybercrime, encryption, emerging technology and digital citizenship. It carries a NCCRS credit recommendation for 3 semester credits.
Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Enroll now → Course details Download the syllabus All Computer Science courses
Compare: Cybersecurity vs Ethics in Technology · Which ethics course?
Why take it here
Why students take Ethics in Technology with UPI Study
Where the credit goes
Universities that work with NCCRS
Ethics in Technology carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Thomas Edison State UniversitySigned partner
University of the PeopleSigned partner
Charter Oak State CollegeSigned partner
Strayer UniversitySigned partner
Columbia Southern UniversitySigned partner
Indiana TechSigned partner
Penn StateWorks with NCCRS
Southern New Hampshire UniversityWorks with NCCRS
SUNYWorks with NCCRS
+many more cooperating universitiesFind my college →Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.
FAQ
Questions students ask us
About Techniques for Extracting Meaningful Business Intelligence from Enormous Datasets
What are the four types of data analytics?
What is the difference between a data warehouse and a data lake?
What is the difference between ETL and ELT?
What makes a business intelligence dashboard effective?
About the Ethics in Technology course
How many college credits is Ethics in Technology?
How much does Ethics in Technology cost and what is included?
Which universities accept UPI Study credits?
How long does Ethics in Technology take?
How are the exams done?
How do my credits reach my college?
Ready for the full course? Earn 3 credits in Ethics in Technology.
116 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.
3 credits · NCCRSEnroll →