UPI Study › Courses › Introduction to Artificial Intelligence › Free lessons › Lesson 1.5 Free with a free account
Techniques for Computing Logarithmic Values
Introduction to Artificial Intelligence · Chapter 1: Foundations of AI and Core ConceptsSection 1.5 · lesson 5 of 6917 min · free with a free account
Free preview
Create a free account to keep watching
It takes a minute and costs nothing. Your free account unlocks all of Chapter 1 and 3 more lessons of Introduction to Artificial Intelligence.
Lesson complete
Earn the 3 college credits for Introduction to Artificial Intelligence
Enrolling adds a quiz after each lesson, the graded work and the official transcript. Recommended for credit by NCCRS.
Next lesson: Core Ideas in Probability and Chance →← How Algorithms Power Computer ProgramsNext lesson: Core Ideas in Probability and Chance →
In this lesson
What logarithms mean, the log laws, and computing logs for AI.
- Define a logarithm as an exponent and evaluate simple logs in bases 2, e, and 10
- Apply the product, quotient, power, and change of base rules while avoiding common mistakes
- Compute logs using tables, series, calculators, and code libraries
- Explain how logs compress scale, prevent underflow in likelihoods, and count halving steps
Free study notes
Get this lesson's notes by email
The key points and 4 practice questions with answers, in your inbox. One email, free.
One email. Unsubscribe in one click.
Transcript
Everything said in the video, section by section (2,337 words). Read along, or search it.
Opening Question
Halving Searches and Vanishing Probabilities
A search that checks one item at a time needs a million steps for a million items. A search that halves the list at each step needs only about twenty. And a model that multiplies a thousand small probabilities gets a number so tiny, the computer rounds it to zero. In both stories, the logarithm comes to the rescue. So it's worth knowing what logs mean, and how to compute them.
Roadmap
Definition, Laws, Methods, and Uses
We'll build it in four parts. What a log is. The log laws, and change of base. Four ways to compute a log. And three places where logs do real work in AI.
Definition
A Log Is an Exponent
Start with a question. How many times must 2 be multiplied by itself to reach 8? Three times: 2 times 2 times 2 is 8, so 2 cubed is 8. A logarithm asks that question in short form. Log base 2 of 8 is 3. In general, log base b of x equals y means b to the y equals x. The log is the exponent. The base must be positive, and not 1, and x must be greater than zero.
Key Facts
Log of One and Log of the Base
Two facts follow. Log base b of 1 is 0, since b to the zero is 1. And log base b of b is 1, since b to the first is b. And a positive base raised to any power, even a negative one, gives a positive result. So the log of zero, or of a negative number, doesn't exist.
Three Common Bases
Binary, Natural, and Common Logs
Any positive base except 1 works, but in practice, three bases dominate. Base 2 gives the binary log, written log 2 of x. It counts bits and halving steps, so it's the natural base for computer science. Base e, about 2.718, gives the natural log, written ell-en of x. It's the base of calculus, probability, and most machine learning formulas. Base 10 gives the common log, written log 10 of x. Each step up of 1 means ten times bigger.
Common Mistake
Plain Log Has No Single Meaning
A warning: the plain word log has no single meaning. Many calculators mean base 10. Many computer science books mean base 2. And Python's math dot log means the natural log. When it matters, check which base is meant.
Patterns
Values Across Three Bases
Here are values in all three bases, and three patterns. First, the log of 1 is zero in every base. Below 1, logs are negative. Above 1, they're positive. Second, logs rise slowly. From 10 to 1,000, the input grows a hundred times, but log base 10 only goes from 1 to 3. Third, a bigger base gives a smaller log: above 1, log base 2 beats the natural log, which beats log base 10.
Log Laws
Product, Quotient, and Power Laws
Because a log is an exponent, it inherits the rules of exponents. Product law: the log of x times y is log x plus log y. So log 2 of 8 times 4 is 3 plus 2, which is 5. Quotient law: the log of x over y is log x minus log y. Log 10 of a thousand over 10 is 3 minus 1. Power law: the log of x to the k is k times log x. Log 2 of 4 cubed is 3 times 2, which is 6. Also, the log of 1 over x is minus log x. All of these need the same base on both sides.
Worked Example
Combining Three Base 2 Logs
Put the laws to work. Evaluate log 2 of 32, plus log 2 of 8, minus log 2 of 4. Step one: read off each log. 32 is 2 to the fifth, so 5. Then 3, and 2. Step two: combine. 5 plus 3 minus 2 is 6. Step three: check with the laws. It's log 2 of 32 times 8 over 4, which is log 2 of 64: 6 again. Two routes, one answer.
Worked Example
Expanding a Natural Log Expression
Now go the other way, and expand. Write ln of x cubed times y, over z squared, as separate logs. Step one, the quotient law: ln of x cubed y, minus ln of z squared. Step two, the product law on the first part: ln of x cubed, plus ln of y, minus ln of z squared. Step three, the power law: 3 ln x, plus ln y, minus 2 ln z.
Quick Check
Which Line of Sixteens Is Right
Quick check. Three students each simplify a log of sixteens, in base two. Only one of these lines is right. Which line is it? Pause, and decide. Line B. 16 times 16 is 256, which is 2 to the eighth, and 4 plus 4 is 8: the product law, on a product. Line A is tempting, but its log holds a sum. 16 plus 16 is 32, and log 2 of 32 is 5. In line C, the square sits outside the log. 4 squared is 16, not 8.
Common Mistakes
Four Classic Log Errors
Those were two of four classic mistakes. One: the log of a sum is not the sum of the logs. Two: a quotient of logs is not the log of a quotient. Log 2 of 8, over log 2 of 4, is 1.5, not 1. Three: a log raised to a power is not the power law. Four: there's no log of zero, or of a negative number.
Change of Base
Computing Any Base From Another
Most calculators offer only a few bases. The change of base rule computes a log in any base, from a log you do have. Log base b of x equals log c of x, divided by log c of b, for any base c, like 10 or e. Why? Let y be log base b of x, so b to the y is x. Take log c of both sides. The power law gives y times log c of b, equals log c of x. Divide, and that's the rule.
Worked Example
Log Base 2 of 50
Try it. Find log base 2 of 50, using natural logs. Ell-en of 50 is 3.9120, and ell-en of 2 is 0.6931. Divide, and you get 5.6439. Use base 10 logs instead, and you get the same answer. The new base cancels out, so the choice doesn't matter.
Constant Factors
Bases Differ by a Fixed Multiplier
This also shows that logs in different bases differ only by a constant factor. Multiply ell-en by 1.4427 to get log base 2. Multiply log base 10 by 3.3219 for the same. And ell-en times 0.4343 gives log base 10. So people often just say the work grows like log n, without naming a base.
Method One
Printed Tables and Interpolation
Now, four ways to compute a log. Method one: printed tables of base 10 logs, for numbers from 1 to 10. Any positive number is m times 10 to the n, with m from 1 up to 10. Then log 10 of x is n, plus log 10 of m. Take 4,720. That's 4.72 times 10 cubed, so n is 3. The table gives 0.6739 for 4.72. Add 3: 3.6739. Between two entries, interpolate: 4.725 is halfway, so its log is halfway too, 0.6744.
Historical Use
Turning Multiplication Into Addition
Tables mattered because of the product law. To multiply 4,720 by 318, look up both logs. Add them: 3.6739 plus 2.5024 is 6.1763. Reverse the log: 10 to the 6.1763 is about 1.501 million, very close to the exact product. A hard multiplication became an easy addition.
Method Two
A Series for ln of 1 Plus x
Method two: a series, an endless sum of shrinking terms. Stop after a few, and you're close. The classic one: ell-en of 1 plus x equals x, minus x squared over 2, plus x cubed over 3, and so on. It works best when x is near 0. Try ell-en of 1.1, so x is 0.1. One term gives 0.1. Two give 0.095. Each extra term shrinks the error. By five terms, the total is 0.095310, and the error is under two ten-millionths.
Faster Methods
A Faster Series and Range Reduction
Far from 1, the terms stop shrinking, and the series fails. Two fixes help. A faster series, built on z equals x minus 1, over x plus 1, works for any x above zero. For ell-en of 2, five terms give 0.693146. And range reduction writes x as m times 2 to the k, with m near 1. Then ell-en of x is ell-en of m, plus k times ell-en of 2. For 20, that's 1.25 times 2 to the fourth, giving 2.995732, the true value. Libraries use refined versions of exactly this idea.
Method Three
Calculator Log and ln Keys
Method three: a calculator, with two log keys: log, for base 10, and ell-en, for base e. Other bases need change of base. For log 2 of 50: ell-en 50, divide, ell-en 2. That gives 5.6439. Check by raising the base back: 2 to the 5.6439 should be near 50, and it is. The common slip is pressing log of 50 over 2, the log of 25, instead of log 50 divided by log 2.
Method Four
Python math and NumPy Logs
Method four, the one AI work almost always uses: a code library. Python's math module handles single numbers, and NumPy handles whole arrays. math dot log of 100 is the natural log, 4.605 and so on. math dot log 2 of 1,024 is 10.0. math dot log 10 of 1,000 is 3.0. And math dot log of 50, comma 2, is log base 2: 5.6439, to four places. NumPy's log 2 turns a whole array of powers of two into their exponents, in one call. And log 1 p of x gives ell-en of 1 plus x, accurately, even for tiny x.
Library Cautions
Rounding, Invalid Inputs, and Tiny Values
Libraries come with three cautions. First, rounding: computers store most numbers approximately. math dot log of 125, comma 5, gives a hair more than 3. So use the dedicated log 2 and log 10, and compare with a tolerance, not exact equality. Second, invalid inputs. math dot log of zero raises an error, while NumPy returns minus infinity, or not a number, with a warning. Check inputs first. Third, tiny values. math dot log of 1 plus 10 to the minus 16 returns 0.0, because the sum rounds to 1. Log 1 p gets it right.
Choosing a Method
Digits of Accuracy by Method
So which to use? Tables give 4 or 5 digits, and show the ideas. A series gives as many digits as terms you add, and lives inside libraries. Calculators give 10 to 12 digits, for quick checks. Code libraries give 15 to 16, for everyday AI work.
Use One
Compressing Huge Ranges of Scale
Now, three places where logs do real work in AI. Use one: compressing scale. Populations, word counts and incomes span enormous ranges, and the giants swamp everything else. In log base 10, each step of 1 means ten times bigger. Illustrative cities from a thousand people up to ten million become just 3 up to 7. AI work uses this on skewed features, with log 1 p for counts that can be zero. It spaces learning rates evenly on a log scale, and draws log-scale charts.
Use Two
Log-Likelihood of a Model
Use two: log-likelihood. The likelihood is how probable a model found everything that actually happened. For independent outcomes, it's the product of the probabilities. Say a model gave 0.9, 0.8, 0.7, 0.9 and 0.6 to five outcomes. The likelihood is their product, 0.27216. The log-likelihood is the log of that product: by the product law, the sum of the logs. That sum is minus 1.3014. Nearer zero means better predicted. And since probabilities are at most 1, it's never positive.
Why Logs Help
Avoiding Underflow in Long Products
Why bother with logs? First, they avoid underflow, where a number too close to zero becomes zero. In Python, 400 probabilities of 0.001 multiply to exactly 0.0. Their logs add to minus 2,763.1: perfectly usable. Second, a sum, with one term per data point, keeps the formulas for improving a model simple. Third, logs never reverse an order, so the best likelihood is also the best log-likelihood. Training often minimizes the negative log-likelihood, the same thing with the sign flipped. Logs also sit inside entropy, which you'll meet later.
Quick Check
One Confident Miss Sinks a Model
Quick check. Two models score three outcomes that all happened. Model P gave them 0.9, 0.9 and 0.1. Model Q gave each 0.6. Using log-likelihood, which model fits better? Pause, and decide. Model Q. Its log-likelihood is 3 times ell-en of 0.6: about minus 1.53. P's is about minus 2.51, further from zero. Its two confident hits are tempting, but its 0.1 alone costs minus 2.30. One confident miss can sink a model. The products agree: 0.216 for Q, and 0.081 for P.
Use Three
Counting Halving Steps
Use three: measuring complexity. Guess a secret number from 1 to 1,000. Asking, is it 1, is it 2, could take a thousand guesses. Asking, is it above 500, and halving each time, takes at most 10, because 2 to the tenth is 1,024. In general, halving n items takes log base 2 of n steps, rounded up. A million items take 20, a billion 30, and eight billion just 33. That's the halving search from the start.
Bits and Codes
Bits Needed to Label n Items
The same count labels things: giving n items each its own binary code takes log base 2 of n bits, rounded up. And because bases differ only by a constant, people just say the work grows like log n, and skip the base.
Worked Example
Sixteen Bits for 50,000 Words
Let's put it together. A language system has 50,000 words. How many bits give each word its own code, using only ell-en and log keys? Step one, name it: log base 2 of 50,000. Step two, choose a method: change of base, ell-en of 50,000 over ell-en of 2. Step three, compute: 10.8198 over 0.6931 is 15.61. Step four, interpret: bits are whole, so round up to 16. Step five, verify: 2 to the 15 is 32,768, too small, and 2 to the 16 is 65,536, enough. Name, choose, compute, interpret, verify: five steps for any log problem.
Key Takeaways
Exponents, Laws, Methods, and Uses
To sum up. A logarithm is an exponent, and its input must be positive. The log laws turn products into sums, quotients into differences, and powers into multiples. Never use them on a sum. Change of base gets any log from ell-en or log. Tables, series, calculators and libraries all compute logs, and libraries need care. And in AI, logs compress scale, keep likelihoods from underflowing, and count halving steps.
Reflection
Finding Log Base 7 of 300
Here's something to think about. Your calculator has only log and ln keys. How would you find log base 7 of 300? And why doesn't it matter which of the two keys you use?
About Introduction to Artificial Intelligence at UPI Study
Introduction to Artificial Intelligence (Computer Science 373) is a self-paced online course of 69 video lessons in 9 chapters, from intelligent agents and search to game strategy, neural networks, logic and reasoning, machine learning, language, robotics and AI ethics. It carries a NCCRS credit recommendation for 3 semester credits.
Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Enroll now → Course details Download the syllabus All Computer Science courses Every course with PRO →
Compare: AI vs Current Trends in CS and IT
Why take it here
Why students take Introduction to Artificial Intelligence with UPI Study
Where the credit goes
Universities that work with NCCRS
Introduction to Artificial Intelligence carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
Thomas Edison State UniversitySigned partner
University of the PeopleSigned partner
Charter Oak State CollegeSigned partner
Strayer UniversitySigned partner
Columbia Southern UniversitySigned partner
Indiana TechSigned partner
Penn StateWorks with NCCRS
Southern New Hampshire UniversityWorks with NCCRS
SUNYWorks with NCCRS
+many more cooperating universitiesFind my college →Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.
FAQ
Questions students ask us
About Techniques for Computing Logarithmic Values
What is a logarithm in simple terms?
How do you calculate a logarithm in a base your calculator does not have?
What are common mistakes with logarithm rules?
Why does machine learning use log-likelihood instead of likelihood?
About the Introduction to Artificial Intelligence course
How many college credits is Introduction to Artificial Intelligence?
How much does Introduction to Artificial Intelligence cost and what is included?
Which universities accept UPI Study credits?
How long does Introduction to Artificial Intelligence take?
How are the exams done?
How do my credits reach my college?
Ready for the full course? Earn 3 credits in Introduction to Artificial Intelligence.
69 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.
3 credits · NCCRSEnroll →