Image-Focused Learning with Convolutional Networks, Explained

UPI Study › Courses › Introduction to Artificial Intelligence › Free lessons › Lesson 5.6 Free with a free account

Image-Focused Learning with Convolutional Networks

Introduction to Artificial Intelligence · Chapter 5: Neural Networks and Learning ArchitecturesSection 5.6 · lesson 37 of 6911 min · free with a free account

← Training Networks: Backpropagation and Gradient MethodsNext lesson: Sequence-Aware Learning with Recurrent Networks →

Earn 3 college credits for Introduction to Artificial IntelligenceRecommended for credit by NCCRS · Computer Science 373 · self-paced · one payment, lifetime access
Enroll in Introduction to Artificial Intelligence · $250 →Download the syllabus (PDF)

In this lesson

How filters, stride, padding and pooling let networks read images.

  • Compute a feature map value by sliding a filter over an image patch
  • Calculate a convolution's output width from input size, filter size, padding and stride
  • Compare max pooling with average pooling and explain what pooling achieves
  • Describe how LeNet-5, AlexNet, VGG and ResNet shaped CNN design

Free study notes

Get this lesson's notes by email

The key points and 4 practice questions with answers, in your inbox. One email, free.

One email. Unsubscribe in one click.

Transcript

Everything said in the video, section by section (1,517 words). Read along, or search it.

Opening Question

Finding Patterns Anywhere in a Photo

To a computer, a photo is just a grid of numbers. Yet a network can learn to spot an edge, an eye, or a wheel. And it can spot them wherever they happen to appear: top left, bottom right, anywhere. How? By scanning the image with small pattern detectors, the same detector at every position. That's a convolutional neural network, or CNN.

Image Basics

Pixels, Grids and Color Channels

A grayscale image is one grid, with one number per pixel. Larger numbers mean brighter pixels. A color image is three grids stacked together: red, green and blue. Each grid is called a channel. Even a small photo holds hundreds of thousands of numbers. Looking at all of them at once would be wasteful. A CNN looks at small patches instead.

Definition

Filters Slide to Make Feature Maps

The pattern detector is a filter, also called a kernel: a small grid of numbers, often three by three, or five by five. Convolution slides the filter across the image. At each position, multiply each filter number by the pixel beneath it, and add up the products. That single number goes into a new grid, the feature map. A large value means: the pattern this filter looks for is here.

Worked Example

A Vertical Edge Filter Scores 27

Here's a tiny five by five image. The two left columns are dark, zero, and the three right columns are bright, nine. So there's a vertical edge. The filter has minus one in its left column, zero in the middle, and plus one on the right. It responds when pixels get brighter from left to right. Lay it over the top-left window. Each row gives minus zero, plus zero, plus nine: nine. Three rows make 27. Slide one step right: the window still straddles the edge, so it's 27 again. One more step, over the uniform bright area, and minus 9 cancels plus 9: zero. So edges produce strong responses, and flat areas give zero. Different filters find different things. A horizontal-edge filter, with minus ones on top and plus ones on the bottom, gives zero everywhere on the image we just used, because every row is identical. And in a real CNN, nobody writes the filters by hand. The filter numbers are weights, and backpropagation learns them, so the network discovers which patterns are useful.

Quick Check

Horizontal Edge Filter Gives 18

Quick check, with a new window. Its top row is 2, 2, 2. The middle row is 5, 5, 5. And the bottom row is 8, 8, 8. Apply the horizontal-edge filter: minus ones on top, zeros in the middle, plus ones on the bottom. What value goes into the feature map? Pause, and work it out. Eighteen. The top row gives minus 6. The middle row is multiplied by zeros, so it adds nothing. The bottom row gives plus 24. Minus 6 plus 24 is 18: the image gets brighter going down, so this filter responds. Zero is what the vertical-edge filter would give here. And 24 forgets the top row.

Layer Parameters

Many Filters Stacked as Channels

A convolutional layer uses many filters at once. Each makes its own feature map, so a layer with K filters outputs K maps, stacked as channels. A filter also spans every channel of its input. On a color image, a three by three filter holds three times three times three, 27 weights, plus one bias. So a layer has K times, F times F times C, plus 1, parameters. 64 filters of three by three on a color image: 64 times 28, which is 1,792. After convolution, an activation such as ReLU is applied to every value, as in other networks.

Output Size

Stride and Padding Set Map Width

The stride is how many pixels the window moves each step. Stride 1 visits every position. Stride 2 skips every other one, so the map is about half as wide and tall. Padding adds a border of zeros around the image, so edge pixels are covered as often as central ones, and the output can keep the input's size. For input width N, filter width F, padding P and stride S, the output width is N plus 2 P minus F, over S, rounded down, plus 1. A five-wide image with a three-wide filter gives 3 at stride 1, and 2 at stride 2. Pad by 1, and stride 1 keeps all 5.

Pooling

Max and Average Pooling Shrink Maps

Pooling shrinks a feature map by summarizing small neighborhoods. Usually a two by two window slides with stride two, and keeps one number from each window. Max pooling keeps the largest. From this four by four map, the four windows give 6, 5, 9 and 8. Average pooling keeps the mean: 3.5, 2, 5.25 and 3.75. Pooling has no weights to learn. It leaves a quarter of the values, it tolerates small shifts, and each later value summarizes a wider part of the image.

Typical Architecture

Convolution, Pooling, Then Dense Layers

These pieces usually fit together one way. Convolution and pooling repeat, building up features, and fully connected layers at the end turn them into scores. Along the way, the feature maps get smaller, while the number of channels usually grows.

Quick Check

Width After Padding and Pooling

Quick check. An image is 32 pixels wide. It goes through a five-wide filter, with stride 1 and padding 2. Then two by two max pooling, with stride 2. How wide is it after each step? Pause, and work it out. 32, then 16. Padding 2 adds four columns: 32 plus 4, minus 5, over 1, plus 1, is 32. Pooling halves it, to 16. 28 is tempting, but it forgets the padding. That's exactly what padding 2 with a five-wide filter is for: keeping the size.

Landmark Networks

From LeNet-5 to ResNet

A few landmark networks show how these ideas grew. Several were judged on ImageNet: over a million labeled photos in a thousand categories, used in a yearly contest. LeNet-5, in 1998, set the recipe: alternate convolution and pooling, then fully connected layers. About 60 thousand parameters, reading handwritten digits. AlexNet, in 2012, won ImageNet by a wide margin, with about 60 million parameters, ReLU, dropout and graphics processors. It's widely seen as the start of the deep learning boom in vision. VGG, in 2014, stacked only small three by three filters, 16 or 19 weight layers deep. And ResNet, in 2015, added skip connections, and trained networks up to 152 layers.

Design Choice

Why VGG Stacks Small Filters

Why small filters? Two three by three layers in a row let each output see a five by five patch. Three of them see seven by seven. With 64 channels in and out, one five by five layer needs 102,400 weights. Two three by threes need only 73,728. One seven by seven layer needs 200,704. Three three by threes need 110,592. Fewer weights, the same view, and extra ReLU steps in between.

Residual Learning

Skip Connections Pass Inputs Through

As networks grew deeper, researchers hit a surprise: beyond some depth, adding layers made the training error worse, not better. ResNet's fix is the residual block. It computes a correction, F of x, and adds it to its own input. The output is F of x, plus x. If a block has nothing useful to add, it learns F of x near zero, and passes its input straight through. The skip path also gives gradients a direct way back. ResNets up to 152 layers won the 2015 ImageNet contest, with a top-five error of about 3.6 percent.

Key Ideas

Four Reasons Convolution Suits Images

Why does convolution suit images so well? Four reasons. Local connections: each output looks only at a small patch, because nearby pixels are related. Weight sharing: one filter is reused at every position, so a pattern is recognized wherever it appears, with far fewer weights. Shift-friendly maps: move an object in the image, and its feature map moves by the same amount. And a hierarchy of features: early layers respond to edges and colors, middle layers to textures and parts, and deep layers to whole objects.

Parameter Count

Dramatic Savings in Weights

The saving in weights is dramatic. On a 28 by 28 grayscale image, a fully connected layer of 100 neurons needs 78,500 parameters. Six five by five filters need 156. On a 224 by 224 color image, 1,000 dense neurons need about 150.5 million. 64 three by three filters need 1,792. The two layers produce different outputs, so this shows how cost scales, not a one-for-one swap. Fewer weights mean less memory, faster training, and less overfitting.

Limitations

Rotation, Scale and Position Matter

Two limits. Convolution handles shifts, but not rotation or size changes by itself. Networks usually learn those from training images that include them. And sometimes position matters. If a feature means something only in one region, sharing a filter everywhere ignores that, and later layers must supply it.

Key Takeaways

Filters, Pooling and Landmark Designs

To sum up. A filter slides across an image, multiplying and adding, to make a feature map that marks where a pattern appears. Stride and padding set the output size, and pooling shrinks maps by keeping one value per window. LeNet set the recipe, AlexNet proved it at scale, VGG stacked small filters, and ResNet added skip connections. Local connections, weight sharing and a hierarchy of features cut parameters enormously.

Think About It

When Sharing a Filter Fails

Something to think about. An eye can appear almost anywhere in a photo of a face. Why does sharing one filter across all positions make sense for finding eyes? And describe a task where sharing a filter everywhere could be a poor choice, because the position of a feature matters.

About Introduction to Artificial Intelligence at UPI Study

Introduction to Artificial Intelligence (Computer Science 373) is a self-paced online course of 69 video lessons in 9 chapters, from intelligent agents and search to game strategy, neural networks, logic and reasoning, machine learning, language, robotics and AI ethics. It carries a NCCRS credit recommendation for 3 semester credits.

Every lesson transcript is free on pages like this one, and the videos are free with a free UPI Study account. Enrolling adds a practice quiz after each lesson, the graded work and final exam that earn the credit, lifetime access, and an official transcript on completion. UPI Study has helped over 130,000 students with college credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Enroll now → Course details Download the syllabus All Computer Science courses Every course with PRO →

Compare: AI vs Current Trends in CS and IT

Why take it here

Why students take Introduction to Artificial Intelligence with UPI Study

How machines search, reason and learnIntroduction to Artificial Intelligence covers how AI systems are built: intelligent agents, search and heuristics, game strategy, logic, neural networks, machine learning, language and perception, and the ethics of AI. It is a common upper-level course in computer science degrees.
Credit at a fraction of the priceThe same 3 credits can cost $750 or more at a US college. Here it is one payment, self-paced, with lifetime access.
Finish your degree soonerTransfer credits reduce the courses you still owe, so you can graduate earlier or lighten a heavy semester.
Strengthen an applicationA completed college-level course on an official transcript shows admissions and scholarship committees you can do the work.
Study on your own scheduleNo start dates, no deadlines. Watch the lessons, practice, and take the final when you are ready.
130,000+students helped with college creditsACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Where the credit goes

Universities that work with NCCRS

Introduction to Artificial Intelligence carries a NCCRS credit recommendation. These universities work with NCCRS; some are UPI Study's signed partners. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.

Thomas Edison State UniversityThomas Edison State UniversitySigned partnerUniversity of the PeopleUniversity of the PeopleSigned partnerCharter Oak State CollegeCharter Oak State CollegeSigned partnerStrayer UniversityStrayer UniversitySigned partnerColumbia Southern UniversityColumbia Southern UniversitySigned partnerIndiana TechIndiana TechSigned partnerPenn StatePenn StateWorks with NCCRSSouthern New Hampshire UniversitySouthern New Hampshire UniversityWorks with NCCRSSUNYSUNYWorks with NCCRS +many more cooperating universitiesFind my college →

Find my college → Not listed? Ask Athena in the corner of this page, or book a 15-minute call.

FAQ

Questions students ask us

About Image-Focused Learning with Convolutional Networks

What is a convolutional neural network?
A convolutional neural network, or CNN, is a neural network that scans an image with small learned filters, applying the same filter at every position. Each filter produces a feature map that marks where its pattern appears. Repeated convolution and pooling build features from edges up to whole objects, and fully connected layers at the end turn those features into class scores.
How do you calculate the output size of a convolutional layer?
Use output width = floor((N + 2P - F) / S) + 1, where N is the input width, F the filter width, P the padding and S the stride. For a 32-pixel input with a 5-wide filter, padding 2 and stride 1, that gives (32 + 4 - 5) / 1 + 1 = 32, so the size is kept. Forgetting the padding is a common mistake and gives 28 instead.
What is the difference between max pooling and average pooling?
Max pooling keeps the largest value in each window, while average pooling keeps the mean. Usually a 2 by 2 window slides with stride 2, so a quarter of the values remain. For a window holding 1, 3, 2 and 6, max pooling gives 6 and average pooling gives 3. Pooling has no weights to learn, and it helps the network tolerate small shifts.
Why do CNNs need fewer parameters than fully connected networks?
CNNs use local connections and weight sharing, so one small filter is reused at every image position. On a 28 by 28 grayscale image, a dense layer of 100 neurons needs 78,500 parameters, while six 5 by 5 filters need just 156. A convolutional layer has K times (F times F times C plus 1) parameters, so 64 filters of 3 by 3 on a color image need 1,792.

About the Introduction to Artificial Intelligence course

How many college credits is Introduction to Artificial Intelligence?
Introduction to Artificial Intelligence (Computer Science 373) carries a NCCRS credit recommendation for 3 semester credits. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities.
How much does Introduction to Artificial Intelligence cost and what is included?
$250, paid once, with lifetime access: all 69 video lessons, a practice quiz after each, the graded work and final exam, and an official transcript when you finish. Students who want several courses choose UPI Study PRO instead.
Which universities accept UPI Study credits?
UPI Study has signed partner agreements with universities such as Thomas Edison State University, University of the People and Charter Oak State College, and works with universities such as Penn State, SNHU and SUNY. ACE and NCCRS have over 2,100 cooperating colleges that will consider accepting credits; the final decision is with the universities. Use "Find my college" on this page to check yours.
How long does Introduction to Artificial Intelligence take?
It is self-paced with no deadlines. The 69 lessons total about 16 hours of video; most students add reading and practice time and finish in a few weeks to a few months, on their own schedule.
How are the exams done?
Quizzes and assignments are online in the course. The final exam is taken online with proctoring; a person on the UPI Study team checks assignments, usually within two days.
How do my credits reach my college?
When the course shows 100% complete, request the official transcript from your dashboard. It is sent to the college you name, and you can send it to more than one. Check your college first with "Find my college" on this page, or ask Athena.

Ready for the full course? Earn 3 credits in Introduction to Artificial Intelligence.

69 video lessons, a quiz after each one, and the graded work that earns the credit. Recommended for credit by NCCRS.

Enroll in Introduction to Artificial Intelligence →Download the syllabusEvery course with PRO →
Introduction to Artificial Intelligence
3 credits · NCCRSEnroll →