Understanding Convolutional Layers: Forward and Backward Passes with Multiple Filters

In actual convolutional neural networks (CNNs), multiple filters are used in each convolutional layer. Each filter learns to detect different features in the input data. Here’s a more detailed explanation of how this works:

Multiple Filters in Convolutional Layers

Forward Pass with Multiple Filters

• Input: Let’s consider a single-channel (grayscale) image of size 5×5.
• Filters: Suppose we have 2 filters, each of size 3×3.

The resulting feature maps are stacked together to form a multi-channel output.

Backward Pass with Multiple Filters

During the backward pass, each filter is updated based on the gradient of the loss with respect to the filter’s weights.

Steps in the Backward Pass:

1. Compute Loss: Calculate the loss using the output of the network and the actual labels.
2. Calculate Gradients:
   • For each feature map, calculate the gradient of the loss with respect to the feature map.
   • Backpropagate these gradients through the convolutional layer to get the gradients with respect to each filter’s weights.
3. Update Filter Weights:
   • For Filter 1, calculate the gradient of the loss with respect to each element in the filter.
   • Update the filter weights using an optimization algorithm.

Example Update:

Summary

• Multiple Filters: Convolutional layers use multiple filters to learn different features from the input.
• Forward Pass: Each filter produces a feature map, and these maps are stacked together.
• Backward Pass: Each filter’s weights are updated using the gradients of the loss with respect to the weights.

This process enables the CNN to learn and extract various features from the input data, improving its ability to make accurate predictions.