What is Image Filtering?

Filtering replaces each pixel with a value based on some function performed on it’s local neighbourhood.

What is Image Filtering?

Used for smoothing and sharpening…

What is Image Filtering?

Estimating gradients…

What is Image Filtering?

Removing noise…

Linear Filtering

Linear Filtering is defined as a convolution.

This is a sum of products between an image region and a kernel matrix:

\[g(i, j) = \sum_{m=-a}^{a}\sum_{n=-b}^{b} f(i - m, j - n) h(m, n)\]

where \(g\) is the filtered image, \(f\) is the original image, \(h\) is the kernel, and \(i\) and \(j\) are the image coordinates.

Convolution

Typically:

\[a=\lfloor \frac{h_{rows}}{2} \rfloor, ~ b=\lfloor \frac{h_{cols}}{2} \rfloor\]

So for a 3x3 kernel:

\[ \text{both } m, n = -1, 0, 1\]

Kernel Matrix

Convolution

1. Scan image with a sub-window centred at each pixel.

   • The sub-window is known as the kernel, or mask.

2. Replace the pixel with the sum of products between the kernel coefficients and all of the pixels beneath the kernel.

   • Sum of products only for linear filters

3. Slide the kernel so it’s centred on the next pixel and repeat for all pixels in the image.

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

Convolution

What about the edges?

What about the edges?

What about the edges?

What about the edges?

What would the filtered image look like?

No change!

What would the filtered image look like?

Shifted left by 1 pixel.

What would the filtered image look like?

Blurred…

Mean Filter

replace each pixel with the mean of local neighbours:

\[ h = \frac{1}{9}\times \begin{bmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{bmatrix} \]

Mean Filter

Mean Filter

we can increase the size of the kernel to get a smoother image:

\[ h = \frac{1}{25}\times \begin{bmatrix} 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \end{bmatrix} \]

Mean Filter

Mean Filter

Gaussian blur

Similar to mean filter:

• Replace intensities with a weighted average of neighbours.
• Pixels closer to the centre of the kernel have more influence.

Gaussian blur

Gaussian blur

Gaussian blur

Image Smoothing

Smoothing effectively low pass filters the image.

• Only really practical for small kernels
• Blurring also destroys image information
• Difference between the mean and Gaussian filter is subtle, but Gaussian is usually preferred

Image Smoothing

If we have many images of the same scene:

• Use idea of averaging to reduce noise.
• Average pixel intensities across images rather than across the spatial neighbour.

Image Smoothing

• Effectively increases the signal-to-noise ratio.
• Useful in applications where image signal is low.
  • E.g., imaging astronomical objects.

What would the filtered image look like?

Image Sharpening

We can control the amount of sharpening:

\[ h_{sharp} = \begin{bmatrix} 0 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 0 \end{bmatrix} + \begin{bmatrix} 0 & -1 & 0\\ -1 & 4 & -1\\ 0 & -1 & 0 \end{bmatrix} * amount \]

More Kernel Examples

There is a nice interactive tool to view kernel operations here: https://setosa.io/ev/image-kernels/

The ImageMagick documentation has a nice list of kernels: https://legacy.imagemagick.org/Usage/convolve/

Unsharp Masking

A high pass filter formed from a low pass filtered image.

• Usually preferred over kernel sharpening filter.
• A legacy of the pre-digital period.

Unsharp Masking

Low pass filter removes high-frequency detail.

Unsharp Masking

Unsharp Masking

The sharpened image is the original image plus the unsharp mask multiplied by some factor.

• The difference image is the unsharp mask!

Unsharp Masking

Unsharp Masking

Unsharp Masking

Unsharp Masking

Generally don’t want to boost all fine detail as noise would also be enhanced.

• Adjust the Gaussian parameters.
• Threshold the difference image.
• Care is required to avoid artefacts (e.g. halos).

Image Filters as Templates

If the filter kernel contains a picture of something you want to locate inside the image (a template), the filter response should be maximised at the local region that most closely matches it.

• We can use image filtering for object location
• Known as Template Matching.

Image Filters as Templates

Algorithm:

• subtract the mean from the image and template
• convolve the template with the image
• find the location of the maximum response

Image Filters as Templates

Image Filters as Templates

Image Filters as Templates