C H A P T E R 9
Third Computer Lab
In this lab, you will import images into Python and display them. You will also explore an important operation in image analysis: convolution. If you’ve ever played with photographic software such as GIMP1(or a commercial alternative), you may have used convolutions to smooth or sharpen images but perhaps you didn’t know it. Your eyes and brain are also doing convolutions whenever you look at anything.
Not all visual data comes from photographs. Many kinds of experimental measurements generate arrays of numbers associated with points in space. Such data can be communicated to our brains as images, as in atomic force microscopy, computed tomography, and magnetic resonance imaging.
Image processing is useful for making these images more meaningful to humans.
Our goals in this lab are to
• Explore the effects of various kinds of local averaging on an image;
• See how to use such averaging to decrease noise in an image; and
• Use specialized filters to emphasize specific features in an image.
9.1 Convolution 107 When we convolve an image with a filter, we get another image. The expression in Equation 9.1 is a set of instructions for constructing this new image: To create each pixel in C, we take the pixels from a subset of the original image, multiply them by their respective weights in the filter, and add up the result. It’s a simple recipe that can have very different results depending on the filter.
9.1.1 Python tools for image processing
Chapter 8 explained how to load, display, and save images with PyPlot. We also performed a mathematical operation on an array to modify the image. Now we will expand our capabilities by importing a module with several image processing functions:
import scipy.ndimage as sim
We chose the nicknamesimas a mnemonic for “SciPy image library.”
One useful function in this module issim.convolve. Usehelpto learn about this function and its options. All of the filters we will use in this lab accept similar arguments.
In this lab, you will explore several filters and convolutions. As you proceed through the exercises, you will see a photograph transformed by each operation. To get an idea of what is happening on a pixel-by-pixel level, you can apply the same convolution to a single dot. (This is theimpulse response of the filter.) This will allow you to see the shape of the filter and better understand what you are doing to the photograph.
Try the following now:
impulse = np.zeros( (51, 51) ) impulse[25, 25] = 1.0
my_filter = np.ones( (3, 3) ) / 9
response = sim.convolve(impulse, my_filter)
5 plt.figure()
plt.imshow(response)
We will explain this filtering operation in the next section. For now, compare the size of two arrays, impulse andresponse. They are the same! You found in Your Turn 9A that the convolution of an image with a filter is at least as large as the original image, and usually larger. Where do the extra points go in Python? Where did they come from in the mathematical derivation?
Refer back to Equation 9.1 and look atC[0,0]. The allowed values ofk and`for this point give only one contribution. Likewise,C[M+m-2,N+n-2]has only one contribution:
C[0, 0] is F[0, 0] * I[0, 0]
C[M+m-2, N+n-2] is F[m-1, n-1] * I[M-1, N-1].
The points at the edges of the convolved image receive contributions from fewer points in the original image than do the points in the interior. This may lead to distortions at the edges of the convolved image. Python crops the edges and returns only the central portion of the convolved image. This has two advantages. First, every point in the convolved image uses at least a quarter of the filter. Second, convolution does not change the size or shape of the original image.
Even with this cropping, however, points near the edge must be treated differently from those in the center. The functionsim.convolveand its relatives offer several options. The simplest is to imagine that the image is surrounded by an infinite black border. That means we effectively enlarge our image array to whatever size it needs to be to provide the same number of points for every Jump to Contents Jump to Index
108 Chapter 9 Third Computer Lab
pixel in the convolution and set the values of all the new points to0. To select this behavior, use the keyword and valuemode='constant'when callingsim.convolve. This overrides the default behavior, which is to pretend that the image is surrounded by reflections of itself.
9.1.2 Averaging
A very simple filter assigns the same weight to each pixel in a fixed region, as in the impulse response example above. Each pixel in the convolved image is an average of its neighbors in the original image.
Assignment:
Follow the instructions of Section 4.1 to obtain the data set 16catphoto. Copy the filesbwCat.tif, gauss filter.csv, andREADME.txtinto your working directory.
a. Make a3×3arraymy_filterin which each element equals1/9. Why does it make sense to choose the value1/9? We’ll call this array the “small square filter.”
b. Usesim.convolveto convolve your new filter with the image you downloaded, and display the re-sult. How does the image change? Try to achieve the same result by using sim.uniform_filter.3 [Hints: You can retain the previous picture for comparison by using plt.figure() to create a new figure before displaying the second. Another option is to display the plots side by side with plt.subplot. (Consult help(plt.subplot) and Section 4.3.7.) You need to set the color map separately for each figure window that you create. (Consulthelp(plt.colormaps).)]
c. Repeat part (a) using a15×15array (the “large square filter”). Be sure to use a constant value appropriate to this larger array.How does the image change? How does it compare with the result of the smaller filter?
d. Use the definition in Equation 9.1 to show that these convolutions produce images in which each pixel is the average of some of the neighboring pixels in the original.
9.1.3 Smoothing with a Gaussian
Now we will look at a slightly more complex filter, called a Gaussian filter. Load the file gauss filter.csv into a NumPy array called gauss. Then, review Section 6.4.3 to refresh your memory on the functionplot_surface.
Assignment:
a. Display the convolution of gauss with the original image. (Also try sim.gaussian_filter with the keyword optionsigma=5.)
b. Use plt.imshow to compare the convolutions of a single dot with a Gaussian filter and the square filter you used in question 9.1.2b.
c. Use plot_surfaceto view the convolved images from part (b) in three dimensions. This is actually a plot of the filters themselves. Use the definition of convolution to explain why. Then, explain how convolution with a Gaussian filter differs from a square filter, and under what circumstances one might prefer a Gaussian filter.
3The functionssim.uniform_filterandsim.gaussian_filtercan be significantly faster thansim.convolve for large filters, but the output is not identical.