1

ME 360: FUNDAMENTALS OF SIGNAL PROCESSING, INSTRUMENTATION AND CONTROL

Experiment No. 5 – Equalizing Audio Signals, Edge Detection in Images

1. CREDITS

Originated: G. Bahl, November 2019

Last Updated: S. S. Igram, October 2020

2. OBJECTIVES

(a) This project is designed to exercise your knowledge in practical signal processing.

(b) Build an understanding of representing signals in the frequency domain.

(c) Become familiar with transforming different types of signals using the Fourier transform.

(d) Practice obtaining desired information from various signals by filtering.

3. KEY CONCEPTS

(a) Signals have distinctive aspects at different frequency ranges. Discrete Fourier Transforms are commonly used to obtain frequency representations of signals.

(b) Filters can be designed to isolate frequency-specific aspects such as isolating beats (low frequency components) in an audio file or detecting edges (high-frequency components) in an image.

(c) Signal processing can lead to artifacts (such as image echoes and aliasing). These artifacts can be alleviated using appropriate filters.

4. SYNOPSIS OF PROCEDURE

4.1 Audio Processing

(a) Build a playable and editable audio file.

(b) Use Matlab’s Fast-Fourier Transform algorithm to construct the DFT of an audio file and produce a single-sided magnitude spectrum plot.

(c) Use the DFT and the inverse-Fourier function to filter specific frequency ranges and reconstruct an audio file.

4.2 Image Processing

(a) Convert an image to grayscale and compute its 2-D DFT.

(b) Construct a 2D geometric HPF to ‘detect’ the edges of an image, removing the rest.

(c) Reconstruct an image of the detected edges, removing any artificial echoes from the reconstructed image by refining the HPF you design.

2

5. Background

The Discrete Fourier Transformation

The Discrete Fourier Transform (DFT) is designed to perform analysis of finite-length discrete time signals using a discretized frequency space representation. This is in contrast to the DTFT which is a continuous function.

𝑋 𝑘1𝑁

𝑥 𝑛 𝑒 .

Note that 𝑁 is the number of data points in both the input vector 𝑥 𝑛 and the DFT 𝑋 𝑘 . Note also that the DFT vector 𝑋 𝑘 is in general complex-valued. This DFT can be obtained very easily in Matlab using the fft() function as follows:

Fk = fft( fn );

The above operation uses the Fast Fourier Transform (FFT), which is simply one of many algorithms that can be used to compute the DFT. The resulting DFT has indices 1 to N. This can be easily related to the Discrete-Time Fourier Transform (DTFT), which is given by

𝑋 Ω 𝑥 𝑛 𝑒 or equivalently 𝑋 F 𝑥 𝑛 𝑒 ,

where Ω 2𝜋𝐹 represents discrete frequency. 𝑋 Ω is periodic with periof Ω 2𝜋 or 𝐹 1 . The frequency indices 𝑘 1 to 𝑁 in DFT correspond to the first principal period of the equivalent DTFT with discrete frequency Ω 2πF spanning from 0 to 2𝜋. More precisely the frequency 𝑘 in DFT relates to Ω

2𝜋 , or equivalently 𝐹 . This relation implies that high frequencies are near Ω π or 𝐹 1/2 ; i.e.

exactly in the middle of the DFT 𝑋 𝑘 . The low frequency components are therefore at 0 and 2𝜋 (remember the periodicity of the DTFT) which correspond to the 1st and Nth positions in 𝑋 𝑘 .

Since this principal period may sometimes be inconvenient for visualization, Matlab provides an fftshift() function that rearranges the transform (using the periodicity property) by shifting the zero-frequency to the center of the vector. It does this quite easily by just exchanging the two halves of the array (blue, orange) as shown in the illustration below. You can use this function if you ever need to.

3

Fourier Analysis and Image Processing

Fourier analysis is also very useful in image processing. All we have to do is remember that images are simply two-dimensional signals. In this lab, we will also learn to perform a simple image processing operation in Matlab. While we certainly could do all processing in color, for simplicity we will choose only to use grayscale here.

Now, let us take a moment to consider the fact that any grayscale image is simply a matrix of numbers (each number is the brightness of the corresponding pixel) and that we should be able to take a Fourier transform of this “signal”. The image is stored using a 2-dimensional matrix 𝐺 𝑚,𝑛 where 𝑚 and 𝑛 are the pixel location in 𝑌 and 𝑋 coordinates respectively. By extension of the 1-D case, we can define a 2-dimensional Discrete Fourier Transform (2D DFT) as follows:

𝐺 𝑘, 𝑙1𝑀𝑁

𝑔 𝑚.𝑛 𝑒 .

Just as we did with the 1-dimensional audio file, we can now do frequency-domain operations with this 2D transform of the image. See visualizations of this in the figures below. Once again, Matlab provides a convenient function called fft2( ) that can perform the above 2D DFT operation for us very quickly.

ImageTransform = fft2( grayscale_data );

Remember that the frequency is now in units of 1/pixel. Just like the 1D DFT, the highest frequencies are in the middle of the 2D DFT. The low frequencies lie on the corners. This is illustrated in the graphic below.

As before, you can use the fftshift( ) function in Matlab to move the zero frequency to the middle and the high frequencies to the corners (and vice versa).

4

6. PROCEDURE

6.1 Audio Equalizer

6.1.1 Audio File in the Frequency Domain

(a) A 15-second electronic music file Electronic.mp4 is provided along with the project description. You can load this file into Matlab using the command: [y_electronic, Fs] = audioread('Electronic.mp4'); where Fs is the sampling rate information.

(b) The audio can be played back using the following commands playerObj = audioplayer( y_electronic, Fs); % build audio play(playerObj); % play the audio

(c) Use the Matlab fft( ) function to compute the DFT, 𝑋 𝑘 , of this music file.

(d) Plot the double-sided magnitude spectrum with a logarithmic vertical axis, using semilogy( ). The frequency-axis should be presented in continuous frequency units, i.e. you must correctly use the sampling rate information to convert from the array index to frequency.

(e) Provide a clean and commented version of your Matlab code (the actual code please, not just a screen capture).

6.1.2 Bass/Low-Frequency Amplification

(a) Let’s say that you like the beat in this music and you want to hear more of it. For this, the goal is to create a filter that amplifies the bass between 0 and 1200 Hz (for this exercise we will aim for uniform amplification by a factor of 3).

(b) This filtering operation can be achieved easily since you already have the DFT of the audio file (*from above). Working directly with the DFT, you should selectively amplify only the frequencies in the desired range. Leave the rest of the frequencies untouched i.e. gain = 1. Do not forget that the DFT contains both positive and negative frequencies, so ensure all are accounted for. Show/explain in detail your calculations for how you identify the correct range of 𝑘 in 𝑋 𝑘 to achieve this operation.

• Common pitfall: You must ensure that the DFT is modified perfectly symmetrically for both positive and negative frequencies. If you are not careful (e.g. if you are off by 1 index on either side), the resulting signal becomes complex-valued since the opposing complex-exponentials can no longer cancel out their imaginary parts. The resulting audio reconstruction may sound wrong - avoid this!

(c) After applying the filter, you can put the result into a new spectrum (e.g. FilteredDFT). Finally, you can reconstruct the bass-amplified audio using the inverse DFT operation ifft( ) like this: y_reconstructed = ifft( FilteredDFT );

(d) Play your filtered sound clip to your TA to verify your filter is correct using playerObjrecon = audioplayer( y_reconstructed, Fs); % build audio play(playerObjrecon); % play the audio

(e) Feel free to produce DFT figures (for yourself) as needed to make sure that your operation is correct and play the filtered audio back for yourself so that you know that you are getting it right. You should be able to hear the difference clearly. Again, no need to submit these figures if you choose to produce them – just good housekeeping!

(f) Make sure to submit your Matlab code in the report.

6.1.3 Let the Bass Drop(Out) / High-Pass Filtering

(a) Now, create a new filter that only retains the higher frequencies without the bass from 0 to 1200 Hz. You can do this by simply setting the DFT or gain values in the corresponding frequency range to zero while leaving the other frequencies untouched.

(b) Play your filtered sound clip to your TA to verify your filter is correct using playerObjrecon2 = audioplayer( y_reconstructed2, Fs); % build audio play(playerObjrecon2); % play the audio

(c) Listen to the original audio file and the filtered files. Describe the difference in the music between the audio files (i) the original and the bass-amplified and (ii) the original and the bass deleted.

(d) What frequency range corresponds to treble (you can use internet search to find this out)? Would you design a low-pass or a high-pass filter to remove treble from an audio file?

(e) Make sure to submit your Matlab code in the report.

5

6.2 Image Edge Detection Edges in an image are nothing more than rapid changes of color or intensity. In other words, edges are characterized by higher frequency components. As a comparison, regions of smooth or continuous color in an image are only composed of lower frequencies that represent gradual changes. Our goal in this exercise is to detect these edges automatically.

Figure 6.2.1 Example image that has been converted to grayscale. Zoomed inset identifies multiple edges.

6.2.1 Load and Convert Image

(a) First, load the image ‘peppers.png’ (it is built in to Matlab) by using the command: rgb_data = imread('peppers.png');

(b) Convert the image to grayscale using the example script code provided in the appendix.

(c) Plot this grayscale image of the peppers and submit with your report.

6.2.2 DFT High-Pass Filter

(a) Compute the 2D DFT matrix of the grayscale image using the fft2( ) function. To confirm this, open the DFT matrix in the variable window by double clicking the variable name.

(b) Now build an ideal high-pass filter that sets the lowest frequencies in the transform to zero. This can be done by setting to zero all of the frequency components that lie inside a circle of some fixed radius from the corners. The frequency domain representation (DFT) of the filter should look similar to Fig 6.2.2. You may also consider other shapes such as an ellipse or even squares, rectangles. Try them all and see which is best for you.

• Hint: A sample piece of pseudo-code using an if statement looks like the following: if sqrt(((x‐x_0)^2)+((y‐y_0)^2)) <= radius % distance from center 

shiftedDFT(y,x) = 0; % set freq to 0 

  end

(c) Record and submit what circle radius you are using for your filter. Explain the spatial frequencies (in 1/pixels) that correspond to the filtered band.

Figure 6.2.2 Example frequency domain representation of an ideal high-pass filter.

Zeros Zeros

Zeros

Zeros

Ones everywhere else

1 N

M

1

DFT of the high-pass filter

6

6.2.3 Image Edge Reconstruction

(a) After the filter is applied, you can reconstruct the image using the inverse DFT operation with the help of the ifft2( ) function:

ReconstructedImage = abs( ifft2(FilteredDFT) );

• Note: that we are applying abs( ) here to avoid any issues with plotting complex values.

(b) Plot the edge-reconstructed image of peppers.png after filtering and include it with your report, along with your Matlab Code.

• Hint: You may consider using the fftshift( ) function twice to make this operation simpler.

Figure 6.2.3 Example image with highlighted edges.

The reconstructed image (of peppers!) may look like the version of our example image in Figure 6.2.3 when plotted. This depends on the color map and the filtering radius you choose. Note how only the edges get highlighted since we have eliminated the low frequencies!

6.2.4 Image Echoes

(a) If you look closely at the edges in the filtered image, you may see some echoes that look like Figure 6.2.4. Show a zoomed-in example of these echo in your report.

(b) Provide a mathematical explanation of why this effect appears. Hint: Think about if something similar might appear in standard 1D signal filtering operations using an ideal box-shaped filter.

Figure 6.2.4 Zoomed-in area showing (LEFT) artificial echoes and (RIGHT) artifacts removed with gradual filtering

6.2.5 Eliminating Echoes

(a) If you make the filter more gradual (rather than a precise box), you can remove most of the echo artifact identified above.

(b) Propose such a filter either mathematically or graphically. Implement it on peppers.png and show that it works.

(c) Submit both your updated image and any accompanying Matlab code.

(d) Suppose you were asked instead to eliminate the edges from an image. Describe how construction and implementation of the DFT of the corresponding Low-Pass filter might differ.

Echoes

7

7. APPENDIX

7.1 Grayscale Image Script

To load an image and plot it in grayscale, use the following code. Note that Matlab stores images in a (y,x) coordinate system. % Load and plot the image rgb_data = imread('enter_filename_here.jpg'); % You can learn more about the imread() function in Matlab help figure; image(rgb_data); axis off; axis image; % We may need to know the size of the image for later operations [Ysize, Xsize, Cols] = size(rgb_data); % Once again, use the Matlab help to learn more about size() % If you need to access the individual color channels in the image, here’s how: % All values range from 0 to 255 red_data = rgb_data(:,:,1); green_data = rgb_data(:,:,2); blue_data = rgb_data(:,:,3); % Let’s create and plot a grayscale version of the image grayscale_data = rgb2gray(rgb_data); figure; image(grayscale_data); colormap gray(256); axis off; axis image; % You can certainly play around with the colormap function in Matlab. % See the Matlab help files for more information.