ECE Department: University of Massachusetts, Amherst

ECE 354

Lab 2: Capturing and Displaying Digital Image

2ECE 354

Big Picture Introduction

Understand the existing SOPC builder setup with new modification

Capturing image using camera Storing the captured image in the frame buffer Display the image on a CRT monitor with the use of a VGA

controller Perform simple image processing Primarily written in C code

3ECE 354

Additional Hardware

TRDB_D5M Digital Camera Serial Connector and

Cable CRT Monitor

4ECE 354

Skills to learn

Integrate code with preexisting code Understanding previously written code Connect to a device that you did not design

5ECE 354

Steps taken to complete project

Begin by looking over camera documentation Understand SOPC builder setup given Write a C program to convert the color image from camera

to black and white C program to transfer data from FIFO to flash memory Gain knowledge of how the DE2_NIOS_HOST_MOUSE_VGA

project displays an image to the CRT monitor Use that knowledge to display your image Finally have fun performing image processing

6ECE 354

Lab2 Setup

[1] TRDB_D5M_Userguide http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGuide.pdf

[1]

7ECE 354

Overview of DE2_NIOS_HOST_MOUSE_VGA project Implements a monochrome display, with a preloaded image,

where the user can draw on it with a mouse USB mouse should be connected USB HOST port CRT monitor should be connected to the VGA port

You will integrate your code so that the project displays the image received from digital camera

8ECE 354

SOPC builder setup: Based on DE2_NIOS_HOST_MOUSE_VGA The program code is stored in SRAM memory Parallel Input/output interface(PIO) for LED, switches Camera Avalon Interface(Camera_IF) helps in transferring

image data The dedicated SDRAM is used as a frame buffer that holds

a single frame of video at the time. The SDRAM is controlled and accessed by using the four-

port controller hardware Controller can be used to read or write to the SDRAM

through its four FIFO buffers. Flash controller is used to control the data transfer to

flash memory VGA controller displaying data on to VGA

9ECE 354

Block diagram of Terasic camera system

TRDB_D5M_Userguide http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGuide.pdf

10ECE 354

Hardware Modules for this Lab

SDRAM-Multiport Controller:This module is a generic 4 port SDRAM controller. It uses two FIFO buffers for inputs (writing) and

two FIFO buffers for outputs. The FIFO buffers are made using Altera MegaFunctions and are 16 bit wide and can hold up to 512 words of this size. In this system the SDRAM controller is used as a frame buffer to hold a single picture frame

Flash Controller VGA controller Camera Avalon Interface module

11ECE 354

To get started with the lab Run the DE2_NOIS_MOUSE_VGA_project present in the

project folder given to you Download the project content files put up online under lab2 Run SOPC builder and understand each of the components

added and configuration present Hardware component files are present under IP folder

• It contains SDRAM_4 port controller components• Camera Avalon Interface

Software components are present under the software folder• hello_led_0 has all the required software component files• Some part of code is given in this project which is present in file

hello_led.c

12ECE 354

Information for C program C code to capture camera data and store it in frame buffer Picture taken at 640x480 has 3 data per pixel times 8 bits

(at least) each, this would mean a total of 7.37 Mbit or 921.6 Kbyte

So efficient storage design required Recommended C code function

• Camera Capture• Copying RGB(pixels) data from FIFO to Flash memory• Using flash memory to store the pixel data before

transferring to VGA output Program space memory is less (SRAM).Avoid unnecessary

buffers. Write programs as efficient as possible

13ECE 354

Displaying image through VGA controller

Look through VGA.h for VGA functions

Write C code similar to that shown below that calls functions from VGA.h to display each pixel

14ECE 354

Reading pixels from Flash Memory(hello_led.c) The methods

called inside this file are NIOS II APIs.

Use this skeleton to develop your code.

15ECE 354

Function.c

This file needs to be updated with your code.

Various skeleton functons are defined. You need to modify/extend these functions.

16ECE 354

Additional information

The image should be 1-bit per pixel with a resolution of 640x480

Uncompressed

17ECE 354

Image Processing

It is required that you implement two forms of image processing (out of 5)

Each bullet below correspond to one image processing technique you need to show on the VGA

Recommendations include:• Add timestamp onto image• Counter to keep track of number of pictures taken• Rotate, mirror, invert image• Simple edge detection (challenging)• Detect changes in images (challenging)

18ECE 354

References for software and hardware design in Lab2

I recommend looking over section II of the NIOS II Software Developer’s Handbook

Go through project titled- Embedded Demonstrator for Video Presentation and Manipulation by Cato Marwell Jonasse (Google it) to get clear insight on hardware description for this project

(You can go through chapter 9 specifically)

DE2_NIOS_HOST_MOUSE_VGA project

NIOS II Software Developer’s Handbook• http://www.altera.com/literature/lit-nio2.jsp

19ECE 354

Where does this project lead?

Next lab will be on the topic sending data over a network• Sending image between DE2 boards• Don’t worry if you have not taken computer networking

20ECE 354

Questions and Comments

21ECE 354

Back up

22ECE 354

Readout modes: D5M camera

The Terasic D5M camera supports a function called :

1. binning -This function reduces the resolution of the image by averaging pixels together

2. Skipping - reduces the output resolution without affecting the field-of-view. It does this by not sampling entire rows and columns of pixels. A skip 2X mode skips one of pixels for every pair of output.