Polarization Processing for Low Light Imaging

Abstract:An important property of still picture cameras and video cameras is their ability to present an image that clearly depicts different objects. Normal cameras have the ability to discern between items in bright settings. When ambient light is very dim, pictures can be taken with a flash to illuminate the environment.

However, there are many scenarios where there is little light and the use of a flash is either prohibited or can compromise a position. Additionally, video cameras do not even have the option, as a flash cannot be triggered for every frame. In these situations, normal cameras lack the necessary technology to distinguish one object from another.

The approach this system takes to overcome the constraints of normal cameras is to process polarization information collected by three cameras. By placing three filters oriented at 0, 45, and 90 degrees on top of the cameras, polarization information is collected for each angle, resulting in total polarization knowledge for every pixel.

With the data collected, an overall image is generated through image recombination and various algorithms. This image displays far more information than that rendered by traditional cameras using only the intensity of light. The additional contrast provided results in the user being able to differentiate between two objects in low light situations that were previously indistinguishable.

Authors:Darren Wang (EE ’08)Anujit Shastri (EE ’08)

Advisors: Professor Jan Van der Spiegel

Viktor GruevZheng Yang

Polarization:

z

y

xE

k

Polarization is defined as the direction of the electric field (E) of an electromagnetic wave (k). While it resides in the plane transverse to the direction of the wave’s propagation, it changes over time.

Degree of Polarization (p):If the polarization of a wave is unpredictable, it is said to be randomly polarized, and its degree of polarization is thus zero. Conversely, if a wave’s polarization is completely predictable, it is perfectly polarized, and has a degree of polarization of 1. Partially polarized waves reside in between these two extremes.

Angle of Polarization (θ):The tip of the electric field of polarized waves will trace out an ellipse over time. The angle of polarization is the angle of the major axis of the ellipse with respect to the horizontal.

Demo Day Times:10:30, 11:00, 1:30,

2:00, 2:30(Group 3)

New Frame Started?

Poll FPGA Frame State

No

Enable FPGA SDRAM Read

Enable FPGA SDRAM Write

Grab FPGA Data

Begin Render Thread

Yes

Project Setup:Physical Setup ComputerFPGA

Light enters the system and passes through three lenses. Behind each of these lenses is a polarization filter oriented at 0, 45, or 90 degrees. A sensor, placed behind each filter, will thus record intensities of polarized light. This decomposition allows for per pixel calculation of the polarization parameters. The sensors output 8 bits of data per pixel in parallel format.

The FPGA acts as an intermediary in this system. Its purpose is to transfer image data from the sensors to the computer. It also controls the sensors by providing the clock and programming the registers through a serial interface. Data traverses a series of FIFO’s and is stored in the on- board SDRAM until ready for transfer to the computer.

Once data has been acquired by the FPGA, it signals the computer for collection. The computer program is perpetually scanning for incoming FPGA signals, upon which a separate rendering thread is invoked to visually display the captured sensor information. The separate images are recombined via user matching of similar points between the three images, such that pixels match each other in space. The polarization parameters are then calculated and displayed.

0 90

2 2

45 90

2

/

U

A U U

A U

I II

I I I I I

p I I

45

90

arctan

2

U

U

I I

I I

Usage of three sensors required development of a printed circuit board (PCB) for mounting of sensors and routing of information to the FPGA. PCB design and layout done using the OrCad Cadence suite of tools and fabricated from a third-party production facility.

Coded in Verilog, the logic is the driving force behind the FPGA. Data coming from the sensors is in binary and valid only at certain times. The FPGA stores only valid data in the SDRAM that resides on its board. A series of control signals control the FPGA from the computer.