3/9/2007EECS150 Lab Lecture #81 Wireless Video Conferencing EECS150 Spring 2007 - Lab Lecture #10 Neil Warren Allen Lee

3/9/2007 EECS150 Lab Lecture #8 1

Wireless Video Conferencing

EECS150 Spring 2007 - Lab Lecture #10

Neil WarrenAllen Lee


Checkpoint #3 Information(1)

Check off Requirement: #1) Be able to set 2 sets different source

and destination addresses on 2 boards and communicate. (CP4 video streaming)

#2) Be able to set both sides’ destination address to 0xFF and a unique source address on each and have communications occur. (CP4 Handshake)


Checkpoint #3 Information

If you’re behind, you can forfeit checkpoint #3 and receive a black box. You will receive a 0 for CP3 but will be

able to move on to CP4 which is worth many more points.

Talk to Neil if you want this option. You lose the points before you get the BB.


Welcome to Checkpoint 4

Combine CP2 and CP3 to make two-way wireless video conferencing system

Note: Specification has changed to make your life much easier – based on questions from students so far. Make sure you work with the non-draft version of the Checkpoint4.doc


Overview

Revisit CP2 and CP3 Communications protocol Video compression Picture-in-picture display


Recall from Checkpoint 2


Checkpoint 4 Datapath


Our Advice (1)

Separate the work between you and your partner to make the best use of your time. Communications System Graphics System

Agree on an interface and make sure you conform to it.


Our Advice (2) Debug Independently!

Communications System: Assume that the graphics system works perfectly

(conforms to your interface). Hardcode in data and requests using timers and

debug the system by using the packet sniffer. Graphics System:

Assume that the communications system works perfectly.

Hardcode transmission inputs, and whatever interface method you use for send/receive acknowledgements.

Much easier to debug each piece, then just put the 2 working pieces together. If you conform to your interface it should work!


Communications Module (1)

Responsibilities Maintain communications link.

Lost packet robustness Provide cross-compatibility

Conform to communications protocol Encode and Decode packets and pass

data to graphics engine Handle the initial handshake between

master and slave.


Communications Module (2) Recommended Pieces:

State Machine Keeps track of the current state of the communications

module. Handles handshaking and properly re-synchronizing the

communications link after lost packet. Sends commands to Packet Encoder, receives updates

from Packet Decoder Packet Encoder

Generate payload data for each type of packet and handle handshake with transceiver.

Take commands from state machine. Packet Decoder

Passes data to the graphics engine and gives status updates to the state machine.


Communications Module (3) Initial Handshaking

Purpose: To allow any 2 boxes without knowledge of the

other’s address to communicate. To set up the timing of master and slave send and

receives.

Note: At the beginning both the master and slave begin with their destination addresses being 0xFF and their source addresses being unique


Communications Module (4) Handshake Method:

Master broadcasts “Master-Seek” packets once every 2.4ms. When a slave receives a “Master-Seek” packet it sends a “Slave-ReqAck”

When the master receives the “Slave-ReqAck” packet it changes its destination address to the source address of the slave and then transmits a “Master-Ready” packet with the new destination address.

When the slave receives a “Master-Ready” packet the slave sends a “Slave-Ready” packet and moves into a state where it can receive active video packets from the master. The “Slave-Ready” packet should be sent with its destination address as the source of the address of the master.

When the master receives the “Slave-Ready” ready packet, it sends its first packet of active video data.

Now the communication should alternate between the master and slave transmitting active video data.


Communications Module (5) Communications Protocol

Purpose: To allow cross-compatibility between different

implementations. Abstracts away the communications aspect of the

project for the graphics module. The 8-bit header designates the type of

packet. The 8-bit line number indicates which line the

respective active video data corresponds to. The 80-bit data contains the AV data for 32

pixels. (16 pixel pairs) 8 bit headers and line numbers make it

easy to view on the packet sniffer!


Communications Module (6)Type 8-Bit

Header8-Bit Line Number 80-Bit Data Payload

Active Video 1 8’d1 8’b{Xmit Line} SAMPLE # 0-31





Active Video 6 8’d6 8’b{Xmit Line} SAMPLE # 160-176*

Master-Seek 8’d23 XX 80’h2

Slave-ReqAck 8’d24 XX 80’h1

Master-Ready 8’d25 XX 80’h2

Slave-Ready 8’d26 XX 80’h1


Communications Module (7) Time Outs – let you know if a communication loss has

occurred. After the expiration of 10ms from the last AV packet

transmission, if the slave not does receive a packet it should return to a state where it can receive AV packets from the master.

If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred.

After the expiration of 10ms from the last AV packet received, if the master does not receive a packet it should send its next AV packet.

If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred.


Communications Module (8)

Tips: Make use of the packet sniffer to

determine what is *actually* flying through the air.

The communications protocol was re-designed specifically for this purpose.


Video Compression (1)

Wireless transfer is the bottleneck Uncompressed video:

507x720x16 = 5,840,640 bits per frame

That’s ~175 Mbits per second! Theoretical max bandwidth for

wireless is 256kbps, so ~22.8 seconds per frame.



Simple lossy compression algorithm

Resolution: 176x127 5 bits grayscale per pixel pair



Compressed Video: 176x127x5/2 = 55,880 bits per frame Approximately 105:1 compression At theoretical max wireless

bandwidth, that’s 4.5 frames per second


Video Compression (4) Vertical compression

Sample every other odd active line (254 active odd lines)/2 = 127 lines

Horizontal compression Sample every fourth pixel pair (except

for the last eight) (360-8)/4 = 88 pixel pairs = 176 pixels For each sampled pixel pair, keep only

the upper five bits of the average luminance



Why 176x127 instead of 180x127? Good question. There’s a good

answer, figure it out!


Video Decompression

Reverse of compression, but lossy due to lack of chrominance

5-bit grayscale represents a pixel pair Decompresses to 32 bits, with

appropriate padding for chrominance values


Writing Compressed Frames

Set aside an area in SDRAM for sending and receiving compressed frames

Update the stored frame to send only after the entire frame has been sent

It is okay to lose a local frame to store a compressed one


Writing Compressed Frames Sequence for handling compressed

frames: Wait for SDRAM and ChipCon initialization Store a compressed frame Tell transmitter FSM that frame in SDRAM is

ready Wait as compressed frame is sent Receive signal from transmitter that last

packet for frame has been sent Once entire compressed frame is sent, store

a new compressed frame


Writing Compressed Frames Two options for implementing compressed

video controller: Merge with VD processor, using the same FIFO

Advantage: Arbiter sees one write port for local video Drawback: Need to cross clock boundaries

Separate module with own FIFO Advantage: May not need to cross clock boundaries Drawback: Arbiter sees two write ports for local video

Pick the one that makes more sense to you


Dual-Clock Communication Modules running on the camera clock and

the system clock cannot safely communicate with direct control signals.

If your modules need to communicate but are on different clocks, you must either implement a safe handshaking protocol or use the memory element we provide.

For your convenience, we’ve included a dual-clocked one-bit-wide FIFO to safely exchange control signals, but you don’t have to use it.


Sender and Receiver FSMs Sender/Transmitter

Waits for frame to be available in SDRAM

Reads from SDRAM and packages data into a packet

Requests a new frame once last packet for frame is sent

Receiver Decodes the compressed data Writes received data into SDRAM


Displaying Compressed Video


Displaying Compressed Video

Constantly display compressed frames to avoid SDRAM decay

This way, you don’t have to auto-refresh!


Multi-Port Arbiter

At least two more ports are needed for the sender and receiver FSMs.

A fifth port may be needed depending on how you handle writing compressed frames.


Tips, Hints, & Common Pitfalls

Start early! If you’re already done with CP3, start

now! We are not providing any Verilog

besides the optional one-bit-wide memory element.

Run the TA solution to see what the final result should look like.



Split up the project into graphics and communications. These parts are fairly independent so

you and your partner can work in parallel.

Do not combine until you’ve verified each part works independently.

It can take up to 25 minutes to push to board after combining!


Tips, Hints, & Common Pitfalls Work Incrementally

Display a sectioned screen Display only compressed video at full frame

rate Simultaneously display local compressed

and uncompressed video at full frame rate Display uncompressed video at full frame

rate and local compressed at 1 FPS Test sender and receiver FSMs by sending

local “packets” to yourself



Be careful how you handle the half-full packet (packet 6): Make sure the receiver knows which

half of the payload to look at for the video data



2 Clocks with phase offset: Camera Clock != Normal Clock Use the 1-bit wide dual-clocked fifo

we provide to cross clock boundaries.


Requirements Must be TA Solution Compatible.

EC can be non-compatible, but you must still develop a compatible solution first.

1 bit file. Both master and slave systems should contained on 1 bit file.

Onscreen prompts. Should display to the user the communications status (see TA solution for more info)

Some method of outputting the (max) time between new frames.


Extra Credit Maximum 20% Early check off 10% Some Ideas (EC% TBD):

Faster frame rate Larger packet sizes Faster transmission Non-full duplex communications

UI options Menus Video output modifications (change contrast with

buttons, etc.) Use N64 controller for extra buttons to add functionality

We will provide a black box to interface Open-ended! Come up with something special,

cool, clever, unique, and/or challenging.


Questions?

Documents

3/9/2007EECS150 Lab Lecture #81 Wireless Video Conferencing EECS150 Spring 2007 - Lab Lecture #10 Neil Warren Allen Lee