Upload
miranda-stephens
View
234
Download
0
Tags:
Embed Size (px)
Citation preview
nus.soc.cs5248Ooi Wei Tsang
1
Intermedia Synchronization
Ooi Wei Tsang
2
Overview
• Why we need to sync?• How close must we sync?• How to sync?• How to measure sync?• Applications
3
Synchronization
Intrastream Interstream
Local playback Networked
4
Problem
• Ensure that audio/video captured at the same instance at the source, are displayed at the same instance at the receivers.
nus.soc.cs5248Ooi Wei Tsang
5
Why we need Synchronization?
6
Why not Pre-sync?
V A V A
Sender Receiver
7
Seperating A/V Flows
Sender Receiver
V
A
V
ASender Receiver
• Can receive audio without video• Can prioritize audio over video
8
May Come from Different Sources
• UNLV Access Grid
Sender Receiver
V
A
V
ASender Receiver
9
or Displayed at Different Sources
Sender Receiver
V
ASender Receiver
A
V
10
More Complications
• Different decoding time for video and audio
• Network delay and jitter• Different clocks
11
Bottom Line
• Must synchronize at the receiver
nus.soc.cs5248Ooi Wei Tsang
12
Synchronization Requirements
13
Reference
• Human Perception of Jitter and Media Synchronization
by Ralf Steinmetz IEEE JSAC 14(1) 1996
14
Results “Badness”
A/V offset-80ms +80ms
15
What matters
• Video ahead of audio vs. audio ahead of video
• Size of head• Occupancy
16
What does not matter
• Language• Speaking or Singing• Hobby• Speed
17
Conclusion
• -80ms to +80ms acceptable• -160ms to +240ms noticable
but not annoying
• - audio > video• + video > audio
nus.soc.cs5248Ooi Wei Tsang
18
How to Synchronize?
19
Receiver needs
• Exact time a sample is generated
• Each packet needs a timestamp
20
RTP Protocol
Media TimestampRTPHdr
21
But..
• Media timestamp has a random offset
22
RTCP Protocol
:
Media Timestamp
NTP Timestamp
RTCPHdr
:
23
NTP
• Network Time Protocol• Synchronize clock up to few
milliseconds
24
Sender Audio
Sender Video
Receiver Audio
Receiver Video
Audio RTCP
Video RTCP
Audio RTP
Video RTP
25
Sender Audio
Sender Video
Receiver Audio
Receiver Video
Audio RTCP
Video RTCP
Audio RTP
Video RTP
Synchronization Delay
26
Multiple Receivers
• Everyone must have the same synchronization delay
27
How to set Sync Delay?• Receivers tell each other their
network delay• Synchronization Delay =
max {Network Delays} + Buffer
28
Buffer Control
29
No Delay No Jitter
30
Delay, No Jitter
31
Delay, Jitter
32
If Buffer too Full/Empty• Send message “Play faster!” or
“Play slower!” to other receivers
33
Must Synchronize!
• Send message “Play faster at time T!” or “Play slower at time T!” to other receivers
nus.soc.cs5248Ooi Wei Tsang
34
How do we Measure Synchronization?
35
Reference
• A Method and Apparatus for Measuring Media Synchronization
• by Brian Smhmidt etc.
• NOSSDAV 1995
36
System under Test
Measurement System
Generator
nus.soc.cs5248Ooi Wei Tsang
37
Beyond Video Conferencing
38
HQ Video and Audio
• Multiple Channels• Multiple Displays
39
Network Musical Performance
BOS
BOS
SF
BOS
SF
ACM Multimedia ‘94
40
NASDAQ
• Synchronize display of stock quotes