1

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

How to synthesize temporal visual signals as transient input traffic for discrete event simulation?

Mauritz Panggabean and Leif Arne RønningenDepartment of Telematics, NTNU

3rd International Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) 2011Budapest, Hungary, 5-7 October 2011

Synthesizing Transient Traffic of Temporal Visual Signals for Discrete Event Simulation

2

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Outline

• Introduction– Background and vision

– Framework: the big picture

– Why transient traffic?

• Preparation and experiment– Type of input: the scenes

– Processing the clips

– Modeling using linear and power functions

• Results: synthetic transient traffic using power function

• Conclusion and the next work

3

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Background and vision

• Futuristic tele-immersive environment system for real-time artistic collaborations with near-natural quality– Arrays of auto-stereoscopic 3D displays and high-end cameras on all

surfaces of the collaboration space

– More than one performers in a collaboration space

– Multiview with view activated by the gaze of the performers

• Challenge: modeling such environment in discrete event simulation prior to construction– Simulation for feasibility study with (un)compressed input data

• First step: a model for synthetic transient input traffic from human motion and camera modes

Begin with the end in mind.Stephen Covey

4

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Framework: the big picture

If everyone is thinking alike, then somebody isn't thinking.George S. Patton

Synthesis of multiobject

human motion

Uncompressed visual traffic: Multiview

projected body-areas based on eye gazes

Incorporating effects from

image compression

Compressed visual traffic: Multiview projected body-areas based on eye gazes

Discrete-event simulation of complex multiview multiobject distributed tele-

immersive collaboration system

Synthesis of eye gazes

Humanoid model

Stochastic and deterministic human motion

Formal model

Synthetic multiobject human

body motions

Synthetic eye gazes

Computation of projected areas based on eye gazes

Rate-distortion curves of image compression

5

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Why transient traffic?

• Stationary traffic nullified by interframe compression

• Important: transient slopes, variations, extreme values and duration (not in mean values)

• Piecewise analysis: a general formal model of a few parameters to synthesize each piece as input traffic (GOAL)

Have you got a problem? Do what you can where you are with what you've got.Theodore Roosevelt

6

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Types of input: the scenes

• Only one static camera, one view and one object• Three scenes as input video clips:

1. Clip PANNING• Object enters the scene from left and disappears on the right side• Equivalent to camera panning

2. Clip ZOOM• Object gradually moves closer to the camera • Equivalent to camera zoom

3. Clip MOTION• The object performs some motions with the limbs at the center of the

scene

Everything should be made as simple as possible, but not simpler.Albert Einstein

7

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Processing the clips

It is an acknowledged truth in philosophy that a just theory will always be confirmed by experiment. Thomas Malthus

TranscodingDe-interlacing

Original color clips (1920x1080, 30Hz)

AVI color clips (1280x720, 30Hz)

Objectsegmentation

Uncompressed input traffic

- Off-the-shelf camera- Interlaced, uncompressed- One person- Uniform color background

- Matlab- PC (2.99GHz, 8.0 GB RAM)

- Object only (the person)- 8 bits per pixel- RGB channels

8

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Input traffic from the clips

No amount of experimentation can ever prove me right; a single experiment can prove me wrong. Albert Einstein

20 25 30 35 40 45

1 50 100 150 200 250

1 25 55 118 127 145

9

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Modelling as power function

In much of society, research means to investigate something you do not know or understand. Neil Armstrong

Power function f(x) = axb as the general model for synthesis

a = 1.108, b = 0.8518RMSE = 0.09655

c = 1.116, d = 0.03048RMSE = 0.104

Power function f(x) = axb and linear function f(x) = cx + d with normalized frame number

a = 0.9533, b = 1.339RMSE = 0.03234

c = 0.9854, d = -0.08577RMSE = 0.04202

a = 1.134, b = 0.8762RMSE = 0.1677

c = 1.145, d = 0.02072RMSE = 0.172

10

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Research is to see what everybody else has seen, and to think what nobody else has thought.Albert Szent-Gyorgyi

Power function for transient input traffic synthesis in simulation• Transient traffic with increasing trend

f(x) = axb + d + e• Transient traffic with decreasing trend

f(x) = a(1-xb) + d + e

• Parameters:– b, to control the curve bending: 0.5 < b < 1

– a = Dmax – Dmin, d = Dmin: Dmin and Dmax as the min/max data

– e, to control the random smoothness of the curve at point x.

emin < e < emax is uniformly distributed, emin ≤ emax and e < a.

– Frame rate F and simulation time S (sec): t = 1/F and x = t/S

11

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Synthetic transient input traffic

An experiment disproving a prediction is a discovery. -Enrico Fermi

Panninga = 4b = 0.65c = 0F = 19

Zooma = 12b = 1.34c = 8F = 230

MotionIncreasing parta = 3, b = 0.75, c = 7F = 9

Decreasing parta = 2.5, b = 2.5, c = 7.5F = 11

For allemin = 0 emax = 0.5S = 1

Adjacent pieces of arbitrary transient traffic connected by setting their values of a and c

12

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Conclusion and the next work

Research is what I'm doing when I don't know what I'm doing. -Wernher von Braun

• Power function as a general formal model for transient traffic synthesis as input for discrete event simulation

• Covers human motion, camera panning and zoom for one view

Synthesis of multiobject

human motion

Uncompressed visual traffic: Multiview

projected body-areas based on eye gazes

Incorporating effects from

image compression

Compressed visual traffic: Multiview projected body-areas based on eye gazes

Discrete-event simulation of complex multiview multiobject distributed tele-

immersive collaboration system

Synthesis of eye gazes

Humanoid model

Stochastic and deterministic human motion

Formal model

Synthetic multiobject human

body motions

Synthetic eye gazes

Computation of projected areas based on eye gazes

Rate-distortion curves of image compression

13

3rd Int’l Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) | Budapest, Hungary | Oct 5-7, 2011

Thank you

The real voyage of discovery consistsnot in seeking new landscapes,but in having new eyes.- Marcel Proust