Kurento Real Time Media Stream Processing · •WebRTC, RTP, HTTP (video tag), etc. •Standard...

Kurento Real Time Media Stream

Processing

Juan Ángel Fuentes

Software Developer. Stream Oriented GE

jafuentes@naevatec.com

Introducing multimedia infrastructures

1

Peer-to-Peer Application (without media infrastructure)

WebRTC video stream

Application with media infrastructure

Media infrastructure

Multimedia infrastructures for the

Future Internet

2

Media is

hereMedia got

there

Media got

there

Analyse

Transform

Store

Transcoding

MCU

Recording

Enrich

Augment

Adapt

Sensors

Context

Events

Media is

here

Enrich, augment, adapt, analyse, transform,

store: what’s the problem?

3

Complexity

The Stream Oriented Generic Enabler –

SO GE

4

• Interoperable media exchange (multiplatform/multiprotocol)

• WebRTC, RTP, HTTP (video tag), etc.

• Standard capabilities

• Transcoding, MCU, recording

• Advanced capabilities

• Computer vision, augmented reality, mixing, blending, etc.

Multimedia infrastructure

• REST API

• JavaScript API

• Java API

APIs

• LGPL 2.1

Is distributed through a flexible FOSS license

Kurento: The equation

5

Future Internet

Multimedia Infrastructur

e

Simple Developmen

t APIs

5

General Architecture

6

77

Key Concepts: Media Elements

Functional unit performing a specific action on a media stream

Developers abstract from the low level implementation

Able to receive media form other elements

Able to send media to other elements

8

Key Concepts: Types of Media Elements

Input Endpoints: injects media

Filters: Transform or analyze media

Hubs: Manage multiple media flows

Output Endpoints: take the media stream out

9

Key Concepts: Media Pipeline

Media Pipeline: is a chain of media elements, where the output

stream generated by one element (source) is fed into one or more

other elements input streams (sinks)

10

Key concepts: media elements and

pipelines

11

Media Element

• Provides a specific media functionality

› Send/receive media. These are the Endpoints

› Process media

› Transform media

• Ready to be used

• New media elements can be added

Media pipeline

• Chain of media elements implementing the

desired media logic

• The Media Server provides the capability

of creating media pipelines by joining

media elements of the toolbox

Media Element

Sin

k

SR

C

Media Pipeline

1212

Agnostic Media adapter

In charge of making possible that Stream Oriented GE APIS allow

developers to combine media elements to create the desired

pipeline

Fully abstracts all the complexities of media codecs and formats

13

Developers create applications just connecting

elementsApplication 2

Application 3

Application 1

Toolbox of media elements

14

Media Server (KMS): The nucleus of

Kurento

15

•KMS is a middleware for media streams:

•Receives the stream

•Process the stream

• Issues the stream

Send

Receive

Analyze

Augment

Enrich

Transform

Transcode

Record

Process

Replicate

Media

SourceMedia

Sink

KMS

Media plane and Signaling plane

16

How to control the media server?

17

•REST API

•JSON RPC

•Websocket interface in KMS

API Implementations

18

For now…

Multimedia Application Architecture in SO GE

Web application three layers parallelism:

Presentation layer: Multimedia presentation and capturing

Application logic: Multimedia logic in charge of building the pipeline

Service Layer: Kurento Media Server

19

20

Kurento Protocol

JSON-RPC over WS

Media Traffic

Application code

Media Traffic

Client Kurento

Media ServerApplication

Server

Java Kurento

Client

JavaScript

Kurento

Client

Application codeApplication code

Kurento

Protocol

JSON-RPC

over WS

Signaling

protocol

Kurento

Media Server

Client

Kurento Applications

Application execution flow

21

Client

Code

Kurento API Media

Server (KMS)

I want this media in this type

Commands requesting

the creation of a pipeline

(JSON-RPC request)

What you want is here

Pipeline

creation

phase

Media

exchange

phase

1

2

Specific logic

(e.g. AAA)

Media

pipeline

creation

Media exchange between client and server

Campus Party Brazil

22

23

Campus Party Brazil

24

Let’s get started!

25

Tutorials

http://doc-kurento.readthedocs.io/en/stable/tutorials.html

What do we need?

26

An instance of the Kurento Media Server

•FIWARE-LAB

•Local installation

Maven

NPM

Bower

Your favorite IDE

27

Tutorial 1 - Hello world

28

Tutorial 1 - Hello world

Tutorial 1 - Hello world, server

29

Tutorial 1 - Hello world, client

30

31

Tutorial 1 - Hello world (Node.js)

Tutorial 1 - Hello world (Node.js)

32

Tutorial 2 - WebRTC magic mirror

33

Tutorial 2 - WebRTC magic mirror

34

35

Tutorial 2 - WebRTC magic mirror

Tutorial 3 - WebRTC one-to-one video call with

recording and filtering

36

Tutorial 3 - WebRTC one-to-one video call with

recording and filtering

37

Tutorial 3 - WebRTC one-to-one video call with

recording and filtering

38

Kurento and the FIWARE-LAB

39

• FIWARE-LAB

– Working instance of FI-WARE enabling free experimentation with

technology

– http://lab.fiware.org

• Creating a Kurento instance from an image

– Use latest version of Kurento images.

• Creating a Kurento instance using recipes

– Use Ubuntu 14.04 LTS clear image

– Use latest version of Kurento recipes

Kurento and the FIWARE-LAB

40

To learn more…

41

Thank you!

http://fiware.org

Follow @FIWARE on Twitter