Project: Python Video Conferencing Client - pyVCC

Project: Python Video Conferencing Client - pyVCC

Adrian Beutner1 andChristoph Kuhr2

Summer Semester 2013

[email protected]@web.de

Abstract

In this document, the concept and design of the Python Video Conferencing Client (pyVCC)and it’s development process is described. PyVCC utilizes the Session Initiation and Descrip-tion Protocols (SIP, SDP) to establish a session between two clients and a call server (CS).The two pyVCC clients share their audio and video (AV) data with the Realtime TransportProtocol (RTP).

Python Video Conferencing Client Content Beutner - Kuhr

Contents

1 Introduction 21.1 Workpackages and Schedueling . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Project Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Concept and Design 32.1 Communication Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 SIP and SDP - Kuhr . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.1.2 RTSP and SDP- Kuhr . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.3 RTP - Beutner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Callserver and MCU - Beutner . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Software Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3.1 libVLC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3.2 SIPSimple Client SDK . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3.3 UML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.4 Proof of Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.4.1 Virtualbox - Firewall - Beutner . . . . . . . . . . . . . . . . . . . . . . 152.4.2 Callserver and MCU - Beutner . . . . . . . . . . . . . . . . . . . . . . 152.4.3 libVLC RTSP Streaming . . . . . . . . . . . . . . . . . . . . . . . . . 152.4.4 SIP Simple SDK - Blink . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3 Discussion 213.1 Callserver and MCU - Beutner . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.1.1 OpenVCS and Asterisk . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.2 Medooze . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.2 pyVLCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.1 libVLC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.2 SIP Simple SDK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.3 RTSP SDP vs SIP SDP . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.4 Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

pyVCC i

Python Video Conferencing Client Content Beutner - Kuhr

List of Figures

2.1 SIP Trapezoid [5, p. 4] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 SDP Body Scheme [4, p. 30] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 TEMP SIP Body Fields [4, p. 30] . . . . . . . . . . . . . . . . . . . . . . . . . 42.4 TEMP Field Values [4, p. 31] . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.5 RTP Wireshark Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.6 Use Case Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.7 Activity Diagram pyVCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.8 Component Diagram pyVCC . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.9 OpenVCS Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.10 pyVLCC GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.11 Blink GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.12 SIP Signaling Wireshark Capture . . . . . . . . . . . . . . . . . . . . . . . . . 233.13 RTSP Signaling Wireshark Capture . . . . . . . . . . . . . . . . . . . . . . . 233.14 SIP SDP Wireshark Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.15 RTSP SDP Wireshark Capture . . . . . . . . . . . . . . . . . . . . . . . . . . 24

List of Tables

3.1 This table shows some data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

pyVCC 1

Python Video Conferencing Client Introduction Beutner - Kuhr

1 Introduction

1.1 Workpackages and Schedueling

A list of single worksteps to perform:

MS2 04/18/2013 Concept and Design:

• Understand RTP (Beutner)

• Understand SIP, RTSP, SDP, libVLC , SIPSimple Client SDK (Kuhr)

• Develope communication concept (Beutner)

• Design software (Kuhr)

MS3 06/27/2013 Implementation and Verification:

• Install call server(Beutner)

• Implement software (Kuhr)

• Install media control unit (Beutner)

• Test software (Beutner, Kuhr)

MS4 07/11/2013 Conclusions and Presentation:

• Conclusions (Beutner, Kuhr)

• Presentation (Beutner, Kuhr)

pyVCC 2


1.2 Project Description

PyVCC is a video conferencing client written with the Python programming language. Theapplication script pyVCC.py is executed within the Python interpreter CLI. Thus, the inter-action and debugging iniformation is printed to the Python CLI. The required functionalityfor SIP sessions management shall be implemented with the SIPSimple Client SDK [?]. TheSIPSimple Client SDK provides an interface for external codecs, called IMediaStream. TheRTP AV data transport is implemented with the libVLC library and is called via the IMedi-aStream API by the SIP components. libVLC capture the local AV data, streams it to theremote client and plays back the AV data stream received by the remote client.With the Session Initiation and Description Protocols (SIP, SDP), two pyVCC Clients es-tablish a session, managed by a Call Server (CS). The Call Server shall be realized with theOpenSource Application openVCS. OpenVCS provides SIP functionaity along with a MCU(Multipoint Control Unit). The MCU manages RTP AV streams encoded with the h264 codecfor video and the G711 codec for audio.

2 Concept and Design

2.1 Communication Concept

2.1.1 SIP and SDP - Kuhr

SIP describes two types of communication, the most common way with the so called SIPtrapezoid and communication which is forwarded to a Multipoint Control Unit (MCU).In the most commonly used case, Voice over IP, the user clients send and receive their SIPmessages to and from their next SIP Proxy. The SIP Proxy forwards the SIP messages tothe appropriate remote SIP Proxy to reach the remote URI. After the session is initializedwith the SDP parameters, the audio data ist transported, via RTP, to the remote user clientbased on a peer-to-peer connection on different ports.

Figure 2.1: SIP Trapezoid [5, p. 4]

The second case is used for video conferencing environments, where different RTP AVCProfile streams, the amount depends on the users registered to the processed session, are

pyVCC 3


mixed to one RTP AVC Profile stream. The mixed stream is then return to each conferenceparticipant. This concept is described in detail in chapter 2.2 Callserver and MCU.

The SDP part of a SIP INVITE message containes different lines of parameters to nego-tiate a RTP AVC Stream. Figure 2.2 shows the rule, by which each line of the SDP body isbuild. In figure 2.3, an example for a SDP body is given.

Figure 2.2: SDP Body Scheme [4, p. 30]

Figure 2.3: TEMP SIP Body Fields [4, p. 30]

pyVCC 4


The fields shown in figure 2.3 have the following meaning:

Figure 2.4: TEMP Field Values [4, p. 31]

2.1.2 RTSP and SDP- Kuhr

A RTSP stream uses HTTP like signaling to negotiate session parameters as well as SIP. Butthe signaling with RTSP is different.The mechanism to start a session uses a different data transport for signaling and mediatransmission, like with SIP signaling.For RTSP port 8554 is used over TCP and the ports 9000 and 9001 are used for RTP audioand video transport over UDP.

Mandatory to start a presentation stream, which means a combination of for exampleaudio and video streams, is a presentation description file. This presentation description fileis provided as a SDP file. It has to be communicated, over a seperate channel like email or awebpage, before the actual start of the session.

Once the servers presentation file is known, the requesting user client sends an OPTIONrequest to the server. It contains information about the requested presentation stream by thepreshared SDP file and the used user client.

The server responses with a 200 OK message. In this message the server gives informationabout the server application that is providing the presentation stream. Also a field namedPUBLIC is contained in the response. Th field PUBLIC is a list of the operations, which areimplemented by the server.

The most important operations, taken from the RTSP RFC2326 [7, p. 11f], are listed below:

pyVCC 5


SETUP:Causes the server to allocate resources for a stream and start an RTSP session.

PLAY and RECORD:Starts data transmission on a stream allocated via SETUP.

PAUSE:Temporarily halts a stream without freeing server resources.

TEARDOWN:Frees resources associated with the stream. The RTSP session ceases to exist on theserver.

RTSP methods that contribute to state use the Session header field (Section 12.37)to identify the RTSP session whose state is being manipulated. The server generatessession identifiers in response to SETUP requests (Section 10.4).

The most important RTSP messages and their responses are listed below:

DESCRIBE:The DESCRIBE method retrieves the description of a presentation or media objectidentified by the request URL from a server. It may use the Accept header to specify thedescription formats that the client understands. The server responds with a descriptionof the requested resource. The DESCRIBE reply-response pair constitutes the mediainitialization phase of RTSP.

Example:C->S: DESCRIBE rtsp://server.example.com/fizzle/foo RTSP/1.0

CSeq: 312

Accept: application/sdp, application/rtsl, application/mheg

RESPONSE with SDPS->C: RTSP/1.0 200 OK

CSeq: 312

Date: 23 Jan 1997 15:35:06 GMT

Content-Type: application/sdp

Content-Length: 376

v=0

o=mhandley 2890844526 2890842807 IN IP4 126.16.64.4

s=SDP Seminar

i=A Seminar on the session description protocol

u=http://www.cs.ucl.ac.uk/staff/M.Handley/sdp.03.ps

[email protected] (Mark Handley)

pyVCC 6


c=IN IP4 224.2.17.12/127

t=2873397496 2873404696

a=recvonly

m=audio 3456 RTP/AVP 0

m=video 2232 RTP/AVP 31

m=whiteboard 32416 UDP WB

a=orient:portrait

The DESCRIBE response MUST contain all media initialization information for theresource(s) that it describes. If a media client obtains a presentation description froma source other than DESCRIBE and that description contains a complete set of mediainitialization parameters, the client SHOULD use those parameters and not then requesta description for the same media via RTSP.

SETUP:The SETUP request for a URI specifies the transport mechanism to be used for thestreamed media. A client can issue a SETUP request for a stream that is already playingto change transport parameters, which a server MAY allow. If it does not allow this,it MUST respond with error ”455 Method Not Valid In This State”. For the benefitof any intervening firewalls, a client must indicate the transport parameters even if ithas no influence over these parameters, for example, where the server advertises a fixedmulticast address.

Since SETUP includes all transport initialization information, firewalls and other inter-mediate network devices (which need this information) are spared the more arduous taskof parsing the DESCRIBE response, which has been reserved for media initialization.

The Transport header specifies the transport parameters acceptable to the client fordata transmission; the response will contain the transport parameters selected by theserver.

C->S: SETUP rtsp://example.com/foo/bar/baz.rm RTSP/1.0

CSeq: 302

Transport: RTP/AVP;unicast;clientport = 4588 − 4589

S->C: RTSP/1.0 200 OK

CSeq: 302

Date: 23 Jan 1997 15:35:06 GMT

Session: 47112344

Transport: RTP/AVP;unicast;

clientport = 4588 − 4589; serverport = 6256 − 6257

The server generates session identifiers in response to SETUP requests. If a SETUPrequest to a server includes a session identifier, the server MUST bundle this setuprequest into the existing session or return error ”459 Aggregate Operation Not Allowed”(see Section 11.3.10).

PLAY:The PLAY method tells the server to start sending data via the mechanism specified

pyVCC 7


in SETUP. A client MUST NOT issue a PLAY request until any outstanding SETUPrequests have been acknowledged as successful. The PLAY request positions the normalplay time to the beginning of the range specified and delivers stream data until the endof the range is reached. PLAY requests may be pipelined (queued); a server MUSTqueue PLAY requests to be executed in order. That is, a PLAY request arriving while aprevious PLAY request is still active is delayed until the first has been completed. Thisallows precise editing. For example, regardless of how closely spaced the two PLAYrequests in the example below arrive, the server will first play seconds 10 through 15,then, immediately following, seconds 20 to 25, and finally seconds 30 through the end.

C->S: PLAY rtsp://audio.example.com/audio RTSP/1.0

CSeq: 835

Session: 12345678

Range: npt=10-15


CSeq: 836

Session: 12345678

Range: npt=20-25


CSeq: 837

Session: 12345678

Range: npt=30-

See the description of the PAUSE request for further examples. A PLAY request withouta Range header is legal. It starts playing a stream from the beginning unless the streamhas been paused. If a stream has been paused via PAUSE, stream delivery resumes atthe pause point. If a stream is playing, such a PLAY request causes no further actionand can be used by the client to test server liveness. The Range header may also containa time parameter. This parameter specifies a time in UTC at which the playback shouldstart. If the message is received after the specified time, playback is started immediately.The time parameter may be used to aid in synchronization of streams obtained fromdifferent sources. For a on-demand stream, the server replies with the actual rangethat will be played back. This may differ from the requested range if alignment of therequested range to valid frame boundaries is required for the media source. If no rangeis specified in the request, the current position is returned in the reply. The unit of therange in the reply is the same as that in the request. After playing the desired range,the presentation is automatically paused, as if a PAUSE request had been issued. Thefollowing example plays the whole presentation starting at SMPTE time code 0:10:20until the end of the clip. The playback is to start at 15:36 on 23 Jan 1997.

C->S: PLAY rtsp://audio.example.com/twister.en RTSP/1.0

CSeq: 833

Session: 12345678

Range: smpte=0:10:20-;time=19970123T153600Z

pyVCC 8


S->C: RTSP/1.0 200 OK

CSeq: 833

Date: 23 Jan 1997 15:35:06 GMT

Range: smpte=0:10:22-;time=19970123T153600Z

For playing back a recording of a live presentation, it may be desirable to use clock units:

C->S: PLAY rtsp://audio.example.com/meeting.en RTSP/1.0

CSeq: 835

Session: 12345678

Range: clock=19961108T142300Z-19961108T143520Z

S->C: RTSP/1.0 200 OK

CSeq: 835

Date: 23 Jan 1997 15:35:06 GMT

A media server only supporting playback MUST support the npt format and MAY sup-port the clock and smpte formats.

PAUSE:The PAUSE request causes the stream delivery to be interrupted (halted) temporarily.If the request URL names a stream, only playback and recording of that stream ishalted. For example, for audio, this is equivalent to muting. If the request URL namesa presentation or group of streams, delivery of all currently active streams within thepresentation or group is halted. After resuming playback or recording, synchronizationof the tracks MUST be maintained. Any server resources are kept, though servers MAYclose the session and free resources after being paused for the duration specified withthe timeout parameter of the Session header in the SETUP message.

Example: C->S: PAUSE rtsp://example.com/fizzle/foo RTSP/1.0

CSeq: 834

Session: 12345678

S->C: RTSP/1.0 200 OK

CSeq: 834

Date: 23 Jan 1997 15:35:06 GMT

The PAUSE request may contain a Range header specifying when the stream or pre-sentation is to be halted. We refer to this point as the ”pause point”. The header mustcontain exactly one value rather than a time range. The normal play time for the streamis set to the pause point. The pause request becomes effective the first time the serveris encountering the time point specified in any of the currently pending PLAY requests.If the Range header specifies a time outside any currently pending PLAY requests, theerror ”457 Invalid Range” is returned. If a media unit (such as an audio or video frame)starts presentation at exactly the pause point, it is not played or recorded. If the Rangeheader is missing, stream delivery is interrupted immediately on receipt of the message

pyVCC 9


and the pause point is set to the current normal play time. A PAUSE request dis-cards all queued PLAY requests. However, the pause point in the media stream MUSTbe maintained. A subsequent PLAY request without Range header resumes from thepause point. For example, if the server has play requests for ranges 10 to 15 and 20 to29 pending and then receives a pause request for NPT 21, it would start playing thesecond range and stop at NPT 21. If the pause request is for NPT 12 and the server isplaying at NPT 13 serving the first play request, the server stops immediately. If thepause request is for NPT 16, the server stops after completing the first play request anddiscards the second play request. As another example, if a server has received requeststo play ranges 10 to 15 and then 13 to 20 (that is, overlapping ranges), the PAUSErequest for NPT=14 would take effect while the server plays the first range, with thesecond PLAY request effectively being ignored, assuming the PAUSE request arrivesbefore the server has started playing the second, overlapping range. Regardless of whenthe PAUSE request arrives, it sets the NPT to 14. If the server has already sent databeyond the time specified in the Range header, a PLAY would still resume at that pointin time, as it is assumed that the client has discarded data after that point. This ensurescontinuous pause/play cycling without gaps.

TEARDOWN:The TEARDOWN request stops the stream delivery for the given URI, freeing theresources associated with it. If the URI is the presentation URI for this presentation,any RTSP session identifier associated with the session is no longer valid. Unless alltransport parameters are defined by the session description, a SETUP request has tobe issued before the session can be played again.

Example: C->S: TEARDOWN rtsp://example.com/fizzle/foo RTSP/1.0

CSeq: 892

Session: 12345678

S->C: RTSP/1.0 200 OK

CSeq: 892

2.1.3 RTP - Beutner

The Real-Time Transport Protocol is a widely used Protocol to carry media data. It worksUDP-based. Poorly described, RTP completes UDP-datagrams with sequence-numbers, times-tamps and unique synchronization source identifiers to allow end to end transmission: Bychecking the sequence numbers, RTP notices if there was packet loss. It can synchronizeAudio- and Video-data by using the timestamps as well. In addition, there is informationabout the Payload Type, for example which codecs are used.RTP is used in conjuncion with RTCP:

2.2 Callserver and MCU - Beutner

To allow videoconferencing you need user agents, a call server and a multipoint control unit.

pyVCC 10


img/rtp_cap.png

Figure 2.5: RTP Wireshark Capture

2.3 Software Design

2.3.1 libVLC

The libVLC library fully covers the functionality of the VLC player. It provides access to thecomplete internal VLC architecture. This way an application for capturing and displayingAV data, can rely on the processing of libVLC. libVLC is a C library with a complete PythonAPI to it.

Installation:sudo apt-get libvlc libvlccore5

pyVCC 11


2.3.2 SIPSimple Client SDK

The SIPSimple Client SDK is a SIP stack completely written in Python. It is an open sourceproject which is funded by the European Union. SIPSimple completey handles a SIP sessions.Simple point to point session are possible, as well as conferences with multiple participants.Since AV data is handled by a seperate interface, it is possible to add custom AV data trans-port functionality to it.

Installation:

Add the following repository:deb http://ag-projects.com/ubuntu precise main

deb-src http://ag-projects.com/ubuntu precise main

sudo apt-get update

sudo apt-get install python-sipsimple libgcrypt11

pyVCC 12


2.3.3 UML

Figure 2.6: Use Case Diagram

Use Cases:

UC10: Call UserUser 1 calls User 2 by entering the URI of the remote User. PyVCC uses the SIPsignaling prtocol to establish a session between user agent 1 and user agent 2.

UC20: Incoming Call from UserUser 2 receives a call by User 1 and acknowledges it, so that a SIP session is established.

UC30: RTP AV Data TransportIf a SIP session is established, the AV data shall be transported between the two useragents with the RTP using the codecs h264 for video and G711 for audio data.

UC40: Configuration, Maintance

pyVCC 13


Figure 2.7: Activity Diagram pyVCC

Figure 2.8: Component Diagram pyVCC

pyVCC 14


2.4 Proof of Concept

2.4.1 Virtualbox - Firewall - Beutner

betriebssytem virtualbox (guest, switch XP/Linux, Asterisk Now)firewall (ports)router (portforwarding, problematik fh (8080,22))

2.4.2 Callserver and MCU - Beutner

OpenVCS OpenVCS (Open Source Video Conferencing Server) is a SIP based MultipointControl Unit that runs on Windows Systems. The App can manage various MCUs.Each MCU (it’s called Bridge here) can manage up to 64 participants. You can managethe Bridges via an xml config-file.For every Bridge, you have to set up the following:

id: The number that identifies the bridge

realm: The domain name

protocol: The Network Protocol that is used for the Bridge

ipver: Version of the IP Protocol

port: Portnumber that identifies the SIP-stack

After the App has started, these values are loaded from the config-file and the Bridgeis added. Now, you can set up a connection from a SIP-Client to the Bridge by calling(SIP INVITE) sip:[email protected]. Then, the OpenVCS App establishes a connectionbetween the SIP clients.Any SIP-client software that supports the Codecs G711 for Audio and H264 for Videocan be used.

2 Medooze

Figure 2.9: OpenVCS Communication

2.4.3 libVLC RTSP Streaming

This RTSP signaling is processed by libVLC itself, within the RTP streaming module. Tomake use of the RTSP signaling, a deep packet inspection has to be done to intercept thoseDESCRIBE messages, in order to take the appropriate measures.

For the purpose of deep packet inspection, the Python API to the C library libpcap is used.

pyVCC 15


import pcap

A simpler processing of the incoming Ethernet frames is done with the Python module Im-packet.

from impacket.ImpactDecoder import EthDecoder

from impacket.ImpactPacket import IP, TCP

PCAP deep packet inspection thread.

class PCAP_Worker(QtCore.QThread):

def __init__(self , net_dev , parent = None):

QtCore.QThread.__init__(self , parent)

self.instance = parent

self.decoder = EthDecoder ()

self.p = pcap.pcapObject ()

Open the Ethernet or WLAN interface.

self.p.open_live(string.join(net_dev ,’’), 1600, 0, 100)

Filter TCP messages.

self.p.setfilter(’tcp’ ,0,0)

self.p.setnonblock (1)

self._running=False

#_________________________________________________________

Get the IP address of network interface in use.

def get_interface_ip(self , ifname):

s = socket.socket(socket.AF_INET , socket.SOCK_DGRAM)

return socket.inet_ntoa(fcntl.ioctl(s.fileno (), 0x8915 ,struct.pack(’

256s’, ifname [:15]))[20:24])

#_________________________________________________________

def get_lan_ip(self):

ip = socket.gethostbyname(socket.gethostname ())

if ip.startswith("127."):

interfaces = ["eth0","eth1","eth2","wlan0","wlan1"]

for ifname in interfaces:

try:

ip = self.get_interface_ip(ifname)

break;

except IOError:

pass

return ip

#_________________________________________________________

def stop(self):

self._running=False

#_________________________________________________________

def run(self):

self._running=True

pyVCC 16


while(self._running ==True):

self.p.dispatch(1, self.callback)

#_________________________________________________________

Inspect packets for DESCRIBE message and save the remote IP address.

def callback(self , jdr , data , something):

l2 = self.decoder.decode(data)

l3 = l2.child ()

if (l3.get_ip_src ()!=self.get_lan_ip ()):

if isinstance(l3, IP):

l4 = l3.child()

if isinstance(l4 , TCP):

if l4.get_th_dport () == 8554:

user_data = l4.get_data_as_string ()

data_lines = user_data.split(’\n’)

data_fields = data_lines [0]. split(’ ’)

if(data_fields [0] == "DESCRIBE" and not self.instance.

mediaplayerRemote.is_playing ()):

stream_address = data_fields [1]. split(’/’)

protocol = stream_address [0]. split(’:’)

socket = stream_address [2]. split(’:’)

session = stream_address [3]

ipaddress = socket [0]

port = socket [1]

recon_stream_address = "%s://%s:%s/%s" %( protocol

[0],l3.get_ip_src (),port ,session)

self.instance.stream_address =

recon_stream_address

self.instance.receive_remote ()

#_________________________________________________________

And the libVLC implementation:

import vlc.py

class pyVLCC(QtGui.QMainWindow):

Local client configuration.

def __init__(self):

QtGui.QMainWindow.__init__(self)

self.setWindowTitle("pyVLCC")

self.instance1 = vlc.Instance ()



self.mediaplayerLocal = self.instance1.media_player_new ()

self.mediaplayerLocalStreamer = self.instance2.media_player_new ()

self.mediaplayerRemote = self.instance3.media_player_new ()

self.media_local = None

self.media_loopback = None

self.media_remote = None

self.video_device = "v4l2 :/// dev/video0"

self.audio_device = "alsa ://hw:0,0"

self.network_device = "lo"#"eth0"

pyVCC 17


self.remote_address = "localhost"#"139.6.19.157"

self.stream_address = "rtsp ://%s:8554/ session.sdp" %(self.

remote_address)

self.ports = dict({"audio":9000, "video":9001, "rtsp":8554})

self.protocols ={ socket.IPPROTO_TCP:’tcp’, socket.IPPROTO_UDP:’udp’,

socket.IPPROTO_ICMP:’icmp’}

self.createLocalSectionUI ()

self.config ()

self.transmit_local ()

self.start_prev ()

self.pcap_rtsp_thread = PCAP_Worker(self.network_device , self)

self.pcap_rtsp_thread.start ()

#_________________________________________________________

AVC Profile 3.0.

def config(self):

self.audio_codec="alaw"

self.audio_sample_rate="48000"

self.audio_channels="1"

self.video_codec="h264"

self.video_bit_rate="512"

#_________________________________________________________

Start or end a call.

def call(self):

self.receive_remote ()

#_________________________________________________________

def hang_up(self):

self.stop()

self.pcap_rtsp_thread.start ()

self.start_prev ()

#_________________________________________________________

Start the webcam preview in the top left video frame.

def start_prev(self):

self.media_loopback = self.instance1.media_new(unicode("rtsp ://

localhost :%s/session.sdp" %self.ports["rtsp"]))

self.mediaplayerLocal.set_media(self.media_loopback)

self.media_loopback.parse ()

if not self.mediaplayerLocal.is_playing ():

if self.mediaplayerLocal.play() >=0:

self.mediaplayerLocal.play()

#_________________________________________________________

Provide the local recorded media on the specified port.

def transmit_local(self):

self.media_local =

self.instance2.media_new(unicode(self.video_device))

self.media_local.add_option(":input -slave =%s" % self.audio_device)

pyVCC 18


self.media_local.add_option(":live -caching =100")

self.media_local.add_option(":sout=# transcode{vcodec =%s,{ vcodec =%s,

venc=x264{profile=baseline ,level =1.3,nocabac ,nobframes ,ref=1},vb=%

s,scale=0,acodec =%s,samplerate =%s,channels =%s}:rtp{sdp=rtsp ://:%s/

session.sdp}"

% % (self.video_codec ,self.video_bit_rate ,self.audio_codec ,self.

audio_sample_rate ,self.audio_channels ,self.ports["rtsp"]))

self.media_local.add_option(":sout -keep")

self.mediaplayerLocalStreamer.set_media(self.media_local)

self.media_local.parse ()

if not self.mediaplayerLocalStreamer.is_playing ():

if self.mediaplayerLocalStreamer.play() >= 0:

self.mediaplayerLocalStreamer.play()

#_________________________________________________________

Receive remote media, described by the incoming RTSP session and play it back in the mainvideo frame.

def receive_remote(self):

self.media_remote = self.instance3.media_new(unicode(self.

stream_address))

self.mediaplayerRemote.set_media(self.media_remote)

self.media_remote.parse ()

if not self.mediaplayerRemote.is_playing ():

if self.mediaplayerRemote.play() >= 0:

self.mediaplayerRemote.play()

#_________________________________________________________

def stop(self):

self.mediaplayerLocalStreamer.stop()

self.media_local.release ()

self.mediaplayerRemote.stop()

self.media_remote.release ()

#_________________________________________________________

Start the main loop.

if __name__ == "__main__":

app = QtGui.QApplication(sys.argv)

vlcc = pyVLCC ()

vlcc.show()

vlcc.resize (1024, 600)

sys.exit(app.exec_())

Qt GUI:

2.4.4 SIP Simple SDK - Blink

Blink is a SIP user agent that is written with the SIP Simple SDK. It has the functionalitiesfor VoIP, messaging, chat and file transfer.

pyVCC 19

Python Video Conferencing Client Concept and Design Beutner - Kuhr

Figure 2.10: pyVLCC GUI

Figure 2.11: Blink GUI

A direct connection between blink and the well known Open Source Software Linphone,could be established.

pyVCC 20


3 Discussion

3.1 Callserver and MCU - Beutner

3.1.1 OpenVCS and Asterisk

broken Video mixer Implementation

3.1.2 Medooze

most recent open source version from 2003

broken dependencies due to library updates

pyVCC 21


3.2 pyVLCC

3.2.1 libVLC

The Python API of libVLC is direct and automatically generated from the libVLC C API.Thus the Python API provices access to all functionality, that is available from the C API.An integration within the IMediaStream interface of the SIP Simple SDK would be possible.

3.2.2 SIP Simple SDK

Due to complexity, the SIP Simple implementation failed, although SIP Simple SDK hasa well documented and structured API. The comlexity of intercoorperation of the differentcomponents is not as simple, as the name suggests.An error, that could be identified, was the registration of user accounts on a SIP registrar.Since there has been no SIP Registrar to register at, SIP Simple returned a timeout. SIPSimple continues trying to register any user, that has been using the API. Even when theprogram and the Python interpreter get terminated and restarted. The error seems to be afault usage, but could not be solved in the given time.Another, but not yet traceable error, was the usage of the IMediaStream interface. The mech-anisms that are used to control the RTP streams, that are implemented in libVLC, are notvery well documented and lack of simple examples. Only the implementation of the nativeSIP Simple SDK RTP audio interface was available as an example. But very complex andwithout any comments. This made it very hard to work around the problems that occured.No SIP Invite message could be generated, due to these both errors.

3.2.3 RTSP SDP vs SIP SDP

A SIP INVITE message contains a SDP body like it is explained in figure 2.2 and 2.4. Incontrast the SDP body of a RTSP REPLY message is shown in figure 3.15.

The following figure shows a session initiation capture with SIP, between th SIP SimpleSDK software blink and linphone-3.0.

In the SDP body, show in figure 3.14, of the SIP INVITE message contains the followingparameters:

3.2.4 Python

In chapter 2.4.3 Proof of Concept - libVLC RTSP Streaming the deep packet inspection isdescribed, which implements a simple way of signaling the RTSP streaming to both userclients within 80 lines of code.The code that is required to stream two different RTSP streams with libVLC, is written inabout 100 lines of code. Since libVLC is a C library, the performance issues are not causedby the Python interpreter.The graphical user interface, written in PyQt, takes another 100 lines of code.

pyVCC 22


img/sip_signaling_capture.png

Figure 3.12: SIP Signaling Wireshark Capture

Figure 3.13: RTSP Signaling Wireshark Capture

img/sip_sdp_body.png

Figure 3.14: SIP SDP Wireshark Capture

Taken these aspect into account, makes the Python programming language a simple andfast way to write user client applications, even with a graphical interface.

3.2.5 Performance

The user client without hardware acceleration is only stable for about 30 seconds, whiletranscoding the local video stream in AVC Baseline Profile 1.3.The following standard output of the x264 library, which is used for transcoding by libVLC,

pyVCC 23

Python Video Conferencing Client Discussion Beutner - Kuhr

Figure 3.15: RTSP SDP Wireshark Capture

indicates in line three, that no hardware acceleration is used.

x264 [warning]: frame MB size (40x30) > level limit (396)

x264 [warning]: MB rate (30000) > level limit (11880)

x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2 AVX

x264 [info]: profile Constrained Baseline, level 1.3

x264 [info]: final ratefactor: 29,40

x264 [info]: using SAR=1/1

Because the CPU is used for transcoding, it has heavy load by the Python interpreter.The following excerpt from the standard output of the command top shows this:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

30968 root 20 0 1606m 131m 46m S 175 3.4 8:44.65 python

Table 3.1: This table shows some data

But even if the VLC player GUI the CPU usage rises to 70 percent.

pyVCC 24

Python Video Conferencing Client References Beutner - Kuhr

References

[1] Hubert Zimmermann, OSI Reference Model -The ISO Model of Architecture for OpenSystems Interconnection, April 1980.

[2] Prof. Dr. Dipl. Ing. Andreas Grebe, NGN Introduction and Basics, July 2013.

[3] Prof. Dr. Dipl. Ing. Andreas Grebe, TCP/IP Protocols and Multimedia Services, July2013.

[4] Prof. Dr. Dipl. Ing. Andreas Grebe, Service Signaling in NGN, July 2013.

[5] Prof. Dr. Dipl. Ing. Andreas Grebe, SIP Routing, Network Address Translation andGateway Technologies in NGN, July 2013.

[6] Prof. Dr. Dipl. Ing. Andreas Grebe, Migration Scenarios in NGN and Corporate Net-works, July 2013.

[7] IETF, http://tools.ietf.org/html/rfc2326, Real Time Streaming Protocol, April 1998.

pyVCC 25

Documents

Project: Python Video Conferencing Client - pyVCC