Unified Approach for Next Generation Multimedia Based Communication Components Integration with Signaling and Media Processing Framework

Chandrasekar Sakthivel Senior Technical Architect

Aricent Technologies Bangalore ,India

Chandrasekar.sakthivel@aricent.com

Selvaraj Kesavan Technical Leader

Aricent Technologies Bangalore

Selvaraj.kesavan@aricent.com

Abstract—Modern digital technology makes it possible to manipulate multi-dimensional signals with systems that range from simple digital circuits to advanced parallel computers. It allows the user to modify and gives the excellent results for user experience as well as other commercial and security applications. Communication Technologies like Circuit Switched Video Telephony, IMS based multimedia Applications like Video Share, VoIP, VVoIP, Video on Demand etc are Multimedia based method provides real time Audio ,Video and Data , it is growing in the current mobile and Broadband technologies.

Current Multimedia based communication technologies are supporting low bandwidth error prone communication mechanism between terminals. Video Telephony supports signaling and the data transmission from peer to peer over low data rate flow in the network. Media control components are required to integrate the communication signaling and media processing.

This paper proposes generic approach across various multimedia framework and signaling modules. A new media control interface layer enables seamless data and signaling flow between various multimedia data processing frameworks and signaling modules and improves media communication processing. This layer increases the data flow and improves the quality of media data processing. It helps to integrate communication signaling module with media processing. This paper analyzes the media control interface based media communication based mechanisms like CS Video Telephony, IP based Multimedia Technologies like Video Share, VOIP etc. Media control Interface layer helps to integrate easily various Multimedia applications such as CS Video Telephony, VOIP etc into various media processing multimedia framework such as DirectShow, GStreamer, Opencore etc.

It also helps to integrate multimedia middleware stacks such as Media Transfer protocol [MTP], Digital Living Network Alliance [DLNA] , Image Processing Pipeline Algorithms with the Application and Native Layer.

Keywords: VideoTelephony, VOIP, MultimediaFramework

I. INTRODUCTION

Many development Platforms come in many shapes and sizes with different features capabilities. But when then telecommunication system access and core networks are concerned, it must be clearly defined in terms of both content and characteristics. This means that the choices of technology and configurations must be verified in a controlled environment.

Multiple level of abstraction helps us to port the

solution easily into different platforms and saves the time. Each level of abstraction adds the over head in terms of performance. Any Multimedia Framework which helps application to handle the resource scheduling and sharing is in programmatically and efficient way. For example most of multimedia framework runs in single thread to platform/operating system but internal to framework many components are attached and framework scheduler has been maintained internally to manage the resources.

Multimedia abstraction in terms of Platform

abstraction Layer [PAL], Hardware abstraction layer [HAL] ,Network Abstraction Layer[NAL] makes framework porting simpler and easier. It saves the processing time and better in terms of scheduling and resource sharing. When we integrate any external multimedia specific source and sink functionality oriented components into multimedia framework, it will become challenge to integrate the specific component into specific multimedia framework.

There is multiple open source as well as proprietary

multimedia frameworks in consumer Electronics/Mobile market. Direct show from Microsoft and MMF from Symbian are some of the proprietary Multimedia Framework. From the GNU community GStreamer, Packet Video Open core are some of the open source based Multimedia frameworks. The entire media framework internally is doing more specific resource scheduling/resource management and inters process communication. But the way it manages the resources are different and it brings significance different between among the frameworks. There are data source component which usually takes the compressed/ uncompressed data from File input/output agent or media input/output reader and pass the corresponding the compress /uncompress media data using intermediate component for processing. Sink or data

2010 International Conference on Signal Acquisition and Processing

978-0-7695-3960-7/10 $26.00 © 2010 IEEE

DOI 10.1109/ICSAP.2010.65

177

consumer usually takes the data in compressed/uncompressed format and stores and provides data into external world. The entire data transfer and component signaling / controlling is internal to framework and scheduled by framework itself.

Each framework names in different way for data provider/consumer and data compress/ uncompress logic. It is called as component/ plug-in node. This plug-in node format helps the framework to get the resource and handle it for data transfer or data retrieval.

There is requirement to approach unified way to the entire framework in order to reduce the integration effort during such a scenario. When video telephony application is integrated into any of the multimedia framework, the unified approach helps us to make one time effort to develop common plugin/ media control layer. Video Telephony application is interrelated with signaling as well as media components. This paper brings the experiment results with respect unified plug-in and universal approach to make easy integration layer between multiple different frameworks.

II. REQUIREMENT OF MEDIA CONTROL INTERFACE: This paper approaches the multimedia functionalities in

unified way and experiments in terms of performance and effort required. Four different multimedia frameworks have been taken as experimental approach to integrate and measure the performance of Unified plug-in based multimedia application. Unified plug-in is called as media control interface layer and it interfaces between the framework engine and external multimedia based applications like CS Video Telephony and IMS based Applications.

III. OPEN PLATFORM APPROACH: Media control interface layer helps to approach in unified

and it brings open platform concept. When multimedia based applications require integrating with the framework in order to achieve the combined solution, Open platform helps to have unified integration approach among different frameworks. Media control interface implements the control signaling and data management.

.

IV. ARCHITECTURE OF MEDIA CONTROL INTERFACE The below architecture is shown Media Control Interface

layer with respect to Android mobile Platform. Third Party components like Video Telephony core stack(3G-324M),IMS signaling stack SIP ,Streaming stacks such as RTSP,RTP,RTCP are integrated in the platform using the unified Approach. Media Control Interface layer extends the common interface from Framework. The below diagram is

shown opencore Multimedia Framework used for third party component integration layer

Figure 1. Architecture of Media Control Interface

Third party applications and Media Control Interface layer is exposed to Application developer to develop User Interface Applications.

Media control needs to interface with signaling entity and media framework. Incase of Video Telephony

Applications, Media Engine will be created and controlled by Media control Interface. This layer needs to be generic

abstraction of the media framework. Usually Media framework runs in single thread concept and it maintains

scheduler mechanism to schedule the internal components.

Application/User Interface

Middleware Stacks [3G-324M,SIP..]

Core Multimedia Framework

N

etw

ork

Abs

tract

ion

L

ayer

[NA

L]

N

ativ

e N

etw

ork

La

yer/M

odem

Platform Abstraction Layer [PAL]/Operation System Compatible Layer

Linux Kernel

Unified Media Control Layer

MediaControl-Application interface

Signaling Manager

Media Manager

Control Flow

Data Flow

178

Media control interface components architecture is designed based on multiple framework and its engine control mechanism. This brings the generic abstraction and the question of generic approach suitability.

Interestingly experimented multimedia framework and its

signaling, data and resource management mechanism allow us to approach in generic way.

In other terms, Media control interface layer acts as application for the media framework engine. Incase of Video Telephony Application, Application Interface has been exposed by Video Telephony Terminal. This terminal controls the signaling entity and data management.

Media control interface contains the following

components init. 1. Interface Layer for Media Framework 2. Interface to the Application/Application Framework 3. Interface Layer for external signaling/data entity 4. State and Signal Management

A. Interface layer for Media Framework:

It is abstraction of heart of the media framework. Usually it creates the core engine and respective components. It will create the scheduler and attaches the components with the scheduler. Each component is responsible for creating the buffer / data flow component. This will differ based on multimedia framework. Android PV framework has the abstraction above Engine component. Driver will provide the required data for engine. But 3rd Party proprietary media framework, the core scheduler and media engine manager needs to be created and then it should create the respective components and attach together.

Media control Interface layer should identify the media

framework capability and it differs in the implementation level

B. Framework Layered Architecture: The below layered architecture is shown and it explains

the advantages of media control interface for multiple applications including DLNA, MTP, IMS based Applications and Video Telephony Application. The below figure show the Platform components.

Figure 2. Framework Layered Architecture

It brings the advantages of modularity and easy

integration with respect to multiple components. The below diagram shows the internal architecture of generic media control interface based Applications

Media control Interface layer provides session control

and session management. It is required for Applications like IMS based Application. It brings robust state control and mechanism for each session. Media control interface provides the integration for ICF, IMS and other applications. API exposed by MCIL will ensure the reusability. It provides the Multimedia Framework independent approach and it helps seamless integration method. It extends the communication signaling.

VT Interface

IMS Stack Interface

MTP Interface

DLNA Interface

Opencore/DShow/GStreamer Multimedia Framework etc

Generic Application Interface

Media Layer/Engine Interface for

Player, VT, IMS

Middleware Core Stacks [DLNA,MTP,3G-H324M,SIP,..]

State manager

Application/Application Framework

Platform Abstraction/Network Abstraction /Device Layer

Control Flow

Data Flow

179

Figure 3. Generic Media Control based Applications

C. Media Control Interface Functionalities :

static MediaCtrlInterface& GetInstance(); MediaPlayer*MediaCtrlInterface::CreatePlayerSession( const char* mediasource, tPlayerConfigInfo *Config_info, Playerfxncallback callbackfxn, void *callbackdata); MediaRecorder*MediaCtrlInterface:: CreateRecorderSession(const char*mediasource, tRecorderConfigInfo *Config_info,

Recorderfxncallback callbackfxn, void *callbackdata);

VtTerminal* MediaCtrlInterface::CreateVtTerminal (tVTConfigInfo *Config_info,Vtterminalfxncallback callbackfxn, void *callbackdata); IMSAppManager* MediaCtrlInterface::CreateIMSAppSession(tIMSConfigInfo *Config_info,eIMSSessionType SesType,

IMSAppfxncallback callbackfxn,void*Callbackdata);

DlnaInterface* MediaCtrlInterface::CreateDLNASession(); MTPInterface* MediaCtrlInterface::CreateMTPSession();

void ClosePlayerSession (MediaPlayer* media_player); void CloseRecorderSession (MediaRecorder* media_recorder); void CloseVtTerminal (VtTerminal* vt_terminal); void CloseIMSAppSession (IMSAppManager* ims_appsession); void CloseDLNASession (DlnaInterface* dlna_sessionl);

void CloseMTPSession (MTPInterface* mtp_session);

Media control Interface extends the single ton mechanism and it provides the various media processing like media

player/recorder and MTP sessions.

D. Advantages: The following are the advantages in Media Control Interface

as listed below.

1. Code maintainability 2. Easy migration to new releases 3. Framework independent Approach 4. Extendable based on Middleware functionality 5. Less risk during code merging and Code scalability 6. System stability and Opencore independent

middleware functionality API to Application developers

7. Easy to adhere any framework architecture and integrate the components into the system

8. Creating the necessary components and attaching with the existing framework is easy and abstracted out in the higher level

9. Easy to integrate with 3rd party Multimedia components

10. Sync with future Gstreamer/opencore/media framework based releases will be easy.

CONCLUSION In this paper ,a unified approach to develop Media Control Interface Layer which abstracts the common functionalities of the Frameworks and to smooth integration of the existing frameworks with other media or signal processing module were presented. This open platform approach suits for any media framework helps to extendable /integrate with other media processing and signal processing modules.

Mid

dlew

are

Com

pone

nts

3G-3

24M

, SIP

,RTP

,RTS

P, D

LNA

, Application/JNI Interface

Application Framework/Media Service Layer

Media Control Interface Layer(Abstraction of Framework, Signaling Module API)

Media Control

Engine I/F

Media Control

Engine I/F

Media Framework Bi-directional /Player/Recorder Engine

Fra

mew

ork

S

ched

uler

Network/Device Abstraction Layer

Tx Comp

Video Enc

Audio Enc

Video Capt

Audio Capt

Rx. Comp

Video Dec

Audio Dec

Video Sink

Audio Sink

180

REFERENCES

[1] TS 26.110 V3.1.0 3GPP, "Codec for circuit switched multimedia telephony service; General description", R`99

[2] ISO/IEC. 21000-7:2004 IT - Multimedia framework (MPEG-21) - Part 7: Digital Item Adaptation, 2004

[3] Vetro, A., et al. Signal Processing Magazine, IEEE - Special Issue on Universal multimedia access, 20 (2), 2003.

[4] Wiegand, T., et al. Overview of the H.264/AVC video coding standard. Circuits and Systems for Video Technology, IEEE Transactions on, 13 (7). 560, 2003.

[5] Gstreamer Framework : http://gstreamer.freedesktop.org [6] Pesce, Mark D. (2003). Programming Microsoft DirectShow for

Digital Video and Television [7] Packet Video Framework http://www.packetvideo.com/products [8] Symbian OS - the mobile operating system

http://www.symbian.com/index.jsp. [9] http://gstreamer.freedesktop.org, Gstreamer User’s Manual, v.0.8 [10] OpenMAX – http://www.khronos.org/openmax

181