Phonon - static.kdenews.org

Phonon

Matthias Kretz

What is Phonon?

Design of PhononCore Classes

Code Examples

User Visible Features

How to Write a Backend

Phonon

Matthias Kretz

former KView maintainer

work on aRts and KDE Multimedia

Student of Physics and Computer Science at the University of Heidelberg

KDE Multimedia Meeting 2006

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Outline

What is Phonon?

Design of PhononCore ClassesCode ExamplesUser Visible FeaturesHow to Write a Backend

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Multimedia API

I task-oriented designI 80/20I easy multimedia developmentI no “competition” for GStreamer/NMM like media

frameworksI in KDE SVN: trunk/KDE/kdelibs/phonon (or

branches/work/kdelibs4_snapshot/phonon)

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Multimedia API




Phonon

Matthias Kretz

What is Phonon?


Code Examples



Multimedia API




Phonon

Matthias Kretz

What is Phonon?


Code Examples



Multimedia API




Phonon

Matthias Kretz

What is Phonon?


Code Examples



Multimedia API




Phonon

Matthias Kretz

What is Phonon?


Code Examples



Motivationthe user’s perspective

I A user should be able to playback any media withoutconfiguration steps

I “power users” want great flexibilityI additional multimedia hardware should be available

to all applications without any further stepsI users need to decide what device to use for what

purpose/programI KDE should not get in the way of “media producers”

Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples



Motivationthe developer’s perspective

I Qt/KDE style APII developers need APIs that are straightforward, easy

to use and understandI applications need a multimedia API that works on

UNIX systems (including OS X) and WindowsI ABI changes should not hinder KDE from using the

newest version of some media framework

Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples








Phonon

Matthias Kretz

What is Phonon?


Code Examples



Phonon Backends

Xine

Application

Phonon

Phonon−Xine

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Phonon Backends

GStreamer

Application

Phonon

Phonon−GStreamer

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Phonon Backends

NMM

Application

Phonon

Phonon−NMM

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Outline

What is Phonon?


Phonon

Matthias Kretz

What is Phonon?


Code Examples



The Core Classes

AudioOutput

VideoOutput

send to

send to

AudioEffect

VideoEffect

VideoPath

AudioPathprocess audio in

process video in

MediaObjectMediaObject

Phonon

Matthias Kretz

What is Phonon?


Code Examples



The Core Classes

AudioOutput

VideoOutput

send to

send to

AudioEffect

VideoEffect

VideoPath


process video in


VideoPath


process video in

Phonon

Matthias Kretz

What is Phonon?


Code Examples



The Core Classes

AudioOutput

VideoOutput

send to

send to

AudioEffect

VideoEffect

VideoPath


process video in


VideoPath


process video in

AudioOutput

VideoOutput

send to

send to

Phonon

Matthias Kretz

What is Phonon?


Code Examples



The Core Classes

AudioOutput

VideoOutput

send to

send to

AudioEffect

VideoEffect

VideoPath


process video in


VideoPath


process video in

AudioEffect

VideoEffect

AudioOutput

VideoOutput

send to

send to

Phonon

Matthias Kretz

What is Phonon?


Code Examples



A Glance at the Phonon Classes

I BackendCapabilitiesI AbstractMediaProducer

I MediaObjectI ByteStreamI AvCapture

I AbstractAudioOutputI AudioOutputI AudioDataOutput

I AbstractVideoOutputI VideoWidgetI VideoDataOutput

I AudioPathI VideoPathI AudioEffect

I VolumeFaderEffect

I VideoEffect

I NameDescriptionTupleI AudioOutputDeviceI AudioCaptureDeviceI VideoOutputDeviceI VideoCaptureDeviceI AudioEffectDescriptionI VideoEffectDescription

I Ui::VideoWidgetI Ui::SeekSliderI Ui::VolumeSliderI Ui::MediaControlsI Ui::EffectWidget

Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples









I VolumeFaderEffect

I VideoEffect



Phonon

Matthias Kretz

What is Phonon?


Code Examples



Outline

What is Phonon?


Phonon

Matthias Kretz

What is Phonon?


Code Examples



Audio Playbackthe simplest case

SimplePlayer

SimplePlayer* player = new SimplePlayer;player->play( "file:///home/user/song.ogg" );

seek/pause/stop

player->seek( milliseconds );player->pause();player->stop();

volume

float volume = player->volume();volume *= 0.5;player->setVolume( volume );

Phonon

Matthias Kretz

What is Phonon?


Code Examples




SimplePlayer


seek/pause/stop


volume


Phonon

Matthias Kretz

What is Phonon?


Code Examples




SimplePlayer


seek/pause/stop


volume


Phonon

Matthias Kretz

What is Phonon?


Code Examples



Audio Playbackwith more control

define the output

output = new AudioOutput;output->setCategory( Phonon::MusicCategory );path1 = new AudioPath;path1->addOutput( output );

AudioOutputAudioPathMediaObject

AudioOutputAudioPath

simple MediaObject use

media1 = new MediaObject;media1->addAudioPath( path1 );media1->setUrl( "file:///home/user/song.ogg" );media1->play();

Phonon

Matthias Kretz

What is Phonon?


Code Examples




define the output


AudioOutputAudioPathMediaObject AudioOutputAudioPath



Phonon

Matthias Kretz

What is Phonon?


Code Examples




define the output


AudioOutputAudioPathMediaObject AudioOutputAudioPath



Phonon

Matthias Kretz

What is Phonon?


Code Examples




define the output



AudioOutputAudioPath



Phonon

Matthias Kretz

What is Phonon?


Code Examples



Audio Playbackadding a Fader

recall the AudioPath

path1 = new AudioPath;

AudioOutput

AudioEffect

AudioPathMediaObject AudioOutputAudioPathMediaObject

insert the Fader

fader1 = new VolumeFaderEffect;fader1->setVolume( 1.0 );path1->insertEffect( fader1 );

Phonon

Matthias Kretz

What is Phonon?


Code Examples






AudioOutput

AudioEffect

AudioPathMediaObject AudioOutputAudioPathMediaObject

insert the Fader


Phonon

Matthias Kretz

What is Phonon?


Code Examples






AudioOutput

AudioEffect

AudioPathMediaObject


insert the Fader


Phonon

Matthias Kretz

What is Phonon?


Code Examples



Audio Playbackprepare next MediaObject for Crossfade

path2 = new AudioPath;path2->addOutput( output );media2 = new MediaObject;media2->addAudioPath( path2 );media2->setUrl( "file:///home/user/moremusic.

ogg" );fader2 = new VolumeFaderEffect;fader2->setVolume( 0.0 );path2->insertEffect( fader2 );

AudioOutput

AudioEffect

AudioEffect

AudioPath

AudioPath

MediaObject

MediaObject

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Audio Playbackprepare next MediaObject for Crossfade

path2 = new AudioPath;path2->addOutput( output );media2 = new MediaObject;media2->addAudioPath( path2 );media2->setUrl( "file:///home/user/moremusic.

ogg" );fader2 = new VolumeFaderEffect;fader2->setVolume( 0.0 );path2->insertEffect( fader2 );

AudioOutput

AudioEffect

AudioEffect

AudioPath

AudioPath

MediaObject

MediaObject

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Audio PlaybackCrossfade 2s before the first song ends

media1->setAboutToFinishTime( 2000 );connect( media1, SIGNAL( aboutToFinish( long )

), SLOT( crossfade( long ) ) );

void MyPlayer::crossfade( long remaining )

fader1->fadeOut( remaining );fader2->fadeIn( remaining );media2->play();

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Outline

What is Phonon?


Phonon

Matthias Kretz

What is Phonon?


Code Examples



The Mixer

I Every AudioOutput has a volume controlI Volume can be read and written using IPCI Central “desktop-mixer” can then control the volume

of all Phonon applicationsI To not let the number of volume controls explode they

are be combined into the categories Notifications,Music, Movies, Games and Communication

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Devices

I central place for device selectionI select device per categoryI applications can override the selection

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Devices

I central place for device selectionI select device per categoryI applications can override the selection

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Plug Your USB Headset

I hotplugging a device will change outputs to the newdevice if it is preferred

I consider VoIPI call comes inI you answer the call using the internal soundcardI plug in the USB headsetI notification shows that the device has been switchedI you can use your headset for the conversation now

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Configurationor: Hiding tedious configuration work from the user

I applications should not need to configure the soundsystem, selecting the device to use is enough

I one central place for configurationI configuration options include

I default video output deviceI default capture devicesI whether to use a soundserver (the soundserver will

then be started by KDE and made available as adevice)

I backend specific options

I For system wide integration a shared configuration isneeded for the cases where hardware mixing orALSA dmix is unavailable

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Network and Special Routingor: How to integrate NMM

I NMM provides for a high degree of networktransparency

I Out of scope for the Phonon APII IPC hooks in the NMM backendI NMM-Phonon control application

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Outline

What is Phonon?


Phonon

Matthias Kretz

What is Phonon?


Code Examples



Where do I Start?

I many classes to implementI starting point: Backend class

I mediaframework initializationI works as factory for all other classes

I then implement MediaObject, AudioPath andAudioOutput : enough for audio playback

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Main Audio Playback Classes

MediaObject

I read and decode mediafileI play, pause, stop, seek, tickI takes multiple Audio - and VideoPath s

AudioPath

I defines routing (and signal processing)I takes multiple AudioOutput s

AudioOutput

I defines audio sinkI software volume control

Phonon

Matthias Kretz

What is Phonon?


Code Examples




MediaObject


AudioPath


AudioOutput


Phonon

Matthias Kretz

What is Phonon?


Code Examples




MediaObject


AudioPath


AudioOutput


Phonon

Matthias Kretz

What is Phonon?


Code Examples



Optimized

I MediaObject::setUrlI start decoding the media to fill audio and video

buffersI emit the length signalI prepare for calls to hasVideo(), seekable(),

availableAudioStreams(),availableVideoStreams(),availableSubtitleStreams and totalTime()

I MediaObject::add{Audio,Video}PathI preprocess audio/video buffers with effects defined

for the pathI prepare for output to the audio device if the

AudioOutput is known

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Optimized cont.

I AbstractMediaProducer::play()I the backend should start playing immediatelyI the tick signal should be emitted - if possible without

pollingI effect change while playing

I buffers will make reaction sluggishI first priority: no dropoutsI second: no latency - overwrite as much of the buffers

with changed audio/video data as possible

Phonon

Matthias Kretz

What is Phonon?


Code Examples



other MediaProducers

subclasses of AbstractMediaProducerI ByteStream

I same as MediaObjectI instead of reading media data located using a URL→ media data is passed (streamed) from theapplication (or frontend)

I AvCaptureI Backend i18n(“anbieten”) audio and video capture

devicesI one video and/or one audio capture device can be

selectedI implicitly synchronized

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Two Designs

I 1:1I Phonon objects create and hold objects of the media

frameworkI good when there’s a 1:1 mapping between Phonon

classes and media framework classesI Phonon objects as description

I Phonon objects describe what the application wantsI Backend object(s) look at what the user wants and

wire media framework objects accordingly

Phonon

Matthias Kretz

What is Phonon?


Code Examples



Summary

using Multimedia functionality in KDE 4 will be easier forI developersI users

Outlook

I network interfaces, DVD-/TV-Support, OSDI backend developmentI reviewI more tests, esp. backend “certification” tests

Documents

Phonon - static.kdenews.org