Embed Size (px)
Citation preview
11 Centricular
GStreamer in the living roomand in outer space
LCA 2016, GeelongMultimedia and Music MiniConf
1 February 2016
Sebastian Dröge <[email protected]>
222 Centricular
Introduction
333 Centricular
Who?
● Long-term GStreamer core developer and maintainer since 10 years now– Did the last few releases and worked on every part until now
● Co-founder of Centricular Ltd– Consultancy offering services around GStreamer, graphics and
multimedia related free software
444 Centricular
What is this all about?
● What is GStreamer?● Cool stuff people are doing with it
555 Centricular
GStreamer? What is it?
AKA: Very short introduction!http://gstreamer.freedesktop.org for more details
666 Centricular
GStreamer
● Pipeline-based multimedia framework● Basic building blocks: elements connected via pads
● Set of libraries with abstract API● Plugins for specific features
– Often wrapping other libraries, e.g. libav/ffmpeg, OpenCV, various codec libraries
777 Centricular
GStreamer (cont'd)
● Open Source, LGPL v2.1+– Proprietary applications and plugins allowed
● Cross-platform– Linux, embedded Linux, Windows, OSX, iOS, Android, …
– Binary SDKs for Windows, OSX, iOS, Android
– x86, ARM, PowerPC, SPARC, …
● Stable GObject based C API/ABI● Bindings for many languages
– Python, C++, JavaScript, .NET, Java, Go, …
888 Centricular
GStreamer (cont'd)
● Not a media player or playback library● Not a codec and protocol library● Not a transcoding tool● Not a streaming server
● But can be (and is) used to implement all that
999 Centricular
GStreamer (cont'd)
● Goals– Flexible and extensible design
– Easy to integrate with other software (in both directions)
● Large, active developer and user community● Large commercial ecosystem
– Users
– Contributors
– Consultancies
101010 Centricular
Cool stuff peopleare doing with GStreamer
111111 Centricular
Desktop Environments and Apps
● GNOME, Enlightenment, XFCE, KDE– Media players, screen capturing, camera usage, …
– First larger scale usage
– Camera/screen sharing with sandboxing (pinos)
● Qt● OpenJDK● Libre/OpenOffice
121212 Centricular
The Web
● WebKit– GTK+, EFL, Qt, Windows/Cairo ports
● Blink/Chromium backend– https://github.com/Samsung/ChromiumGStreamerBackend/
● OpenWebRTC (http://openwebrtc.io)– WebRTC implementation by Ericsson
– Available with sample apps for Linux, Android, iOS and OSX
– Work in progress, WebKit integration underway
131313 Centricular
Server Side
● Kurento (http://www.kurento.org)– WebRTC / HTML5 streaming server
● Streaming & transcoding servers– RTSP, DASH/HLS, etc.
– Commercial hardware boxes
● DLNA server implementation Rygel– https://wiki.gnome.org/Projects/Rygel
141414 Centricular
Video Editing
151515 Centricular
Video Editing
● PiTiVi (http://pitivi.org)– GTK+ based Non-linear video editor
– Completely written in Python
● GStreamer Editing Services Library– Foundation of PiTiVi
– Also used in commercial products
1616 Centricular
Music Composing & Audio Editing
1717 Centricular
Music Composing & Audio Editing,Audio/video remixing and creation
● Buzztrax: Sequencer, synthesizer & tracker– http://buzztrax.org
● Put together various instruments graphically, define control curves for properties and let them run
● Processing.org: Language for visual arts– Look at the website for details and examples, it's awesome!
– Lots of videos and pictures to be watched there
1818 Centricular
Ingest / Playout Servers
CC BY 2.0
Citytv & OMNI TelevisionRogers Studios – Interior
By Nayu Kim
191919 Centricular
Ingest / Playout Servers
● Live recording, broadcasting, mixing– Defined latency
● Scheduling of recorded shows● Overlaying, subtitling, news ticker● GPU usage (codecs, composition, etc)● Needs to run reliably 24/7● SDI input/output, file or network
– Blackmagic Decklink cards, and many others
202020 Centricular
Set-top Boxes & TVs
● Live TV, DVB, IPTV● Personal Video Recorder● Catch-up and video on demand
● Low-cost embedded SoCs● Dreambox, YouView/BBC, free.fr, … STBs● LG WebOS, Samsung Tizen
2121 Centricular
Integrated Entertainment Systems
CC BY-SA 2.0
JAL Economy cabin withMAGIC systems
By Hideyuki KAMON
2222 Centricular
Integrated Entertainment Systems
● In-flight entertainment (IFI)● In-vehicle infotainment (IVI)● Video-on-demand● Sharing media between users● Public announcements● Synchronized multi-device playback
● GENIVI Alliance
2323 Centricular
Video walls & distributed speaker systems
CC BY 2.0
CES 2012 - LG video wall
By Doug Kline
2424 Centricular
Video walls & distributed speaker systems
● Frame-accurate synchronized output– Also for audio
● Aurena: https://github.com/thaytan/aurena
● Control & command rooms● Video walls● Digital signage● Classroom presentations
252525 Centricular
Drones / UAV
● Most known for military and espionage use● Increasingly non-military, civil use cases
– Rescue & emergency
– Law enforcement
– Package delivery
262626 Centricular
Augmented Reality
http://www.ericsson.com/research-blog/5g/remote-excavation-using-webrtc-real-time-video-eye-5g/
272727 Centricular
Home Security Systems
● Security cameras● Motion detection● Recording & streaming
282828 Centricular
Industrial image processing
● Automated production pipelines● Classification of quality of products
– Also fruits and vegetables, or counterfeit money
● Hyper-spectral image processing● Making use of OpenCV via GStreamer
292929 Centricular
IP Conferencing & Communication
● Low-latency, multi party conferencing– Also in commercial products
● WebRTC● SIP
– Farstream framework as used by e.g. pidgin
– https://wiki.freedesktop.org/www/Software/Farstream/
303030 Centricular
Event Recording & Broadcasting,Media Management & Publishing
● Timvideos– e.g. used for the presentation recording here
– http://timvideos.us
● Mediagoblin– Deploy your own YouTube, Flickr, SoundCloud
– http://www.mediagoblin.org
● Also a few commercial services based on GStreamer
313131 Centricular
Media Management & Publishing
CC BY 3.0
Image from Caminades
From the MediaGoblin website
323232 Centricular
Mobile & Embedded Devices
● Various iOS and Android apps– Also integrated into some Android devices, e.g. Samsung Galaxy
Xcover
● Part of many embedded Linux SDKs– Raspberry Pi, Freescale iMX, RDK, TI, …
● Support for lots of hardware and APIs– OpenGL integration, V4L2, OpenMAX, OpenSL, …
333333 Centricular
Gravitational Wave Research
3434 Centricular
Gravitational Wave Research
● LIGO (http://ligo.org)– Laser Interferometer Gravitational-Wave Observatory
– Trying to detect gravitational waves from neutron stars twins
● Large scale signal processing– Pipelines with 1000s of filters
● Large parts free software
353535 Centricular
International Space Station
● “Hi! Over the last few months I've been working on a project for the European Space Agency that is scheduled to be launched next year for use on the International Space Station, providing astronauts with a mobile procedure viewer in the form of a slightly modified Nexus 5 smartphone."
● Scheduled to go up in Q1/2015● http://www.hfes-europe.org/wp-content/uploads/2014/06/Chintamani.pdf
363636 Centricular
Thanks!
Any questions?
Find more information at http://gstreamer.freedesktop.orgor write me a mail [email protected]
373737 Centricular
