Upload
others
View
21
Download
0
Embed Size (px)
Citation preview
From Zero to Zero-CopyAccelerating Your Embedded Media Player
Lukas RusakSeptember 2018
whoamiLukas Rusak
Bachelors of Applied Science in Mechanical Engineering
Team Kodi Member
Team LibreELEC Member
LibreELEC Board Member
Okanagan Region of B.C.
Mission Hill Vineyard
Kelowna B.C and Okanagan
Lake
Agenda
● Introducing zero-copy● Current proprietary methods● Open source solutions● Video decoding pipeline● Rendering methods● Board specifics
Zero-copy
"Zero-copy" describes computer operations in which the CPU does not perform the task of copying data from one memory area to another. [1]
● Embedded devices with minimal CPU power trying to run 4K HEVC video
● Critical for devices with limited memory bandwidth
● Available in proprietary and open source solutions
Current Proprietary Methods
● Raspberry Pi (broadcom) dispmanx○ libbcm_host
● i.MX6 Vivante○ Blob that does everything for windowing and rendering○ libvpuwrap for decoding
● Amlogic libamcodec○ Userspace library dependent of vendor BSP Kernel (3.14)
● Impossible to fix bugs● Locked to vendor kernels● No chance to update● Code maintenance for vendor specific methods
Open Source Solutions
● Video 4 Linux 2 (V4L2) Driver○ Access to onboard decoding capabilities○ Memory-to-Memory devices○ Codecs are hardware and driver dependent
● Direct Rendering Manager (DRM) Driver○ Only required if utilizing direct to plane rendering
● OpenGLES 2.0 and EGL○ Only required if utilizing EGL image import to OpenGLES○ EXT_image_dma_buf_import [2]○ OES_EGL_image_external [3]○ NV12 or YUYV or YU12 or similar YUV format support
Video Decoding Pipeline
● V4L2 interface directly using IOCTL’s● Gstreamer
○ dma-buf ready● FFmpeg
○ dma-buf patches under review
● Using Gstreamer or FFmpeg shifts video decoding out of your application
Decoded Frame
AVDRMFrameDescriptor
Compressed Frame
Decoder
fd
Direct to Plane RenderingPositives
● Embedded devices may support YUV formats directly
● Most performant (no empirical evidence)● No OpenGLES/EGL context required
○ Skips the GPU entirely● Video overlays are possible using DRM Planes● Very straight forward when using atomic DRM
Negatives
● Requires* in driver scaling capabilities● No OpenGLES capabilities (shaders)● More complex if using multiple planes
Video + GUI
GUI Only
Primary Overlay Cursor (DRM Planes)
OpenGLES/EGL Image Import
Positives
● Shader support● Only one DRM Plane required
Negatives
● More expensive (requires EGL context and GLES operations)● Requires extensions that are less common
Mali Proprietary Blob
● Open source kernel module paired with closed source binary blob for OpenGLES
● Licensed from ARM● Bundled library containing either:
○ x11○ Wayland/GBM○ GBM
● Impossible to fix bugs● Lima and Panfrost open source efforts underway
Allwinner
● DRM Driver○ “sun4i”○ Supports scaling○ Supports overlays○ Supports NV12
● V4L2 Driver○ Patches pending (requires V4L2 request API)○ Supports NV12
● OpenGLES Driver○ “mali”○ Supports EGL import○ Supports NV12
Orange Pi PC
Amlogic
● DRM Driver○ “meson”○ Supports Scaling (WIP)○ Supports Overlays (WIP)○ Supports NV12○ Neil Armstrong <[email protected]>
● V4L2 Driver○ “Meson” (WIP)○ Supports NV12○ Supports HEVC, H264○ Maxime Jourdan <[email protected]>
● OpenGLES Driver○ “mali”○ Supports EGL import○ Supports NV12
Libre Computer Le Potato (S905)
Broadcom
● DRM Driver○ “vc4”○ Supports scaling○ Supports overlays○ Supports NV12 and YUV420
● V4L2 Driver○ “Bcm2835-v4l2-codec” (WIP)○ Supports NV12 and YUV420
● OpenGLES Driver○ “vc4” ○ No YUV import
Raspberry Pi 3B+
NXP
● DRM Driver○ “imx” and “etnaviv”○ No scaling (available via a separate ip)○ Supports overlays○ Supports NV12
● V4L2 Driver○ “coda”○ Supports NV12
● OpenGLES Driver○ “etnaviv”○ Supports EGL import○ Supports NV12 (WIP)
■ Target mesa 18.3■ gc3000 good performance■ gc2000 ok performance
Solidrun Cubox i4-Pro (gc2000)
Qualcomm
● DRM Driver○ “msm”○ Supports scaling○ Supports overlays○ Supports NV12
● V4L2 Driver○ “venus”○ Supports NV12
● OpenGLES Driver○ “freedreno”○ Supports EGL import○ Supports NV12
Dragonboard 410c
Others
HiSilicon
MediaTek
Renesas
Rockchip
Asus Tinkerboard S (RK3288)
Renesas R-Car
Index
1. https://en.wikipedia.org/wiki/Zero-copy
2. https://www.khronos.org/registry/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import.txt
3. https://www.khronos.org/registry/OpenGL/extensions/OES/OES_EGL_image_external.txt
Kodi Implementation Links:
https://github.com/xbmc/xbmc/tree/master/xbmc/windowing/gbm
https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIME.cpp
https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIMEGLES.cpp
https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp
Q/ATheme By: Samfisher Design