26
Document Number: 608837-001 Intel® Media Accelerator Reference Software for Windows* White Paper Document Number: 608837-001 Document Revision: 1.0 May 2019

Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Document Number: 608837-001

Intel® Media Accelerator Reference

Software for Windows*

White Paper

Document Number: 608837-001

Document Revision: 1.0

May 2019

Page 2: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

White Paper May 2019

2 Document Number: 608837-001

You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products

described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject

matter disclosed herein.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product

specifications and roadmaps.

The products described may contain design defects or errors known as errata which may cause the product to deviate from published

specifications. Current characterized errata are available on request.

Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by

visiting: http://www.intel.com/design/literature.htm.

Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation.

Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or

retailer.

No computer system can be absolutely secure.

Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.

*Other names and brands may be claimed as the property of others.

Copyright © 2019, Intel Corporation. All rights reserved.

Page 3: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 3

Contents

1.0 Introduction ............................................................................................................................. 6

1.1 Objective ............................................................................................................................................................. 6

1.2 Terminology and Products....................................................................................................................... 7

2.0 Requirements and Materials ........................................................................................ 10

2.1 Related Files and Resources ............................................................................................................... 10

3.0 How to Use This Document........................................................................................... 12

4.0 Decode, Post-process, and Encode Video ............................................................ 13

5.0 Microsoft Media Foundation* Framework ........................................................... 14

5.1.1 Pipeline Components ............................................................................................................. 14

6.0 Composite and Render Video ...................................................................................... 16

7.0 Intel® Media Accelerator Reference Software ................................................. 18

7.1 Features and Support .............................................................................................................................. 18

7.2 Architectural Overview ........................................................................................................................... 19

8.0 Usage Models ........................................................................................................................ 21

9.0 Conclusion .............................................................................................................................. 24

10.0 How to Acquire the Intel® Media Accelerator Reference Software ..... 25

Appendix A Text ............................................................................................................................................. 26

A.1 Platform Configuration ............................................................................................................................ 26

A.2 Bibliography ................................................................................................................................................... 26

Figures

Figure 1. Pipeline Accelerator Model (Intel® Corporation, 2016) .................................................. 13 Figure 2. Media Foundation* Operations on a Multiplexed File ........................................................ 15 Figure 3. Composite Video Example .................................................................................................................... 16 Figure 4. Composited Stream with Custom Compositor (Intel® Corporation, 2012) ...... 17 Figure 5. Intel® Media Accelerator Reference Software Architecture ......................................... 20 Figure 6. Interactive Whiteboard Media Foundation Pipeline Flow ................................................ 21 Figure 7. Interactive Whiteboard Usage Model............................................................................................ 22 Figure 8. Digital Signage Example........................................................................................................................ 23

Page 4: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

White Paper May 2019

4 Document Number: 608837-001

Tables

Table 1. Media Process Terminology .................................................................................................................... 7 Table 2. Acronyms ............................................................................................................................................................ 8 Table 3. Products, SDKs, and Distributions ..................................................................................................... 8 Table 4. Software ........................................................................................................................................................... 10 Table 5. Hardware .......................................................................................................................................................... 10 Table 6. Related Files and Resources ............................................................................................................... 10 Table 7. Media Foundation Components (Polinger, 2011) ................................................................. 14 Table 8. Platform Configuration for Performance Testing .................................................................. 26

Page 5: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 5

Revision History

Date Revision Description

May 2019 1.0 Initial Release

§

Page 6: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Introduction

Intel® Media Accelerator Reference Software

White Paper May 2019

6 Document Number: 608837-001

1.0 Introduction

The Intel® Media Accelerator Reference Software, is a reference media player

application for digital signage, interactive whiteboards (IWBs), vending, and kiosk usage

models. It harnesses fixed-function hardware acceleration for superior media streaming

performance, improved workload balance and resource utilization, and customizable

GPU pipeline solutions.

The architecture of the implementation includes the Intel® Media SDK, a high-level API,

which improves developer experience by promoting portability and compatibility with a

range of processors. In addition, Intel® Media Accelerator Reference Software integrates

with Microsoft Media Foundation* to create a pipeline for media encoding, decoding,

and post-processing with the use of built-in and custom Media Foundation*

components. Available for Windows® 10, Intel® Media Accelerator Reference Software

also contains an implementation for compositing various media content streams on a

single display to create a rich user experience.

This white paper describes the architecture of Intel® Media Accelerator Reference

Software, a software reference solution applicable to many different market usage

models.

1.1 Objective

Intel’s fixed-function hardware acceleration, with integrated GPU, promotes efficient

workload balancing of a system’s GPU and CPU resources. Offloading media and

graphics tasks from the CPU to the GPU frees available CPU cycles for other activities,

such as video analytics and AI and creates opportunity for developing new usage

models.

In addition, potential developer integration of Intel® Distribution of OpenVINO™ toolkit

enhances computer vision (CV) and analytics possibilities. Many different usage models,

from simple audience counters at a kiosk to emotion detection in an IWB, employ video

analytics and AI. For example, developers of educational models can employ computer

vision and behavioral analysis in IWBs to monitor student response to presented

content. Over time, collected data can be used to design and calibrate content and

interactions. With behavioral data reference implementations, solution providers can

gain insight into user responses and craft improved messaging, personalization,

prediction, and more.

Page 7: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Introduction

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 7

1.2 Terminology and Products

Table 1. Media Process Terminology

Term Description

Collage display A feature of Intel® hardware and device drivers that enables the use of multiple independent displays to share one image.

Demultiplexing (Demux)

A process by which a single stream is separated into its original streams (e.g., video, audio, subtitles).

Multiplexing (Mux) A process by which two or more media assets, such as video and audio, are combined into a single output resulting in a container (e.g., MP4*).

Video compositing A process by which visual elements from two or more sources are combined into a single video plane.

Video decode A process by which encoded video is converted to its original RAW format.

Video encode or transcode

A process by which a video’s original format is converted to another digital format to enable compatibility with target media players and devices.

Video post-processing

A set of processes by which video is altered to increase perceived quality (e.g., de-interlacing, de-noise, sharpness, etc.).

Video transform A technique of using a particular algorithm or standard for splitting video into batches of frames for processing as well as reassembling processed video.

Page 8: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Introduction

Intel® Media Accelerator Reference Software

White Paper May 2019

8 Document Number: 608837-001

Table 2. Acronyms

Acronym Description

DASH Dynamic Adaptive Streaming over HTTP

(also known as MPEG-DASH)

A technique employing adaptive bitrate streaming to deliver media content over the Internet using HTTP web servers.

EVR Enhanced Video Renderer

A Microsoft Media Foundation* component that renders videos. Media Foundation provides two forms of renderers, EVR media sink and EVR filter.

HDMI* and HDMI-IN* High Definition Multimedia Interface* (Input*)

A proprietary digital interface for transmitting types of media data, both video and audio, as well as control information. It replaces earlier analog methods and offers improvement in image quality, signal integrity, and form factor (i.e., single cable).

HDR High Dynamic Range

An imaging technique that attains a luminosity range similar to that of human vision.

MFT Media Foundation* Transforms

A flexible Microsoft Media Foundation* component for processing media data. The component operates between Media Foundation* sinks and sources and includes encoding, decoding, and digital-signal processing.

RTSP Real Time Streaming Protocol

A network protocol for communication between media servers streaming video and/or audio. The protocol controls the creation of streaming sessions between two end points.

Table 3. Products, SDKs, and Distributions

Term Description

Direct3D* 11 Microsoft* Direct3D* 11

An API in DirectX* for 3D rendering in applications that require visualization and graphics, such as games and CAM/CAD engineering.

DXVA* Microsoft* DirectX* Video Acceleration

An API specification that enables hardware acceleration of video decoding through a pipeline to a GPU.

Intel® Distribution of OpenVINO™ toolkit

A software toolkit featuring computer vision, neural network inference, and deep learning deployment capabilities along with hardware acceleration.

Intel® Graphics Technology

A series of processors that contain an integrated graphics processor on the same die as the CPU.

Page 9: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Introduction

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 9

Term Description

Intel® Media Accelerator Reference Software

Intel® Media Accelerator Reference Software

A software reference implementations for use of Intel® Quick Sync Video.

Intel® Media SDK A set of libraries, tools, and samples for enhancing video encoding, decoding and post-processing.

Intel® Quick Sync Video

A technology that uses fixed-function Intel® Graphics Technology to perform video encode and decode.

Microsoft Media Foundation*

A multimedia framework and platform for digital media on Windows* operating systems.

§

Page 10: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Requirements and Materials

Intel® Media Accelerator Reference Software

White Paper May 2019

10 Document Number: 608837-001

2.0 Requirements and Materials

Table 4 and Table 5 list and describe the materials used to achieve hardware

acceleration in the applicable use cases for Intel® Media Accelerator Reference

Software.

Table 4. Software

Software Description

Operating System Microsoft* Windows® 10 64-bit

Media Reference Solution

Intel® Media Accelerator Reference Software

Intel® Media SDK for Windows*

2017 R2

2018 R2

Intel® Media SDK

Table 5. Hardware

Hardware Description

Processors Intel® Core™ Processor (beginning with 6th generation)

Intel® Atom® Processor

Intel® Xeon® Processor with Integrated graphics

Recommended Memory

Minimum of 8 GB Dual Channel DDR3 or DDR4

2.1 Related Files and Resources

Table 6 lists other files and resources related to Intel® Media Accelerator Reference

Software.

Table 6. Related Files and Resources

File Document No./Location

Developing Microsoft Media Foundation Applications

By Anton Polinger

Bibliography

Digital Signage Media Player Application: Media Decode Using Intel® Media SDK and Compositing Using DXVA-HD*

327201-001US

Intel® Media SDK Intel® Developer Zone

Page 12: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

How to Use This Document

Intel® Media Accelerator Reference Software

White Paper May 2019

12 Document Number: 608837-001

3.0 How to Use This Document

This document builds on key concepts involved in media processing before detailing

Intel® Media Accelerator Reference Software architecture and its potential usage

models. Each topic is presented in overview style.

SECTION 4.0: Decode, Post-process, and Encode Video

Describes elementary streams in Intel® Media SDK, scheduling and asynchronous

workloads, and the importance of workload consolidation.

SECTION 5.0: Microsoft Media Foundation* Framework

Provides an overview of Media Foundation* pipeline components.

SECTION 6.0: Composite and Render Video

Covers sub-streams and composited streams.

SECTION 7.0: Intel® Media Accelerator Reference Software

Lists features and illustrates component parts of the implementation.

SECTION 8.0: Usage Models

Illustrates an IWB pipeline example and describe potential usage models.

SECTION 9.0: Conclusion

Reiterates the potential of the reference implementation.

See Bibliography in the appendix for sources referenced in this document.

For more in depth coverage of topics see Bibliography and Related Files and

Resources.

§

Page 13: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Decode, Post-process, and Encode Video

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 13

4.0 Decode, Post-process, and Encode Video

Intel® Media Accelerator Reference Software employs the Intel® Media SDK, an API for

developing applications that leverage optimized hardware acceleration. Optimization

occurs not in the application layer but in the low-level libraries and the GPU driver,

increasing the likelihood that developers can create portable software that scales

across a wide-range of platforms.

Intel® Media SDK Focus

DecodePost-

processEncode

DecodePost-

processEncode

Demux/Split

MuxVideo Elementary Stream or Simple Transcoding

Audio Elementary Stream

Container file input

Container file output

Figure 1. Pipeline Accelerator Model (Intel® Corporation, 2016)

The SDK handles three operations: decode, video post-process, and encode. Figure 1

illustrates the SDK performance optimization focus. These operations correspond to

GPU pipeline stages in which the output from one stage provides input to the next. The

SDK employs an asynchronous framework to schedule work in the pipeline. As a result,

each stage may have multiple frames in some phase of processing on the GPU (Intel®

Corporation, 2016).

The design of Intel® Media SDK focuses on workload consolidation of the three

elementary video stream operations, as this is where optimization is most needed.

When using this accelerator model, developers create applications that establish

sessions and schedule work in the pipeline, but the applications do not control

precisely when the work occurs. Applications set parameters for the processes via the

API, and low-level architecture of the SDK handles the details of transcoding sessions

to balance workloads, ensuring optimal performance of CPU and GPU (Intel®

Corporation, 2016).

§

Page 14: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Microsoft Media Foundation* Framework

Intel® Media Accelerator Reference Software

White Paper May 2019

14 Document Number: 608837-001

5.0 Microsoft Media Foundation* Framework

The Intel® Media Accelerator Reference Software application layer interfaces to

Microsoft Media Foundation*, a platform for enabling developers to create advanced

video and audio processing applications on Windows*. Media Foundation* uses the

component object model (COM) and provides a pipeline layer, consisting of three main

components, to manage and process media data (Microsoft* Corporation, 2018).

5.1.1 Pipeline Components

Intel® Media Accelerator Reference Software uses built-in Media Foundation*

components as well as custom implementations to achieve maximum media streaming

flexibility. Table 7 lists and describes the operations of the three main components of

Media Foundation.

Table 7. Media Foundation Components (Polinger, 2011)

Media Foundation* Component

Purpose

Media Foundation* Source

Accepts data streams from a file, network, or hardware device.

Separates data into video and audio streams.

Transfers data to other Media Foundation* objects.

Media Foundation* Transforms

Transforms media data.

Executes decoding, encoding, and digital signal processing.

Media Foundation* Sinks

Renders content on screen or audio card using an enhanced video renderer (EVR) or streaming audio renderer (SAR).

Writes to a file on disk or sends to network for external consumption.

Page 15: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Microsoft Media Foundation* Framework

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 15

Media Foundation* Source

Media Foundation* Transform

Media Foundation* Sink

Container(File)

Multiplexed File

EVRSAR

Demultiplex

Decode

Render

Figure 2. Media Foundation* Operations on a Multiplexed File

Figure 2 illustrates a generalized Media Foundation* pipeline with component

operations on a multiplexed file. In this example, a multiplexed file is demultiplexed,

separated into video and audio streams, by the Media Foundation* Source. The

decoders, Media Foundation* Transform components, un-compress the streams, and

two sinks, an EVR and SAR, render the streams (Microsoft* Corporation, 2018).

§

Page 16: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Composite and Render Video

Intel® Media Accelerator Reference Software

White Paper May 2019

16 Document Number: 608837-001

6.0 Composite and Render Video

Traditionally the compositing process involves the creation of a single image from

multiple source images in order to provide the illusion of a single image. For example, a

composite of figures in a landscape may consist of a large landscape image as the

background, with multiple smaller images of figures in the foreground. Intel® Media

Accelerator Reference Software uses video compositing to combine multiple types of

streams on a single display.

Figure 3. Composite Video Example

Figure 3 depicts a composite from multiple sources combined with a ticker tape text

scroll at the bottom of the screen. In this example, static images are combined with

streaming media types and text to produce a multi-stream display for digital signage.

Page 17: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Composite and Render Video

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 17

Multi-stream Display

CustomCompositor

Primary Stream

Sub-stream 1

Sub-stream 2

Sub-stream n

-----

CompositedStream

Figure 4. Composited Stream with Custom Compositor (Intel® Corporation, 2012)

Figure 4 illustrates the compositing process in Intel® Media Accelerator Reference

Software, which is designed to accept various media types, such as text streams and

video content. The compositor of the Intel® Media Accelerator Reference Software

architecture creates the composited stream by combing a Primary Stream with one or

more Sub-streams. The resulting composited frame is pipelined to Direct3D* 11 for

rendering.

§

Page 18: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

Intel® Media Accelerator Reference Software

White Paper May 2019

18 Document Number: 608837-001

7.0 Intel® Media Accelerator Reference Software

Beginning with 2nd generation products, Intel® Core™ processors contained integrated

graphic processors (IGPs) or GPUs known collectively as Intel® Graphics Technology.

With these GPUs, Intel provided fixed-function hardware acceleration for various types

of media processing. The GPUs significantly boosted graphics performance and

improved overall system efficiency by freeing up the CPU to focus on other operations.

Intel® Quick Sync Video uses Intel® Graphics Technology to deliver media encoding,

decoding, and post-processing. Developers of digital signage, vending, and IWB

application software can use these capabilities to improve graphics performance

significantly. For example, testing at Intel demonstrated that hardware acceleration

reduced CPU loading from 70 percent to 10 percent (Intel® Corporation, 2012) as

compared to software-only video decoding (See Appendix, Table 8).

Intel® Media Accelerator Reference Software is a reference implementation of Intel®

Media SDK, a set of APIs optimized for Intel® Graphics Technology. The implementation

consists of the Intel® Media Accelerator Reference Software application and source

code and demonstrates how to employ Intel® Quick Sync Video to deliver fixed-

function hardware performance for digital signage, IWBs, and other market solutions.

The solution is released under an Intel® Proprietary License.

7.1 Features and Support

Intel® Media Accelerator Reference Software has support for various features and

streaming protocols including:

4K or Ultra High Definition (UHD) Resolution

High-Dynamic Range (HDR) imaging

Multiple displays, up to three, which can be different, tiled, or identical

Multiple zones

Video overlay and alpha blending

Hardware-accelerated video decode, encode, and post processing

Real Time Streaming Protocol (RTSP)

Ticker tape scrolling text

Widgets, RSS Feed, HTML5

Transcode/trans-rating and real-time transcoding/trans-rating.

Dynamic Adaptive Streaming over HTTP (DASH)

Page 19: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 19

Additionally, Intel® Media Accelerator Reference Software contains support for various

input sources:

IP camera

High-definition Multimedia Interface (HDMI-IN*) Capture Card

SDI/3G camera

USB camera

7.2 Architectural Overview

Figure 5 presents a high-level overview of the Intel® Media Accelerator Reference

Software implementation and its relationship to hardware components (Intel®

Corporation, 2012):

The software runs on Intel® Core™, Intel® Atom®, and Intel® Xeon processors featuring

hardware-accelerated video encoding and decoding with support for various codec

standards.

Drivers, Microsoft DXVA*, and Microsoft Direct3D* 11 exploit hardware acceleration

capabilities.

The Intel® Media SDK creates an abstraction layer that enables developers to take

advantage of hardware acceleration and optimizations in the layers below while

hiding implementation details that hinder development of platform-independent

code.

The Media Foundation* application layer interfaces with the Intel® Media SDK with

custom transform for media processing while using built-in source and sinks for end-

to-end (E2E) pipeline.

The custom compositor accepts various media content creators or multiple content

streams, such as text feeds, video, and gallery images, and creates a composited

frame on a display.

Page 20: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Intel® Media Accelerator Reference Software

Intel® Media Accelerator Reference Software

White Paper May 2019

20 Document Number: 608837-001

Custom Compositor

Optimized Media

Library (SW)

Optimized MediaLibrary

(HW-Accelerated) Decode/Encode

Intel® Media SDK

Dispatcher(libmfx)

Transforms

Sink

Microsoft Media Foundation*

DX11* Render

(SW)

DX11* Render

(HW Accelerated)

MicrosoftDXVA*

Microsoft Direct3D* 11

Drivers

CPU GPU

Inte

l® M

edia

SD

KO

S &

Dri

vers

Mu

ltim

ed

iaF

ram

ew

ork

Source

Multi-stream Display

Ap

pli

cati

on

Lay

er

Inte

l®C

om

pu

te

Intel® software

Intel® hardware

Intel® Media Accelerator Reference Software

Microsoft*

Figure 5. Intel® Media Accelerator Reference Software Architecture

Page 21: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Usage Models

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 21

8.0 Usage Models

Intel® Media Accelerator Reference Software uses Media Foundation components to

provide a pipeline for rendering media playback or writing data to a file or disk. Intel®

Media Accelerator Reference Software architecture leverages Media Foundation*

components, both renderer and archiver sinks, to support its multi-zone display usage

models.

Local Content Play

Compose

Streaming/SharingContent from

Remote Device

Encode (H.264)

Decode (H.264)

Decode (H.264)

Streaming/SharingContent from Remote

Device

Display Content

Save to Disk(Storage)

SINKSTRANSFORMSSOURCES

Figure 6. Interactive Whiteboard Media Foundation Pipeline Flow

Figure 6 illustrates a generalized look at a typical IWB pipeline, while Figure 7

demonstrates the resulting display, which includes two contents zones with content

being saved to disk.

ZONE 1Local Content(in-classroom)

ZONE 2Content from a Remote Device

(remote classroom)

x x

Save to Disk

Page 22: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Usage Models

Intel® Media Accelerator Reference Software

White Paper May 2019

22 Document Number: 608837-001

Figure 7. Interactive Whiteboard Usage Model

The Intel® Media Accelerator Reference Software reference implementation offers

source code and an application to developers to promote IWB and digital signage

usage models. The software aims to promote an expansive view of ideas that scale to

other video composite scenarios. With developer integration of Intel® Distribution of

OpenVINO™ toolkit, analytics can increase possibilities. For example, Intel® Media

Accelerator Reference Software software could be used to support:

Multi-zone IWBs: Previously recorded and stored educational content can be

composited and streamed remotely to an interactive white board solution in a

classroom setting. In future versions of the product, CV software may enhance the

experience for educators with student facial analysis.

IWBs and Creative Content Editing: Input streams from multiple cameras can be

composited, cropped, and rendered on multiple screens to generate immersive

experiences in meetings and educational settings.

IWBs and Effective Collaboration: The processing of multiple camera streams of

media content, such as video, images, and HTML, presents opportunities for more

collaborative video conferences and remote meetings.

IWBs and Efficient Meetings: Statistical analysis of meeting room occupancy,

frequency of use of media devices, length of meetings, and more helps organizations

manage content, conference rooms, and resources.

Digital Signage in Sports: A digital signage display for sports stadiums can combine

recorded athlete interviews with score updates as ticker text.

Digital Signage in Retail: A digital signage display in a mall can combine wayfinding

maps with future CV features like a statistical count of users and user demographics.

Digital Signage in Hospitality: A digital signage display can advertise spa services

and restaurant menus in a hotel.

Page 23: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Usage Models

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 23

Zone 1: 1x 4K, HEVC, 60FPS, 18MBps Video Playback

Zone 4: 1x 4k image

Zone 3: Ticker Text scolling at 40 words per minute.

Zone 2: 1x 4K, HEVC, 30FPS, 7Mbps (bit rate) Video Playback

Figure 8. Digital Signage Example

Figure 8 presents a detailed usage model from a demo for digital signage featuring

multiple zones of both streaming and static content.

§

Page 24: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

Conclusion

Intel® Media Accelerator Reference Software

White Paper May 2019

24 Document Number: 608837-001

9.0 Conclusion

The Intel® Media Accelerator Reference Software reference implementation combines

the performance optimizations of Intel® Media SDK with Media Foundation* pipeline

integration to deliver enhanced developer experience. The Intel® Media SDK enables

software developers and solution providers to future-proof code with an API that

abstracts low-level details that traditionally slow development time and create

portability hassles over the life of products. Media Foundation integration fosters

greater compatibility with Windows* video and audio devices.

Interoperability between Intel® Media SDK and Media Foundation* along with

composited video streams creates the building blocks for a variety of market solutions

and enhanced user experience. With the integration of Intel® Distribution of

OpenVINO™ toolkit, developers can expand existing usage models further with

analytics, using neural network inference and deep learning deployment.

Contact your Intel representative to obtain a copy of Intel® Media Accelerator Reference

Software

Page 25: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

How to Acquire the Intel® Media Accelerator Reference Software

Intel® Media Accelerator Reference Software

May 2019 White Paper

Document Number: 608837-001 25

10.0 How to Acquire the Intel® Media Accelerator

Reference Software

In order to acquire the Intel® Media Accelerator Reference Software, you’ll need to

contact your Intel representative to see if you qualify for access.

§

Page 26: Intel® Media Accelerator Reference Software for …...graphics tasks from the CPU to the GPU frees available CPU cycles for other activities, such as video analytics and AI and creates

How to Acquire the Intel® Media Accelerator Reference Software

Intel® Media Accelerator Reference Software

White Paper May 2019

26 Document Number: 608837-001

Appendix A Text

A.1 Platform Configuration

Table 8. Platform Configuration for Performance Testing

Features Description

Platform NUC7i7DNKE

Processor Intel® Core™ i5-U Processor

Memory 8 GB DDR4

Operating System Microsoft* Windows* 10 RS3

Graphics Driver 24.20.100.6229

A.2 Bibliography

Intel® Corporation. (2012). Digital Signage Media Player Application.

Intel® Corporation. (2016). Intel® Media Developers Guide.

Microsoft* Corporation. (2018). Microsoft Media Foundation Programming Guide.

Retrieved from Microsoft Media Foundation Programming Guide:

https://docs.microsoft.com/en-us/windows/desktop/medfound/media-

foundation-programming-guide

Polinger, A. (2011). Developing Microsoft Media Foundation Applications.

§