OpenGL/ES and SGL: 2D/3D Graphics Accelerating Technologies

in Android Platform

Shao-Yi Chien (簡韶逸)

Media IC & System Lab

Graduate Institute of Electronics Engineering

National Taiwan University

Outline

• Introduction to Android

• Related 2D/3D graphics API

• SGL

• OpenGL/ES

• Integration of hardware accelerators

• Commercial products

• Our new developed GPU

Introduction to Android

• Android

– A software stack for mobile devices

– Includes an operating system, middleware and key applications

• The Android SDK

– Provides the tools and APIs necessary to begin developing applications on the Android platform using the Java programming language.

Architecture of Android

Related 2D/3D Graphics API

• 2D graphics library– SGL is the underlying 2D graphics engine

– The most common 2D graphics APIs can be found in the drawable package (android.graphics.drawable).

• 3D graphics library– An implementation based on OpenGL ES 1.0 APIs

– OpenGL APIs are available from the Khronos OpenGL ES package (javax.microedition.khronos.opengles, javax.microedition.khronos.egl)• Similar to the J2ME JSR239

– Plus some Android OpenGL utilities (android.opengl)

SGL

• Scalable Graphics Library (Skia Graphics Library)

• SGL is the low-level graphics library implemented in native code that handles rendering– Text, Geometries, and Images

• Developed by Skia Inc., which is acquired by Google in 2005.

• SGL is feature-set compatible with existing 2D standards, making it ideal to serve as a back-end for public formats such as SVG, PDF, and OpenVG

• Employed in Android and Chrome

Khronos Group

Khronos Group

OpenGL

OpenCL

OpenGL ES Family(OpenGL for Embedded Systems)

OpenGL 1.3

OpenGL ES 1.0

OpenGL ES 1.1

OpenGL 1.5

OpenGL 2.0

OpenGL ES 2.0

They will exist at the same time.

They won’t replace each other

Widely available

cross-platform 3D

graphics API

Enable software AND hardware implementations -

- including small-footprint, low-end fixed point

platform

What’s New in OpenGL ES 1.1

• Enhance effects– Multi-texture

– Vertex skinning

– Particle effect

– Bump shading…

• Enhance power– Buffer object – support render to texture

– Swap interval

– Power event

Roles for Each Family Members

• OpenGL ES 1.0: low cost devices, can be implemented with software

• OpenGL ES 1.1: must be implemented with some graphics accelerator

• OpenGL ES 2.0: support shading language. For high-end products

Integration of Hardware Accelerators:OpenGL as an Example

• If there is no 3D hardware acceleratorApplication Program

OS

Services

I/O

ServicesGDI OpenGL

Software

Rasterizer

Display

Device

Integration of Hardware Accelerators:OpenGL as an Example

• If there is 3D hardware acceleratorApplication Program

OS

Services

I/O

ServicesGDI OpenGL

Hardware

Driver

Display

Device

Commercial Products

• PowerVR MBX

– Support OpenGL ES 1.1 and OpenVG 1.0

• ARM Mali family

Commercial Products

Our New Developed GPU

• A unified architecture to support multimedia applications

• Quad-core stream processor SoC– 1 configurable filtering unit (CFU)– 2 stream processing units (SPUs)– 1 general-purpose RISC core

• Proposed techniques– Unified stream kernel– Configurable memory array (CMA)– Adaptive task scheduling– Adaptive multi-threading– Power aware frequency scaling (PAFS)– Configurable filtering unit (CFU)

System Architecture

AHB System Bus

Off-Chip

Memory

External

Memory

Bus

Unified Stream Processing Subsystem

RISC

CPU

Power

Aware

Frequency

Scaling

Memory

Controller

(MC)

Vertex

Fetch

Unit

Universal

Rasterizer

Rop

Unit

Configurable Memory Array (CMA)

Unified Shader 0

(US0)

Unified Shader 1

(US1)

I$ D$

T$Texture Unit with CFU Mobile

GPU

Coef.

Unified Stream Kernel (USK)

• Different level of scalability

19

Configurable Memory Array (CMA)

• Intra-level configuration

– Adjust different size of registers/cache depending on the application

• Inter-level configuration

– Assist to balance the workload in the two cores

20

Adaptive Task Scheduling (ATS)

• An application can always be partitioned into a series of cascaded tasks.

• If the workloads of the two stages are unbalance, the hardware utilization and performance will decrease– ATS technique is proposed

21

Adaptive Task Scheduling (ATS)

• USK0 is the bottleneck stage

• USK1 is the bottleneck stage

22

Adaptive Multi-Threading (AMT)

• Hide the instruction latency to increase hardware utilization and performance

23

Power Aware Frequency Scaling (PAFS)

• Level 1– Frequency scaling for low

battery state

– 30% Power Reduction

• Level 2– Clock gating for idle block

– 16% Power Reduction

• Nearly 50% power reduction

24