OpenGL ES on AndroidMinho Kim
Dept. of Computer Science
University of Seoul
Resources
• Short developer guide by Androidhttps://developer.android.com/guide/topics/graphics/opengl.html
• Tutorial by Android (Android framework) https://developer.android.com/training/graphics/opengl/index.html
• “OpenGL ES 2.0 Programming Guide” (2009) http://opengles-book.com/(requires an SVN client to download samples)
• Learn OpenGL ES by Kevin Brothaler (GLES ver. 2.0) http://www.learnopengles.com/archives/
• “OpenGL ES 2 for Android” by Kevin Brothaler (2013)
• OpenGL ES 2.0 course (spring, 2013) @Univ. of Seoul http://www.minho-kim.com/courses/13sp43.527/
OpenGL ES SDK
• ARM Mali http://malideveloper.arm.com/develop-for-mali/sdks/opengl-es-2-sdk-for-android/
• PowerVR SGX http://www.imgtec.com/powervr/insider/sdk/KhronosOpenGLES2xSGX.asp
• NVIDIA Tegra https://developer.nvidia.com/tegra-android-development-pack
• Qualcom Adreno https://developer.qualcomm.com/mobile-development/mobile-technologies/gaming-graphics-optimization-adreno/tools-and-resources
Environment Setup
• Install ADT bundle https://developer.android.com/sdk/index.html• JRE is required to execute Eclipse
• Samples can be installed using SDK Manager https://developer.android.com/tools/samples/index.html
• Install USB driver
GLSL Comparison
GLSL4.3 GLSL3.3 GLSL1.5 GLSL ES 3.0 GLSL ES 1.0Tess shader o o x x xGeo Shader o o o x xVAO o o o o xStorage Qualifiers (in, out) o o x o xUniform block o o x o xDirectly Accessing Buffer Data(glMapBuffer)
o o x o x
Instanced Rendering o o x o xMultisample Fragment Operations(glSampleCoverage)
o o x o x
Transform feedback o o x o x
OpenGL ES 2.0 on Android
• Supported through Android framework API (Java) and NDK (C)
• OpenGL API class – android.opengl.GLES20
OpenGL ES Development
• Required to create a view container by implementing both GLSurfaceViewand GLSurfaceView.Renderer
• GLSurfaceView• a view container for graphics drawn with OpenGL• needs to be extended to implement the touch listener to capture touch screen
events
• GLSurfaceView.Renderer• an interface that controls what is drawn within that view• Three methods are required to be implemented
• onSurfaceCreated() ≈ init()• onDrawFrame() – display()• onSurfaceChanged() – reshape()
Minimal OpenGL ES App (Java)
Minimal OpenGL ES 2.0 App from Scratch
• Create an empty Android App Project with “Create custom launcher icon” and “Create Activity” checkbox unchecked
• Minimum required SDK: 10 (Gingerbread 2.3.3)
• Create folder such as “src/com/example/android/opengl” (New Folder)
• Refresh the project
• Implement Activity, GLSurfaceView, and GLSurfaceView.Renderer as in the following slides• Implementing your GLSurfaceView is optional if no UI is required• Use “Ctrl-Shit-O” for automatic imports
• Click AndroidManifest.xml and select “Application” tab
• Add an “Activity” to “Application Nodes” and select the Activity you implemented as its “Name” (e.g. “com.example.android.opengl.MyGLActivity”)
• Add an “Intent Filter” to the Activity you added
• Add an “Action” to the Intent Filter you added with its name “android.intent.action.MAIN”
• Add an “Category” to the Intent Filter you added with its name “android.intent.category.LAUNCHER”
Implementing Activity
• New Class• Package – browse the folder (e.g. com.example.android.opengl)
• Name – choose yours (e.g. MyActivity)
• Superclass – android.app.Activity
package com.example.android.opengl;
import android.app.Activity;import android.opengl.GLSurfaceView;import android.os.Bundle;
public class MyActivity extends Activity {private GLSurfaceView mGLSurfaceView;@Override protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);mGLSurfaceView = new GLSurfaceView(this);mGLSurfaceView.setEGLContextClientVersion(2);mGLSurfaceView.setRenderer(new MyGLRenderer(this));setContentView(mGLSurfaceView);
}@Override protected void onResume() {
super.onResume();mGLSurfaceView.onResume();
}@Override protected void onPause() {
super.onPause();mGLSurfaceView.onPause();
}}
MyActivity.java
Implementing GLSurfaceView.Renderer
• New Class• Package – browse the folder (e.g. com.example.android.opengl)
• Name – choose yours (e.g. “MyGLRenderer”)
• Superclass – none
• Add interface “Renderer”
package com.example.android.opengl;import java.nio.ByteBuffer;import java.nio.ByteOrder;import java.nio.FloatBuffer;
import javax.microedition.khronos.egl.EGLConfig;import javax.microedition.khronos.opengles.GL10;
import android.opengl.GLES20;import android.opengl.GLSurfaceView.Renderer;import android.content.Context;
public class MyGLRenderer implements Renderer {private int mProgramObject;private FloatBuffer mVertices;
private final float[] mVerticesData ={ -0.90f, -0.90f , // Triangle 1
0.85f, -0.90f ,-0.90f, 0.85f ,0.90f, -0.85f , // Triangle 20.90f, 0.90f ,-0.85f, 0.90f };
public MyGLRenderer(Context context) {mVertices = ByteBuffer.allocateDirect(mVerticesData.length *
4).order(ByteOrder.nativeOrder()).asFloatBuffer();mVertices.put(mVerticesData).position(0);
}@Override public void onDrawFrame(GL10 gl) {
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);GLES20.glUseProgram(mProgramObject);GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, 6);
}@Override public void onSurfaceChanged(GL10 gl, int width, int
height) {GLES20.glViewport(0, 0, width, height);
}
@Override public void onSurfaceCreated(GL10 gl, EGLConfig config) {String src_vert =
"attribute vec4 vPosition; \n"+ "void main() \n"+ "{ \n"+ " gl_Position = vPosition; \n"+ "} \n";
String src_frag = "precision mediump float; \n"
+ "void main() \n"+ "{ \n"+ " gl_FragColor = vec4 ( 0.0, 0.0, 1.0, 1.0 );\n"+ "} \n";
int h_vert = GLES20.glCreateShader(GLES20.GL_VERTEX_SHADER);GLES20.glShaderSource(h_vert, src_vert);GLES20.glCompileShader(h_vert);
int h_frag = GLES20.glCreateShader(GLES20.GL_FRAGMENT_SHADER);GLES20.glShaderSource(h_frag, src_frag);GLES20.glCompileShader(h_frag);
int h_prog = GLES20.glCreateProgram();GLES20.glAttachShader(h_prog, h_vert);GLES20.glAttachShader(h_prog, h_frag);GLES20.glBindAttribLocation(h_prog, 0, "vPosition");GLES20.glLinkProgram(h_prog);
GLES20.glUseProgram(h_prog);GLES20.glVertexAttribPointer(0, 2, GLES20.GL_FLOAT, false, 0,
mVertices);GLES20.glEnableVertexAttribArray(0);
mProgramObject = h_prog;
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 0.0f);}
}
Implementing GLSurfaceView
• New Class• Package – browse the folder (e.g. com.example.android.opengl)
• Name – choose yours (e.g. “MyGLSurfaceView”)
• Superclass – GLSurfaceView
• MyGLSurfaceView.javapackage com.example.android.opengl;
import com.example.android.opengl.MyGLRenderer;
import android.content.Context;import android.opengl.GLSurfaceView;
public class MyGLSurfaceView extends GLSurfaceView {public MyGLSurfaceView(Context context) {
super(context);}
}
GLSurfaceView
• Manages an EGL display, which enables OpenGL to render into a surface.• Calls EGL functions internally
(https://android.googlesource.com/platform/frameworks/base/+/master/opengl/java/android/opengl/GLSurfaceView.java)
• Renders on a dedicated thread to decouple rendering performance from the UI thread.
• UI handling• Requires to subclass GLSurfaveView• Requires cross-thread communication using queueEvent()
• EGL configuration using setEGLConfigChooser before setRenderer()• default: RGB_888 color buffer + (at least) 16-bit depth buffer
• Requires to register a renderer using setRenderer
• Rendering mode – continuous or on-demand (set by setRenderMode())
• Requires to be paused/resumed when the activity is pause/resumed
GLSurfaceView.Renderer
• Responsible for making OpenGL calls to render a frame
• Called on a separate thread than the UI thread communication by queueEvent()
• EGL context lost• happens typically when device wakes up after going to sleep
• needs to re-create any lost resources in onSurfaceCreated()
Minimal OpenGL ES App (NDK)
Building a Minimal OpenGL ES App (NDK)
• NDK Setup• Download & unzip NDK (https://developer.android.com/tools/sdk/ndk)
• Set NDK location in Eclipse• Window Preferences Android NDK NDK Location (where you unzipped the
NDK files)
• Samples
• hello-gl2 by Google (included in NDK)
MyActivity.java
onCreate()onResume()onPause()
MyGLRenderer.java
onDrawFrame(){
MyGLJNI.display();}onSurfaceChanged(){
MyGLJNI.reshape(…);}onSurfaceCreated(){
MyGLJNI.init();}
MyGLJNI.java
static {System.LoadLibrary(“my_gl_jni”);
}native init();native reshape(…);native display();
my_gl_jni.cpp
init_JNI()reshape_JNI(…)display_JNI()JNICALL Java_com_example_test_MyGLJNI_init(…){ init_JNI(); }JNICALL Java_com_example_test_MyGLJNI_reshape(…){ reshape_JNI(…); }JNICALL Java_com_example_test_MyGLJNI_display(…){ display_JNI(); }
Android.mk
LOCAL_MODULE := libmy_gl_jniLOCAL_SRC_FILES := my_gl_jni.cpp
src/com/example/test
jni
Overall Procedure
1. Create an Android Application Project (with black activity)• Don’t include “_” in the package name!
2. Implement an Activity class (onCreate(), onResume(), onPause())• Set activity properly in AndroidManifest.xml as before
3. Implement a GLSurfaceView class (optional)
4. Implement a GLSurfaceView.Renderer class (onDrawFrame(), onSurfaceChanged(), onSurfaceCreated())
5. Implement a JNI interface library
6. Add native support: Right click Android Tools Add Native Support… Library Name: my_gl_jni(“my_gl_jni.cpp” and “Android.mk” are created in “jni” folder)
7. Modify “my_gl_jni.cpp” to implement OpenGL ES source code (init_JNI(), display_JNI(), reshape_JNI())
8. Modify “Android.mk”
9. AndroidManifest.xml Manifest Manifest Extras Uses Sdk Set “Min SDK version” to “14”
MyActivity.java
private GLSurfaceView mGLSurfaceView;@Override protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);mGLSurfaceView = new GLSurfaceView(this);mGLSurfaceView.setEGLContextClientVersion(2);mGLSurfaceView.setRenderer(new MyGLRenderer());setContentView(mGLSurfaceView);
}@Override protected void onResume() {
super.onResume();mGLSurfaceView.onResume(); }
@Override protected void onPause() {super.onPause();mGLSurfaceView.onPause(); }
MyGLSurfaceView (optional)
public MyGLSurfaceView(Context context) {super(context);
}
MyGLRenderer.java
@Overridepublic void onDrawFrame(GL10 gl) {
MyGLJNI.display();}
@Overridepublic void onSurfaceChanged(GL10 gl, int width, int height) {
MyGLJNI.reshape(width, height);}
@Overridepublic void onSurfaceCreated(GL10 gl, EGLConfig config) {
MyGLJNI.init();}
MyGLJNI.java
• Implement a JNI library class• New Class
• Name – e.g. “MyGLJNI”
• No superclass
static {System.loadLibrary("my_gl_jni");
}public static native void reshape(int width, int height);public static native void display();public static native void init();
my_gl_jni.cpp
• Create “jni” folder
• Create my_gl_jni.cpp which implements init_JNI(), display_JNI(), reshape_JNI()
• Add following (e.g. package name is “com.example.test”, JNI interface java class name is “MyGLJNI”)
extern "C" {JNIEXPORT void JNICALL Java_com_example_test_MyGLJNI_init(JNIEnv * env, jobject obj);JNIEXPORT void JNICALL Java_com_example_test_MyGLJNI_reshape(JNIEnv * env, jobject obj, jint width, jint height);JNIEXPORT void JNICALL Java_com_example_test_MyGLJNI_display(JNIEnv * env, jobject obj);
};JNIEXPORT void JNICALL Java_com_example_test_MyGLJNI_reshape(JNIEnv * env, jobject obj, jint width, jint height){ reshape_JNI(width, height);}JNIEXPORT void JNICALL Java_com_example_test_MyGLJNI_display(JNIEnv * env, jobject obj){ display_JNI();}JNIEXPORT void JNICALL Java_com_example_test_MyGLJNI_init(JNIEnv * env, jobject obj){ init_JNI();}
Android.mk
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := libmy_gl_jniLOCAL_CFLAGS := -WerrorLOCAL_SRC_FILES := my_gl_jni.cppLOCAL_LDLIBS := -llog -lGLESv2
include $(BUILD_SHARED_LIBRARY)
EGL
• Native Platform Interface• https://www.khronos.org/egl• https://www.khronos.org/registry/egl/sdk/docs/man/xhtml/
• GLSurfaceView calls EGL functions internally
• NDK without GLSurfaceView• https://code.google.com/p/android-native-egl-example/• Examples by Imagination (PowerVR)• EGL initialization in JNI
• FreeGLUT for Android• http://freeglut.sourceforge.net/docs/android.php
OpenGL ES 2.0 Version Checking
• Needs to check if GLES 2.0 is supported after creating GLSurfaceViewby checking
• Android emulator bug – returned value is always 0
• If supported, set GLES version to 2.0 before setRenderer()
((ActivityManager)getSystemService(Context.ACTIVITY_SERVICE)).getDeviceConfigurationInfo().reqGlEsVersion >= 0x20000
mGLSurfaceView.setEGLContextClientVersion(2);
Compiling Shaders
• Similar to the desktop OpenGL
Examples
Filling Buffers
• Use ByteBuffer class
• https://developer.android.com/training/graphics/opengl/shapes.html
Transformations
• android.opengl.Matrix class has helper functions for model-view transformations & projections
• Steps1. float[] matMVP = new float[16];
2. android.opengl.setIdentity(matMVP, 0);
3. android.opengl.perspectiveM(matMVP, …);
4. <more transformations in reverse order…>
5. android.opengl.translateM(matMVP, …);
6. GLES20.glUniformMatrix4fv(…,FloatBuffer.wrap(matMVP));
Animation
• Set setRenderMode(GLSurfaceView.RENDERMODE_WHEN_DIRTY); (e.g. in the constructor of your GLSurfaceView class)
• Call requestRender() whenever the screen needs to be refreshed
• https://developer.android.com/training/graphics/opengl/touch.html
UI
• To handle touch input, implement onTouchEvent() method in your GLSurfaceView class
• ApiDemos/Graphics/OpenGL ES/Touch Rotate
• “Responding to Touch Events” (Android tutorial)
• http://www.learnopengles.com/android-lesson-five-an-introduction-to-blending/
Texture Loading
• android.opengl.GLUtils to load texture from Bitmap class
• ApiDemos/Graphics/OpenGL ES/Textured Triangle
• http://www.learnopengles.com/android-lesson-four-introducing-basic-texturing/
Texture Compression
• ETC1• standard compression format (no alpha channel supported)• ETC1Util utility class
• ATITC(ATC)
• PVRTC
• S3TC(DXTn/DXTC)
• 3DC
• Should be declared in AndroidMenifest.xml using <supports-gl-texture> tag for Google Play filtering service
• ApiDemos/Graphics/OpenGL ES/Compressed Texture
Text Rendering
• ApiDemos/Graphics/OpenGL ES/Sprite Text