Inside Android's User Interface

8/12/2019 Inside Android's User Interface

1/44

1

Inside Android's UIEmbedded Linux Conference Europe 2012

Karim Yaghmour@arim!aghmour

[email protected]


2/44

2

These slides are made available to you under a Creative Commons Share-Alike 3. license. The !ull terms o! this license are here"https"##creativecommons.org#licenses#by-sa#3.#

Attribution re$uirements and misc.% &'(AS( )(A*"

This slide must remain as-is in this speci!ic location +slide ,2% everythingelse you are !ree to change including the logo "-

/se o! !igures in other documents must !eature the belo0 riginals at/)' immediately under that !igure and the belo0 copyright notice 0hereappropriate.

4ou are !ree to !ill in the *elivered and#or customi5ed by space on theright as you see !it.

4ou are 6)78*(9 !rom using the de!ault About slide as-is or any o! itscontents.

4ou are 6)78*(9 !rom using any content provided by 3rd

parties 0ithoutthe (:&'8C8T consent !rom those parties.

+C Copyright 212% persys inc.

These slides created by" ;arim 4aghmour

riginals at" 000.opersys.com#community#docs

*elivered and#or customi5ed by


3/44

3

About

Author o!"

8ntroduced 'inu< Trace Toolkit in 1=== riginated Adeos and relay!s +kernel#relay.c

Training% Custom *ev% Consulting% ...


4/44

>

Agenda

Android?s /8% 0hat?s that

Architecture 7asics

*isplay Core

pen'

8nput 'ayer

)elevant Apps and Services System Startup

)e!erences and &ointers


5/44

B

1. Android?s /8% 0hat?s that

"oC # $%U&ouch inpuLC(Ke!board

)))


6/44

1.1. Dhat 8?m 9T covering

Eedia layer

Stage6right

Fideo playback

Camera

*)E

(tc.


7/44

G

2. Architecture 7asics

Hard0are used to run Android

AS&

7inder

System Services

HA'


8/44

I


9/44

=


10/44

1


11/44


12/44

12


13/44

13

#!rame0orks#base#services#Java#...

#!rame0orks#base#services#Jni#

#hard0are#libhard0are#

#device#KEA9/6.L#K*(F8C(L

#sdk#emulator#

;ernel or module

#!rame0orks#base#core#...

AS&-providedAS'

Eanu!.-providedEanu!. license

Eanu!.-providedGPL-license


14/44

1>

3. *isplay Core

*isplay Hard0are

Classic 'inu< display stack

*isplay stack in Android

;ernel driver

HA' de!inition

HA' module

Sur!ace 6linger Dindo0 Eanager

Dalkthrough


15/44

1B

3.1. *isplay Hard0are

EE/

8EE/


16/44

1

3.2. Classic 'inu< display stack

3 3 *isplay stack in Android


17/44

1G

3.3. *isplay stack in Android


18/44

1I


19/44

1=

3.>. ;ernel driver

Fideo memory management Eode setting

Checking o! parameters

Eotorola :oom"

#dev#nvhdcp1

#dev#nvhost-ctrl

#dev#nvhost-display

#dev#nvhost-dsi

#dev#nvhost-gr2d

#dev#nvhost-gr3d

#dev#nvhost-isp

#dev#nvhost-mpe

#dev#nvhost-vi

#dev#nvmap

#dev#tegra-crypto

#dev#tegraMavp

#dev#tegraMcamera

#dev#tegraM!use

#dev#tegraMrpc

#dev#tegraMsema

... 0hatever hides in h0composer HA' module


20/44

2

3.B. HA' *e!inition

#hard0are#libhard0are#include#hard0are#h0composer.h

struct h0cMprocs"

invalidate+

vsync+

struct h0cMcomposerMdevice"

prepare+

set+

dump+

register&rocs+

$uery+

N+


21/44

21

3.. HA' module

Skeleton #hard0are#libhard0are#modules#h0composer.cpp

#system#lib#h0#h0composer.7A)*.so

#system#lib#h0#gralloc.7A)*.so

(


22/44

22

3.G. Sur!ace 6linger

Actual server" #!rame0orks#native#services#sur!ace!linger

Client side"

#!rame0orks#native#libs#gui Client # Server inter!ace"

8Sur!aceComposerClient.cpp

8Sur!aceComposer.cpp

This is 9T an aidl?ed service

All communication is manuallymarshalled#unmarshalled


23/44

23

3.I. Dindo0 Eanager

Server side" #!rame0orks#base#services#Java#com#android#server#0m#

Dindo0EanagerService.Java

Session.Java

Client side"

#!rame0orks#base#core#Java#android#vie0# Dindo0Eanager.Java

Dindo0Eanager8mpl.Java

Fie0)oot8mpl.Java

8nter!aces"

8Dindo0Eanager.aidl 8Dindo0Session.aidl

&arameters +incl. 5-order"

See Dindo0Eanager.Java


24/44

2>

3.=. Dalkthrough

Activity Eanager relies on Activity Thread AT calls on attach+ and makeFisible+

makeFisible does 0m.addFie0+

vm.addFie0+ - this also called by Status7ar to display itsel!

Creates a ne0 Fie0)oot8mpl

Call on its setFie0+

setFie0+ calls on sDindo0Session.add+...

This results in call to DE?s addDindo0+

Fie0)oot8mpl?s per!ormTraversals+

Calls on relayoutDindo0+

Calls to DE session?s relayout+

Call to DE?s relayoutDindo0+

Call to createSur!ace'ocked+

ne0 Sur!ace+...


25/44


26/44

2

>.1. Dhat?s pen'

8t?s Just an A&8 ... nothing but an A&8 ...

Check out Dikipedia

Eultiple versions out

(S versions !or embedded use

/p to (S 3.

Android support up to (S 2.


27/44

2G

>.2. Dhat?s in a modern-day &/

A tremendous amount o! parallel processing units

S8E*-like instruction set

Fideo decoding#encoding capabilities

...


28/44

2I

>.3. So!t0are layers involved

;ernel driver

' libraries

9ative ' A&8

Oava ' A&8


29/44


30/44

3

>.B. (' libs #!rame0orks#base#native#opengl#libs

(ntry point" #system#lib#lib('.so 'ooks !or #system#lib#egl#egl.c!g

#system#lib#egl - Eot :oom"

egl.c!g

lib('Mper!hud.so

lib('Mtegra.so lib'(SMandroid.so

lib'(Sv1MCEMper!hud.so

lib'(Sv1MCEMtegra.so

lib'(Sv2Mper!hud.so

lib'(Sv2Mtegra.so

elg.c!g"

tegra


31/44

31

>.. 9ative inter!ace

#!rame0orks#native#opengl#include ('

(TC1

'(S '(S2

;H)


32/44

32

>.G. Oava inter!ace

' libs re$uired by libandroidMruntime.so

#!rame0orks#base#opengl#Java#android#opengl"

...


33/44

33

>.I. So!t0are ' implementation

#!rame0orks#native#opengl#libagl


34/44

3>

B. 8nput 'ayer

;ernel side - std 'inu< input layer" #dev#input#N

9o HA' use

9ative lib"

libinput

#!rame0orks#base#services#input

8nput Eanager Service"

#!rame0orks#base#services#Java#com#android#server#input

Started and directly tied to Dindo0 Eanager

Speci!ic con!ig !iles +see source.android.com

So!t keyboard"

#!rame0orks#base#core#Java#android#inputmethodservice

8nput methods"

#packages#inputmehods

http"##developer.android.com#guide#topics#te


35/44

3B

. )elevant Apps and Services

'auncher Status7ar

Dallpaper Eanager Service

9oti!ication Service

App Didgets


36/44

3

.1. 'auncher

An app like any other See #packages#app#'auncher2


37/44

3G

.2. Status7ar

A uni$ue app See #!rame0orks#base#packages#System/8

Connects to Status 7ar Eanager and gives an

inter!ace it can use to call back into Status 7ar Can use set8con+ to display icons on the right

&rovides a C&/ usage add-on that renders

straight on rest o! display using higher 5-order


38/44


39/44

id


40/44

>

.B. App Didgets

See#!rame0orks#base#services#Java#com#android#server#AppDidgetService.Java

G S S


41/44

>1

G. System Startup

;ernel 8nit

7oot animation

'auncher

G 1 7 t i ti


42/44

>2

G.1. 7oot animation

Started by Sur!ace 6linger bootanim binary

#!rame0orks#base#cmds#bootanimation

)elies on bootanimation.5ip 0# &9s +nothing but

Seehttps"##github.com#CyanogenEod#androidMvendorMcm#tree#Jellybean#prebuilt#common#bootanimatino

Eust contain a desc.t


43/44

>3

I. )e!erences and &ointers

/se the source% 'uke Oim Huang?s Android raphics

7enJamin Rores? 'inu< Eaga5ine # 6rance articles

E8&S article on graphics internals" http"##developer.mips.com#212#>#11#learning-about-

android-graphics-subsystem#

Stphane Earchesin?s 'inu< raphics *rivers" an

8ntroduction http"##source.android.com#tech#input#inde


44/44

>>

Thank you ...

[email protected]

Documents

Inside Android's User Interface