Mobile Phone is Transforming to a Multimedia Computer

Yrjo Neuvo

2 © 2007 Nokia

Towards the 4 Billion Milestone

Mobile phone subscriptions globally, millions

Source: Nokia0

-92 -93 -94 -95 -96 -97 -98 -99 -00 -02-01 -03 -04 -05 -07

200

400

600

800

1 000

1 200

1 400

1 600

1 8002 000

2 2002 400

2 600

2 800

3 000

Index

• Architecture of a mobile phone• Engine power management• Software side of the story• Speech, audio and image processing• Future directions• Closing remarks

Two different devices, different requirements

N92• Up to 40 MB internal dynamic

memory • Up to 2 GB mini-SD cards• Large 2.8" QVGA (320x240

pixels) TFT display 16 million colors and 1.0” (128 x 36 pixel) LCD capable of 65,536 colors

• WiFi, WCDMA,DVB-H• Bluetooth v.2.0, USB 2.0• 2 megapixel camera (1600x

1200 pixels) and CIF camera (352x288pixel) for video calling

• Integrated flash & Auto focus• MP3/AAC/AAC+/eAAC+/WMA/

M4A/AMR-NB/AMR-WB• Stereo FM radio, Visual Radio• Nokia Web Browser

Nokia 6030• Phonebook with up to

500 entries• No expandable memory• Display 128x128 pixel,

65536 color display• GSM / GPRS• No local connectivity• FM radio• xHTML browser

Architecture of a mobile device

Lowest Cost Architecture

• Limited interface support• Single-chip is attractive, if it meets cost,

yield and performance targets

RFDSP

BBRF

DISPLAY

Charger

Headset / audio

Application, Core and Protocol SW

Architecture of a Cellular Device… etc …

FM radioDVB-H

WLAN

Display

Camera

Input Device

ASIC

System LogicDSPMCUSRAM

RFIF

Audio CodecBB regulatorsEnergy mgmt

ASIC

SDRAMSTD

FlashSTDSTD

System ChipGSM/EDGE/WCDMA/CDMA2000

Analog Chip

Antennas,RF

FlashSTD

HW acceler.DSPMCUSRAM

ASICMultimedia Chip

Application SW

Symbian OS

GPS

Protocol SW

Bluetooth

Memory

Card

Open InterfacesMobile Industry Processor Interface (MIPI) promotesfaster and broader adoption of interface standards

MIPI CompliantMultimedia Chip

FlrDA

I2S

I2CModem I/F

SDRAM Controller

NAND Controller

GPIOsLCD I/F

USB OTG

MMCSD Camera IF

UART

PCM

Keyboard

I2C orSPI Audio

Codec

Power

Management

Modem

BlueToothMobileDDR

NANDFlash

LCD CtrlUSB

LCDDisplay

IrDA

Block Diagram of a Cellular PhoneBiCMOS

GaAs HBT

0.25µm CMOS

0.13µm CMOS

720mAh

~ ÷

Battery ChargingControl

Regulators

PowerSupply

ADCDAC

AudioCodec

BottomConnector

SIM card

Mixed-Signal

BBMemory

ControlInterfacesLogic

DSP160 MHz

MCU50 MHz

SRAM

Flash

Keyboard

LCDBacklight

LCD

64 Mbit

128x12864k colors

Infra Red

Vibra

Microphone

Earpiece

HandsfreeSYNTH

PA

RF1900

1800

900

900

1800/1900

Engine powermanagement

Engine Development for Cellular Phones

Single band Single band1993 2004

Direct conversion Rx and Tx

Triple band

Superheterodyne

IF subsampling Rx IQ- modulation & upconversion Tx

Dual band

(RF PWB only)

Current Consumption in Different States, please note, old product

GSM idle 3.5mA• WCDMA idle 3.4mA• GSM call

–(TX lev 5)275mA–(TX lev 7)222mA–(TX lev 19)144mA

• WCDMA call–(21 dBm) 418mA–(10 dBm) 286mA–(0 dBm) 271mA

• MP3 playback105mA

Power Consumption & Battery CapacityPower Consumption @ Max. Transmitter Output Power [W]Battery Capacity Index [%]

1W100%

2W200%

3W300%

6150

04020098 06

Single-ban

d

Dual-ban

d

Triple-

band

WCDMAVoice Call

orStand-Alone

Application Space GSM

Voice Call

Multimedia Call Space

Approximate power limit of a handheld device

Battery Capacity Gap or

Reduced Operation

Time

GPRS2 SlotsUplink

Nokia 6000 Series

Introduced

An Enabler of

Size Shrink

130c

c

97cc 123c

c

60cc

Battery Capacity

Gross Power Consumption Exceeds Dissipation Capability of Mobile Device

POWER,W

3

2

6

4

1

5

COOLING REQUIRED

100cc plastic monoblock

100cc metal monoblock

100cc plastic clamshell,open

Cellular RF Cellular RF

Miscellaneous

Cellular BBCellular BB

Local Connectivity

Local Connectivity

Local Connectivity

Display+backlight

Display+backlight

Display+backlight

Camera

Camera

Cellular RF

Audio

Audio

Audio

Apps Engine

Apps Engine

Mass Memory

Mass Memory

Power conversion

Power conversion

Power conversion

Cellular BB

Large plastic communicator,open

From the body to the soul

Software side of the story

Converged Device

Series 40

Fulfilling needs with customer-centric devices

Features

Price

Nokia 5500

Nokia E61

Nokia 2610

Nokia 1112

Nokia 5300

Series 30

Entry Phone Feature Phone

Nokia 2323

Nokia 6086

Nokia N93

Maemo

Nokia N800

True Web experiences: S60 open source

browser

New experiences

The most extensible open platform

Open & extensible platform

Open platform offers innovation potential for

licensees, operators and developers

Expand from High to Mid range

Platform expands to meet greater market

opportunities

Nokia 6290

Nokia E62

Handheld of the year 2005

Maemo:Innovation platform

Development environment

100+ applications created by open source developers supported

Open technologies

Based on standard Linux and other open source technologies including

GNOME

New product category

Nokia N800 Internet Tablet - Dedicated device for Internet

browsing and communications

Nokia OS

Java (CDLC)

Application Services

Series 40 UE

C2P

P2P

3rd Party AppsUser

Interface

Symbian OS

Common Language Bindings

Application Services (AS)

Java App Env.

Middleware Services Platform

Other App Env.

C++ App Env.

User Experience

C2P: Browsing, M-media, PIM

P2P: Rich Call & Messaging

3rd Party Applications

Series 60 Scalable UI

Series 80 UI

End-to-End Platform ArchitectureSeries 40 S60

N800 Open Architecture

GNOME subsetGTK+/Hildon, Theming,

Pango/Internationalization, Input Methods, GNOME-VFS, Gconf

Applicationengines

Application UI

Application framework

Proprietary

Partner

GPL/LGPLCore (Linux Kernel, X server, D-BUS etc)

• Open source is not an afterthought or an add-on but the most fundamental design principle: Open if no reasons to close

BrowserStandard Internet technologies

Flash Lite Easy to create great user experience

Java Portability across device categories

Native C++Best performance and

functionality

Open application runtimes

Speech, audio and

imageprocessing

Audio Design• Speaker sizes:

– 20, 16, 13 mm round– 11 x 15 mm rectangular

• Earpieces– 13, 8 mm round– 7 x 11, 4,8 x 10 mm

• Electret microphones• Signal processing• Mechanical

– 1-1.5 cc, shape, stiffness, material

Speech Processing in (GSM) Mobiles

A/D

D/A

Speech Coding(AMR, EFR, FR, HR)

VAD/DTX

Channel CodingInterleaving

Burst Building Modulation (GMSK)

RF

Speech DecodingBad Frame HandlingDTX Comfort Noise

DemodulationDe-interleaving

Channel Decoding

TX

RX

SpeechEnhancements

Acoustic EchoCancellation

Noise Suppression

Equalizing

…

Advances in speech coding/quality

• The increase in DSP MIPS has enabled the use of more sophisticated speechcoding algorithms

• Current narrowband qualityon par with PSTN (withEFR/AMR codecs)

• AMR-WB provides superiorwideband quality exceedingPSTN quality

>> PSTN7 kHz audio

30AMR-WB6.6-23.85 kbpsACELP

=PSTN12AMR4.75–12.2 kbpsACELP

=PSTN12EFR12.2 kbpsACELP

<PSTN20Half-Rate5.6 kbps, VSELP

< PSTN3Full-Rate13 kbps, RPE-LTP

QualityMIPSCodec

Speech codecs in GSM

Video coding - standards•Video coding standards

– MPEG-2: digital TV, DVD– H.263: 3GPP Multimedia, IP video conferencing– MPEG-4 Visual: 3GPP, includes H.263 Baseline– H.264, MPEG-4 AVC: Most advanced, multi-purpose standard

3GPP, broadcasting,HD-DVD, conferencing– VC-1, WMV9 Microsoft codec for streaming & broadcast– MPEG-4 AVC provides the same quality with less than half of the

bitrate compared to H.263 and MPEG-4 Visual Simple Profile – MPEG-4 AVC outperforms WMV9 clearly in quality / compression

efficiency

•Two main standardization tracks: – ITU-T: H.261, H.262, H.263, H.264 (Group: Video Coding Experts

Group, VCEG)– ISO/IEC: MPEG-1, MPEG-2, MPEG-4 Visual, MPEG-4 AVC

(Group: Moving Picture Experts Group MPEG)

Video Codec Evolution

20

30

50

40

50

Full

qual

ity c

ompr

essi

on ra

tio

1992 1997 2002 2005 year

MPEG-1

H.261

MPEG-2

H.263

MPEG-4H.263+

RealVideo9 WMV9

MPEG-4 AVC

H.264AVS 1.0

AVS-M

Dominant format in consumer electronics

Dominant format in mobile phones

Dominant format in streaming over the internet

3GPP Rel-6, 3GPP2, IP datacasting over DBB-H, video conferencing, DVD High-Definition,…

Scalability

MPEG-4

SVC

Complete Video ChainImage capture

Postprocessing Errorresilience

Preprocessing

Multiplexing

Colorconversiondownscaling

cycles per pixel(cpp)

Demultiplexing

channel

UpscalingColorconversion

Coding

Decoding

Displaying

5020-100

50-500

50-1500

20-10050-300

300

20

20

70

cycles (RISC)

AVC Compared to Earlier Standards

2727282930313233343536373839

0 50 100 150 200 250Bit-rate [kbit/s]

ForemanQCIF 10Hz

QualityY-PSNR [dB]

MPEG-2H.263

MPEG-4H.264/MPEG-4 AVC

Panorama with mobile phonecamera

• 25 frames, 320x240 resolution. Final image 705x262

The way forward

1 2 3 4 5

118

Antennas

2G/3GCellular

WLANBlue toot

hDVB-H GP

S

FMUWB

RF ID

diversityRX

7

9 10

6

Diverging application needs are driving for diversity of radios

2G/3G together with UWB, WLAN, RFID, Bluetooth, FM Radio, DVB-H, GPS, …

Multiradio

Long Term Evolution

WCDMA384 kbps DL384 kbps ULRTT ~150msCS/PS

HSDPA14.4 Mbps peak

RTT ~100msPS

HSUPA5.7 Mbps peak

RTT ~50msPS

UTRA evolution: WCDMA 5MHz

UTRA LTE: Up to 20MHz BW

DL improvements

to WCDMA

UL improvements

to WCDMA

Control and user plane latency improvements

Capacity enhancements

EUTRA100Mbps peak DL50Mbps peak ULRTT ~10msPS

New radio access

techniques

HSPA evolution28/42 Mbps peak DL11 Mbps peak UL

Towards 4G

LTE performance targets [3GPP 25.913]

Scalable 1.25-20MhzBandwidth

100msIdle to active latency

10msRadio network latency

2-3 x R62-3 x R6User throughput at cell edge 5%

2-3 x R63-4 x R6Spectral efficiency

50 Mbps100 MbpsPeak data rate (20MHz)

UplinkDownlink

35 © 2007 Nokia

Antenna & RF frontend

Longer-term. Technologies improve gradually:•Configurable antennas•Sampling frontend•Tunable RF filters

Analog-Digital conversion

Technology progress is promising.•Improving ADC performance enables moving mixers and filters to digital side

Digital basebandTechnology is maturing. •Programmable baseband processors may replace or complement custom ASIC solutions.•Digital RF logic will be moved to CPU

“Ideal” Software Defined radioLNA = Low-noise amplifierPA = Power amplifierADC = Analog-Digital converterDAC = Digital-Analog converter

LNA

PA

Antenna

ADC

DACDigital

RF section Digital RF/IF and basebandTX - Transmitter

RX - Receiver

Programmable baseband

GPS

DVB-T

GSM

UMTS

802.11a

GalileoDoppler

EDGE,GPRS

HSDPA, MIMO

11n (MIMO)

0.1 0.3 1 3 10 30 GIPS

Mobile Pentium @ 1 GHz (~10W)

802.11a: 3 Pentiums, 30WUMTS: 10 Pentiums, 100W

⇒Need for specialized DSP

Source: Kees van Berkel, et. al., Philips, SDR Technical Conference 2004

Cellular protocols set very high demands on computing capacity

Closing remarks

Likely time to reachplateau

2006-20072007-20082008-20092009-

Hype chart

Handset Sensors

NFC

Mobile Broadcast TVAjax Mash-ups / Web 2.0

FMC / VoWLAN / UMA / IMS /Mobile SIP

3.9 / 4G

Mobile Linux

MVNOs

”Long Tail”

Mobile music

Mobile search

WiMax

Enterprise mobile apps

Wirelessemail

VoIP

LBS

RSS

Blogs

Technology Trigger

Peak of inflated expectations

Trough of disillusionment

Slope of enlightment

Plateau of productivity

MATURITY

Visi

bilit

y