Technology for Developing Regions

Eric BrewerTier Group, UC Berkeley

CITRISCITRISOctober 1, 2008

Thanks to the Sonesh Surana Yahel-Ben DavidThanks to the Sonesh Surana, Yahel-Ben Davidand the TIER Students & Faculty

The “Base of the Pyramid”The Base of the Pyramid• 3-4 billion peoplep p

– Equivalent purchasing power < $2/day• Could swell to 6-8 billion over next 25 yearsy• Most live in rural villages or urban slums

– Movement towards urbanizationMovement towards urbanization• Education levels are low or non-existent

– Especially for womenEspecially for women => high birth rates => more poverty

• Markets are hard to reach, disorganized2

a ets a e a d to eac , d so ga ed

Traditional DevelopmentTraditional Development

• Very top downVery top down– International agencies fund big projects

Often with stings and debt attached– Often with stings and debt attached– Managed by questionable governments

Littl l f hi h t h l• Little role for high technology– (some role for medicine, agricultural tech)– Agencies have almost no technical capacity– Technology chosen/used by large contractors

3

Cellular Success StoryCellular Success Story

• 3B cellular users already3B cellular users already– Only 1.1B in the developed world

Africa is the fastest growing market– Africa is the fastest growing market• Not a development project!• Driven by bottom-up demand:

– Ease of use (voice)– Need for communications: work, remittances– Prepaid minutes and scratch-off cards

4

Bottom-up FinancingBottom-up Financing

5

RemittancesRemittances• Money sent back home to

l tirelatives• Remittance flows:

– (into) India: $17.4B– North Africa: $17.6B– West Africa: $10.4B– Central Africa: $2.7B

E t Af i $5 9B$40B

– East Africa: $5.9B– South Africa: $2.0B

• Many small paymentsC iti l t f th• Critical part of the economy– Largely informal

Legal?

6

– Legal?

Microfinance: Grameen BankMicrofinance: Grameen Bank• Owned entirely by the poory y p

– Began in one village in 1976– 2.6 million borrowers (95%

women), over 1,000 branches in over 42 000 villages 12 000 staffover 42,000 villages. 12,000 staff.

• Has loaned more than US$3.9B since inception

Over US$3 5B repaid with interest– Over US$3.5B repaid with interest (98.75% recovery rate); $290M loaned in the last 12 months.

• Has never accepted any charityp y y— has always been a profitable social enterprise

• 46.5% of borrowers have

7

46.5% of borrowers have crossed the poverty line

Bottom-up BusinessesBottom-up Businesses

8

Grameen TelecomC ll l i l B l d hCellular in rural Bangladesh

• ‘Village Phone’Village Phone– Buy phone, rent to your neighbors

• So far over 95,000 loans of average US$200 have beenaverage US$200 have been given to buy mobile phones.

• Covers 50,000 of 68,000 villages– 60M users– => Scales!

• Phone Lady income up by 2x• Phone Lady income up by 2x– Maintains the system– => Works!

9– Example of new rural income

• Medical transcription • Rural Kerela:Medical transcription• Audio in, text out• Customers in US India

Rural Kerela:– 30km from a small city– cheaper than Bangalore, Customers in US, India p g

Chennai, etc.• Pays 5-8 cents/line

– about $8/hour• Transcribe 3 times then

!merge!

10

WiFi enabled rural income

The Urban-Rural DivideThe Urban Rural Divide

• Sadly cellular is an urban phenomenonSadly, cellular is an urban phenomenon– Need user density to pay for basestations

Grameen Telecom is not an exception• Grameen Telecom is not an exception– Ignores basestations (subsidized)

( )– Bangladesh is a very dense country (not rural)• The urban world might be “flat”

– Bangalore is more like LA than Indian village– But rural world is on a different plane

11

WiFi for Rural ConnectivityWiFi for Rural Connectivity

12

Why WiFi?Why WiFi?

• Very low cost due to huge volumeVery low cost due to huge volume• Unlicensed spectrum (mostly!)• Incremental deployment• Incremental deployment

– Limited capital– Start small grow over time– Start small, grow over time

13

WiLD: Slotted WiFi MACWiLD: Slotted WiFi MAC• Move to TDMA with coarse slots

– Rate-based flow control (due to high RTT)– Exponential backoff is bad for voice

• Move to bulk acks• Move to bulk acks– Turn off automatic acks– Normal ack timeouts too short

• Add some forward-error correction• Support multiple antennas per pole

“burst synchronization” => all send or all receive– burst synchronization => all send or all receive– Turn off carrier sense (to co-transmit with neighbor)

• Working on:

14

g– better MAC for mixed voice/data– point-to-multipiont

15

16NSDI 2007: WiLDNet Results

New World Record:

382 Kms

17

New World Record – 382 Kms

Pico El Aguila, Venezuela

18Elev: 4200 meters

AirJaldi Rural WiFi ISP• North India• Tibetan Community• WiLD links + APs• Links are 10–40 km long• Achieve 4–5 Mb/s per link• VoIP + Internet• 10,000 users

Routers used: (a) Linksys WRT54GL, (b) PC Engines Wrap Hybrid: closed mesh for backhaul

19

Boards, Costs: (a) $50, (b) $140

Hybrid: closed mesh for backhaul

Rural Telemedicine• Aravind Eye Hospitals

– Tamil Nadu, India– 5 hospitals– But too far for most to walk

• Need:– 15M blind in India

70% f bli d t t bl– 70% of blindness treatable– 7% in rural areas get care

• Goals:– 50 rural vision centers

Diagnosis and prevention

20

– Diagnosis and prevention

Aravind Eye Hospital Network

• All WiLD links• Links 1-15 km long• Achieve 4-5 Mb/s per link• Video-conferencing• 3000 consultations/month3000 consultations/month

Routers used: PC Engines Wrap boards, 266 Mhz CPU, 512 MB

21

Cost: $140

22

Real Impact• Over 80,000 patients so far• Centers are cash-flow positive• Over 14000 thousand patients have recovered sight• Growing to 50 centers covering 2.5M people• Hoping to replicate in Lumbini Nepal• Hoping to replicate in Lumbini, Nepal

23

Some Challenges

• Bad quality grid powerq y g p• Limited local expertise• Remote locations• Open Spectrum?• Open Spectrum?

24

Poor Quality Powere

age

Ran

geVo

lta

Number of Instances seen over 6 weeks

Spikes and Swells:

• Lost 50 power adapters

Low Voltages:

• Incomplete boots

Frequent Fluctuations:

• CF corruptions

25• Burned 30 PoE ports • HW watchdog fails • Battery Damage

What about UPS?

• “Affordable” UPS systems ($300) are of standby-type– Primary source is grid– Secondary source is battery– Good quality power supplied only during outage!

26

Good quality power supplied only during outage!– 2006 fault episodes include use of UPS

Power Solutions• Low-voltage disconnect circuit

– Disconnects power when voltage is too lowp g– Designed by AirJaldi, $2

• Low-cost Solar Power ControllerLow cost Solar Power Controller– PPT (peak power tracking) => 15% more power draw– LVD + trickle charging => Doubles battery life– Voltage regulator => No spikes and swellsVoltage regulator > No spikes and swells– Power-over-Ethernet => Remote Mgmt– $70 (compared to $300 commercial units)

• Have not lost any routers in last year

W ki bi d id/ l t ll27

• Working on combined grid/solar controller

Local expertise: Training and Migration

TIER, UC BerkeleyA i d

Migration at AravindAravindLocal Vendor

g

Maintenance

Installation

Management

Equipment Supply

28

Jan’06 – Jun’06

2007: 5 more clinic links

Jun’07 – Dec’07Jul’06 – Dec’06 Jan’07 – Jun’07

Open Spectrum MattersOpen Spectrum Matters• Telecom problems aboundp

– Very limited competition (often none)– Weak regulators (telecoms go above them)

• Telecom is often largest tax payer politically astuteTelecom is often largest tax payer, politically astute– Hinders both VoIP and open spectrum

• Rural areas can’t wait for carriersF d t i t d l i iti l– Freedom to innovate, deploy is critical

– (No one buys rural spectrum intentionally!)– Interference is not a problem in rural areasp

• WiFi is the right place to focus for open spectrum– The lowest cost data solution (140m units in 2005)

Lower training deployment costs29

– Lower training, deployment costs

Other Tier TechnologiesOther Tier Technologies

• Delay Tolerant Networking (DTN)Delay Tolerant Networking (DTN)• Phone-based Applications

H lth d– Health records– Voice-messaging phones

• Education– Learning English from on a cellphone– Multiple Mice per PC improves learning

30

Education: multiple mice per PC

Most PCs are used by 5-8 kids• Educational value?

– Appears to reduce attrition

Give each child their own mouse (joint work with MSR India)– Significant impact on boysSignificant impact on boys– … but not for competitive games– Most effective game: all users

must pick correct answerp– Recent work to make it work with

legacy software

Decentralized DevelopmentDecentralized DevelopmentMore development should be bottom-up• Decentralized financing exists and is large:

– $40B/year in remittances to Africa– $10+B/year in microfinance loans

• Cellular and the Internet are enablers– Millions of entrepreneurs that we can help!

• Bottom-up projects WORK– Tend to fit the actual needs– Inherently localized

• IT, but also power, clean water, ???32

SummarySummary

• Technology has a larger role to playTechnology has a larger role to play– Rural areas are the right focus

• Many needs:Many needs:– Spectrum, power solutions, applications– Lower-cost international bandwidthLower cost international bandwidth – Capacity building: continuous training/support

• TIER projects have real impact todayTIER projects have real impact today• Berkeley should lead the movement

toward more decentralized development33

toward more decentralized development

BackupBackup

34

Education: EnglishEducation: English• English as a second language is very

valuablevaluable– Arguably defines “middle class” in India

• Goal: teach English via cell phone games– Cell phones are a better platform than PCs– Games can be effective with limited teachers– Must localize the content– Also developing continuous speech

recognition on the phone

Learning Assessment (Preliminary)• Substantial vocabulary gains:

M ( t t) 3 7 t f 12

g ( y)

– Mean score (pre-test): 3.7 out of 12– Mean score (post-test) : 8 out of 12– p-value < 0 001– p-value < 0.001– std. dev. = 3.7

Matthew KamDivya Ramachandran J h CJohn Canny

Remote ManagementRemote Management

• Really don’t want to visit nodes to fix themReally don t want to visit nodes to fix them– Even if you live in the same country!– Remotes locations, risky travelRemotes locations, risky travel

• Solutions:– Robust engineering (e.g. watchdogs)Robust engineering (e.g. watchdogs)– Need preventative maintenance– Remote diagnosis helps someg p– Alternate paths to remote nodes

• Sometimes we use cellular

38

Hardware Faults at Aravind, 2006

Instances* Description Total Downtime

63 Router board not powered 63 days

7 Router powered but hung 10 days

21 Router powered but not connected to remote LAN (burned ethernet ports)

34 days( p )

3 Router on, but wireless cards not transmitting (low voltage)

2 days

3 Router on but pigtails not connected 45 days3 Router on, but pigtails not connected 45 days

1 Router on, but antenna Line-of-Sight blocked

8 weeks

39

*Conservative Estimate

>90% of faults are power-related

Africa

40

International BandwidthInternational Bandwidth

• $3000 by Mb/s per month (in Africa)$3000 by Mb/s per month (in Africa)– Almost all via satellite

• What about fiber?What about fiber?– Undersea fiber along the western side (SAT3)– But priced just below satellite!!But priced just below satellite!!

• Benin: cheap at $2000 for Mb/s/month• Nitel (Nigeria) was $8000 until recently

– Very big place to lay fiber…• Need to decide intranet or Internet?

41

WiFi Internet for Africa?WiFi Internet for Africa?

• Key idea:Key idea:– Put in a WiFi backbone

Convert segments to fiber over time– Convert segments to fiber over time• Much lower capital up front than fiber!• When link is full, you can fund fiber!

• Crazy? maybe

42

43

44

45

SW Faults

Software Faults at Aravind (in 2006)

Instances* Description Total Downtime

4 No default gateway specified 4 days

3 Wrong ESSID, channel, mode 3 days

2 W IP dd 2 d2 Wrong IP address 2 days

2 Misconfigured routing 3 days

*Conservative Estimate*Conservative Estimate

46

2. Data Collection and Monitoring

• Designed Push-based PhoneHome systemDesigned Push based PhoneHome system– Measures node, link, network level properties– Posts to server every 3 hours

• Also used for remote managementWith e er post opens re erse ssh t nnel– With every post, opens reverse ssh tunnel

• Has helped with some diagnosisHas helped with some diagnosis– E.g. 2 wireless cards do not work together during low

voltage situations

47

48

3. Alternate Network Entry Points

• Getting into the network is itself a challenge– And once in, other points may not be reachable

• Enable alternate points using Satellite (V-SAT) and Cellphone– V-SAT up only 60% of time at Aravind– GPRS phone (console access) at AirJaldi

U i bi ti ith d d h b h l i• Use in combination with cascaded hop-by-hop logins– Advantage: Does not depend on system-wide routing or

underlying IP configuration

• Can recover from SW configuration errors• No recourse for physical faults, except visits

But might still gain useful information to help preparation of visit

49

– But might still gain useful information to help preparation of visit

4. Recovery Mechanisms

• HW watchdogHW watchdog– Low voltage is still problematic for on-board HW watchdog– Use LVD, so the router is not hung, but disconnected– At Airjaldi, built external HW watchdog for < $1 for routers

without on-board HW watchdogs

• SW watchdog– Run by cron every 4 minutes– Configuration file lists what parameters to measure and

what services to restart

50

Operational ResultsFault Incident Counts

60

40

50Before: Jan 07 - Jun 07

After: Jul 07 - Dec 07

20

30

40

Coun

t

10

20

0WeeklyManualReboots

Number ofPower-relatedRouter Faults

ProlongedDowntimesgreater than

CF CardCorruptions

(Aravind)

51

(AirJaldi) (Aravind)g

1day (Aravind)( )

Incidents

Lesson 1: Reduce Need For Trained Staff

• Reduce downtimes through better powerReduce downtimes through better power

• Nodes reboot reliablyy

• Data collection and monitoringg

• Remote management

• Tackle lack of connectivity via backchannels

52

Lesson 2: Simple Re-design Is Often EnoughEnough

• WiLDNet MACWiLDNet MAC

• Improved Low Voltage Disconnectp g

• External HW watchdogg

• Low-cost Power Controller

53

Lesson 3: Real Cost of Power Is Cleaning

• Grid power is bad qualityGrid power is bad quality

• Cleaning it up is costly and recurringg p y g– Need UPS, battery replacements– Upto 20X the monthly cost of grid power!

• Solar becomes more competitive and viableEqual to cost of UPS and batteries– Equal to cost of UPS and batteries

– And gives us the ability to generate power in remote areas

54

Thank you!

Resources:

• http://tier.cs.berkeley.edu/wiki

• sonesh@cs.berkeley.edu

Acknowledgments:

• Ermanno Pietrosemoli VenezuelaErmanno Pietrosemoli, Venezuela

• Alan Mainwaring, Intel Research

• Matt Podolsky, Intel Researchy,

• AirJaldi

• Aravind Eye Care System

55

Is it sustainable?• Vision center setup cost: $4000 (once)• Cost of Link: $1000 (once), towers $2000 ( ),

(sometimes)• Operating Cost: $150/month

B k• Break even– Assuming $10000 capital cost per center– About 2 5 yearsAbout 2.5 years

• Next Step: 50 Vision Centers– Implies 50,000 patients with significant vision

56

improvement• ~ 500,000 other patients with lesser issues

The Wireless HypothesisThe Wireless Hypothesis

1. Wireless infrastructure is the first viable1. Wireless infrastructure is the first viable infrastructure for rural areas

– Much lower cost than rail, water, electricity, roads– (ignores dirt roads and mandates, e.g. aqueducts)

2. … but it can lead to the rest– By improving health care, education, government– And by creating rural income.

And particularly WiFi

57