Integrating Telecom Big Data: Challenges and Lessons Learned

By Fabíola Pereira

Motivation

Telecom + Internet

Motivation

Where is

9663-2161?

I want a service that, when given the cell phone number of my friend John, I receive a

map of his current location.

Motivation

How about an application that every day, at 8 am, makes me a call and plays

my favorite song?

Motivation

Weather Forecast

Mother’s Day Sale

Video Conference

My playlistCall

& vote

Where is it?MMS

Closest drugstore

Blogging

Motivation

BirthdayReminder

Voice Mail

Weather Forecast

Mother’s Day Sale

Video Conference

Virtual Pabx

My playlist

Horoscope

Call & vote

Where is it?Any news?

House Monitoring

Warn my vendors!

SMS

MMS

Closest drugstore

Mail

Random videos

Blogging

Daily feeds

Motivation

Let’s build a co-creation service platform

Innovation

Profitability

Time to

market

Popularity

and

dynamism

Velocity

VolumeVariety

Motivation

BIG DATA

Outline

Algar Telecom

Coreo Project

Coreo Architecture

Lessons Learned

Challenges

Conclusion

Algar Telecom

58 years in operation More than 1900 associates More than 800 thousand customers

Publicly traded*

* Via debentures

Business and Areas of Operation

Concession - 87 communities in 4 statesAuthorization area - offices expanding into key states

* In partnership with Algar Tecnologia

Data and voice

IT Services*

Businesses

Retail

Brazil

Central Brazil

Pioneering

Cellular telephone

Pre-paid telephone

Digital telephone

1 Gb broadband

Wireline broadband

3G deployment

Commercial fibre optic

Speech recognition

Leader in broadband penetration

Pioneer in NGN deployment – Next Generation Network

Brazilian Interior

Latin America

Outline

Algar Telecom

Coreo Project

Coreo Architecture

Lessons Learned

Challenges

Conclusion

Coreo Platform

Factsheet

• Dynamic Platform for the Composition of Applications

• ~1,5 years of development

• 6 software developers

• In homologation phase

• Innovative project

Coreo Platform

Components – the smallest units

Send MMS

Send SMS

Translate

Web Scrapper

Web Service

Send Mail RSS

On ReceiveSMS

On ReceiveMMS

Math

Make Call

Get and PostDTMF

Database

Concat

Announcement

Telecom

Internet Utils

Media

Storage

Coreo Platform

Applications – make your own composition via a Service Creation Environment

Location(LBS)

Where is it?

Coreo Platform

Applications – make your own composition via a Service Creation Environment

Funny Alarm

Web Scrapper

Make Call

Announcement

Coreo Platform

Coreo Platform

Coreo Platform

Outline

Algar Telecom

Coreo Project

Coreo Architecture

Lessons Learned

Challenges

Conclusion

Coreo Architecture

App Engine

Event Handler

Serv

ices

Mon

itorin

g an

d St

atisti

cs

Logs

and

SD

Rs

Portal SCEWeb

ServicesGadgets

Core

o Id

entit

y Se

rver

3rd Party and Web AppsUsers

COREO Platform

File Manager

Scheduler

WS Broker

DB Manager

Telecom PlatformsSDP

Controller

Components

Coreo Architecture

Coreo Architecture – front-end

Coreo Architecture – front-end

• Run apps via• Portal, • WS or • External gadgets.

Coreo Architecture – front-end

• Run apps via• Portal, • WS or • External gadgets.

• Coreo Accounts (IS)• Ldap user store;• Memcached session store;• Identity Server personalized.

Coreo Architecture – core

Coreo Architecture – core

• Messaging passing, non-blocking calls

Coreo Architecture – core

• Messaging passing, non-blocking calls

• NoSQL for states storage: stateless architecture

Coreo Architecture – core

• Messaging passing, non-blocking calls

• NoSQL for states storage: stateless architecture

• WSO2 ESB for services exposure and communication with Telecom Platforms

Coreo Architecture – back-end

Coreo Architecture – back-end

• WSO2 BAM 1.3 for monitoring and statistics• most popular apps;• most used components;• endpoint errors.

Coreo Architecture – back-end

• WSO2 BAM 1.3 for monitoring and statistics• most popular apps;• most used components;• endpoint errors.

• Structured data into DB and unstructured data into HDFS via Scribe• Service Detail Records (SDRs –

similar to Call Detail Records);• system logs.

Coreo Architecture – discussion

WSO2 Tools

Outline

Algar Telecom

Coreo Project

Coreo Architecture

Lessons Learned

Challenges

Conclusion

Lessons Learned

1. Be careful what you count when measuring data growth

2. Continuous improvement. It never ends!

3. CAP Theorem: what are your requirements? What do you really need?

4. Open source rocks!

1. Become more data-driven

Challenges

2. Move from reactive to proactive, turning data into actions

3. How to deal with legacy telecom platforms? How to adapt them? How to coexist?

4. Make Coreo happen!

Outline

Algar Telecom

Coreo Project

Coreo Architecture

Lessons Learned

Challenges

Conclusion

Conclusion

Built over WSO2 tools

Distributed and scalable system – ready for Big Data

Coreo Platform

Telecom Innovation

Thank you!www.algartelecom.com.br

Fabíola Pereira Cesar Alvarenga Emilio Dias{fabiolas,cesarw,emilio}@algartelecom.com.br