View
4.209
Download
13
Category
Tags:
Preview:
DESCRIPTION
CDR-Stats is a free and open source call detail record analysis and reporting software for Freeswitch, Asterisk and other types of VoIP Switch. It allows you to interrogate CDR to provide reports and statistics via a simple to use powerful web interface.It is based on the Django Python Framework, Celery, SocketIO, Gevent and MongoDB.
Citation preview
Call Data Analysisfor Asterisk & FreeSWITCH
with MongoDB
Arezqui Belaid @areskib<info@star2billing.com>
Problems to solve
- Millions of Call records
- Multiple sources
- Multiple data formats
- Replication
- Fast Analytics
- Multi-Tenant
- Realtime
- Fraud detection
Why MongoDB- NoSQL - Schema-Less
- Capacity / Sharding
- Upserts
- Replication : Increase read capacity
- Async writes : Millions of entries / acceptable losses
- Compared to CouchDB - native drivers
What does it look like? Dashboard
Hourly / Daily / Monthly reporting
Compare call traffic
World Map
Realtime
Under the hood- FreeSWITCH (freeswitch.org)- Asterisk (asterisk.org)
- Django (djangoproject.com)
- Celery (celeryproject.org)
- RabbitMQ (rabbitmq.com)
- Socket.IO (socket.io)
- MongoDB (mongo.org)
- PyMongo (api.mongo.org)
- and more...
Our Data - Call Detail Record (CDR)1) Call info : 2) BSON :
3) Insert Mongo : db.cdr.insert(CDR);
CDR = { ... 'callflow':{ 'caller_profile':{ 'username':'1000', 'destination_number':'5578193435', 'ani':'71737224', 'caller_id_name':'71737224', ... }, ... }, 'variables':{ 'mduration':'12960', 'effective_caller_id_name':'Extension 1000', 'outbound_caller_id_number':'0000000000', 'duration':'3', 'end_stamp':'2012-05-23 15:45:12.856527', 'answer_uepoch':'1327521953952257', 'billmsec':'12960',...
'hangup_cause_q850':'20', 'hangup_cause':'NORMAL_CLEARING', 'sip_received_ip':'192.168.1.21', 'sip_from_host':'127.0.0.1', 'tts_voice':'kal',7', 'accountcode':'1000', 'sip_user_agent':'Blink 0.2.8 (Linux)', 'answerusec':'0', 'caller_id':'71737224', 'call_uuid':'adee0934-a51b-11e1-a18c-00231470a30c', 'answer_stamp':'2012-05-23 15:45:09.856463', 'outbound_caller_id_name':'FreeSWITCH', 'billsec':'66', 'progress_uepoch':'0', 'answermsec':'0', 'sip_via_rport':'60536', 'uduration':'12959984', 'sip_local_sdp_str':'v=0\no=FreeSWITCH 1327491731\n' },...
Pre-Aggregate
Pre-Aggregate - Daily CollectionProduce data easier to manipulate : current_y_m_d = datetime.strptime(str(start_uepoch)[:10], "%Y-%m-%d") CDR_DAILY.update({ 'date_y_m_d': current_y_m_d, 'destination_number': destination_number, 'hangup_cause_id': hangup_cause_id, 'accountcode': accountcode, 'switch_id': switch.id, },{ '$inc': {'calls': 1, 'duration': int(cdr['variables']['duration']) } }, upsert=True)
Output db.CDR_DAILY.find() :{ "_id" : ..., "date_y_m_d" : ISODate("2012-04-30T00:00:00Z"), "accountcode" : "1000", "calls" : 1, "destination_number" : "0045277522", "duration" : 23, "hangup_cause_id" :9, "switch_id" :1 }...
- Faster to query pre-aggregate data- Upsert is your friend / update if exists - insert if not
Map-Reduce - Emit Step- MapReduce is a batch processing of data- Applying to previous pre-aggregate collection (Faster / Less data)
map = mark_safe(u''' function(){ emit( { a_Year: this.date_y_m_d.getFullYear(), b_Month: this.date_y_m_d.getMonth() + 1, c_Day: this.date_y_m_d.getDate(), f_Switch: this.switch_id }, {calldate__count: 1, duration__sum: this.duration} ) }''')
Reduce Step is trivial, it simply sums up and counts :
reduce = mark_safe(u''' function(key,vals) { var ret = { calldate__count : 0, duration__sum: 0, duration__avg: 0 }; for (var i=0; i < vals.length; i++){ ret.calldate__count += parseInt(vals[i].calldate__count); ret.duration__sum += parseInt(vals[i].duration__sum); } return ret; } ''')
Map-Reduce - Reduce Step
Map-ReduceQuery :
out = 'aggregate_cdr_daily'calls_in_day = daily_data.map_reduce(map, reduce, out, query=query_var)
Output db.aggregate_cdr_daily.find() :
{ "_id" : { "a_Year" : 2012, "b_Month" : 5, "c_Day" : 13, "f_Switch" :1 }, "value" : { "calldate__count" : 91, "duration__sum" : 5559, "duration__avg" : 0 } }{ "_id" : { "a_Year" : 2012, "b_Month" : 5, "c_Day" : 14, "f_Switch" :1 }, "value" : { "calldate__count" : 284, "duration__sum" : 13318, "duration__avg" : 0 } }...
Roadmap
- Quality monitoring
- Audio recording
- Add support for other telecoms switches
- Improve - refactor (Beta)
- Testing
- Listen and Learn
WAT else...?
- Website : http://www.cdr-stats.org - Code : github.com/star2billing/cdr-stats - FOSS / Licensed MPLv2 - Get started : Install script Try it, it's easy!!!
Questions ?
Twitter : @areskibEmail : areski@gmail.com
Recommended