37
Logging at the TB Scale RJ Auburn, CTO

Logging at the TB Scale (Voxeo at splunk>live)

Embed Size (px)

DESCRIPTION

What do you do if you generate over 1 terabyte of logging data each day? How do you begin to analyze that volume of data? At the "splunk>live" event in Orlando, FL, on March 10, 2010, Voxeo CTO RJ Auburn spoke about how Voxeo uses Splunk to analyze logging data and monitor network operations. He also described how Splunk is integrated into Voxeo's on-premise Prophecy communications platform. More info about Voxeo can be found at http://www.voxeo.com/ and http://blogs.voxeo.com/

Citation preview

Page 1: Logging at the TB Scale (Voxeo at splunk>live)

Logging at the TB Scale RJ Auburn, CTO

Page 2: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

About Voxeo

  Founded in 1999

  World’s largest hosted VoiceXML and CCXML platform – Over 82,000 hosted ports globally; hundreds of premise deployments

  Over 100,000 developers using Voxeo platforms

  The Voxeo difference: Unlocked Communications, Customer Obsession Teams, Communications Passion

Page 3: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Mission

We make communications simple.

Page 4: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Unlocked Communications

Page 5: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Open Door Culture

  Executive access and attention for fast answers, readily available guidance and visionary insight

  Transparent communication for perfect alignment

  Collaborative, passionate problem-solving

  Focus on getting things done, not titles and hierarchy

Page 6: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Customer Obsession Teams

Page 7: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Free 24x7 Developer Support

“THANK YOU for such wonderful support. I can't remember the last time I experienced support like this, even when paying premium rates. Your department is definitely unique.”

“Your support is definitely the fastest on the planet.”

“Voxeo and Voxeo support is freaking incredible.... Wow and you can quote me on that.”

“Really good tutorials matched by world-class support! You guys rock.”

Page 8: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Communications Passion

Page 9: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Telecom 3.0

ClackPoint

Page 10: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Design Once − Deploy Anywhere

Page 11: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 11

Slough, UK Frankfurt, Germany

Singapore

Las Vegas, NV Atlanta, GA Orlando, FL

Voxeo Prophecy Hosting

Hong Kong

Page 12: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 12

Page 13: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 13

Page 14: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 14

Page 15: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 15

Page 16: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 16

Page 17: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 17

Page 18: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Security

  PCI compliant hosting platform since April 07

  Support for HIPAA, SOX, Gramm-Leach-Bliley, ISO 17799

  Nightly security scans of the entire hosting network

  Standard support for features like encrypted call recording

Page 19: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Architecture Diagram

19

Voxeo Platform Voxeo Platform Voxeo Platform Voxeo Platform

UI

Developer Portal NOC

Page 20: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Server Building Blocks – Storage

  SGI C1001 + eSAS RAID •  2x 4 core 2.3ghz Xeon

(L5410) •  8GB RAM •  150gb 10k/rpm boot drives •  LSI raid card •  eSAS RAID:

•  3U half depth •  Upto 16x 1TB SATA Drives

Page 21: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Operations Team

  Hundreds of scheduled searches run every 5 minutes

  24 x 7 NOC team members monitor Splunk dashboards, reports and alerts

  Splunk serves as one of the primary network monitoring tools

  We can watch for trends and build in optimization and capacity planning

21

Page 22: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

NOC Dashboard screen

22

Page 23: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

NOC Dashboard screen

23

Page 24: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Easy Creation of Custom Dashboards

Page 25: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Serving (And Securing) Our Developers

  Over 100,000 Developers access our platform—no easy task to help them troubleshoot their apps

  Splunk is integrated with our custom Authentication and single signon system

  There are access and search restrictions for every user to insure full multi-tenant secure access

  Ensures secure, self-service on-demand access

25

Page 26: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Customer Portal

26

Page 27: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Customer Portal

27

Page 28: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Ad-hoc reporting

Page 29: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Ad-hock reporting

Page 30: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Hosted or Premise - Why choose?

  ONE platform, hosted or premise

  Easy migration of applications between deployment models

  Hybrid offers best of both worlds •  Disaster recovery / business

continuity

•  Overflow for seasonal, peaks

Page 31: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Embedded in Premise Product

31

Page 32: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Embedded in Premise Product

32

Page 33: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation

Embedded in Premise Product

33

Page 34: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 34

Page 35: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 35

Page 36: Logging at the TB Scale (Voxeo at splunk>live)

© Voxeo Corporation © Voxeo Corporation © Voxeo Corporation 36

Page 37: Logging at the TB Scale (Voxeo at splunk>live)