Machine Learning The Key Ingredient to Self-Driving Data Center

Machine Learning as The Key Ingredient to

“Self-Driving” Data Center Sergey A. Razin

CTO, SIOS Technology Corp.

2

Who is SIOS?

Who is SIOS?

3

How did IT

start?

How did IT start?

5

How did IT started?

6

How did IT started?

7

9

10

API

11

Operational Scale & Complexity in Virtualized IT Underlies the Problem

12

# of Data Points for 7 day (1 week) (over 60B for 10,000 VMs)

VMs

# of Objects

# of Metrics

0

10,000,000,000

20,000,000,000

30,000,000,000

40,000,000,000

50,000,000,000

60,000,000,000

70,000,000,000

1001,000

3,0005,000

10,000

Axi

s Ti

tle

100 1,000 3,000 5,000 10,000

VMs 100 1,000 3,000 5,000 10,000

# of Objects 2,000 15,000 50,000 100,000 200,000

# of Metrics 604,800,000 4,536,000,000 15,120,000,000 30,240,000,000 60,480,000,000

13

What is Machine

Learning?

Does ML exist today?

14

Gap In Tools

15

16

17

Goodbye, Big Data!

Hello, Smart Data!

We have a problem here…

We can’t see it UNLESS we have tools

Conventional Tools

Copyright @ 2015 SIOS Technology Corp. All rights reserved. 19

Data

Still Dealing With Noise

Conventional Tools

State of The Productized Art





What is the problem? Why Conventional Tools Can’s Solve It?

• Understanding of Data Semantics because: • Data is Big

• Data is complex

• Data is high-dimensional (with a lot of interrelationships and interplay)

• Data is features rich

• individual measurable property of a phenomenon being observed (examples of the data features: latencies, IOPS, etc.)

• Solution required different type of tool


What if there was a solution?


Topological Behavior Analysis


Topological Data Analysis (TDA)


TDA

Mathematics (Topology) Data

Shape



NOISE



DATA HAS SHAPE


DATA HAS MEANING


Now back to user’s problem…



Topological Behavior Analysis (TBA)



IT Operations Is A Big Data Problem


Noise


SIOS iQ – Applying Big Data Principles Topological Behavior Analysis


Topological Behavior Analysis & Machine Learning


Topological Behavior Analysis & Machine Learning



39

What’s Next?

Self-Driving Datacenter

40

41

Why should IT

brush up on

Algorithms?

Expert Knowledge + Machine Intelligence

42

43

Stay Relevant - Education

Machine Learning Course: https://www.coursera.org/learn/machine-learning

https://www.coursera.org/learn/machine-learning



44

Stay Relevant - Education

Data Scientist Certification Program: https://www.coursera.org/specialization/jhudatascience/1

https://www.coursera.org/specialization/jhudatascience/1

DIY – Machine Learning

http://www.r-project.org/

http://www.cs.waikato.ac.nz/ml/weka/

http://madlib.net/

https://mahout.apache.org/

http://jubat.us/en/







http://madlib.net/

https://mahout.apache.org/

http://jubat.us/en/

46

How about those latencies?

Anomaly Detection – K-means Clustering

47

Outlier Detection Using K-means

Anything outside 3 standard deviations is an outlier

Social Graph -> IT Graph

http://neo4j.com/

FlockDB

https://github.com/twitter/flockdb

http://giraph.apache.org/

http://neo4j.com/

http://neo4j.com/

https://github.com/twitter/flockdb

http://giraph.apache.org/

49

Neo4j

https://www.vmware.com/support/pubs/sdk_pubs.html

API

https://www.vmware.com/support/pubs/sdk_pubs.html

50

Want to see it

in Action? http://us.sios.com/

SIOS iQ™ Machine Learning Analytics for optimizing

VMware environments

52

SIOS iQ™ Machine Learning Analytics for optimizing VMware environments

Participate in our SIOS iQ™

usability study and have a coffee on us!

53

• Always Interested

Job Opportunities @ SIOS

Great People. Exciting and Innovative Projects.

Data & Analytics

Machine Learning The Key Ingredient to Self-Driving Data Center