Elatic{on}16 recap

Elastic{ON} ’16 Recap

Shravanthi Denthumdas

Solution Architect

@shravanthid

Agenda

• Elastic Landscape Evolution

• What’s New in

– Elasticsearch

– Logstash

– Beats

– Kibana

• My Favorite Talks/Use Cases

ELK Stack is now…

Elastic Stack

Versions

Versions

Extensions

Shield, Marvel, Watcher, Graph and more..

Extensions

bin/kibana-plugin install file:../xpack-5.0.zip

bin/elasticsearch-plugin install file:../xpack-5.0.zip

Extensions

Cloud

• Elastic Cloud (formerly “Found”)

• Elastic Cloud Enterprise (beta)

What’s Evolving?

0.x to 1.xCorruptionSplit BrainSecurity Exploits OOM

1.4Checksums EverywhereCluster ResiliencyScripting Locked DownDoc Values made faster

2.X – Fix All The Things

• Mappings Refactoring• Translog Reliability• Usability

– Query DSL– Merge Autothrottling– Structured Exceptions – Admin Safeguards

• Heap Usage - Better Data Structures• Security• Improved Test Coverage• Opinionated and Lenient• Query Profiler

Reindex API

Update by Query

and more…

5.0• Ingest Node

5.0• Ingest Node API

5.0

• Painless Scripting

• Java HTTP Client

• Data Structures

– String Mappings

{ "type": "text", "index: true }

{ "type": "keyword", "index: true }

– Point Field Encoding

and more…

What’s Latest?

Data Delivery

Today

Solution

1.x

5.0

2.x

5.x - Management

• Node Stats API

• Hot Thread API

• Monitoring Logstash in Kibana

• Better Config Management

– Dynamic config reloading

– Centralizing configs as documents in Elasticsearch

• Clustering and Load Balancing

and more…

What’s Brewing?

What is Beats?

“Beats is the platform for building lightweight, open source data shippers for many types of operational data you want to enrich with Logstash, search and analyze in Elasticsearch, and visualize in Kibana.”

- Elastic

Elastic Beats

Packetbeat Live Demo

Community Beats

5.0

• MetricBeat

• Redis & Kafka output support

What’s cookin?

Plugins

FiltersPin Filters

Edit Filters

Aliases

CustomizationColors

Axis Labeling

And More..

• Manageability

– Export Everything

– Status Page

Sessions

64 sessions in 3 days

My top 5• Fighting Human Trafficking, Fraud, & More: Elastic @

Giant Oak• How HotelTonight Finds the Best Hotels in the Moment• Bringing Healthcare Analytics to the Point-of-Care @

Mayo Clinic• Dude where are my messages? Message Analytics

@Netflix• How the Elastic Stack Changed Goldman Sachs

Questions?

References

The content in this presentation is based on Elastic{ON} ’16 talks.