Better, Faster, Stronger! Migration to 3.0
© 2015 A
madeus IT G
roup SA
Upgrade MongoDB in the enterprise environment
Attila Tozser, Amadeus Data Processing GmbH, MongoDB Days
Germany, 20/10/2015
Amadeus IT Group1
© 2015 A
madeus IT G
roup SA
Amadeus is a technology company dedicated to the
global travel industry.
We are present in 195 countries with a worldwide team of more than 13,200 people.
Our solutions help improve the
business performanceof travel agencies, corporations, airlines,
airports, hotels, railways and more.
_ Introduction1Amadeus in a few words
© 2015 A
madeus IT G
roup SA
Our solutions facilitate journeys and enrich the travel
experience for hundreds of millions of people every year.
to shape the future of travel
_ Introduction1
We are working together with
our customers and partners
We have a clear purpose
© 2015 A
madeus IT G
roup SA
Page 5
A history of shaping the future of travel _ Amadeus history2
1987Amadeus is founded
1995World leader in number of travel agency locations
19981 MILLIONbookings made on a single day for the first time
2010Amadeus diversifiesinto IT for hotel, railand airport
1992First booking made through Amadeus
1996e-Commerce division launched
2000Partnership with BA and Qantas to launch Amadeus Altéa – our core Airline IT offering
2014Contracts with Ryanair and Southwest Airlinesand a strategic technology relationship with IHG
© 2015 A
madeus IT G
roup SA
Page 6
_ Amadeus today3Robust global operations
© 2015 A
madeus IT G
roup SA
1.6+billiondata requests processed per day
526milliontravel agencybookings processed in 2014
695millionPassengers Boarded (PBs) in 2014
95%of the world’s scheduled network airline seats
Page 7
Close to our customers
© 2015 A
madeus IT G
roup SA
Page 8
A global team of 13,200+ professionals
_ 118 nationalities
_ Cultural diversity
_ Clear focus on recruiting and developing talent
_ Values based culture
MongoDB at Amadeus2
© 2015 A
madeus IT G
roup SA
Page 10
Business needsGathered from customers
user-friendly way to query any datain our main operational databases
_ Unpredictable ad-hoc search_ Many advanced reporting requirements
A way to store orders of magnitude more different data from various sources, but still maintain low latency
Sustain Organic growth of data and throughput (time-to-time 2X or more a year) without degradation
Page 11
Runs on standard x86 architecture
C++, Python & Java drivers
Enterprise-grade security_ SSL encryption
_ Kerberos authentication
_ Data-at-rest encryption
MongoDB is perfect match to our needs
IntegrationIn the Amadeus standards
New features in MongoDB 3.02
© 2015 A
madeus IT G
roup SA
© 2015 A
madeus IT G
roup SA
© 2015 A
madeus IT G
roup SA
WiredTiger our observations• Compression ratio 3x-10x• IO characteristic tunable
• Better latency• Enhanced security
© 2015 A
madeus IT G
roup SA
In-memory our observations• On OOM instant crash
• Sustained 150K/4KB IOPSOn a single box
• 1-6ms latency
Upgrade MongoDB in the enterprise environment
2
© 2015 A
madeus IT G
roup SA
© 2015 A
madeus IT G
roup SA
AmadeusClients
© 2015 A
madeus IT G
roup SA
SS
S
S
S
S
Application servers MongoDB
© 2015 A
madeus IT G
roup SA
S
S
S
S
S
S
Database sever:Operating SystemMongoDB binariesMisc. processes:
Ops manager agentsMonitoring codeAutomation code
© 2015 A
madeus IT G
roup SA
S
S
S
S
S
S
This appserver consist of• Operating System• Application code• MongoDB driver• Mongos binary
© 2015 A
madeus IT G
roup SA
MongoDB Cluster
MMS/Ops Manager
© 2015 A
madeus IT G
roup SA
MongoDB Cluster
MMS/Ops Manager
_Always Sharded and replicated
_Over 10 test levels
_Monitoring
_Backup_(automation)
Mongodb 3.0 needs Ops Manager > 1.6.0
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
MMS application database
BackupDatabase
BackupDatabase
BackupDatabase
MMS App Server MMS App Server MMS App Server
Backup Daemon
Backup Daemon
Backup Daemon
Backup agents
Monitoringagents
Automationagents
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
MMS application database
BackupDatabase
BackupDatabase
BackupDatabase
MMS App Server MMS App Server MMS App Server
Backup Daemon
Backup Daemon
Backup Daemon
Backup agents
Monitoringagents
Automationagents
Upgrade separately as any other MongoDB cluster3.0 only from MMS 1.6
The MMS server process contains the new versions
Should be on the same version: Upgrade together
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
MMS App Server
REST API
Corporate Monitoring
API callsMetrics
Corporate Automation
API callsConfig options
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
MMS App Server
REST API
Corporate Monitoring
API callsMetrics
Corporate Automation
API callsConfig options
Test the changes of the API to avoid bad surprises
© 2015 A
madeus IT G
roup SA
SS
S
S
S
S
http://docs.mongodb.org/manual/release-notes/3.0-upgrade/
_Upgrade MongoDB
_Be prepared for fallback funnies_Backup, MONGODB-CR auth
_Lucky case: Give a try to Ops Manager automation
_Less lucky case: Do it by hand_Start from a 2.6 installation
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
S
Upgrade the Mongos processes
Config 1 Config 2 Config 3
S, 3.xUpgrade cluster metadata
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
S
Upgrade the Mongos processes
Config 1 Config 2 Config 3
S, 3.x
Upgrade all the mongos processes
Upgrading MMS3
© 2015 A
madeus IT G
roup SA
S
Upgrade the Mongos processes
Config 1 Config 2 Config 3
S, 3.x
Upgrade config
3
© 2015 A
madeus IT G
roup SA
PrimaryMMAP v1
2.6
SecondaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
Install binaries
Install binaries
Install binaries
SecondaryMMAPv1
2.6
3
© 2015 A
madeus IT G
roup SA
PrimaryMMAP v1
2.6
SecondaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
SecondaryWT 3.0
3
© 2015 A
madeus IT G
roup SA
PrimaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
SecondaryWT 3.0
SecondaryMMAP v1
2.6
3
© 2015 A
madeus IT G
roup SA
PrimaryMMAP v1
2.6
SecondaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
SecondaryWT 3.0PrimaryWT 3.0
SecondaryWT 3.0
SecondaryWT 3.0
3
© 2015 A
madeus IT G
roup SA
PrimaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
SecondaryWT 3.0
SecondaryMMAPv1
2.6
PrimaryWT 3.0
3
© 2015 A
madeus IT G
roup SA
PrimaryMMAP v1
2.6
SecondaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
SecondaryWT 3.0PrimaryWT 3.0
SecondaryWT 3.0
SecondaryMMAPv1
2.6
SecondaryWT 3.0
3
© 2015 A
madeus IT G
roup SA
SecondaryMMAP v1
2.6
SecondaryMMAP v1
2.6
Upgrade the Shards
SecondaryWT 3.0
PrimaryWT 3.0
SecondaryWT 3.0
Look into the future3
© 2015 A
madeus IT G
roup SA
3
© 2015 A
madeus IT G
roup SA
MY MongoDB wishlist
_ Cross cluster replication
_ Filtered / delayed / batched
_Multimaster (AP operation)
_Multithreaded queries
_ Transactions
_On multi update
Divider title here
1
© 2015 A
madeus IT G
roup SA
Thank you!
Questions?