Upload
couchbase
View
137
Download
4
Tags:
Embed Size (px)
Citation preview
INTUIT’S JOURNEY TO MICRO-SERVICES
Umed Zokirov | IntuitJustin Michaels l Couchbase
©2015 Couchbase Inc. 3
Agenda
Micro-Services and Couchbase Technology Justin Michaels, Solution Architecture Team from
Couchbase Use Case: Intuit’s Journey
Umed Zokirov, Lead Developer and Application Architect from Intuit
Micro-Services and Couchbase Technology
©2015 Couchbase Inc. 5
Framing the Discussion
• Micro-Services • Architecture strategy that provides individual services
focused on supporting finite independent services.
• Monolithic • Single application handling UI presentation, business
logic, data access and integration
©2015 Couchbase Inc. 6
Framing the Discussion
• The Art of Scalability• Martin Abbott … formerly COO of Quigo, spent nearly six
years at eBay, most recently as SVP of Technology/CTO.• Michael Fisher … veteran software and technology
executive, spent two years as CTO of Quigo. Previously, as VP of Engineering & Architecture for PayPal.
• “Scale Cube”
©2015 Couchbase Inc. 7
Micro-Services: Scale Cube
Y-Axis – Scale by isolating services
X-Axis – Scale by System/Application Duplication
Z-Axis – Scale by partitioning data
©2015 Couchbase Inc. 8
Micro-Services: Scale Cube
Travel Site
Application Container
Load BalancerBrowser User Interface Product Information Airline Schedule Price Information Recommendations
©2015 Couchbase Inc. 9
Micro-Services: Scale Cube
Travel Site
Application Container
Load BalancerBrowserTravel Site
Travel SiteTravel Site
Travel Site
Application ContainerPricing Engine
Application ContainerApplication Container
Schedule Service
Application ContainerApplication Container
Product Catalog
Application ContainerApplication Container
Recommendation Engine
Application ContainerApplication ContainerData Service
Y-Axis – Scale by isolating services
X-Axis – System/Application Duplication
Z-Axis – Scale by partitioning data
©2015 Couchbase Inc. 10
Micro-Services: Couchbase Technology
• Managing the Scale Cube• Partitioning Strategy (Z Axis)
ACTIVE ACTIVE ACTIVE
REPLICA REPLICA REPLICA
Couchbase Server 1 Couchbase Server 2 Couchbase Server 3
ACTIVE ACTIVE
REPLICA REPLICA
Couchbase Server 4 Couchbase Server 5
SHARD5
SHARD2
SHARD SHARD
SHARD4
SHARD SHARD
SHARD1
SHARD3
SHARD SHARD
SHARD4
SHARD1
SHARD8
SHARD SHARD SHARD
SHARD6
SHARD3
SHARD2
SHARD SHARD SHARD
SHARD7
SHARD9
SHARD5
SHARD SHARD SHARD
SHARD7
SHARD
SHARD6
SHARD
SHARD8
SHARD9
SHARD
©2015 Couchbase Inc. 11
Micro-Services: Couchbase Technology
• Managing the Scale Cube• System Duplication (X Axis)
Memory Resources
Cluster Capacity
Compute Resources
Storage Resources
node1 node8
©2015 Couchbase Inc. 12
Micro-Services: Couchbase Technology
• Managing the Scale Cube• Independent services for Query, Index and Data• Independent scalability for capacity per Service• Data access provided by distributed cache
Index ServiceGlobal
Secondary Indexes
Couchbase Cluster
Query Service
Data ServiceViews and Geo Views
node1 node8
©2015 Couchbase Inc. 13
Micro-Services: Couchbase Technology
Heavier indexing (index more fields) : add compute to index service nodes
Increased query load: linearly scale query serviceMore data: linearly scale data service
Couchbase Cluster
node1 node8 node9
Data Service
Index ServiceQuery
Service
©2015 Couchbase Inc. 14
Micro-Services
AirportsTravel
ServicesB2C/B2B
Travel Agent
Application Tier
Mainframe
Couchbase Cluster
Read requests go to Couchbase for high performance and saving MIPS
Data updates are written directly to the mainframe. Event Engine Push UpdatesDocument TTLs
RDMS
Intuit Confidential and Proprietary15
Micro-Services at Intuit
Intuit Confidential and Proprietary16
Intuit operates in 2 market segments
• Consumer Finance Software
• TurboTax – market leader in tax preparation software
• Quicken – day to day financial management
• Mint – personal finance management
• Small Business
• QuickBooks – small business accounting package
• TurboTax & Lacerte – tools for accountants and tax professionals
Introduction
Intuit Confidential and Proprietary17
• TurboTax Online is 20 years old, started its life as the port of the desktop software.
• Monolithic architecture was in its DNA
• Grew to be a 1.5 billion dollar business serving millions of customers
• Peak tax season the application supports 250,000 concurrent users who interact the application on average 35 minutes per session.
• Changes needed to support additional growth.
History – TurboTax Online
Intuit Confidential and Proprietary18
TurboTax Online Legacy Architecture
Intuit Confidential and Proprietary19
• Challenges of Monolithic Architecture
– Each change requires complete build, regression tests
– Difficulties to A/B test, scale
– 4 month tax season and organizational pushback
– Teams aren’t productive or innovative due to legacy code base
• Applied divide and conquer approach, i.e. micro-services around key pieces of functionality
History – TurboTax Online
Intuit Confidential and Proprietary20
• Ease of piece-by-piece deployment vs. monolith
• Teams are free to choose their technology stack
• Flexible testing
• Quick iterations focused on concrete features
• Multiple A/B testing strategies
• App vs. Micro-service. Some services offer UI widgets.
Why Micro-Services
Intuit Confidential and Proprietary21
TurboTax Online = Micro-Services and Apps
Intuit Confidential and Proprietary22
• Increased chatter and traffic between services
• Multiple services request and consume virtually the same data within milliseconds
• Resiliency issues with multiple app and services accessing the data store that can’t be scaled quickly enough.
• Issues with accessing services that we don’t own, external to Intuit.
• Hard to manage end-to-end user experience among multiple apps
Challenges of Approach
Intuit Confidential and Proprietary23
• Alternatives
• Go through and refactor service-by-service and address issues
• Strategically create a cache tier to reduce chatter between services and applications, hold frequently accessed data
• Data mine the request patterns, type of data being demanded, traffic flows and analyze use cases to see if caching is a good fit, what’s their tolerance of possibility of viewing stale data
• Evaluated multiple technology usage within as well as outside Intuit to choose Couchbase
Solution
Intuit Confidential and Proprietary24
Cache Service via Couchbase
Intuit Confidential and Proprietary25
TurboTax Online + Couchbase
Intuit Confidential and Proprietary26
• Reduce read response times by 50%
• Reduced intra-service chatter by 20%
• Cut write response times by factor of 4
Cache Service via Couchbase
Intuit Confidential and Proprietary27
• In-memory key-value store
• Easy to use Java SDK
• Support for XDCR and ease of deployment
• Good monitoring integration that plugs easily to existing monitoring solutions such as AppDynamics and Splunk
Why Couchbase
Intuit Confidential and Proprietary28
• Key part of Service Platform Infrastructure
• More and more teams are interested in using it
• Data Mining and Analytics
• Write-behind cache solution for Data Platform
• Replacement for Memcached, Ehcache
Couchbase At Intuit
Intuit Confidential and Proprietary29
Thank You